File Blob Layer — Physical Storage as a First-Class Object

Every uploaded file on the platform splits into two objects:

• File (fil_files) — the logical file. Identity, ownership, visibility, title, the URL a user links to. Everything a user sees.
• FileBlob (fbb_file_blobs) — the physical bytes underneath it. The stored name on disk / in the bucket, size, content hash, storage driver, and offload state. A File references one blob via fil_fbb_file_blob_id.

Any feature that stores bytes — entity photos, inbound-email attachments, AI chat uploads, profile pictures, Drive — gets this for free by going through the one ingestion path below. No feature touches fbb_file_blobs directly.

Storing bytes

There is exactly one byte-ingestion path. Use it; never write a File row and a file on disk by hand.

// From an uploaded temp file (a $_FILES entry, or any staged path):
$file = File::createFromUpload($tmp_path, $display_name, $mime, $owner_id, $restrictions);

// From bytes you already hold in memory (generated / received / fetched):
$file = File::createFromBytes($bytes, $display_name, $mime, $owner_id, $restrictions);

$restrictions is an array of visibility columns set at creation, e.g. ['fil_private' => true] or ['fil_min_permission' => 5, 'fil_source' => File::SOURCE_ENTITY_PHOTO].

Origin tags — where a file came from

Every file carries fil_source, a short key naming the subsystem that created it (File::SOURCE_*; NULL means a legacy row from before the tags). *Stamp one when you store a file, and declare it in File::source_catalog() — the one place that says what each tag is:

Classification lives on the source rather than on the row because it is a property of what created the file: every sealed search index is internal, and there is no user-visible one. So the tag stamped at creation already carries the answer, with no per-row column for a writer to get wrong.

MultiFile filters on origin three ways, and they resolve to exactly one SQL condition: sources (allowlist), sources_not (blocklist), source_not (its single-value form). An allowlist combined with a blocklist is the allowlist minus the blocked entries. NULL rows survive a blocklist — they are not what it names — and can only enter a result through an explicit File::SOURCE_UNCLASSIFIED in the allowlist, because fil_source IN (...) never matches NULL.

Both route through FileBlob::createFromPath(), which:

1. Hashes the bytes (sha256) and measures the size.
2. Dedup lookup — a live blob with the same (sha256, size, is_private)? Hit → retains that blob (refcount +1) and discards the new bytes. Miss → moves the bytes into the correct visibility directory under a collision-free fbb_stored_name and inserts a new blob.
3. Detects the honest MIME type from the bytes (the caller's $mime is only a fallback) and stores it — so fil_type is trustworthy by construction, never by caller discipline.

Nothing else has to. A variant that does not exist yet is generated the first time it is requested, and only the one size asked for: /uploads/<size>/<name> resolves the file, applies the normal is_viewable() gate, and then calls $file->ensure_variant($size) before serving. So a subsystem can store an image without thinking about sizes at all — an inbound mail attachment, a chat upload — and get_url('avatar') still resolves, while an 80px listing never causes a 1920px copy to be written. Generation sits behind the gate on purpose: resizing is real CPU and the requested name comes from the URL, so a caller who may not view the file cannot spend it.

The on-demand path is for local blobs; a cloud variant means download, resize and re-upload, which does not belong inside a page request. Sealed and client-encrypted files are skipped either way — their stored bytes are a container, so resizing them would produce garbage.

Reference counting and deletion

Lifecycle is code-owned (refcounting isn't expressible in declarative deletion rules):

• File::permanent_delete() calls FileBlob::release($blob_id). Release decrements the count inside a transaction guard and, at zero, deletes the physical original + every image variant (local unlink or cloud driver delete) and the blob row. A blob still referenced by another file is left untouched.
• FileBlob::retain($blob_id) is the increment twin (a new file pointing at an existing blob — the dedup path calls it).

Public vs private is a physical placement — public bytes go to the fast-serve directory / public bucket, private bytes to the restricted directory / verified-private bucket — so it is a blob property, fbb_is_private. But it is derived from the referencing files. The invariant: every file referencing a blob is in the same visibility class. It is maintained by:

• Dedup scoping. A dedup match requires equal (sha256, size, is_private). The same bytes uploaded once public and once private become two blobs — never a shared blob straddling both stores.
• Visibility change (File::move_to_correct_directory(), called after save / soft_delete / undelete). When a file's desired class no longer matches its blob's: - refcount 1 → flip the blob (move its bytes between dirs, or pull home from the wrong bucket). - refcount > 1 → copy-on-write split: copy the bytes to a new blob in the target class, repoint this file, decrement the old blob. The siblings keep the shared blob.
• Soft-delete exception. Soft-deleting one of several references leaves a shared blob where it is — the deleted file's own URLs are already gated by File::is_viewable(), and the siblings keep serving. A refcount-1 soft delete flips the blob private (as before); undelete flips it back.

No two live items may share a name in one folder — folders or files — and a partial unique index is what enforces it. Files are scoped to fil_source = 'drive', because fil_files is platform-wide and an avatar, a store image and a mail attachment all sit at folder NULL where duplicate titles are ordinary.

DriveHelper::folder_name_taken() and file_name_taken() are fast paths that give a clean answer before any work is done, but two requests can both pass one and both reach the insert — so every create, rename, move and restore saves through DriveHelper::save_folder_unless_name_taken() or save_file_unless_name_taken(). On a uniqueness refusal these ask the same question again and, if the name is now taken, return false for the caller to answer exactly as it would have a moment earlier; anything else is rethrown.

A file arriving by upload is created rather than saved, so the two upload paths carry the rule themselves: they ask file_name_taken() first and translate a 23505 from the insert into the same refusal. There are two of them, and both matter — drive_upload_complete for a transferred file, and the dedup short-circuit in drive_upload_init, which creates the file itself when the account already holds identical bytes and never reaches completion at all. Both put the destination folder in the creation, so an upload never occupies the root's namespace on its way to where it was going.

A file refusal carries reason: name_taken in the error data. Sync clients branch on that marker rather than on the message, and treat it as a state that resolves rather than a failure: the sibling holding the name is something the next index walk will tell them about, and landing it is what frees the name. Without the rule, two devices that conflict on one file both upload the same conflicted-copy name, the server takes both, and every device can then materialize only one of them — leaving the rest unsyncable forever with nothing the user can do about it, since they cannot see the second file to rename it.

A trashed folder is not a place

Nothing new may be placed into a folder that is in the trash. Every verb that puts an item somewhere — drive_folder_create, drive_upload_init, drive_upload_complete and drive_move — asks DriveHelper::folder_is_trashed() about its destination and refuses with reason: parent_trashed. load_folder() resolves trashed folders on purpose, since listing the trash and restoring out of it both need them, so the check belongs at each placement site rather than in the loader.

The refusal exists because a live item under a trashed parent is invisible and permanent: no listing walks into a trashed folder, so no client can be told where the item lives, no user can see it to move or delete it, and it holds quota nobody can account for. The state arises from an ordinary two-device race — one device writes into a folder while another deletes it — and once the write is taken there is nothing to repair it. Sync clients treat the marker as Overtaken rather than a failure: the index walk carrying the delete also trashes their copy of the folder, and anything inside it the server never took is rescued out on the way.

drive_upload_complete discards the staged upload as it refuses, rather than leaving it to expire — the destination is gone and the transfer has nowhere left to land. drive_restore is not subject to the rule; restoring into a still-trashed parent re-roots the item instead.

One case the rule cannot reach, stated honestly: a Fortress file's fil_title is an opaque enc-… identifier and its real name lives inside encrypted metadata the server cannot read, so two client-encrypted files can share a real name and differ here. That is the price of the server not being able to see the name, and noticing it is the client's job.

Protection levels

Every folder carries a protection level (fol_protection_level), and every file records the level it was stored at (fil_protection_level). Drive shows three rungs of the platform ladder (includes/ProtectionLevel.php):

A refusal reads a file's effective level — the stronger of its own and its folder's (DriveHelper::effective_file_level()). The two disagree on purpose while a level change converges: the folder's column is the truth about what is promised, the file's about what its bytes are. Public links and member grants are refused on the promise, so nothing is minted in that gap only to be revoked when the batch arrives. Byte work — sealing, unsealing, quota — reads the file's own level, which is the one describing what is on disk. A move that would seal a file carrying a live link or grant is refused outright, naming what is in the way.

Changing a level (drive_level_change) is Standard ↔ Private only — the two the server holds a key wrapping for. The folder changes at once, so everything uploaded from that moment lands at the new level; the files already inside are converted afterwards by repeated drive_level_batch calls, each bounded by a byte budget rather than a row count. Raising needs only the owner's vault public key and so runs locked; lowering decrypts and needs the window. Going Private ends any public links and member grants in the subtree — the first call reports them and does nothing until the caller confirms.

Private is described in full in Drive Encryption, which covers both custody models.

Access and grant semantics

A Drive file is private, so File::is_viewable() returns true for the owner or an admin. Sharing adds one clause: a viewer or editor grant on the file, or on any ancestor folder also grants view. The one grant-reach implementation is DriveHelper::grant_reaches() (a recursive-CTE ancestor walk bounded by drive_max_folder_depth); is_viewable and every drive_* action resolve sharing through it. DriveHelper::can_write() adds the same reach for editor — an editor may rename, save a new version, and upload into a shared folder. Delete, trash, and share stay owner-only; an editor grant never confers them. "Shared with me" is FileAccessGrant::entity_ids_for_user; filename search spans the caller's own files plus granted files and granted folder subtrees.

Trees have a single owner. Everything under a folder belongs to the folder's owner: an upload into a shared folder creates a file owned by (and billed to) the folder owner, whoever performs it (the actor is recorded in the change feed), and drive_move only accepts a destination folder owned by the moved item's owner. That invariant is what makes the folder trash and delete-forever cascades — which select by folder — safe: they can never touch another user's rows. It is also why a grantee browsing a shared folder sees the full listing (children are listed by the folder owner's id). A grantee's breadcrumb is cut at the topmost folder that carries a direct grant for them, so the owner's private ancestor names are never exposed.

Grant management is a full-set reconcile: drive_share_sync takes grants (a JSON object mapping each grantee — a user id or an email, resolved server-side — to a role) and FileAccessGrant::sync_for_entity inserts new grants, updates changed roles, and hard-deletes ones no longer in the set (revocation is a row deletion; there is no delete_time). Newly-granted users get a drive_share notification, gated by their NotificationPreference.

Public links (drive_link_create, owner-only, gated by the drive_share_links tier feature) mint a FileShareLink: the raw token is returned once, only its SHA-256 is stored. A live link (not revoked, not past fsl_expires_time, password satisfied) grants view at /s/{token} — anonymous, no login. The page streams files through a short-lived signed URL and renders folders as a read-only listing scoped to the shared subtree; the link is the durable revocable grant, the signed URL is the transport. drive_link_revoke stamps fsl_revoked_time.

Encrypted files (files inside an encrypted vault folder) add a second layer to the share dialog: a FileAccessGrant grants access, and a FileKeyGrant grants readability — the owner's browser wraps the file key to each recipient's Drive vault public key, and an upload seals the new file's key to the destination's full reader set. An encrypted vault folder exists only at the Drive root or inside another vault, so a plaintext subtree never hides encrypted content. Encrypted files rename via their re-encrypted metadata (drive_rename with encrypted_metadata), never a plaintext name. Public links carry the file key in the URL fragment and are single-file only (encrypted folders can't use them). See Drive Encryption.

Upload protocol (resumable, sequential chunks)

Uploads never overwrite bytes directly — they flow through the one blob-ingestion path, so dedup and accounting are automatic. The protocol is the complete server contract for sync clients:

> This protocol is not Drive-only. It is the platform's route for any file > larger than a single request, and other subsystems use it by passing a purpose > to drive_upload_init — see API § Uploading something that is not a Drive > file. Everything described > below is the drive purpose specifically: quota, folders, encryption and the > dedup short-circuit belong to Drive and do not run for others.

1. drive_upload_init — `{name, folder_id?, file_id?, size_bytes, sha256?, mime_type?}`. Gates folder/file write access, then the per-file size (drive_max_file_bytes; for an encrypted destination the gate is DriveHelper::encrypted_size_ceiling() — the cap plus the client container's fixed per-chunk overhead, since the cap means plaintext bytes and the upload arrives as ciphertext) and quota (`bytes_used + size_bytes <= drive_storage_bytes`) of the owner who will be billed — the target file's owner, else the destination folder's owner, else the actor. If sha256 matches a private blob the actor already possesses it short-circuits: retain the blob, create the File (or a new FileVersion when file_id is set), and return the file — no bytes transferred. Possession means the blob is already referenced by one of the actor's own files or file versions (FileBlob::find_dedup's $possessed_by_user_id): a client-claimed hash is not proof of content, so a foreign hash+size never matches — no cross-user disclosure and no existence oracle. Otherwise it creates a FileUpload and returns {upload_token, chunk_bytes} (the raw token; only its hash is stored).
2. PUT /api/v1/drive_upload/{token} — the raw-body chunk transport (a pre-CRUD branch in api/apiv1.php, the inbound twin of management/backups/fetch). Chunks are sequential only: the request carries `Content-Range: bytes <start>-<end>/<total>, and <start> must equal the server's received_bytes` or the response is 409 with {received_bytes} so the client resumes from the right offset. Appends are serialized per upload with a Postgres advisory lock, and the offset is re-read under the lock, so concurrent PUTs for one token cannot interleave writes. GET on the same path returns {received_bytes, expected_bytes}. Scratch bytes accumulate in a part-file under {site_root}/storage/drive_uploads/, outside the web root. Chunk requests use their own api_upload rate-limit bucket so a large upload never drains the general API budget; the transport's response helpers write the request-log row with the actual outcome (success on 2xx, failure with the status code otherwise).
3. drive_upload_complete — {upload_token}. Verifies the byte count and (when given) the sha256, re-validates write access, and enforces the quota here, where bytes are admitted to storage: under a per-owner advisory lock it recomputes usage fresh and rejects the complete if the upload would land past drive_storage_bytes (the init check is only a fast-fail — N uploads opened while under quota cannot all complete past it; a rejected upload keeps its pending row so the user can free space and retry). Then it ingests through FileBlob::createFromPath (server-side dedup applies — the server hashed the actual bytes), creates the File owned by the folder owner — or a new FileVersion when the upload targeted an existing file — recomputes the owner's usage, records the change, and clears the pending row. Safe to retry, and covered by the standard Idempotency-Key machinery. For an encrypted destination the complete also carries the opaque key/metadata payloads, and an encrypted version upload must reuse the file's key — see Drive Encryption for that contract.

Saving new content to a file demotes the current head blob to a FileVersion and repoints the file at the freshly-ingested blob — the head blob's reference is transferred to the version row, so refcounts stay correct without a retain/release. Restoring swaps the roles back. Both prune to the owner's drive_versioning_depth (oldest versions released and deleted). Because a version IS a reference to its blob, FileVersion::permanent_delete releases it, and the file's fvr_fil_file_id rule is permanent_delete so deleting a file releases every version's bytes. drive_versions lists a file's history; drive_version_restore promotes one back to head. The web UI uploads a new version by sending file_id to drive_upload_init.

Quota accounting

DriveUsage is recomputed, never incremented — DriveUsage::recompute($user) sums the blob sizes of the user's Drive files (fil_source = 'drive') plus the versions of those files, inside the caller's transaction, after every upload / new version / restore / permanent delete. Version bytes bill the file's owner (a version row's fvr_usr_user_id records who saved it — audit only, since an editor may save a version of someone else's file), and an upload into a shared folder bills the folder owner, per the single-owner-tree rule above. Each logical file bills its full size even when its bytes are deduped onto a shared blob (dedup saves disk, not quota); trashed files count until purged. DriveUsage::current_bytes($user) is a row-free read for the storage meter (the /drive page render is a GET and must not create a row). The daily DriveUsageReconcile task re-runs the sum for every file-owning user as a drift backstop.

Trash

Trashing soft-deletes; DriveHelper::soft_delete_folder_cascade deletes the folder first (earliest timestamp in its cascade) then every descendant, so restore_folder_cascade — which restores only descendants with delete_time >= the folder's own — leaves a child trashed independently earlier in the trash. "Delete forever" runs an impact preview (DriveHelper::delete_impact) before permanent_delete_tree destroys the subtree (a raw folder permanent-delete instead orphans its files to root via the fil_fol_folder_id → null rule). The daily retention sweep permanently deletes Drive items (fil_source = 'drive', plus folders) trashed longer than its window (default 30 days) — a soft-deleted file belonging to another subsystem is that subsystem's to reclaim and is never touched. Restoring a folder whose parent is still in the trash re-roots it (with a name-collision suffix when needed) so it never reappears inside an unreachable parent.

Change feed

Every mutation records one FileChange via `FileChange::record($kind, $entity_type, $entity_id, $owner_id, $actor_id) — kinds created, content`, renamed, moved, trashed, restored, deleted, grant_changed. The feed is append-only and the primary key is the cursor. drive_changes takes a {cursor} and returns the changes after it that the caller may see — their own entities plus entities shared to them — with next_cursor. A nonzero cursor that cannot be proven contiguous with the retained window — it points before the earliest retained row, or the log is empty after a purge — returns {reset: true} so the client re-lists from scratch rather than silently missing changes. The daily retention sweep trims rows past drive_change_feed_retention_days (default 30).

Sync clients

A desktop client keeps a folder on a computer matching the user's Drive. Four server surfaces serve it, on top of the change feed above. The client itself — its state model, conflict policy, per-filesystem naming rules, and health model — is Drive Sync Client.

Content identity on exports

DriveHelper::file_export carries three fields a syncing client needs and a browser ignores:

Management. drive_devices, drive_device_rename, drive_device_revoke, surfaced on /profile/security alongside App Sessions. Revoking unlinks the device and* revokes its session key; a device row without its credential revoked would be a list that lies. Bytes already downloaded stay on that computer — what stops is future access.

Liveness comes for free. When the caller of drive_changes maps to a SyncDevice, the handler stamps sde_last_seen_time (throttled to hourly, like apk_last_used_time) and sde_last_cursor from the request. There is no separate heartbeat call and therefore no client that can forget to send one, and the security page can say "last synced 4 minutes ago" — which is what makes a stalled device visible instead of silent.

Settings and tier features

Settings (settings.json): drive_active ('0'), drive_max_folder_depth ('32'), drive_upload_chunk_bytes ('8388608'), api_upload_rate_limit_requests ('10000'), api_upload_rate_limit_window ('3600').

Tier features (includes/core_tier_features.json), read via SubscriptionTier::getUserFeature($uid, $key, $default): drive_storage_bytes (total quota; 0 disables uploads), drive_max_file_bytes (per-file cap; 0 disables), drive_share_links (boolean), drive_versioning_depth (versions kept). All default to the fail-closed value for a tierless member.