File Storage#
A space can hold a number of files. Depending on their space-access-privileges, a collaborator can create (= upload) new files, read, overwrite or delete existing files.
Files#
A file consists of two elements:
- payload
- metadata
The metadata of a file contains:
- the path and filename in absolute terms, based on the root of the space
- the mime type
- size, optional intended size, in Bytes
- optional timestamps (
createdAt,modifiedAt,accessedAt,deletedAt)
When a collaborator renames a file, by changing path and/or name, the virtual directory tree might be modified. As an example, a file can be renamed and moved from /Calculations/draft/marketing v13 beta.xlsx to /Calculations/final/marketing.xslx in one operation.
The path-and-name of all files within a space must be distinct. Creating a file with a path-and-name or renaming a file where the target exists already will be rejected by the server.
The path of an uploaded file must exist already, before a file can be uploaded into it.
Upload and Download#
Upload and download operations with content-range are supported within limits.
- Spaces does not allow random access writes. A file can either be written from start byte 0, or appended to its current final byte.
- Note: Upload-resume, or appending to an existing file, is not yet supported.
- Specifying a start byte of 0 for existing files will overwrite all previously stored payload.
- Concurrent writes to files are not allowed. While one write operation is ongoing, no other write operation is allowed.
- A download request can cover the full payload or request a range to bytes.
Detecting a broken upload#
Spaces allows multiple users to collaborate within the same space. When one person adds or overwrites a file, others are notified and can observe the progress of the upload. An ongoing upload may end prematurely for various reasons:
- Intended: deliberate cancellation of the upload; uploading large files in chunks (Upload resume).
- Unintended: network interruption; client or server outage; mobile app forced into hibernation.
Spaces does not automatically "clean up" or remove partial uploaded files, instead the collaborators of a space must do that. To support them in this decision, the original uploader can specify the intendedSize of the new object prior to uploading it. This value, together with the actual size, is included in the file metadata and space progress events.
| Actual size (x), Intended size (y) | Status | Meaning |
|---|---|---|
x==0, y==0 |
Complete | Zero Byte object, eg. directory. |
x==0, y>0 |
Initiated | Metadata exists. Upload has not yet started. |
x>0, x!=y |
Incomplete | Metadata exists. Upload has been started. As long as ProgressItems arrive, this upload is alive. |
x>0, x==y |
Complete | Metadata exists. Upload has been completed. |
Broken uploads do not trigger post-processing.
Directories#
Spaces emulates the file system functionality of directories by using regular files with the mime-type inode/directory and no payload.
Directory-files must be explicitly created for every element of the path, with the exception of the root-directory. The root-directory cannot be deleted or moved. An existing directory cannot be renamed into the root-directory.
Since directories are implemented as regular files, the same uniqueness constraint applies to them.
Directory-files have no payload.
The metadata of directory-files can be updated as with regular files. However, some operations have wider consequences:
- renaming a directory will recurse into and apply to all its child directories and files
- deleting a directory will recurse into and delete all its child directories and files
Trash#
When a file or directory is deleted by a collaborator, it is removed from the directory-tree of the space and transferred to the trash area. The metadata field deletedAt is set to the current timestamp. The payload is not removed permanently. Files in trash still contribute to the quota of the space (and organization).
In the trash, the unique constraint on path-and-name is lifted. Multiple files with the same path-and-name can exist in the trash.
While in trash, files cannot be retrieved. However, a collaborator can un-delete a file and move it back from trash into the directory tree, at which time the unique-constraint applies again. If the target path-and-name is already used, the collaborator must provide a new path-and-name for the undelete operation.
To free up quota and permanently delete files (and their payload), individual files in trash or the complete contents of the trash can be purged. These objects become orphans, having no connection to other entities anymore. The DeleteOrphanedObjectsJob will clean them up at regular intervals in the background.