Skip to main content

Upload Token System

Upload Token System

CURRENT STATE - GENERIC INTAKE SYSTEM

Project: Generic Intake System

Status: In development

Ultimate goal: A self-service, reusable, aesthetically integrated intake framework that lets authorized nontechnical Ready Room users create purpose-specific public file requests without webmaster assistance. The underlying upload engine remains generic for future uses.

Current accepted functionality:

  • Drupal core managed_file owns automatic initial file transfer.
  • Generic file acceptance is proven. Format-specific diagnosis is closed absent contradictory evidence.
  • Token-specific private storage under private://uploads/[token]/ is proven.
  • Finish submission and permanent-file finalization are proven.
  • Reusable token behavior is proven.
  • The current Pixel/Chrome public upload presentation is accepted.
  • The redundant generic Upload title is closed work.
  • Public Drupal status-popup behavior was corrected toward the established inline PageMessage pattern.

Current implementation slice: request data model, Ready Room Upload Requests dashboard, and simple nontechnical request creation.

Next implementation principle: preserve the proven uploader/finalization engine and evolve the surrounding request-management system. Do not rewrite the working upload path merely to implement the broader framework.

Closed work: extension and MIME enumeration, image-versus-video diagnosis, synthetic upload triggering, custom serializer/queue transfer experiments, generic Upload page title, and repeated reproving of accepted core transfer/finalization behavior.

Reopen closed work only when: new contradictory functional evidence demonstrates that the accepted behavior is no longer true.

Current Agreed Product Decisions

  • The Ready Room menu links directly to an Upload Requests dashboard. Individual public requests do not become menu entries.
  • Granular Drupal permissions, assigned to roles, govern creation, management, reassignment, lifecycle actions, viewing submissions, and permanent deletion.
  • Request-specific instructions are entered once and dynamically reused in both the invitation email and public upload page. The live page is authoritative when instructions change.
  • The underlying uploader remains format-agnostic even when a request expects ZIP files, photographs, video, documents, or other material.
  • Public size language uses industry-standard decimal MB where 1 MB equals 1,000,000 bytes, plus the exact comma-separated byte count where useful.
  • The configured limit is per individual file, not the aggregate of multiple files in a submission.
  • Large collections are split among multiple files or ZIP archives rather than reducing original photo or video quality merely to fit a limit.
  • Removal before Finish submission is required. Each file gets a clear individual Remove action rather than Drupal selection checkboxes.
  • The pre-finish manifest shows filename, MB, exact bytes, status, and Remove, plus file count and aggregate size.
  • Resumable transfer is desirable for large files if a maintained Drupal-compatible solution can provide it simply. Do not build a custom resumable-upload platform.
  • SA controls stored/internal filenames for sortable, collision-resistant recovery. Original contributor filenames remain preserved as metadata. Database fields, not filename parsing, remain authoritative for Views.
  • Each request has an immutable human-friendly ID and each Finish submission has a batch identity.
  • Useful provenance is retained, including original/generated filename, request, batch, known source/contributor information, timestamps, size, and relevant supplied metadata.
  • Exact duplicate hashes may be retained for integrity and later duplicate identification, but exact duplicates are not automatically rejected because provenance can differ.
  • The responsible party determines the intended disposition of each request. Receipt and automatic publication are separate concepts.
  • Original files are preserved by default, while storage and retention policy may be revisited if storage becomes a material constraint.
  • ZIP preservation or extraction depends on purpose, permissions, and downstream workflow rather than one universal rule.
  • Copyright remains with whoever legally owns it. Upload does not itself transfer copyright to SA.
  • Contributors should represent, to the best of their knowledge, that they are entitled to provide the material and are not knowingly violating another persons rights.
  • Applicable photo policy grants SA appropriate continuing non-exclusive permission to preserve, reproduce, migrate or convert, adapt for presentation, publish, and use contributed material in Association activities and publications.
  • SA remains the sole arbiter of whether and under what conditions photographs administered or distributed through the SA website are provided or authorized for third-party use, without claiming copyright SA does not own.
  • Known photographer/source and SA credit may both be used publicly as appropriate, while full provenance remains preserved internally.
  • The upload system does not ask contributors to manage future contact requests. SA may forward a legitimate contact request to the known owner/source. The requester is told that no response within the defined period means no permission or contact. Silence is never consent.
  • Photo-oriented requests may support both a batch description and optional per-photo descriptions plus other appropriate configurable metadata.
  • The exact applicable rights/agreement version, request, batch, and acceptance time are retained.
  • The existing photo-contribution workflow is superseded only after the generic framework proves equivalent or better required functionality.
  • Successful submission normally produces one internal summary per completed batch.
  • Requests may generate up to two sensible expiration reminders, but any successful submission cancels outstanding reminder prompting for that request.
  • The system does not automatically nag the contributor. The creator can resend the active request and current dynamically generated instructions.
  • Successful received material is not automatically deleted. Truly abandoned temporary material may be cleaned up conservatively under documented policy.
  • Arbitrary contributed files are treated as untrusted until appropriately reviewed or scanned. Prefer a simple maintained scanning mechanism rather than custom antivirus infrastructure.
  • Desktop, tablet, and mobile are supported where practical. Device parity must not make important functionality unreasonably complex or impossible.
  • After Finish submission, show a meaningful receipt before offering Upload another batch while the request remains active.
  • Errors remain inline and explain what happened, why, and what the contributor should do. Successfully transferred files are not needlessly retransmitted because another file failed.
  • KISS governs implementation.

Implementation order: request data model and Ready Room dashboard; shared instructions and email workflow; public manifest/removal/size/receipt/error UX; naming/batch/history/accounting/provenance/hash; permissions/expiration/reminders/lifecycle; then evaluate maintained resumable upload and malware scanning; replace photo-contribute only after parity is proven.

Deterministic AI instruction: routine work on this project must begin by rereading the ACTIVE RULES block and this CURRENT STATE block. Deeper historical or diagnostic material is supporting evidence and must not override current accepted truth unless new evidence changes that truth.

Upload Token System

This is the authoritative reference for the generic public upload-token product implemented in skyhawk_site_fixes.

Architecture

  • Public route: /upload/{token}.
  • Admin route: /admin/skyhawk/upload-tokens.
  • The public route is owned by UploadTokenForm.
  • Drupal core managed_file owns file selection, AJAX transfer, validation, and temporary file entities.
  • Files are stored under private://uploads/[token]/.
  • No custom synthetic mouse event or parallel direct-upload controller participates in the live product.

Public Upload Page Workflow

  • The public page has no redundant generic Upload page title. Its identifying heading is Upload Files: [label].
  • The secret token value is never displayed.
  • Selecting files is the upload action. Drupal core managed_file immediately starts AJAX transfer from the file-input change event.
  • Successful initial uploads become temporary Drupal managed files under private://uploads/[token]/.
  • The final action is Finish submission.
  • Skyhawk custom-form results never use Drupal messenger/status popups. Upload/finalization results render inline on the form using the established PageMessageTrait pattern required by Tier-1 Rule 8.3.

Token Lifecycle

  • A token is valid when it exists, has status active, and has not expired.
  • Generated tokens have a minimum lifetime of seven days.
  • A token may be used for multiple upload sessions until expiry or explicit administrative revocation.
  • A successful upload does not consume or invalidate the token.
  • uses_count records cumulative finalized files for administrative information only.
  • uses_allowed = 0 is retained as the unlimited-until-expiry/revocation sentinel for schema compatibility.

Upload Rules

  • Multiple files may be selected.
  • Maximum individual file size is 511 MiB (535,822,336 bytes).
  • The hosting transport ceiling remains 512 MiB for an HTTP POST, so exceptionally large batches may need to be selected/uploaded separately even though each individual file is legal.
  • After Drupal receives temporary managed files, final submission makes them permanent, records each file in skyhawk_upload_file, increments cumulative file accounting, and sends the configured batch notification.
  • Completion notifications contain only the files finalized in that batch, not every historical file associated with the token.

Administration

  • The admin form creates reusable tokens and sends the upload URL by email.
  • The admin form can explicitly revoke an active token.
  • Existing contact and purpose data are reused.
  • The current history table remains operational; conversion to a Drupal View is a later Views-first improvement, not part of the uploader replacement.

Replacement State

The public and admin upload forms were replaced in toto after a discovery audit found contradictory one-use lifecycle logic, a redundant unrouted direct-upload controller, and a custom mobile workaround that duplicated Drupal core behavior.

The old UploadTokenController and upload_token_mobile_fix.js are retired into the timestamped replacement backup rather than destroyed during initial acceptance testing.

Functional acceptance remains pending until the Pixel test verifies: legal multi-file upload, final submission, private storage, history rows, notification, reuse of the same token, explicit revocation, expiry rejection, and oversize rejection.

Current Functional State

Generic initial upload: ACCEPTED. Drupal core managed_file selection and automatic AJAX transfer to token-scoped private storage are proven. File-format diagnosis is closed.

Finish submission permanent-file transition: ACCEPTED. Direct Pixel testing proved selection, automatic upload, Finish submission, permanent managed-file state in private://uploads/[token]/, and reset of the form for another batch.

Public upload-page UX: ACCEPTED on Pixel/Chrome. The redundant generic Upload title is absent. Upload Files: [label] remains as the identifying heading. The repaired public-page presentation has passed direct browser inspection. This branch is closed absent contradictory evidence.

Current acceptance boundary: upload history. The first remaining unproven transition is whether Finish submission creates the required skyhawk_upload_file history row for each finalized file. Do not reopen generic upload, permanent-file finalization, or the accepted Pixel UX while resolving history.

Current Regression Token

No token label is permanently reserved as the regression token. Use a currently active, unexpired test token from the existing token system. Do not create a replacement token merely because an upload test fails.