fotospiel-app/docs/ops/media-storage-spec.md

Media Storage Ops Specification

Purpose

This document explains how customer photo uploads move through the Fotospiel platform, which services handle each stage, and what operators must monitor. It complements the PRP storage chapters by focusing on day-to-day operational expectations.

High-Level Flow

1. Upload (web / API containers)

   • Guest/Tenant requests hit the Laravel app container.
   • EventStorageManager::getHotDiskForEvent() resolves the current hot storage target for the event (usually the local disk mounted at /var/www/html/storage/app/private).
   • Controllers call Storage::disk($hotDisk)->putFile("events/{event}/photos", $file) and immediately write the original file plus a generated thumbnail to the hot disk.
   • EventMediaAsset rows are created with status = hot, capturing disk key, relative path, checksum, size, and variant metadata.
   • Every upload also dispatches ProcessPhotoSecurityScan to the media-security queue for antivirus and EXIF stripping.

2. Metadata & accounting

   • photos table stores user-facing references (URLs, thumbnail URLs, guest metadata).
   • The linked event_media_assets records keep canonical storage information and are used later for archival/restores.
   • media_storage_targets and event_storage_assignments tables describe which disk key is considered “hot” vs “archive” per event.

3. Asynchronous archival

   • The scheduler (either php artisan schedule:work container or host cron) runs storage:archive-pending.
   • That command finds events whose galleries expired or were manually archived and dispatches ArchiveEventMediaAssets jobs onto the media-storage queue.
   • ArchiveEventMediaAssets streams each file from the current disk to the archive disk resolved by EventStorageManager::getArchiveDiskForEvent(), updates the asset row to status = archived, and optionally deletes the hot copy.
   • Archive storage usually maps to an object store bucket using the prefix tenants/{tenant_uuid}/events/{event_uuid}/photos/{photo_uuid}/ (see docs/prp/10-storage-media-pipeline.md).

4. Cold storage / restore

   • When an event is re-opened or media needs to be rehydrated, jobs mark affected assets restoring and copy them back to a hot disk.
   • Restores reuse the same event_media_assets bookkeeping so URLs and permissions stay consistent.

Container & Service Responsibilities

Component	Role
app (Laravel FPM)	Accepts uploads, writes to the hot disk, and records metadata.
media-storage-worker	Runs /scripts/queue-worker.sh media-storage; consumes archival/restoration jobs and copies data between disks. Shares the same app-code volume so it sees /var/www/html/storage.
queue workers	Default queue consumers for non-storage background jobs.
media-security-worker	Processes ProcessPhotoSecurityScan jobs (antivirus + EXIF scrub).
scheduler	Runs php artisan schedule:work, triggering storage:archive-pending, storage:monitor, queue health checks, etc.
horizon	Optional dashboard / supervisor for queue throughput.
Redis	Queue backend for all worker containers.

Key Commands & Jobs

Command / Job	Description
storage:archive-pending	Scans for expired/archived events and dispatches ArchiveEventMediaAssets jobs (media-storage queue).
storage:monitor	Aggregates capacity/queue stats and emails alerts when thresholds are exceeded (config/storage-monitor.php).
media:backfill-thumbnails	Regenerates thumbnails for existing assets; useful before enabling archival on migrated data.
ProcessPhotoSecurityScan	Runs antivirus/EXIF stripping for a photo; default queue is media-security.
ArchiveEventMediaAssets	Copies hot assets to the archive disk, updates statuses, and deletes hot copies if configured.

Configuration Reference

Setting	Location	Notes
Default disk	.env FILESYSTEM_DISK & config/filesystems.php	Hot uploads default to local (/var/www/html/storage/app/private).
Storage targets	media_storage_targets table	Each row stores key, driver, and JSON config; EventStorageManager registers them as runtime disks.
Security queue	.env SECURITY_SCAN_QUEUE & config/security.php	Defaults to media-security.
Archive scheduling	config/storage-monitor.php['archive']	Controls grace days, chunk size, locking, and dispatch caps.
Queue health alerts	config/storage-monitor.php['queue_health']	Warning/critical thresholds for media-storage and media-security queues.
Checksum alerts	config/storage-monitor.php['checksum_validation']	Enables checksum verification alerts and thresholding window.
Container volumes	docker-compose.yml	app, workers, and scheduler share the app-code volume so /var/www/html/storage is common.

Operational Checklist

• Before enabling archival in a new environment

  • Seed storage targets: php artisan db:seed --class=MediaStorageTargetSeeder.
  • Run migrations so event_media_assets and related tables exist.
  • Backfill existing photos into event_media_assets (custom script or artisan command) so archival jobs know about historical files.

• Monitoring

  • Watch storage:monitor output (email or logs) for capacity warnings on hot disks.
  • Use Horizon or Redis metrics to verify media-storage queue depth; thresholds live in config/storage-monitor.php.
  • Review /var/www/html/storage/logs/storage-jobs.log (if configured) for archival failures.
  • Checksum mismatches (hot→archive) are flagged by storage:monitor using checksum_validation thresholds.
  • Ensure media-security queue stays below critical thresholds so uploads aren’t blocked awaiting security scans.

• Troubleshooting uploads

  • Confirm hot disk is mounted and writable (/var/www/html/storage/app/private/events/...).
  • Verify media_storage_targets contains an active is_hot=true entry; EventStorageManager falls back to default disk if none is assigned.
  • Check Redis queue lengths; stalled media-security jobs prevent photos from being marked clean.

• Troubleshooting archival

  • Run php artisan storage:archive-pending --event=<ID> --force to enqueue a specific event.
  • Tail docker compose logs -f media-storage-worker for copy failures.
  • Verify archive disk credentials (e.g., S3 keys) via media_storage_targets.config; missing or invalid settings surface as job failures with status=failed.
  • If hot copies must remain, dispatch ArchiveEventMediaAssets with $deleteSource=false (custom job call).

• Restoring media

  • Assign a hot storage target if none is active (EventStorageManager::ensureAssignment($event)).
  • Dispatch a restore job or manually copy assets back from the archive disk, updating event_media_assets.status to hot or restoring.

Related Documentation

• docs/prp/10-storage-media-pipeline.md — canonical architecture diagram for storage tiers.
• docs/ops/queue-workers.md — how to run media-storage and media-security workers (scripts live in /scripts/).
• docs/ops/deployment/docker.md / docs/ops/deployment/dokploy.md — container topology and volumes.
• config/security.php, config/storage-monitor.php, and config/filesystems.php for runtime knobs.

Keep this spec updated whenever the storage pipeline, queue names, or archive policies change so ops can quickly understand the flow end-to-end.