fotospiel-app/docs/ops/media-storage-spec.md

# Media Storage Ops Specification

## Purpose

This document explains how customer photo uploads move through the Fotospiel platform, which services handle each stage, and what operators must monitor. It complements the PRP storage chapters by focusing on day-to-day operational expectations.

## High-Level Flow

1. **Upload (web / API containers)**
   - Guest/Tenant requests hit the Laravel app container.
   - `EventStorageManager::getHotDiskForEvent()` resolves the current *hot* storage target for the event (usually the local disk mounted at `/var/www/html/storage/app/private`).
   - Controllers call `Storage::disk($hotDisk)->putFile("events/{event}/photos", $file)` and immediately write the original file plus a generated thumbnail to the hot disk.
   - `EventMediaAsset` rows are created with `status = hot`, capturing disk key, relative path, checksum, size, and variant metadata.
   - Every upload also dispatches `ProcessPhotoSecurityScan` to the `media-security` queue for antivirus and EXIF stripping.

2. **Metadata & accounting**
   - `photos` table stores user-facing references (URLs, thumbnail URLs, guest metadata).
   - The linked `event_media_assets` records keep canonical storage information and are used later for archival/restores.
   - `media_storage_targets` and `event_storage_assignments` tables describe which disk key is considered “hot” vs “archive” per event.

3. **Asynchronous archival**
   - The scheduler (either `php artisan schedule:work` container or host cron) runs `storage:archive-pending`.
   - That command finds events whose galleries expired or were manually archived and dispatches `ArchiveEventMediaAssets` jobs onto the `media-storage` queue.
   - `ArchiveEventMediaAssets` streams each file from the current disk to the archive disk resolved by `EventStorageManager::getArchiveDiskForEvent()`, updates the asset row to `status = archived`, and optionally deletes the hot copy.
   - Archive storage usually maps to an object store bucket using the prefix `tenants/{tenant_uuid}/events/{event_uuid}/photos/{photo_uuid}/` (see `docs/prp/10-storage-media-pipeline.md`).

4. **Cold storage / restore**
   - When an event is re-opened or media needs to be rehydrated, jobs mark affected assets `restoring` and copy them back to a hot disk.
   - Restores reuse the same `event_media_assets` bookkeeping so URLs and permissions stay consistent.

## Container & Service Responsibilities

| Component | Role |
| --- | --- |
| `app` (Laravel FPM) | Accepts uploads, writes to the hot disk, and records metadata. |
| `media-storage-worker` | Runs `/docs/queue-supervisor/queue-worker.sh media-storage`; consumes archival/restoration jobs and copies data between disks. Shares the same `app-code` volume so it sees `/var/www/html/storage`. |
| `queue` workers | Default queue consumers for non-storage background jobs. |
| `media-security-worker` | Processes `ProcessPhotoSecurityScan` jobs (antivirus + EXIF scrub). |
| `scheduler` | Runs `php artisan schedule:work`, triggering `storage:archive-pending`, `storage:monitor`, queue health checks, etc. |
| `horizon` | Optional dashboard / supervisor for queue throughput. |
| Redis | Queue backend for all worker containers. |

## Key Commands & Jobs

| Command / Job | Description |
| --- | --- |
| `storage:archive-pending` | Scans for expired/archived events and dispatches `ArchiveEventMediaAssets` jobs (`media-storage` queue). |
| `storage:monitor` | Aggregates capacity/queue stats and emails alerts when thresholds are exceeded (`config/storage-monitor.php`). |
| `media:backfill-thumbnails` | Regenerates thumbnails for existing assets; useful before enabling archival on migrated data. |
| `ProcessPhotoSecurityScan` | Runs antivirus/EXIF stripping for a photo; default queue is `media-security`. |
| `ArchiveEventMediaAssets` | Copies hot assets to the archive disk, updates statuses, and deletes hot copies if configured. |

## Configuration Reference

| Setting | Location | Notes |
| --- | --- | --- |
| Default disk | `.env` `FILESYSTEM_DISK` & `config/filesystems.php` | Hot uploads default to `local` (`/var/www/html/storage/app/private`). |
| Storage targets | `media_storage_targets` table | Each row stores `key`, `driver`, and JSON config; `EventStorageManager` registers them as runtime disks. |
| Security queue | `.env SECURITY_SCAN_QUEUE` & `config/security.php` | Defaults to `media-security`. |
| Archive scheduling | `config/storage-monitor.php['archive']` | Controls grace days, chunk size, locking, and dispatch caps. |
| Queue health alerts | `config/storage-monitor.php['queue_health']` | Warning/critical thresholds for `media-storage` and `media-security` queues. |
| Container volumes | `docker-compose.yml` | `app`, workers, and scheduler share the `app-code` volume so `/var/www/html/storage` is common. |

## Operational Checklist

- **Before enabling archival in a new environment**
  - Seed storage targets: `php artisan db:seed --class=MediaStorageTargetSeeder`.
  - Run migrations so `event_media_assets` and related tables exist.
  - Backfill existing photos into `event_media_assets` (custom script or artisan command) so archival jobs know about historical files.

- **Monitoring**
  - Watch `storage:monitor` output (email or logs) for capacity warnings on hot disks.
  - Use Horizon or Redis metrics to verify `media-storage` queue depth; thresholds live in `config/storage-monitor.php`.
  - Review `/var/www/html/storage/logs/storage-jobs.log` (if configured) for archival failures.
  - Ensure `media-security` queue stays below critical thresholds so uploads aren’t blocked awaiting security scans.

- **Troubleshooting uploads**
  - Confirm hot disk is mounted and writable (`/var/www/html/storage/app/private/events/...`).
  - Verify `media_storage_targets` contains an active `is_hot=true` entry; `EventStorageManager` falls back to default disk if none is assigned.
  - Check Redis queue lengths; stalled `media-security` jobs prevent photos from being marked clean.

- **Troubleshooting archival**
  - Run `php artisan storage:archive-pending --event=<ID> --force` to enqueue a specific event.
  - Tail `docker compose logs -f media-storage-worker` for copy failures.
  - Verify archive disk credentials (e.g., S3 keys) via `media_storage_targets.config`; missing or invalid settings surface as job failures with `status=failed`.
  - If hot copies must remain, dispatch `ArchiveEventMediaAssets` with `$deleteSource=false` (custom job call).

- **Restoring media**
  - Assign a hot storage target if none is active (`EventStorageManager::ensureAssignment($event)`).
  - Dispatch a restore job or manually copy assets back from the archive disk, updating `event_media_assets.status` to `hot` or `restoring`.

## Related Documentation

- `docs/prp/10-storage-media-pipeline.md` — canonical architecture diagram for storage tiers.
- `docs/ops/queue-workers.md` — how to run `media-storage` and `media-security` workers (scripts in `/docs/queue-supervisor/`).
- `docs/ops/deployment/docker.md` / `docs/ops/deployment/dokploy.md` — container topology and volumes.
- `config/security.php`, `config/storage-monitor.php`, and `config/filesystems.php` for runtime knobs.

Keep this spec updated whenever the storage pipeline, queue names, or archive policies change so ops can quickly understand the flow end-to-end.