added prp

2025-11-11 21:23:00 +01:00
parent d271f256b7
commit 0c83b15636
9 changed files with 4823 additions and 0 deletions
--- a/prp/plugin-architecture.md
+++ b/prp/plugin-architecture.md
@@ -0,0 +1,651 @@
+# Plugin Architecture Specification
+
+## Overview
+
+Extensible plugin system for integrating multiple AI service providers, enabling the application to work with different AI platforms (ComfyUI, RunwareAI, etc.) through a unified interface.
+
+## Architecture Pattern
+
+**Strategy Pattern Implementation** with plugin-based architecture allowing runtime selection of AI processing backends.
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                    ImageController                              │
+│  • Orchestrates image processing workflow                       │
+│  • Loads appropriate plugin based on ApiProvider               │
+└─────────────────────┬───────────────────────────────────────────┘
+                      │
+        ┌─────────────▼─────────────────────────────────────────────┐
+        │            PluginLoader                                   │
+        │  • Discovers and instantiates plugins                     │
+        │  • Maps plugin identifiers to implementations            │
+        └─────────────┬───────────────────────────────────────────┘
+                      │
+        ┌─────────────▼─────────────────────────────────────────────┐
+        │         ApiPluginInterface                                │
+        │  • Defines contract for all AI plugins                    │
+        │  • Ensures consistent behavior across providers          │
+        └─────┬─────────────────┬──────────────────────────────────┘
+              │                 │
+        ┌─────▼──────┐   ┌──────▼──────┐
+        │ ComfyUi    │   │ RunwareAi   │
+        │ Plugin     │   │ Plugin      │
+        └────────────┘   └─────────────┘
+```
+
+## Core Interfaces
+
+### ApiPluginInterface
+
+**Purpose**: Defines the contract that all AI provider plugins must implement
+
+**Location**: `app/Api/Plugins/ApiPluginInterface.php`
+
+**Methods**:
+
+#### Core Plugin Information
+```php
+/**
+ * Returns unique plugin identifier
+ */
+public function getIdentifier(): string
+
+/**
+ * Returns human-readable plugin name
+ */
+public function getName(): string
+
+/**
+ * Checks if plugin is enabled and ready for use
+ */
+public function isEnabled(): bool
+
+/**
+ * Enables the plugin
+ */
+public function enable(): bool
+
+/**
+ * Disables the plugin
+ */
+public function disable(): bool
+```
+
+#### Image Processing
+```php
+/**
+ * Main method for processing image style changes
+ * @param Image $image Original image to transform
+ * @param Style $style Style to apply
+ * @return array Processing result with tracking information
+ */
+public function processImageStyleChange(Image $image, Style $style): array
+```
+
+#### Progress & Status
+```php
+/**
+ * Gets current processing status for image
+ * @param string $imageUUID Image identifier
+ * @return array Status information
+ */
+public function getStatus(string $imageUUID): array
+
+/**
+ * Gets processing progress for image
+ * @param string $imageUUID Image identifier
+ * @return array Progress information (0-100 percentage)
+ */
+public function getProgress(string $imageUUID): array
+```
+
+#### Result Retrieval
+```php
+/**
+ * Retrieves final styled image data
+ * @param string $promptId Provider-specific job identifier
+ * @return string Base64 encoded image data
+ */
+public function getStyledImage(string $promptId): string
+```
+
+#### Provider-Specific Features
+```php
+/**
+ * Tests connection to AI service
+ * @param array $data Connection parameters
+ * @return bool Connection success
+ */
+public function testConnection(array $data): bool
+
+/**
+ * Searches for available AI models
+ * @param string $searchTerm Search query
+ * @return array Available models
+ */
+public function searchModels(string $searchTerm): array
+```
+
+## Plugin Implementations
+
+### 1. ComfyUI Plugin
+
+**Plugin Identifier**: `comfyui`
+
+**Purpose**: Integration with ComfyUI web-based AI image generation platform
+
+**Configuration Requirements**:
+```php
+// ApiProvider configuration
+[
+  'name' => 'ComfyUI Production',
+  'api_url' => 'https://your-comfyui-server.com',
+  'plugin' => 'comfyui',
+  'enabled' => true
+]
+```
+
+#### Key Features
+
+**Workflow-Based Processing**:
+- Uses JSON workflow definitions for complex AI pipelines
+- Supports custom node-based processing graphs
+- Parameter injection via placeholder replacement
+
+**WebSocket Progress Updates**:
+- Real-time progress monitoring via WebSocket connection
+- Detailed progress information (current step, completion percentage)
+- Live preview of intermediate results
+
+**Placeholder System**:
+```json
+{
+  "prompt": "__PROMPT__",
+  "filename": "__FILENAME__",
+  "model_id": "__MODEL_ID__"
+}
+```
+- `__PROMPT__`: Style prompt text
+- `__FILENAME__`: Uploaded image filename
+- `__MODEL_ID__`: AI model identifier
+
+#### Processing Flow
+
+1. **Image Upload**
+   ```php
+   POST {api_url}/upload/image
+   Body: multipart/form-data with image file
+   Response: {"name": "uploaded_filename.png"}
+   ```
+
+2. **Workflow Construction**
+   ```php
+   // Merge model and style parameters
+   $mergedParams = array_replace_recursive($modelParams, $styleParams)
+
+   // Replace placeholders
+   $workflow = str_replace('__PROMPT__', $prompt, $workflow)
+   $workflow = str_replace('__FILENAME__', $filename, $workflow)
+   $workflow = str_replace('__MODEL_ID__', $modelId, $workflow)
+   ```
+
+3. **Prompt Queueing**
+   ```php
+   POST {api_url}/prompt
+   Body: {"prompt": $workflow}
+   Response: {"prompt_id": "unique-job-id"}
+   ```
+
+4. **Progress Monitoring**
+   ```php
+   // WebSocket connection
+   ws://{server}/ws
+
+   // Progress messages
+   {
+     "type": "progress",
+     "data": {
+       "prompt_id": "job-id",
+       "value": 50,
+       "max": 100
+     }
+   }
+   ```
+
+5. **Result Retrieval**
+   ```php
+   GET {api_url}/history/{prompt_id}
+
+   // Extract image URL
+   $imageUrl = "/view?filename={$filename}&subfolder={$subfolder}&type=output"
+
+   // Download final image
+   GET {api_url}{$imageUrl}
+   ```
+
+#### Error Handling
+- **Connection Failures**: Retry logic with exponential backoff
+- **Invalid Workflows**: JSON validation before submission
+- **Processing Timeouts**: Configurable timeout limits
+- **WebSocket Disconnections**: Automatic reconnection attempts
+
+---
+
+### 2. RunwareAI Plugin
+
+**Plugin Identifier**: `runwareai`
+
+**Purpose**: Integration with RunwareAI REST API for image style transformation
+
+**Configuration Requirements**:
+```php
+// ApiProvider configuration
+[
+  'name' => 'RunwareAI Production',
+  'api_url' => 'https://api.runwareai.com',
+  'token' => 'your-api-token',
+  'plugin' => 'runwareai',
+  'enabled' => true
+]
+```
+
+#### Key Features
+
+**Direct API Integration**:
+- RESTful API communication
+- Bearer token authentication
+- Synchronous processing with immediate results
+
+**Model Search Capability**:
+- Search available AI models by name/type
+- Filter by model categories (base, checkpoint, etc.)
+- Real-time model availability checking
+
+**Seed Image Processing**:
+- Upload original image as "seed"
+- Apply style transformation using seed as base
+- Return styled result directly
+
+#### Processing Flow
+
+1. **Image Upload**
+   ```php
+   POST {api_url}
+   Headers: Authorization: Bearer {token}
+   Body: {
+     "taskType": "imageUpload",
+     "taskUUID": "unique-id",
+     "image": "data:image/png;base64,{base64_data}"
+   }
+   Response: {
+     "data": [{
+       "imageUUID": "uploaded-image-uuid"
+     }]
+   }
+   ```
+
+2. **Style Transformation**
+   ```php
+   POST {api_url}
+   Headers: Authorization: Bearer {token}
+   Body: {
+     "taskType": "imageInference",
+     "taskUUID": "unique-id",
+     "positivePrompt": "style prompt text",
+     "seedImage": "uploaded-image-uuid",
+     "model": "model-id",
+     "outputType": "base64Data"
+   }
+   Response: {
+     "data": [{
+       "imageBase64Data": "base64-encoded-result"
+     }]
+   }
+   ```
+
+3. **Model Search** (Optional)
+   ```php
+   POST {api_url}
+   Headers: Authorization: Bearer {token}
+   Body: {
+     "taskType": "modelSearch",
+     "search": "search-term",
+     "type": "base",
+     "category": "checkpoint",
+     "limit": 100
+   }
+   Response: {
+     "data": [{
+       "results": [{
+         "name": "Model Name",
+         "air": "model-id",
+         "type": "model-type"
+       }]
+     }]
+   }
+   ```
+
+#### Error Handling
+- **Authentication Failures**: Clear error messages for token issues
+- **Rate Limiting**: Respect API rate limits with proper delays
+- **Invalid Parameters**: Validation of all request parameters
+- **Network Issues**: Timeout handling with retry logic
+
+## Plugin Loading Mechanism
+
+### PluginLoader Class
+
+**Purpose**: Discovers, loads, and manages AI provider plugins
+
+**Location**: `app/Api/Plugins/PluginLoader.php`
+
+**Key Methods**:
+
+#### `getPlugin(string $pluginIdentifier, ApiProvider $apiProvider): ApiPluginInterface`
+- **Purpose**: Returns instantiated plugin for given identifier
+- **Implementation**: Dynamic class loading based on plugin identifier
+- **Error Handling**: Throws exception for unknown plugins
+
+#### `getAvailablePlugins(): array`
+- **Purpose**: Returns list of all available plugin identifiers
+- **Implementation**: Scans plugin directory for implementations
+
+**Plugin Discovery**:
+```php
+// Maps plugin identifiers to class names
+$pluginMap = [
+  'comfyui' => ComfyUi::class,
+  'runwareai' => RunwareAi::class,
+]
+
+// Dynamic instantiation
+$pluginClass = $pluginMap[$identifier] ?? null;
+if ($pluginClass) {
+  return new $pluginClass($apiProvider);
+}
+```
+
+## Parameter Merging Strategy
+
+### Deep Merge Algorithm
+
+**Purpose**: Combines AI model parameters with style-specific overrides
+
+**Implementation**:
+```php
+$modelParams = $style->aiModel->parameters ?? [];
+$styleParams = $style->parameters ?? [];
+$mergedParams = array_replace_recursive($modelParams, $styleParams);
+```
+
+**Example**:
+```php
+// Model parameters (base configuration)
+$modelParams = [
+  "workflow" => [
+    "nodes" => [
+      "1" => ["class_type" => "KSampler"],
+      "2" => ["class_type" => "SaveImage"]
+    ]
+  ]
+];
+
+// Style parameters (overrides and additions)
+$styleParams = [
+  "workflow" => [
+    "nodes" => [
+      "1" => [
+        "inputs" => ["seed" => 12345],
+        "class_type" => "KSamplerAdvanced"  // Override
+      ],
+      "3" => ["class_type" => "PreviewImage"]  // Addition
+    ]
+  ]
+];
+
+// Result: Deep merged configuration
+$mergedParams = [
+  "workflow" => [
+    "nodes" => [
+      "1" => [
+        "class_type" => "KSamplerAdvanced",  // Overridden
+        "inputs" => ["seed" => 12345]       // Added
+      ],
+      "2" => ["class_type" => "SaveImage"],   // Preserved
+      "3" => ["class_type" => "PreviewImage"] // Added
+    ]
+  ]
+]
+```
+
+## Logging Integration
+
+### LoggablePlugin Trait
+
+**Purpose**: Provides consistent logging across all plugins
+
+**Location**: `app/Api/Plugins/LoggablePlugin.php`
+
+**Features**:
+- **Structured Logging**: Consistent log format across plugins
+- **Log Levels**: Debug, Info, Warning, Error
+- **Context Information**: Plugin name, operation details
+- **Performance Tracking**: Execution time monitoring
+
+**Usage**:
+```php
+class ComfyUi implements ApiPluginInterface
+{
+  use LoggablePlugin;
+
+  public function processImageStyleChange(Image $image, Style $style): array
+  {
+    $this->logInfo('Starting ComfyUI style change process', [
+      'image_id' => $image->id,
+      'style_id' => $style->id
+    ]);
+
+    // Processing logic...
+
+    $this->logInfo('ComfyUI style change completed successfully', [
+      'prompt_id' => $result['prompt_id']
+    ]);
+
+    return $result;
+  }
+}
+```
+
+## Error Handling Strategy
+
+### Plugin-Specific Errors
+
+**ComfyUI Errors**:
+- **Workflow Validation**: Invalid JSON structure in workflow
+- **Node Connection**: Missing or invalid node connections
+- **Resource Limits**: GPU memory or processing queue full
+- **Network Timeout**: Server unreachable or slow response
+
+**RunwareAI Errors**:
+- **Authentication**: Invalid or expired API token
+- **Rate Limiting**: Too many requests per minute/hour
+- **Model Availability**: Requested model not available
+- **Image Processing**: Invalid image format or corrupted data
+
+### Error Recovery
+
+**Retry Logic**:
+```php
+$maxRetries = 3;
+$retryDelay = 1000000; // 1 second in microseconds
+
+for ($attempt = 1; $attempt <= $maxRetries; $attempt++) {
+  try {
+    return $this->processImageStyleChange($image, $style);
+  } catch (RetryableException $e) {
+    if ($attempt === $maxRetries) {
+      throw new MaxRetriesExceededException('Processing failed after ' . $maxRetries . ' attempts');
+    }
+    usleep($retryDelay * $attempt); // Exponential backoff
+  }
+}
+```
+
+**Circuit Breaker Pattern**:
+- Track failure rates per provider
+- Temporarily disable failing providers
+- Automatic recovery testing
+
+## Testing Strategy
+
+### Unit Tests
+- **Plugin Interface**: Verify all plugins implement required methods
+- **Parameter Merging**: Test deep merge algorithm with various inputs
+- **Error Handling**: Verify proper exception throwing and recovery
+
+### Integration Tests
+- **End-to-End Processing**: Test complete workflow with mock AI services
+- **Provider Switching**: Verify seamless transition between providers
+- **Error Scenarios**: Test behavior with network failures and invalid responses
+
+### Mock Services
+- **ComfyUI Mock**: Simulated WebSocket and HTTP responses
+- **RunwareAI Mock**: Mock API responses for all endpoints
+- **Network Simulation**: Latency and failure injection
+
+## Future Plugin Development
+
+### Adding New Providers
+
+1. **Implement Interface**:
+   ```php
+   class NewProvider implements ApiPluginInterface
+   {
+     use LoggablePlugin;
+
+     public function getIdentifier(): string { return 'newprovider'; }
+     public function getName(): string { return 'New AI Provider'; }
+     // ... implement all required methods
+   }
+   ```
+
+2. **Register Plugin**:
+   ```php
+   // In PluginLoader
+   $pluginMap['newprovider'] = NewProvider::class;
+   ```
+
+3. **Add Configuration**:
+   - Create ApiProvider record with appropriate settings
+   - Configure authentication and endpoint details
+
+### Plugin Enhancement Opportunities
+
+**Batch Processing Plugin**:
+- Process multiple images simultaneously
+- Queue management and progress aggregation
+- Resource optimization across multiple requests
+
+**Local Processing Plugin**:
+- Integration with local AI models (Stable Diffusion WebUI)
+- Reduced latency for development/testing
+- Offline processing capabilities
+
+**Cloud Provider Plugins**:
+- **AWS Bedrock**: Amazon's AI service integration
+- **Google Vertex AI**: Google's ML platform
+- **Azure Cognitive Services**: Microsoft AI services
+- **Replicate**: Cloud-based model hosting
+
+**Specialized Processing**:
+- **Upscaling Plugin**: Dedicated image enhancement
+- **Style Transfer Plugin**: Specialized style application
+- **Inpainting Plugin**: Content-aware image editing
+
+## Configuration Management
+
+### Provider-Specific Settings
+
+**ComfyUI Settings**:
+```php
+// Via ApiProvider parameters (JSON)
+{
+  "timeout": 180,
+  "retry_attempts": 3,
+  "websocket_enabled": true,
+  "workflow_path": "/custom/workflows"
+}
+```
+
+**RunwareAI Settings**:
+```php
+{
+  "request_timeout": 60,
+  "rate_limit_delay": 1000,
+  "model_cache_ttl": 3600,
+  "supported_formats": ["png", "jpg", "webp"]
+}
+```
+
+### Environment-Based Configuration
+
+**Development**:
+- Local ComfyUI instance
+- Mock API responses for testing
+- Verbose logging enabled
+
+**Production**:
+- Multiple load-balanced providers
+- Failover between providers
+- Performance monitoring and alerting
+
+## Performance Considerations
+
+### Resource Management
+- **Connection Pooling**: Reuse HTTP connections where possible
+- **Timeout Configuration**: Appropriate timeouts for different operations
+- **Memory Management**: Clean up large image data after processing
+
+### Scalability
+- **Concurrent Processing**: Handle multiple simultaneous requests
+- **Queue Management**: Prevent resource exhaustion
+- **Load Balancing**: Distribute requests across multiple provider instances
+
+### Monitoring
+- **Performance Metrics**: Processing time, success rates, error frequencies
+- **Resource Usage**: Memory, CPU, network utilization
+- **Business Metrics**: User satisfaction, feature adoption
+
+## Security Considerations
+
+### Credential Management
+- **Encrypted Storage**: API tokens encrypted in database
+- **Environment Variables**: Sensitive configuration in env files
+- **Access Control**: Restrict plugin configuration to admin users
+
+### Input Validation
+- **Image Validation**: Verify uploaded image integrity
+- **Parameter Sanitization**: Clean and validate all AI parameters
+- **Output Verification**: Ensure generated images are safe for display
+
+### Network Security
+- **HTTPS Enforcement**: All external API calls over TLS
+- **Certificate Validation**: Verify AI service certificates
+- **Request Signing**: Authenticate requests where required
+
+## Deployment Considerations
+
+### Plugin Distribution
+- **Composer Packages**: Each plugin as separate package
+- **Auto-Discovery**: Automatic plugin registration
+- **Version Management**: Plugin compatibility checking
+
+### Environment Setup
+- **Development**: Easy plugin development and testing
+- **Staging**: Pre-production validation of new plugins
+- **Production**: Zero-downtime plugin updates
+
+### Backup and Recovery
+- **Configuration Backup**: Plugin settings and credentials
+- **State Preservation**: In-progress processing state
+- **Failover**: Automatic switching to backup providers