AGENTS.md

AI coding agent instructions for stable-diffusion.cpp-rest

Project Overview

A C++ based REST API wrapper for the stable-diffusion.cpp library, providing HTTP endpoints for image generation with Stable Diffusion models.

Project Type: Node.js Application Primary Language: TypeScript (63% of codebase) Secondary Languages: C (19%), C++ (18%)

Architecture

Project Structure:

src/ - Source code
webui/app/ - Application code
webui/components/ - React/UI components
webui/lib/ - Library code
webui/public/ - Public assets
webui/app/img2img/ - Project files
webui/app/settings/ - Settings
webui/app/text2img/ - Project files
webui/app/upscaler/ - Project files
webui/components/auth/ - Project files
webui/components/ui/ - Project files

Key Files

Configuration:

webui/package.json - Node.js dependencies and scripts

Documentation:

CLAUDE.md
HASHING_IMPLEMENTATION_GUIDE.md
MODEL_DETECTION.md
PAM_AUTHENTICATION.md

Code Style Guidelines

Use camelCase for variables and functions
Use PascalCase for classes and components
Prefer const/let over var
Use async/await over callbacks when possible

Testing

Test Files:

TEST_RESULTS_SUMMARY.md
src/test_model_detector.cpp
test-auth/api_keys.json
test-auth/users.json

Existing Project Guidelines

The following guidelines were found in existing AI configuration files:

AI Agent Guidelines

From CLAUDE.md:

Project Overview

This is a C++ REST API server that wraps the stable-diffusion.cpp library, providing HTTP endpoints for Stable Diffusion image generation. The server is built with a modular architecture featuring three main components: HTTP Server, Generation Queue, and Model Manager.

Initial Setup and Build


# Create build directory and configure
mkdir build && cd build
cmake ..

# Build the project (parallel build)
cmake --build . --parallel

### Build Configuration Options

bash

Build with CUDA support (default: ON)

cmake -DSD_CUDA_SUPPORT=ON ..

Build without CUDA

cmake -DSD_CUDA_SUPPORT=OFF ..

Debug build

cmake -DCMAKE_BUILD_TYPE=Debug ..

Release build (default)

cmake -DCMAKE_BUILD_TYPE=Release ..


### Clean and Rebuild

bash

Clean build artifacts

cd build cmake --build . --target clean

Or delete build directory entirely

rm -rf build mkdir build && cd build cmake .. cmake --build . --parallel


### Threading Architecture
- **Main thread**: Initialization, signal handling, coordination
- **Server thread**: HTTP request handling (in `Server::serverThreadFunction()`)
- **Queue worker threads**: Generation processing (managed by GenerationQueue)
- Signal handler sets global `g_running` flag for graceful shutdown

### External Project Integration
The stable-diffusion.cpp library is built as an external project. Include directories and libraries are configured via the `sd-cpp` interface target:

cmake target_link_libraries(stable-diffusion-rest-server PRIVATE

sd-cpp
ggml
ggml-base
ggml-cpu
${DEPENDENCY_LIBRARIES}

)


When accessing stable-diffusion.cpp APIs, include from the installed headers:
- `#include <stable-diffusion.h>`
- `#include <ggml.h>`

The wrapper class `StableDiffusionWrapper` (`stable_diffusion_wrapper.cpp/h`) encapsulates all interactions with the stable-diffusion.cpp library.

## Model Architecture Detection System
The server includes an automatic model architecture detection system that analyzes checkpoint files to determine their type and required auxiliary models.

### Supported Architectures
The system can detect the following architectures:

| Architecture | Required Files | Command-Line Flags |
|-------------|----------------|-------------------|
| **SD 1.5** | VAE (optional) | `--vae vae-ft-mse-840000-ema-pruned.safetensors` |
| **SD 2.1** | VAE (optional) | `--vae vae-ft-ema-560000.safetensors` |
| **SDXL Base/Refiner** | VAE (optional) | `--vae sdxl_vae.safetensors` |
| **Flux Schnell** | VAE, CLIP-L, T5XXL | `--vae ae.safetensors --clip-l clip_l.safetensors --t5xxl t5xxl_fp16.safetensors` |
| **Flux Dev** | VAE, CLIP-L, T5XXL | `--vae ae.safetensors --clip-l clip_l.safetensors --t5xxl t5xxl_fp16.safetensors` |
| **Flux Chroma** | VAE, T5XXL | `--vae ae.safetensors --t5xxl t5xxl_fp16.safetensors` |
| **SD3** | VAE, CLIP-L, CLIP-G, T5XXL | `--vae sd3_vae.safetensors --clip-l clip_l.safetensors --clip-g clip_g.safetensors --t5xxl t5xxl_fp16.safetensors` |
| **Qwen2-VL** | Qwen2VL, Qwen2VL-Vision | `--qwen2vl qwen2vl.safetensors --qwen2vl-vision qwen2vl_vision.safetensors` |

### Testing the Detection System
A standalone test binary can be built to test model detection:

bash cd build cmake -DBUILD_MODEL_DETECTOR_TEST=ON .. cmake --build . --target test_model_detector

Architecture-Specific Loading

The server will automatically use the correct parameters when loading models based on detected architecture. For architectures requiring multiple auxiliary models (Flux, SD3, Qwen), the server will:

Check if all required models are available
Return warnings via API if models are missing
Display warnings in WebUI with instructions to load missing models
Provide correct command-line flags for manual loading

See MODEL_DETECTION.md for complete documentation on the detection system.

Web UI Architecture

The project includes a Next.js-based web UI located in /webui that provides a modern interface for interacting with the REST API.

Building the Web UI


# Build Web UI manually
cd webui
npm install
npm run build

# Build via CMake (automatically copies to build directory)
cmake --build build --target webui-build

The built UI is automatically copied to build/webui/ and served by the REST API server at /ui/.

WebUI Structure

Framework: Next.js 16 with React, TypeScript, and Tailwind CSS
Routing: App router with static page generation
UI Components: Shadcn/ui components in /webui/components/ui/
Pages:
- /webui/app/text2img/page.tsx - Text-to-image generation
- /webui/app/img2img/page.tsx - Image-to-image generation
- /webui/app/upscaler/page.tsx - Image upscaling
- /webui/app/models/page.tsx - Model management
- /webui/app/queue/page.tsx - Queue status

Important Components

Main Layout (/webui/components/main-layout.tsx)
- Provides consistent layout with sidebar and status bar
- Includes Sidebar and ModelStatusBar components
Sidebar (/webui/components/sidebar.tsx)
- Fixed position navigation (z-index: 40)
- Always visible on the left side
- Handles page navigation
Model Status Bar (/webui/components/model-status-bar.tsx)
- Fixed position at bottom (z-index: 35)
- Positioned with left-64 to avoid overlapping sidebar
- Shows current model status, queue status, and generation progress
- Polls server every 1-5 seconds for updates
Prompt Textarea (/webui/components/prompt-textarea.tsx)
- Advanced textarea with syntax highlighting
- Autocomplete for LoRAs and embeddings
- Suggestions dropdown (z-index: 30 - below sidebar)
- Highlights LoRA tags (<lora:name:weight>) and embedding names

Testing Cache Behavior

First Load (Cache Miss):

# Open browser DevTools → Network tab
# Load page → See all assets with Status 200
# Check Response Headers for Cache-Control and ETag

Reload (Cache Hit):

# Reload page → See assets with Status 200 (from disk cache)
# Or Status 304 (Not Modified) if revalidating

After Rebuild (Cache Invalidation):

# Rebuild UI: cmake --build build --target webui-build
# Restart server
# Reload page → Version checker shows update notification
# Click Refresh → All assets redownloaded with new ETag

PAM Authentication System

The server includes comprehensive PAM (Pluggable Authentication Modules) authentication support for integration with system authentication backends.

PAM Implementation Components

PamAuth Class (include/pam_auth.h, src/pam_auth.cpp)
- Encapsulates all PAM functionality
- Handles PAM conversation callbacks
- Provides conditional compilation stubs when PAM is disabled
- Manages PAM service initialization and cleanup
UserManager Integration (include/user_manager.h, src/user_manager.cpp)
- Provides authenticatePam() method
- Manages PAM authentication enable/disable
- Creates guest users for successful PAM authentication
- Handles conditional compilation with #ifdef ENABLE_PAM_AUTH
AuthMiddleware Integration (include/auth_middleware.h, src/auth_middleware.cpp)
- Routes PAM authentication requests
- Extracts credentials from HTTP requests
- Handles PAM-specific error responses
- Integrates with existing authentication flow
Build System Integration (CMakeLists.txt, cmake/FindPAM.cmake)
- Custom FindPAM.cmake module for PAM library detection
- Conditional compilation with ENABLE_PAM_AUTH option
- Proper linking when PAM is available
- Graceful fallback when PAM is not available

PAM Authentication Flow

Request Reception: HTTP request with credentials in JSON body
Middleware Routing: AuthMiddleware routes to PAM authentication
Credential Extraction: Username and password extracted from request
PAM Authentication: PamAuth class authenticates against system
User Creation: Successful authentication creates/updates user record
Token Generation: JWT token generated for subsequent requests

PAM Service Configuration

Create a PAM service file at /etc/pam.d/stable-diffusion-rest:

# Basic PAM configuration for stable-diffusion-rest
auth    sufficient    pam_unix.so try_first_pass nullok_secure
auth    required    pam_deny.so

account sufficient    pam_unix.so
account required    pam_deny.so

password sufficient    pam_unix.so nullok_use_authtok nullok_secure md5 shadow
password required    pam_deny.so

session required    pam_limits.so
session required    pam_unix.so

Conditional Compilation

PAM support is conditionally compiled based on the ENABLE_PAM_AUTH flag:

#ifdef ENABLE_PAM_AUTH
// PAM-specific code
PamAuthResult PamAuth::authenticateInternal(const std::string& username, const std::string& password);
#else
// Stub implementations when PAM is not enabled
PamAuthResult PamAuth::authenticateInternal(const std::string& username, const std::string& password) {
    PamAuthResult result;
    result.success = false;
    result.errorMessage = "PAM authentication not available (compiled without PAM support)";
    result.errorCode = "PAM_NOT_AVAILABLE";
    return result;
}
#endif

PAM Error Handling

The system provides comprehensive error handling for PAM authentication:

Error Code	Description	PAM Error
AUTHENTICATION_FAILED	Invalid credentials	PAM_AUTH_ERR
USER_NOT_FOUND	User does not exist	PAM_USER_UNKNOWN
CREDENTIAL_EXPIRED	Password expired	PAM_CRED_EXPIRED
ACCOUNT_EXPIRED	Account expired	PAM_ACCT_EXPIRED
PASSWORD_CHANGE_REQUIRED	Password change needed	PAM_NEW_AUTHTOK_REQD
MAX_TRIES_EXCEEDED	Too many failed attempts	PAM_MAXTRIES
AUTHENTICATION_UNAVAILABLE	PAM service unavailable	PAM_AUTHINFO_UNAVAIL

Testing PAM Authentication

Test PAM configuration with pamtester:

# Install pamtester
sudo apt-get install pamtester

# Test authentication
sudo pamtester stable-diffusion-rest username authenticate

# Test account management
sudo pamtester stable-diffusion-rest username acct_mgmt

Security Considerations

No Password Storage: The server never stores user passwords
Memory Management: Passwords are cleared from memory after authentication
Secure Transmission: Always use HTTPS in production environments
PAM Service Security: Restrict permissions on PAM service files
Account Lockout: Configure account lockout in PAM to prevent brute force attacks

Troubleshooting PAM Issues

Check PAM Libraries: Verify PAM libraries are installed
Verify Build Configuration: Ensure server was built with PAM support
Test PAM Configuration: Use pamtester to validate PAM service
Check System Logs: Review /var/log/auth.log or journalctl
Verify Service File: Ensure PAM service file syntax is correct

For detailed PAM authentication setup instructions, see PAM_AUTHENTICATION.md.

AI Coding Assistance Notes

Important Considerations:

Check package.json for available scripts before running commands
Be aware of Node.js version requirements
Consider impact on bundle size when adding dependencies
Project has 85 files across 16 directories
Check build configuration files before making structural changes

This AGENTS.md file was generated by Nanocoder. Update it as your project evolves.

AGENTS.md 13 KB Historique Raw

AGENTS.md

Project Overview

Architecture

Key Files

Code Style Guidelines

Testing

Existing Project Guidelines

AI Agent Guidelines

Project Overview

Initial Setup and Build

Build with CUDA support (default: ON)

Build without CUDA

Debug build

Release build (default)

Clean build artifacts

Or delete build directory entirely

Architecture-Specific Loading

Web UI Architecture

Building the Web UI

WebUI Structure

Important Components

Testing Cache Behavior

PAM Authentication System

PAM Implementation Components

PAM Authentication Flow

PAM Service Configuration

Conditional Compilation

PAM Error Handling

Testing PAM Authentication

Security Considerations

Troubleshooting PAM Issues

AI Coding Assistance Notes

AGENTS.md 13 KB

Historique Raw