### /home/asoliver/desarrollo/normogen/thoughts/research/2026-02-14-backend-deployment-constraints.md ```markdown 1: # Backend Deployment Constraints 2: 3: ## Deployment Requirements 4: 5: ### Docker + Docker Compose 6: - Backend must be containerized using Docker 7: - Use Docker Compose for local development and homelab deployment 8: - Multi-stage builds for optimal image size 9: - Non-root user for security 10: 11: ### Kubernetes Compatibility 12: - Design for future Kubernetes deployment 13: - Use environment-based configuration (env vars) 14: - Health check endpoints (`/health`, `/ready`) 15: - Graceful shutdown handling 16: - Stateless application design 17: - ConfigMap and Secret ready 18: 19: ### One-Command Deployment 20: ```bash 21: # Clone repository 22: git clone https://github.com/yourusername/normogen.git 23: cd normogen 24: 25: # Setup configuration 26: cp config/example.env config/.env 27: # Edit config/.env with your settings 28: 29: # Build and run 30: docker compose build 31: docker compose up -d 32: ``` 33: 34: ## Homelab Deployment (Phase 1) 35: 36: ### Port Configuration 37: - **MongoDB**: Standard ports 38: - Primary: `27017` (default MongoDB port) 39: - This allows connecting with existing MongoDB tools/admin UIs 40: 41: - **Backend API**: `6000` (homelab range) 42: - External: `6000` (host port) 43: - Internal: `8000` (container port) 44: - Mapped in docker-compose.yml 45: 46: - **Future Services**: Port range `6000-6999` 47: - `6000`: Backend API (current) 48: - `6001`: Web UI (future) 49: - `6002`: Admin dashboard (future) 50: - `6003`: Metrics/monitoring (future) 51: 52: ### Network Configuration 53: - Docker network: `normogen-network` 54: - Must communicate with existing Docker services on homelab server 55: - MongoDB accessible to host for backup/admin tools 56: 57: ### Environment Configuration 58: ```bash 59: # config/.env (gitignored) 60: # Server Configuration 61: RUST_LOG=info 62: SERVER_HOST=0.0.0.0 63: SERVER_PORT=8000 64: ALLOWED_ORIGINS=http://localhost:3000,http://localhost:6001 65: 66: # Database Configuration 67: MONGODB_URI=mongodb://mongodb:27017/normogen 68: MONGODB_DATABASE=normogen 69: 70: # JWT Configuration 71: JWT_SECRET=your-super-secret-jwt-key-here 72: JWT_ACCESS_TOKEN_EXPIRY_MINUTES=15 73: JWT_REFRESH_TOKEN_EXPIRY_DAYS=30 74: 75: # Encryption Configuration (for validation only) 76: # Actual encryption happens client-side 77: ENCRYPTION_ALGORITHM=aes-256-gcm 78: 79: # Rate Limiting 80: RATE_LIMIT_REQUESTS=100 81: RATE_LIMIT_DURATION_SECONDS=60 82: ``` 83: 84: ### Docker Compose Structure 85: ```yaml 86: # docker-compose.yml 87: version: '3.8' 88: 89: services: 90: backend: 91: build: 92: context: . 93: dockerfile: docker/Dockerfile 94: container_name: normogen-backend 95: ports: 96: - "6000:8000" # Homelab port range 97: environment: 98: - RUST_LOG=${RUST_LOG:-info} 99: - SERVER_PORT=8000 100: - MONGODB_URI=mongodb://mongodb:27017/normogen 101: - MONGODB_DATABASE=${MONGODB_DATABASE:-normogen} 102: - JWT_SECRET=${JWT_SECRET} 103: - JWT_ACCESS_TOKEN_EXPIRY_MINUTES=${JWT_ACCESS_TOKEN_EXPIRY_MINUTES:-15} 104: - JWT_REFRESH_TOKEN_EXPIRY_DAYS=${JWT_REFRESH_TOKEN_EXPIRY_DAYS:-30} 105: env_file: 106: - config/.env 107: depends_on: 108: mongodb: 109: condition: service_healthy 110: networks: 111: - normogen-network 112: restart: unless-stopped 113: healthcheck: 114: test: ["CMD", "curl", "-f", "http://localhost:8000/health"] 115: interval: 30s 116: timeout: 10s 117: retries: 3 118: start_period: 40s 119: 120: mongodb: 121: image: mongo:6.0 122: container_name: normogen-mongodb 123: ports: 124: - "27017:27017" # Standard MongoDB port 125: environment: 126: - MONGO_INITDB_DATABASE=${MONGO_DATABASE:-normogen} 127: volumes: 128: - mongodb_data:/data/db 129: - mongodb_config:/data/configdb 130: networks: 131: - normogen-network 132: restart: unless-stopped 133: healthcheck: 134: test: ["CMD", "mongosh", "--eval", "db.adminCommand('ping')"] 135: interval: 10s 136: timeout: 5s 137: retries: 5 138: start_period: 10s 139: 140: volumes: 141: mongodb_data: 142: driver: local 143: mongodb_config: 144: driver: local 145: 146: networks: 147: normogen-network: 148: driver: bridge 149: ``` 150: 151: ## Kubernetes Deployment (Phase 2 - Future) 152: 153: ### Manifests Structure 154: ``` 155: k8s/ 156: ├── base/ 157: │ ├── deployment.yaml 158: │ ├── service.yaml 159: │ ├── configmap.yaml 160: │ ├── secrets.yaml 161: │ └── ingress.yaml 162: └── overlays/ 163: ├── homelab/ 164: │ ├── kustomization.yaml 165: │ └── patches/ 166: └── production/ 167: ├── kustomization.yaml 168: └── patches/ 169: ``` 170: 171: ### Deployment Configuration 172: - **Replicas**: 2-3 for high availability 173: - **Liveness Probe**: `/health` (10s interval, 5s timeout) 174: - **Readiness Probe**: `/ready` (5s interval, 3s timeout) 175: - **Resource Limits**: 176: - CPU: 500m-1000m 177: - Memory: 256Mi-512Mi 178: - **Helm Chart**: Optional for easier deployment 179: 180: ### Configuration Management 181: - **ConfigMap**: Non-sensitive config (log levels, ports) 182: - **Secrets**: Sensitive config (JWT secret, DB URI) 183: - **Environment Variables**: All configuration via env vars 184: 185: ## Dockerfile Design 186: 187: ### Multi-Stage Build 188: ```dockerfile 189: # docker/Dockerfile 190: 191: # Build stage 192: FROM rust:1.75-alpine AS builder 193: WORKDIR /app 194: 195: # Install build dependencies 196: RUN apk add --no-cache musl-dev pkgconf openssl-dev 197: 198: # Copy manifests 199: COPY Cargo.toml Cargo.lock ./ 200: 201: # Create dummy main.rs to build dependencies 202: RUN mkdir src && echo "fn main() {}" > src/main.rs 203: RUN cargo build --release && rm -rf src 204: 205: # Copy source code 206: COPY src ./src 207: 208: # Build application 209: RUN touch src/main.rs && cargo build --release 210: 211: # Runtime stage 212: FROM alpine:3.18 213: WORKDIR /app 214: 215: # Install runtime dependencies 216: RUN apk add --no-cache ca-certificates openssl 217: 218: # Copy binary from builder 219: COPY --from=builder /app/target/release/normogen-backend /app/normogen-backend 220: 221: # Create non-root user 222: RUN addgroup -g 1000 normogen && \ 223: adduser -D -u 1000 -G normogen normogen && \ 224: chown -R normogen:normogen /app 225: 226: USER normogen 227: 228: # Expose port 229: EXPOSE 8000 230: 231: # Check 232: HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \ 233: CMD wget --no-verbose --tries=1 --spider http://localhost:8000/health || exit 1 234: 235: # Run application 236: CMD ["./normogen-backend"] 237: ``` 238: 239: ## Configuration Management 240: 241: ### Environment Variables 242: All configuration via environment variables (12-factor app): 243: - Database connections 244: - JWT secrets 245: - API credentials 246: - Feature flags 247: - Log levels 248: 249: ### Configuration Files 250: ``` 251: config/ 252: ├── .env # Gitignored, local dev 253: ├── .env.example # Git committed, template 254: └── defaults.env # Git committed, defaults 255: ``` 256: 257: ### Configuration Validation 258: - Validate required env vars on startup 259: - Fail fast if misconfigured 260: - Log configuration (sanitize secrets) 261: 262: ## Endpoints 263: 264: ### Check Endpoints 265: ```rust 266: // GET /health - Liveness probe 267: // Returns: {"status": "ok"} 268: // Used by: K8s liveness probe, Docker check 269: // Purpose: Is the app running? 270: 271: // GET /ready - Readiness probe 272: // Returns: {"status": "ready", "database": "connected"} 273: // Used by: K8s readiness probe 274: // Purpose: Is the app ready to serve traffic? 275: ``` 276: 277: ## Logging & Observability 278: 279: ### Structured Logging 280: - JSON format for production 281: - Log levels: ERROR, WARN, INFO, DEBUG, TRACE 282: - Request ID tracing 283: - Sensitive data sanitization 284: 285: ### Metrics (Future) 286: - Prometheus endpoint: `/metrics` 287: - Request rate, error rate, latency 288: - Database connection pool status 289: 290: ## Security Considerations 291: 292: ### Container Security 293: - Non-root user (UID 1000) 294: - Read-only filesystem (except /tmp) 295: - Minimal base image (Alpine) 296: - No shell in runtime image 297: - Security scanning (Trivy, Snyk) 298: 299: ### Secrets Management 300: - Never commit secrets to git 301: - Use environment variables 302: - K8s: Use secrets, not configmaps 303: - Docker: Use env_file or Docker secrets 304: - Gitignore: `config/.env` 305: 306: ## Deployment Checklist 307: 308: ### Initial Setup 309: - [ ] Clone repository 310: - [ ] Copy `config/.env.example` to `config/.env` 311: - [ ] Configure environment variables 312: - [ ] Create Docker network: `docker network create normogen-network` 313: - [ ] Generate JWT secret: `openssl rand -base64 32` 314: 315: ### Build & Run 316: - [ ] Build: `docker compose build` 317: - [ ] Run: `docker compose up -d` 318: - [ ] Check logs: `docker compose logs -f backend` 319: - [ ] Check status: `curl http://localhost:6000/health` 320: - [ ] Check ready: `curl http://localhost:6000/ready` 321: 322: ### Verification 323: - [ ] Backend responds on port 6000 324: - [ ] MongoDB accessible on port 27017 325: - [ ] Checks passing 326: - [ ] Database connection working 327: - [ ] JWT authentication working 328: 329: ## Questions for Clarification 330: 331: ### Environment 332: 1. **Reverse Proxy**: Will you use a reverse proxy (Nginx, Traefik, Caddy)? 333: - If yes, should TLS be handled at proxy or backend? 334: - What domain/path for the API? 335: 336: 2. **MongoDB Deployment**: 337: - Is MongoDB already running in your homelab? 338: - Or should docker-compose spin up a dedicated MongoDB instance for Normogen? 339: - If existing: What's the connection string? 340: 341: 3. **Resource Limits**: 342: - Any CPU/memory constraints for the homelab? 343: - Suggested limits: 500m CPU, 512Mi RAM? 344: 345: 4. **Backup Strategy**: 346: - MongoDB backups? (Volume mount, scheduled dump) 347: - Configuration backups? 348: 349: 5. **Monitoring**: 350: - Log aggregation? (ELK, Loki, Splunk) 351: - Metrics collection? (Prometheus, Grafana) 352: - APM? (Datadog, New Relic) 353: 354: 6. **CI/CD**: 355: - Will you use GitHub Actions, GitLab CI, or Jenkins? 356: - Auto-deploy on commit to main branch? 357: - Automated testing before deploy? 358: 359: ### Development Workflow 360: 7. **Hot Reload**: Development mode with hot reload? 361: - Use `cargo-watch` for auto-rebuild on file changes? 362: 363: 8. **Database Migrations**: 364: - Run migrations on startup? 365: - Separate migration tool? 366: 367: 9. **Seed Data**: 368: - Seed development database with sample data? 369: 370: ### Production Readiness 371: 10. **Rate Limiting**: 372: - Per-IP rate limiting? 373: - Per-user rate limiting (after auth)? 374: 375: 11. **CORS Configuration**: 376: - Allowed origins for local dev? 377: - Allowed origins for production? 378: 379: 12. **TLS/HTTPS**: 380: - Local dev: HTTP or HTTPS? 381: - Production: TLS termination at proxy? 382: 383: ## Next Steps 384: 385: Once the above questions are answered, we can proceed with: 386: 387: 1. **Phase 2.1**: Docker Compose Setup 388: - Create Dockerfile (multi-stage build) 389: - Create docker-compose.yml 390: - Create configuration templates 391: - Test local deployment 392: 393: 2. **Phase 2.2**: Axum Server Setup 394: - Initialize Rust project 395: - Setup Axum dependencies 396: - Create check endpoints 397: - Test containerized build 398: 399: 3. **Phase 2.3**: MongoDB Integration 400: - MongoDB connection pooling 401: - Readiness check with database 402: - Test database connectivity 403: 404: 4. **Phase 2.4**: Configuration Management 405: - Environment variable validation 406: - Configuration struct 407: - Secret loading 408: 409: 5. **Phase 2.5**: Deployment Testing 410: - Test on homelab server 411: - Verify checks 412: - Verify database connection 413: - Test restart behavior ``` ## Deployment Decisions (User Confirmed) ### 1. MongoDB Deployment ✅ **New MongoDB instance in docker-compose** - MongoDB will be spun up as part of docker-compose.yml - Volume mounts for data persistence - Standard port 27017 for admin tool access ### 2. Reverse Proxy ✅ **Using reverse proxy for TLS termination** - Backend handles HTTP only - TLS/HTTPS handled by reverse proxy (Nginx/Traefik/Caddy) - CORS configuration needed for allowed origins ### 3. Development Workflow ✅ **Hot reload for local development** - Use `cargo-watch` for auto-rebuild on file changes - Separate docker-compose.dev.yml for development - Production build without hot reload ### 4. Resource Limits ✅ **Homelab resource constraints** - CPU: 1000m (1 CPU core limit) - Memory: 1000Mi (1GB RAM limit) - Configure in docker-compose.yml - Configure in Kubernetes Deployment for future ### 5. Repository & CI/CD ✅ **Forgejo repository** - Git hosting: Forgejo (self-hosted GitHub-compatible) - CI/CD: Forgejo Actions (GitHub Actions compatible) - Single repository for backend, mobile, web ## Updated Monorepo Structure ``` normogen/ # Root repository ├── backend/ # Rust backend │ ├── src/ # Source code │ │ ├── main.rs │ │ ├── auth/ │ │ ├── api/ │ │ ├── db/ │ │ └── config/ │ ├── docker/ │ │ └── Dockerfile # Multi-stage build │ ├── Cargo.toml │ ├── Cargo.lock │ ├── docker-compose.yml # Homelab deployment │ ├── docker-compose.dev.yml # Development with hot reload │ ├── config/ │ │ ├── .env.example │ │ └── defaults.env │ └── k8s/ # Future Kubernetes manifests │ ├── base/ │ └── overlays/ │ ├── mobile/ # React Native (iOS + Android) │ ├── src/ # Source code │ │ ├── components/ │ │ ├── screens/ │ │ ├── navigation/ │ │ ├── store/ │ │ ├── services/ │ │ └── utils/ │ ├── package.json │ ├── tsconfig.json │ ├── App.tsx │ └── android/ │ └── ios/ │ ├── web/ # React web app │ ├── src/ # Source code │ │ ├── components/ │ │ ├── pages/ │ │ ├── store/ │ │ ├── services/ │ │ └── utils/ │ ├── package.json │ ├── tsconfig.json │ └── index.html │ ├── shared/ # Shared TypeScript code │ ├── src/ │ │ ├── types/ # Shared types │ │ ├── validation/ # Zod schemas │ │ ├── encryption/ # Encryption utilities │ │ └── api/ # API client │ └── package.json │ ├── thoughts/ # Research & design docs │ └── research/ │ ├── .gitignore # Root gitignore ├── README.md # Root README └── docker-compose.root.yml # Optional: Run all services ``` ### Updated Docker Compose with Resource Limits ```yaml # docker-compose.yml (production) version: '3.8' services: backend: build: context: . dockerfile: docker/Dockerfile container_name: normogen-backend ports: - "6000:8000" environment: - RUST_LOG=${RUST_LOG:-info} - SERVER_PORT=8000 - MONGODB_URI=mongodb://mongodb:27017/normogen - MONGODB_DATABASE=${MONGODB_DATABASE:-normogen} - JWT_SECRET=${JWT_SECRET} - JWT_ACCESS_TOKEN_EXPIRY_MINUTES=${JWT_ACCESS_TOKEN_EXPIRY_MINUTES:-15} - JWT_REFRESH_TOKEN_EXPIRY_DAYS=${JWT_REFRESH_TOKEN_EXPIRY_DAYS:-30} env_file: - config/.env depends_on: mongodb: condition: service_healthy networks: - normogen-network restart: unless-stopped deploy: resources: limits: cpus: '1.0' # 1 CPU core memory: 1000M # 1GB RAM reservations: cpus: '0.25' memory: 256M check: test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:8000/health"] interval: 30s timeout: 10s retries: 3 start_period: 40s mongodb: image: mongo:6.0 container_name: normogen-mongodb ports: - "27017:27017" environment: - MONGO_INITDB_DATABASE=${MONGO_DATABASE:-normogen} volumes: - mongodb_data:/data/db - mongodb_config:/data/configdb networks: - normogen-network restart: unless-stopped deploy: resources: limits: cpus: '0.5' memory: 512M reservations: cpus: '0.25' memory: 128M check: test: ["CMD", "mongosh", "--eval", "db.adminCommand('ping')"] interval: 10s timeout: 5s retries: 5 start_period: 10s volumes: mongodb_data: driver: local mongodb_config: driver: local networks: normogen-network: driver: bridge ``` ### Development Docker Compose (Hot Reload) ```yaml # docker-compose.dev.yml (development) version: '3.8' services: backend: build: context: . dockerfile: docker/Dockerfile.dev container_name: normogen-backend-dev ports: - "6000:8000" volumes: - ./src:/app/src # Hot reload: Mount source code - ./Cargo.toml:/app/Cargo.toml environment: - RUST_LOG=debug - SERVER_PORT=8000 - MONGODB_URI=mongodb://mongodb:27017/normogen - MONGODB_DATABASE=normogen_dev - JWT_SECRET=dev-secret-do-not-use-in-production - CARGO_WATCH=1 # Enable hot reload depends_on: mongodb: condition: service_healthy networks: - normogen-network restart: unless-stopped mongodb: image: mongo:6.0 container_name: normogen-mongodb-dev ports: - "27017:27017" environment: - MONGO_INITDB_DATABASE=normogen_dev volumes: - mongodb_dev_data:/data/db networks: - normogen-network check: test: ["CMD", "mongosh", "--eval", "db.adminCommand('ping')"] interval: 10s timeout: 5s retries: 5 volumes: mongodb_dev_data: driver: local networks: normogen-network: driver: bridge ``` ### Development Dockerfile (with cargo-watch) ```dockerfile # docker/Dockerfile.dev FROM rust:1.75-alpine WORKDIR /app # Install runtime and build dependencies RUN apk add --no-cache \ musl-dev \ pkgconf \ openssl-dev \ curl \ wget \ git # Install cargo-watch for hot reload RUN cargo install cargo-watch # Copy manifests COPY Cargo.toml Cargo.lock ./ # Create dummy main.rs to build dependencies RUN mkdir src && echo "fn main() {}" > src/main.rs RUN cargo build && rm -rf src # Copy source code (will be mounted as volume in dev) COPY src ./src # Expose port EXPOSE 8000 # Run with hot reload CMD ["cargo-watch", "-x", "run"] ``` ### Forgejo CI/CD (Example) ```yaml # .forgejo/workflows/backend-build.yml name: Build and Test Backend on: push: branches: [main, develop] pull_request: branches: [main, develop] jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Install Rust uses: actions-rust-lang/setup-rust-toolchain@v1 - name: Cache cargo registry uses: actions/cache@v3 with: path: ~/.cargo/registry key: ${{ runner.os }}-cargo-registry-${{ hashFiles('**/Cargo.lock') }} - name: Build backend working-directory: ./backend run: cargo build --verbose - name: Run tests working-directory: ./backend run: cargo test --verbose - name: Build Docker image working-directory: ./backend run: docker build -f docker/Dockerfile -t normogen-backend:${{ github.sha }} . deploy-homelab: needs: build runs-on: ubuntu-latest if: github.ref == 'refs/heads/main' steps: - name: Deploy to homelab run: | echo "Deploy to homelab server" # Add deployment script here ``` ### CORS Configuration (for Reverse Proxy) ```rust // src/config/cors.rs use tower_http::cors::{CorsLayer, Any}; use axum::http::{HeaderValue, Method}; pub fn create_cors_layer(allowed_origins: &str) -> CorsLayer { let origins: Vec<&str> = allowed_origins.split(',').map(|s| s.trim()).collect(); CorsLayer::new() .allow_origin(origins.into_iter().map(|s| { s.parse::().unwrap() }).collect::>()) .allow_methods([Method::GET, Method::POST, Method::PUT, Method::PATCH, Method::DELETE, Method::OPTIONS]) .allow_headers(Any) .allow_credentials(true) .max_age(3600) } ``` ## Updated Deployment Checklist ### Initial Setup - [ ] Clone repository from Forgejo - [ ] Copy `backend/config/.env.example` to `backend/config/.env` - [ ] Configure environment variables - [ ] Generate JWT secret: `openssl rand -base64 32` - [ ] Set resource limits in docker-compose.yml ### Development Setup - [ ] Run development: `docker compose -f docker-compose.dev.yml up -d` - [ ] Check logs: `docker compose -f docker-compose.dev.yml logs -f backend` - [ ] Test hot reload: Make changes to src/, verify auto-rebuild ### Production Deployment - [ ] Run production: `docker compose up -d` - [ ] Check logs: `docker compose logs -f backend` - [ ] Verify resource limits: `docker stats` - [ ] Configure reverse proxy (Nginx/Traefik/Caddy) - [ ] Configure reverse proxy CORS headers - [ ] Test API through reverse proxy