normogen/thoughts/research/2026-02-14-backend-deployment-constraints.md at main

goose 1cf927f527 Docs: Add backend deployment constraints and monorepo structure

- Documented Docker/Kubernetes deployment requirements
- Added homelab configuration (resource limits, ports)
- Configured reverse proxy compatibility
- Designed monorepo structure (backend/mobile/web/shared)

2026-02-14 15:30:13 -03:00

22 KiB

Raw Permalink Blame History

/home/asoliver/desarrollo/normogen/thoughts/research/2026-02-14-backend-deployment-constraints.md

1: # Backend Deployment Constraints
2: 
3: ## Deployment Requirements
4: 
5: ### Docker + Docker Compose
6: - Backend must be containerized using Docker
7: - Use Docker Compose for local development and homelab deployment
8: - Multi-stage builds for optimal image size
9: - Non-root user for security
10: 
11: ### Kubernetes Compatibility
12: - Design for future Kubernetes deployment
13: - Use environment-based configuration (env vars)
14: - Health check endpoints (`/health`, `/ready`)
15: - Graceful shutdown handling
16: - Stateless application design
17: - ConfigMap and Secret ready
18: 
19: ### One-Command Deployment
20: ```bash
21: # Clone repository
22: git clone https://github.com/yourusername/normogen.git
23: cd normogen
24: 
25: # Setup configuration
26: cp config/example.env config/.env
27: # Edit config/.env with your settings
28: 
29: # Build and run
30: docker compose build
31: docker compose up -d
32: ```
33: 
34: ## Homelab Deployment (Phase 1)
35: 
36: ### Port Configuration
37: - **MongoDB**: Standard ports
38:   - Primary: `27017` (default MongoDB port)
39:   - This allows connecting with existing MongoDB tools/admin UIs
40:   
41: - **Backend API**: `6000` (homelab range)
42:   - External: `6000` (host port)
43:   - Internal: `8000` (container port)
44:   - Mapped in docker-compose.yml
45: 
46: - **Future Services**: Port range `6000-6999`
47:   - `6000`: Backend API (current)
48:   - `6001`: Web UI (future)
49:   - `6002`: Admin dashboard (future)
50:   - `6003`: Metrics/monitoring (future)
51: 
52: ### Network Configuration
53: - Docker network: `normogen-network`
54: - Must communicate with existing Docker services on homelab server
55: - MongoDB accessible to host for backup/admin tools
56: 
57: ### Environment Configuration
58: ```bash
59: # config/.env (gitignored)
60: # Server Configuration
61: RUST_LOG=info
62: SERVER_HOST=0.0.0.0
63: SERVER_PORT=8000
64: ALLOWED_ORIGINS=http://localhost:3000,http://localhost:6001
65: 
66: # Database Configuration
67: MONGODB_URI=mongodb://mongodb:27017/normogen
68: MONGODB_DATABASE=normogen
69: 
70: # JWT Configuration
71: JWT_SECRET=your-super-secret-jwt-key-here
72: JWT_ACCESS_TOKEN_EXPIRY_MINUTES=15
73: JWT_REFRESH_TOKEN_EXPIRY_DAYS=30
74: 
75: # Encryption Configuration (for validation only)
76: # Actual encryption happens client-side
77: ENCRYPTION_ALGORITHM=aes-256-gcm
78: 
79: # Rate Limiting
80: RATE_LIMIT_REQUESTS=100
81: RATE_LIMIT_DURATION_SECONDS=60
82: ```
83: 
84: ### Docker Compose Structure
85: ```yaml
86: # docker-compose.yml
87: version: '3.8'
88: 
89: services:
90:   backend:
91:     build:
92:       context: .
93:       dockerfile: docker/Dockerfile
94:     container_name: normogen-backend
95:     ports:
96:       - "6000:8000"  # Homelab port range
97:     environment:
98:       - RUST_LOG=${RUST_LOG:-info}
99:       - SERVER_PORT=8000
100:       - MONGODB_URI=mongodb://mongodb:27017/normogen
101:       - MONGODB_DATABASE=${MONGODB_DATABASE:-normogen}
102:       - JWT_SECRET=${JWT_SECRET}
103:       - JWT_ACCESS_TOKEN_EXPIRY_MINUTES=${JWT_ACCESS_TOKEN_EXPIRY_MINUTES:-15}
104:       - JWT_REFRESH_TOKEN_EXPIRY_DAYS=${JWT_REFRESH_TOKEN_EXPIRY_DAYS:-30}
105:     env_file:
106:       - config/.env
107:     depends_on:
108:       mongodb:
109:         condition: service_healthy
110:     networks:
111:       - normogen-network
112:     restart: unless-stopped
113:     healthcheck:
114:       test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
115:       interval: 30s
116:       timeout: 10s
117:       retries: 3
118:       start_period: 40s
119: 
120:   mongodb:
121:     image: mongo:6.0
122:     container_name: normogen-mongodb
123:     ports:
124:       - "27017:27017"  # Standard MongoDB port
125:     environment:
126:       - MONGO_INITDB_DATABASE=${MONGO_DATABASE:-normogen}
127:     volumes:
128:       - mongodb_data:/data/db
129:       - mongodb_config:/data/configdb
130:     networks:
131:       - normogen-network
132:     restart: unless-stopped
133:     healthcheck:
134:       test: ["CMD", "mongosh", "--eval", "db.adminCommand('ping')"]
135:       interval: 10s
136:       timeout: 5s
137:       retries: 5
138:       start_period: 10s
139: 
140: volumes:
141:   mongodb_data:
142:     driver: local
143:   mongodb_config:
144:     driver: local
145: 
146: networks:
147:   normogen-network:
148:     driver: bridge
149: ```
150: 
151: ## Kubernetes Deployment (Phase 2 - Future)
152: 
153: ### Manifests Structure
154: ```
155: k8s/
156: ├── base/
157: │   ├── deployment.yaml
158: │   ├── service.yaml
159: │   ├── configmap.yaml
160: │   ├── secrets.yaml
161: │   └── ingress.yaml
162: └── overlays/
163:     ├── homelab/
164:     │   ├── kustomization.yaml
165:     │   └── patches/
166:     └── production/
167:         ├── kustomization.yaml
168:         └── patches/
169: ```
170: 
171: ### Deployment Configuration
172: - **Replicas**: 2-3 for high availability
173: - **Liveness Probe**: `/health` (10s interval, 5s timeout)
174: - **Readiness Probe**: `/ready` (5s interval, 3s timeout)
175: - **Resource Limits**: 
176:   - CPU: 500m-1000m
177:   - Memory: 256Mi-512Mi
178: - **Helm Chart**: Optional for easier deployment
179: 
180: ### Configuration Management
181: - **ConfigMap**: Non-sensitive config (log levels, ports)
182: - **Secrets**: Sensitive config (JWT secret, DB URI)
183: - **Environment Variables**: All configuration via env vars
184: 
185: ## Dockerfile Design
186: 
187: ### Multi-Stage Build
188: ```dockerfile
189: # docker/Dockerfile
190: 
191: # Build stage
192: FROM rust:1.75-alpine AS builder
193: WORKDIR /app
194: 
195: # Install build dependencies
196: RUN apk add --no-cache musl-dev pkgconf openssl-dev
197: 
198: # Copy manifests
199: COPY Cargo.toml Cargo.lock ./
200: 
201: # Create dummy main.rs to build dependencies
202: RUN mkdir src && echo "fn main() {}" > src/main.rs
203: RUN cargo build --release && rm -rf src
204: 
205: # Copy source code
206: COPY src ./src
207: 
208: # Build application
209: RUN touch src/main.rs && cargo build --release
210: 
211: # Runtime stage
212: FROM alpine:3.18
213: WORKDIR /app
214: 
215: # Install runtime dependencies
216: RUN apk add --no-cache ca-certificates openssl
217: 
218: # Copy binary from builder
219: COPY --from=builder /app/target/release/normogen-backend /app/normogen-backend
220: 
221: # Create non-root user
222: RUN addgroup -g 1000 normogen && \
223:     adduser -D -u 1000 -G normogen normogen && \
224:     chown -R normogen:normogen /app
225: 
226: USER normogen
227: 
228: # Expose port
229: EXPOSE 8000
230: 
231: # Check
232: HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
233:   CMD wget --no-verbose --tries=1 --spider http://localhost:8000/health || exit 1
234: 
235: # Run application
236: CMD ["./normogen-backend"]
237: ```
238: 
239: ## Configuration Management
240: 
241: ### Environment Variables
242: All configuration via environment variables (12-factor app):
243: - Database connections
244: - JWT secrets
245: - API credentials
246: - Feature flags
247: - Log levels
248: 
249: ### Configuration Files
250: ```
251: config/
252: ├── .env              # Gitignored, local dev
253: ├── .env.example      # Git committed, template
254: └── defaults.env      # Git committed, defaults
255: ```
256: 
257: ### Configuration Validation
258: - Validate required env vars on startup
259: - Fail fast if misconfigured
260: - Log configuration (sanitize secrets)
261: 
262: ## Endpoints
263: 
264: ### Check Endpoints
265: ```rust
266: // GET /health - Liveness probe
267: // Returns: {"status": "ok"}
268: // Used by: K8s liveness probe, Docker check
269: // Purpose: Is the app running?
270: 
271: // GET /ready - Readiness probe
272: // Returns: {"status": "ready", "database": "connected"}
273: // Used by: K8s readiness probe
274: // Purpose: Is the app ready to serve traffic?
275: ```
276: 
277: ## Logging & Observability
278: 
279: ### Structured Logging
280: - JSON format for production
281: - Log levels: ERROR, WARN, INFO, DEBUG, TRACE
282: - Request ID tracing
283: - Sensitive data sanitization
284: 
285: ### Metrics (Future)
286: - Prometheus endpoint: `/metrics`
287: - Request rate, error rate, latency
288: - Database connection pool status
289: 
290: ## Security Considerations
291: 
292: ### Container Security
293: - Non-root user (UID 1000)
294: - Read-only filesystem (except /tmp)
295: - Minimal base image (Alpine)
296: - No shell in runtime image
297: - Security scanning (Trivy, Snyk)
298: 
299: ### Secrets Management
300: - Never commit secrets to git
301: - Use environment variables
302: - K8s: Use secrets, not configmaps
303: - Docker: Use env_file or Docker secrets
304: - Gitignore: `config/.env`
305: 
306: ## Deployment Checklist
307: 
308: ### Initial Setup
309: - [ ] Clone repository
310: - [ ] Copy `config/.env.example` to `config/.env`
311: - [ ] Configure environment variables
312: - [ ] Create Docker network: `docker network create normogen-network`
313: - [ ] Generate JWT secret: `openssl rand -base64 32`
314: 
315: ### Build & Run
316: - [ ] Build: `docker compose build`
317: - [ ] Run: `docker compose up -d`
318: - [ ] Check logs: `docker compose logs -f backend`
319: - [ ] Check status: `curl http://localhost:6000/health`
320: - [ ] Check ready: `curl http://localhost:6000/ready`
321: 
322: ### Verification
323: - [ ] Backend responds on port 6000
324: - [ ] MongoDB accessible on port 27017
325: - [ ] Checks passing
326: - [ ] Database connection working
327: - [ ] JWT authentication working
328: 
329: ## Questions for Clarification
330: 
331: ### Environment
332: 1. **Reverse Proxy**: Will you use a reverse proxy (Nginx, Traefik, Caddy)?
333:    - If yes, should TLS be handled at proxy or backend?
334:    - What domain/path for the API?
335: 
336: 2. **MongoDB Deployment**: 
337:    - Is MongoDB already running in your homelab?
338:    - Or should docker-compose spin up a dedicated MongoDB instance for Normogen?
339:    - If existing: What's the connection string?
340: 
341: 3. **Resource Limits**:
342:    - Any CPU/memory constraints for the homelab?
343:    - Suggested limits: 500m CPU, 512Mi RAM?
344: 
345: 4. **Backup Strategy**:
346:    - MongoDB backups? (Volume mount, scheduled dump)
347:    - Configuration backups?
348: 
349: 5. **Monitoring**:
350:    - Log aggregation? (ELK, Loki, Splunk)
351:    - Metrics collection? (Prometheus, Grafana)
352:    - APM? (Datadog, New Relic)
353: 
354: 6. **CI/CD**:
355:    - Will you use GitHub Actions, GitLab CI, or Jenkins?
356:    - Auto-deploy on commit to main branch?
357:    - Automated testing before deploy?
358: 
359: ### Development Workflow
360: 7. **Hot Reload**: Development mode with hot reload?
361:    - Use `cargo-watch` for auto-rebuild on file changes?
362: 
363: 8. **Database Migrations**:
364:    - Run migrations on startup?
365:    - Separate migration tool?
366: 
367: 9. **Seed Data**:
368:    - Seed development database with sample data?
369: 
370: ### Production Readiness
371: 10. **Rate Limiting**:
372:     - Per-IP rate limiting?
373:     - Per-user rate limiting (after auth)?
374: 
375: 11. **CORS Configuration**:
376:     - Allowed origins for local dev?
377:     - Allowed origins for production?
378: 
379: 12. **TLS/HTTPS**:
380:     - Local dev: HTTP or HTTPS?
381:     - Production: TLS termination at proxy?
382: 
383: ## Next Steps
384: 
385: Once the above questions are answered, we can proceed with:
386: 
387: 1. **Phase 2.1**: Docker Compose Setup
388:    - Create Dockerfile (multi-stage build)
389:    - Create docker-compose.yml
390:    - Create configuration templates
391:    - Test local deployment
392: 
393: 2. **Phase 2.2**: Axum Server Setup
394:    - Initialize Rust project
395:    - Setup Axum dependencies
396:    - Create check endpoints
397:    - Test containerized build
398: 
399: 3. **Phase 2.3**: MongoDB Integration
400:    - MongoDB connection pooling
401:    - Readiness check with database
402:    - Test database connectivity
403: 
404: 4. **Phase 2.4**: Configuration Management
405:    - Environment variable validation
406:    - Configuration struct
407:    - Secret loading
408: 
409: 5. **Phase 2.5**: Deployment Testing
410:    - Test on homelab server
411:    - Verify checks
412:    - Verify database connection
413:    - Test restart behavior

Deployment Decisions (User Confirmed)

1. MongoDB Deployment

✅ New MongoDB instance in docker-compose

MongoDB will be spun up as part of docker-compose.yml
Volume mounts for data persistence
Standard port 27017 for admin tool access

2. Reverse Proxy

✅ Using reverse proxy for TLS termination

Backend handles HTTP only
TLS/HTTPS handled by reverse proxy (Nginx/Traefik/Caddy)
CORS configuration needed for allowed origins

3. Development Workflow

✅ Hot reload for local development

Use cargo-watch for auto-rebuild on file changes
Separate docker-compose.dev.yml for development
Production build without hot reload

4. Resource Limits

✅ Homelab resource constraints

CPU: 1000m (1 CPU core limit)
Memory: 1000Mi (1GB RAM limit)
Configure in docker-compose.yml
Configure in Kubernetes Deployment for future

5. Repository & CI/CD

✅ Forgejo repository

Git hosting: Forgejo (self-hosted GitHub-compatible)
CI/CD: Forgejo Actions (GitHub Actions compatible)
Single repository for backend, mobile, web

Updated Monorepo Structure

normogen/                          # Root repository
├── backend/                       # Rust backend
│   ├── src/                       # Source code
│   │   ├── main.rs
│   │   ├── auth/
│   │   ├── api/
│   │   ├── db/
│   │   └── config/
│   ├── docker/
│   │   └── Dockerfile             # Multi-stage build
│   ├── Cargo.toml
│   ├── Cargo.lock
│   ├── docker-compose.yml         # Homelab deployment
│   ├── docker-compose.dev.yml     # Development with hot reload
│   ├── config/
│   │   ├── .env.example
│   │   └── defaults.env
│   └── k8s/                       # Future Kubernetes manifests
│       ├── base/
│       └── overlays/
│
├── mobile/                        # React Native (iOS + Android)
│   ├── src/                       # Source code
│   │   ├── components/
│   │   ├── screens/
│   │   ├── navigation/
│   │   ├── store/
│   │   ├── services/
│   │   └── utils/
│   ├── package.json
│   ├── tsconfig.json
│   ├── App.tsx
│   └── android/
│   └── ios/
│
├── web/                           # React web app
│   ├── src/                       # Source code
│   │   ├── components/
│   │   ├── pages/
│   │   ├── store/
│   │   ├── services/
│   │   └── utils/
│   ├── package.json
│   ├── tsconfig.json
│   └── index.html
│
├── shared/                        # Shared TypeScript code
│   ├── src/
│   │   ├── types/                 # Shared types
│   │   ├── validation/            # Zod schemas
│   │   ├── encryption/            # Encryption utilities
│   │   └── api/                   # API client
│   └── package.json
│
├── thoughts/                      # Research & design docs
│   └── research/
│
├── .gitignore                     # Root gitignore
├── README.md                      # Root README
└── docker-compose.root.yml        # Optional: Run all services

Updated Docker Compose with Resource Limits

# docker-compose.yml (production)
version: '3.8'

services:
  backend:
    build:
      context: .
      dockerfile: docker/Dockerfile
    container_name: normogen-backend
    ports:
      - "6000:8000"
    environment:
      - RUST_LOG=${RUST_LOG:-info}
      - SERVER_PORT=8000
      - MONGODB_URI=mongodb://mongodb:27017/normogen
      - MONGODB_DATABASE=${MONGODB_DATABASE:-normogen}
      - JWT_SECRET=${JWT_SECRET}
      - JWT_ACCESS_TOKEN_EXPIRY_MINUTES=${JWT_ACCESS_TOKEN_EXPIRY_MINUTES:-15}
      - JWT_REFRESH_TOKEN_EXPIRY_DAYS=${JWT_REFRESH_TOKEN_EXPIRY_DAYS:-30}
    env_file:
      - config/.env
    depends_on:
      mongodb:
        condition: service_healthy
    networks:
      - normogen-network
    restart: unless-stopped
    deploy:
      resources:
        limits:
          cpus: '1.0'              # 1 CPU core
          memory: 1000M            # 1GB RAM
        reservations:
          cpus: '0.25'
          memory: 256M
    check:
      test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s

  mongodb:
    image: mongo:6.0
    container_name: normogen-mongodb
    ports:
      - "27017:27017"
    environment:
      - MONGO_INITDB_DATABASE=${MONGO_DATABASE:-normogen}
    volumes:
      - mongodb_data:/data/db
      - mongodb_config:/data/configdb
    networks:
      - normogen-network
    restart: unless-stopped
    deploy:
      resources:
        limits:
          cpus: '0.5'
          memory: 512M
        reservations:
          cpus: '0.25'
          memory: 128M
    check:
      test: ["CMD", "mongosh", "--eval", "db.adminCommand('ping')"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 10s

volumes:
  mongodb_data:
    driver: local
  mongodb_config:
    driver: local

networks:
  normogen-network:
    driver: bridge

Development Docker Compose (Hot Reload)

# docker-compose.dev.yml (development)
version: '3.8'

services:
  backend:
    build:
      context: .
      dockerfile: docker/Dockerfile.dev
    container_name: normogen-backend-dev
    ports:
      - "6000:8000"
    volumes:
      - ./src:/app/src           # Hot reload: Mount source code
      - ./Cargo.toml:/app/Cargo.toml
    environment:
      - RUST_LOG=debug
      - SERVER_PORT=8000
      - MONGODB_URI=mongodb://mongodb:27017/normogen
      - MONGODB_DATABASE=normogen_dev
      - JWT_SECRET=dev-secret-do-not-use-in-production
      - CARGO_WATCH=1            # Enable hot reload
    depends_on:
      mongodb:
        condition: service_healthy
    networks:
      - normogen-network
    restart: unless-stopped

  mongodb:
    image: mongo:6.0
    container_name: normogen-mongodb-dev
    ports:
      - "27017:27017"
    environment:
      - MONGO_INITDB_DATABASE=normogen_dev
    volumes:
      - mongodb_dev_data:/data/db
    networks:
      - normogen-network
    check:
      test: ["CMD", "mongosh", "--eval", "db.adminCommand('ping')"]
      interval: 10s
      timeout: 5s
      retries: 5

volumes:
  mongodb_dev_data:
    driver: local

networks:
  normogen-network:
    driver: bridge

Development Dockerfile (with cargo-watch)

# docker/Dockerfile.dev

FROM rust:1.75-alpine
WORKDIR /app

# Install runtime and build dependencies
RUN apk add --no-cache \
    musl-dev \
    pkgconf \
    openssl-dev \
    curl \
    wget \
    git

# Install cargo-watch for hot reload
RUN cargo install cargo-watch

# Copy manifests
COPY Cargo.toml Cargo.lock ./

# Create dummy main.rs to build dependencies
RUN mkdir src && echo "fn main() {}" > src/main.rs
RUN cargo build && rm -rf src

# Copy source code (will be mounted as volume in dev)
COPY src ./src

# Expose port
EXPOSE 8000

# Run with hot reload
CMD ["cargo-watch", "-x", "run"]

Forgejo CI/CD (Example)

# .forgejo/workflows/backend-build.yml
name: Build and Test Backend

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main, develop]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Install Rust
        uses: actions-rust-lang/setup-rust-toolchain@v1
      
      - name: Cache cargo registry
        uses: actions/cache@v3
        with:
          path: ~/.cargo/registry
          key: ${{ runner.os }}-cargo-registry-${{ hashFiles('**/Cargo.lock') }}
      
      - name: Build backend
        working-directory: ./backend
        run: cargo build --verbose
      
      - name: Run tests
        working-directory: ./backend
        run: cargo test --verbose
      
      - name: Build Docker image
        working-directory: ./backend
        run: docker build -f docker/Dockerfile -t normogen-backend:${{ github.sha }} .

  deploy-homelab:
    needs: build
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    steps:
      - name: Deploy to homelab
        run: |
          echo "Deploy to homelab server"
          # Add deployment script here

CORS Configuration (for Reverse Proxy)

// src/config/cors.rs
use tower_http::cors::{CorsLayer, Any};
use axum::http::{HeaderValue, Method};

pub fn create_cors_layer(allowed_origins: &str) -> CorsLayer {
    let origins: Vec<&str> = allowed_origins.split(',').map(|s| s.trim()).collect();
    
    CorsLayer::new()
        .allow_origin(origins.into_iter().map(|s| {
            s.parse::<HeaderValue>().unwrap()
        }).collect::<Vec<_>>())
        .allow_methods([Method::GET, Method::POST, Method::PUT, Method::PATCH, Method::DELETE, Method::OPTIONS])
        .allow_headers(Any)
        .allow_credentials(true)
        .max_age(3600)
}

Updated Deployment Checklist

Initial Setup

Clone repository from Forgejo
Copy backend/config/.env.example to backend/config/.env
Configure environment variables
Generate JWT secret: openssl rand -base64 32
Set resource limits in docker-compose.yml

Development Setup

Run development: docker compose -f docker-compose.dev.yml up -d
Check logs: docker compose -f docker-compose.dev.yml logs -f backend
Test hot reload: Make changes to src/, verify auto-rebuild

Production Deployment

Run production: docker compose up -d
Check logs: docker compose logs -f backend
Verify resource limits: docker stats
Configure reverse proxy (Nginx/Traefik/Caddy)
Configure reverse proxy CORS headers
Test API through reverse proxy

22 KiB Raw Permalink Blame History