normogen/thoughts/research/2026-02-14-backend-deployment-constraints.md
goose 1cf927f527 Docs: Add backend deployment constraints and monorepo structure
- Documented Docker/Kubernetes deployment requirements
- Added homelab configuration (resource limits, ports)
- Configured reverse proxy compatibility
- Designed monorepo structure (backend/mobile/web/shared)
2026-02-14 15:30:13 -03:00

786 lines
22 KiB
Markdown

### /home/asoliver/desarrollo/normogen/thoughts/research/2026-02-14-backend-deployment-constraints.md
```markdown
1: # Backend Deployment Constraints
2:
3: ## Deployment Requirements
4:
5: ### Docker + Docker Compose
6: - Backend must be containerized using Docker
7: - Use Docker Compose for local development and homelab deployment
8: - Multi-stage builds for optimal image size
9: - Non-root user for security
10:
11: ### Kubernetes Compatibility
12: - Design for future Kubernetes deployment
13: - Use environment-based configuration (env vars)
14: - Health check endpoints (`/health`, `/ready`)
15: - Graceful shutdown handling
16: - Stateless application design
17: - ConfigMap and Secret ready
18:
19: ### One-Command Deployment
20: ```bash
21: # Clone repository
22: git clone https://github.com/yourusername/normogen.git
23: cd normogen
24:
25: # Setup configuration
26: cp config/example.env config/.env
27: # Edit config/.env with your settings
28:
29: # Build and run
30: docker compose build
31: docker compose up -d
32: ```
33:
34: ## Homelab Deployment (Phase 1)
35:
36: ### Port Configuration
37: - **MongoDB**: Standard ports
38: - Primary: `27017` (default MongoDB port)
39: - This allows connecting with existing MongoDB tools/admin UIs
40:
41: - **Backend API**: `6000` (homelab range)
42: - External: `6000` (host port)
43: - Internal: `8000` (container port)
44: - Mapped in docker-compose.yml
45:
46: - **Future Services**: Port range `6000-6999`
47: - `6000`: Backend API (current)
48: - `6001`: Web UI (future)
49: - `6002`: Admin dashboard (future)
50: - `6003`: Metrics/monitoring (future)
51:
52: ### Network Configuration
53: - Docker network: `normogen-network`
54: - Must communicate with existing Docker services on homelab server
55: - MongoDB accessible to host for backup/admin tools
56:
57: ### Environment Configuration
58: ```bash
59: # config/.env (gitignored)
60: # Server Configuration
61: RUST_LOG=info
62: SERVER_HOST=0.0.0.0
63: SERVER_PORT=8000
64: ALLOWED_ORIGINS=http://localhost:3000,http://localhost:6001
65:
66: # Database Configuration
67: MONGODB_URI=mongodb://mongodb:27017/normogen
68: MONGODB_DATABASE=normogen
69:
70: # JWT Configuration
71: JWT_SECRET=your-super-secret-jwt-key-here
72: JWT_ACCESS_TOKEN_EXPIRY_MINUTES=15
73: JWT_REFRESH_TOKEN_EXPIRY_DAYS=30
74:
75: # Encryption Configuration (for validation only)
76: # Actual encryption happens client-side
77: ENCRYPTION_ALGORITHM=aes-256-gcm
78:
79: # Rate Limiting
80: RATE_LIMIT_REQUESTS=100
81: RATE_LIMIT_DURATION_SECONDS=60
82: ```
83:
84: ### Docker Compose Structure
85: ```yaml
86: # docker-compose.yml
87: version: '3.8'
88:
89: services:
90: backend:
91: build:
92: context: .
93: dockerfile: docker/Dockerfile
94: container_name: normogen-backend
95: ports:
96: - "6000:8000" # Homelab port range
97: environment:
98: - RUST_LOG=${RUST_LOG:-info}
99: - SERVER_PORT=8000
100: - MONGODB_URI=mongodb://mongodb:27017/normogen
101: - MONGODB_DATABASE=${MONGODB_DATABASE:-normogen}
102: - JWT_SECRET=${JWT_SECRET}
103: - JWT_ACCESS_TOKEN_EXPIRY_MINUTES=${JWT_ACCESS_TOKEN_EXPIRY_MINUTES:-15}
104: - JWT_REFRESH_TOKEN_EXPIRY_DAYS=${JWT_REFRESH_TOKEN_EXPIRY_DAYS:-30}
105: env_file:
106: - config/.env
107: depends_on:
108: mongodb:
109: condition: service_healthy
110: networks:
111: - normogen-network
112: restart: unless-stopped
113: healthcheck:
114: test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
115: interval: 30s
116: timeout: 10s
117: retries: 3
118: start_period: 40s
119:
120: mongodb:
121: image: mongo:6.0
122: container_name: normogen-mongodb
123: ports:
124: - "27017:27017" # Standard MongoDB port
125: environment:
126: - MONGO_INITDB_DATABASE=${MONGO_DATABASE:-normogen}
127: volumes:
128: - mongodb_data:/data/db
129: - mongodb_config:/data/configdb
130: networks:
131: - normogen-network
132: restart: unless-stopped
133: healthcheck:
134: test: ["CMD", "mongosh", "--eval", "db.adminCommand('ping')"]
135: interval: 10s
136: timeout: 5s
137: retries: 5
138: start_period: 10s
139:
140: volumes:
141: mongodb_data:
142: driver: local
143: mongodb_config:
144: driver: local
145:
146: networks:
147: normogen-network:
148: driver: bridge
149: ```
150:
151: ## Kubernetes Deployment (Phase 2 - Future)
152:
153: ### Manifests Structure
154: ```
155: k8s/
156: ├── base/
157: │ ├── deployment.yaml
158: │ ├── service.yaml
159: │ ├── configmap.yaml
160: │ ├── secrets.yaml
161: │ └── ingress.yaml
162: └── overlays/
163: ├── homelab/
164: │ ├── kustomization.yaml
165: │ └── patches/
166: └── production/
167: ├── kustomization.yaml
168: └── patches/
169: ```
170:
171: ### Deployment Configuration
172: - **Replicas**: 2-3 for high availability
173: - **Liveness Probe**: `/health` (10s interval, 5s timeout)
174: - **Readiness Probe**: `/ready` (5s interval, 3s timeout)
175: - **Resource Limits**:
176: - CPU: 500m-1000m
177: - Memory: 256Mi-512Mi
178: - **Helm Chart**: Optional for easier deployment
179:
180: ### Configuration Management
181: - **ConfigMap**: Non-sensitive config (log levels, ports)
182: - **Secrets**: Sensitive config (JWT secret, DB URI)
183: - **Environment Variables**: All configuration via env vars
184:
185: ## Dockerfile Design
186:
187: ### Multi-Stage Build
188: ```dockerfile
189: # docker/Dockerfile
190:
191: # Build stage
192: FROM rust:1.75-alpine AS builder
193: WORKDIR /app
194:
195: # Install build dependencies
196: RUN apk add --no-cache musl-dev pkgconf openssl-dev
197:
198: # Copy manifests
199: COPY Cargo.toml Cargo.lock ./
200:
201: # Create dummy main.rs to build dependencies
202: RUN mkdir src && echo "fn main() {}" > src/main.rs
203: RUN cargo build --release && rm -rf src
204:
205: # Copy source code
206: COPY src ./src
207:
208: # Build application
209: RUN touch src/main.rs && cargo build --release
210:
211: # Runtime stage
212: FROM alpine:3.18
213: WORKDIR /app
214:
215: # Install runtime dependencies
216: RUN apk add --no-cache ca-certificates openssl
217:
218: # Copy binary from builder
219: COPY --from=builder /app/target/release/normogen-backend /app/normogen-backend
220:
221: # Create non-root user
222: RUN addgroup -g 1000 normogen && \
223: adduser -D -u 1000 -G normogen normogen && \
224: chown -R normogen:normogen /app
225:
226: USER normogen
227:
228: # Expose port
229: EXPOSE 8000
230:
231: # Check
232: HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
233: CMD wget --no-verbose --tries=1 --spider http://localhost:8000/health || exit 1
234:
235: # Run application
236: CMD ["./normogen-backend"]
237: ```
238:
239: ## Configuration Management
240:
241: ### Environment Variables
242: All configuration via environment variables (12-factor app):
243: - Database connections
244: - JWT secrets
245: - API credentials
246: - Feature flags
247: - Log levels
248:
249: ### Configuration Files
250: ```
251: config/
252: ├── .env # Gitignored, local dev
253: ├── .env.example # Git committed, template
254: └── defaults.env # Git committed, defaults
255: ```
256:
257: ### Configuration Validation
258: - Validate required env vars on startup
259: - Fail fast if misconfigured
260: - Log configuration (sanitize secrets)
261:
262: ## Endpoints
263:
264: ### Check Endpoints
265: ```rust
266: // GET /health - Liveness probe
267: // Returns: {"status": "ok"}
268: // Used by: K8s liveness probe, Docker check
269: // Purpose: Is the app running?
270:
271: // GET /ready - Readiness probe
272: // Returns: {"status": "ready", "database": "connected"}
273: // Used by: K8s readiness probe
274: // Purpose: Is the app ready to serve traffic?
275: ```
276:
277: ## Logging & Observability
278:
279: ### Structured Logging
280: - JSON format for production
281: - Log levels: ERROR, WARN, INFO, DEBUG, TRACE
282: - Request ID tracing
283: - Sensitive data sanitization
284:
285: ### Metrics (Future)
286: - Prometheus endpoint: `/metrics`
287: - Request rate, error rate, latency
288: - Database connection pool status
289:
290: ## Security Considerations
291:
292: ### Container Security
293: - Non-root user (UID 1000)
294: - Read-only filesystem (except /tmp)
295: - Minimal base image (Alpine)
296: - No shell in runtime image
297: - Security scanning (Trivy, Snyk)
298:
299: ### Secrets Management
300: - Never commit secrets to git
301: - Use environment variables
302: - K8s: Use secrets, not configmaps
303: - Docker: Use env_file or Docker secrets
304: - Gitignore: `config/.env`
305:
306: ## Deployment Checklist
307:
308: ### Initial Setup
309: - [ ] Clone repository
310: - [ ] Copy `config/.env.example` to `config/.env`
311: - [ ] Configure environment variables
312: - [ ] Create Docker network: `docker network create normogen-network`
313: - [ ] Generate JWT secret: `openssl rand -base64 32`
314:
315: ### Build & Run
316: - [ ] Build: `docker compose build`
317: - [ ] Run: `docker compose up -d`
318: - [ ] Check logs: `docker compose logs -f backend`
319: - [ ] Check status: `curl http://localhost:6000/health`
320: - [ ] Check ready: `curl http://localhost:6000/ready`
321:
322: ### Verification
323: - [ ] Backend responds on port 6000
324: - [ ] MongoDB accessible on port 27017
325: - [ ] Checks passing
326: - [ ] Database connection working
327: - [ ] JWT authentication working
328:
329: ## Questions for Clarification
330:
331: ### Environment
332: 1. **Reverse Proxy**: Will you use a reverse proxy (Nginx, Traefik, Caddy)?
333: - If yes, should TLS be handled at proxy or backend?
334: - What domain/path for the API?
335:
336: 2. **MongoDB Deployment**:
337: - Is MongoDB already running in your homelab?
338: - Or should docker-compose spin up a dedicated MongoDB instance for Normogen?
339: - If existing: What's the connection string?
340:
341: 3. **Resource Limits**:
342: - Any CPU/memory constraints for the homelab?
343: - Suggested limits: 500m CPU, 512Mi RAM?
344:
345: 4. **Backup Strategy**:
346: - MongoDB backups? (Volume mount, scheduled dump)
347: - Configuration backups?
348:
349: 5. **Monitoring**:
350: - Log aggregation? (ELK, Loki, Splunk)
351: - Metrics collection? (Prometheus, Grafana)
352: - APM? (Datadog, New Relic)
353:
354: 6. **CI/CD**:
355: - Will you use GitHub Actions, GitLab CI, or Jenkins?
356: - Auto-deploy on commit to main branch?
357: - Automated testing before deploy?
358:
359: ### Development Workflow
360: 7. **Hot Reload**: Development mode with hot reload?
361: - Use `cargo-watch` for auto-rebuild on file changes?
362:
363: 8. **Database Migrations**:
364: - Run migrations on startup?
365: - Separate migration tool?
366:
367: 9. **Seed Data**:
368: - Seed development database with sample data?
369:
370: ### Production Readiness
371: 10. **Rate Limiting**:
372: - Per-IP rate limiting?
373: - Per-user rate limiting (after auth)?
374:
375: 11. **CORS Configuration**:
376: - Allowed origins for local dev?
377: - Allowed origins for production?
378:
379: 12. **TLS/HTTPS**:
380: - Local dev: HTTP or HTTPS?
381: - Production: TLS termination at proxy?
382:
383: ## Next Steps
384:
385: Once the above questions are answered, we can proceed with:
386:
387: 1. **Phase 2.1**: Docker Compose Setup
388: - Create Dockerfile (multi-stage build)
389: - Create docker-compose.yml
390: - Create configuration templates
391: - Test local deployment
392:
393: 2. **Phase 2.2**: Axum Server Setup
394: - Initialize Rust project
395: - Setup Axum dependencies
396: - Create check endpoints
397: - Test containerized build
398:
399: 3. **Phase 2.3**: MongoDB Integration
400: - MongoDB connection pooling
401: - Readiness check with database
402: - Test database connectivity
403:
404: 4. **Phase 2.4**: Configuration Management
405: - Environment variable validation
406: - Configuration struct
407: - Secret loading
408:
409: 5. **Phase 2.5**: Deployment Testing
410: - Test on homelab server
411: - Verify checks
412: - Verify database connection
413: - Test restart behavior
```
## Deployment Decisions (User Confirmed)
### 1. MongoDB Deployment
**New MongoDB instance in docker-compose**
- MongoDB will be spun up as part of docker-compose.yml
- Volume mounts for data persistence
- Standard port 27017 for admin tool access
### 2. Reverse Proxy
**Using reverse proxy for TLS termination**
- Backend handles HTTP only
- TLS/HTTPS handled by reverse proxy (Nginx/Traefik/Caddy)
- CORS configuration needed for allowed origins
### 3. Development Workflow
**Hot reload for local development**
- Use `cargo-watch` for auto-rebuild on file changes
- Separate docker-compose.dev.yml for development
- Production build without hot reload
### 4. Resource Limits
**Homelab resource constraints**
- CPU: 1000m (1 CPU core limit)
- Memory: 1000Mi (1GB RAM limit)
- Configure in docker-compose.yml
- Configure in Kubernetes Deployment for future
### 5. Repository & CI/CD
**Forgejo repository**
- Git hosting: Forgejo (self-hosted GitHub-compatible)
- CI/CD: Forgejo Actions (GitHub Actions compatible)
- Single repository for backend, mobile, web
## Updated Monorepo Structure
```
normogen/ # Root repository
├── backend/ # Rust backend
│ ├── src/ # Source code
│ │ ├── main.rs
│ │ ├── auth/
│ │ ├── api/
│ │ ├── db/
│ │ └── config/
│ ├── docker/
│ │ └── Dockerfile # Multi-stage build
│ ├── Cargo.toml
│ ├── Cargo.lock
│ ├── docker-compose.yml # Homelab deployment
│ ├── docker-compose.dev.yml # Development with hot reload
│ ├── config/
│ │ ├── .env.example
│ │ └── defaults.env
│ └── k8s/ # Future Kubernetes manifests
│ ├── base/
│ └── overlays/
├── mobile/ # React Native (iOS + Android)
│ ├── src/ # Source code
│ │ ├── components/
│ │ ├── screens/
│ │ ├── navigation/
│ │ ├── store/
│ │ ├── services/
│ │ └── utils/
│ ├── package.json
│ ├── tsconfig.json
│ ├── App.tsx
│ └── android/
│ └── ios/
├── web/ # React web app
│ ├── src/ # Source code
│ │ ├── components/
│ │ ├── pages/
│ │ ├── store/
│ │ ├── services/
│ │ └── utils/
│ ├── package.json
│ ├── tsconfig.json
│ └── index.html
├── shared/ # Shared TypeScript code
│ ├── src/
│ │ ├── types/ # Shared types
│ │ ├── validation/ # Zod schemas
│ │ ├── encryption/ # Encryption utilities
│ │ └── api/ # API client
│ └── package.json
├── thoughts/ # Research & design docs
│ └── research/
├── .gitignore # Root gitignore
├── README.md # Root README
└── docker-compose.root.yml # Optional: Run all services
```
### Updated Docker Compose with Resource Limits
```yaml
# docker-compose.yml (production)
version: '3.8'
services:
backend:
build:
context: .
dockerfile: docker/Dockerfile
container_name: normogen-backend
ports:
- "6000:8000"
environment:
- RUST_LOG=${RUST_LOG:-info}
- SERVER_PORT=8000
- MONGODB_URI=mongodb://mongodb:27017/normogen
- MONGODB_DATABASE=${MONGODB_DATABASE:-normogen}
- JWT_SECRET=${JWT_SECRET}
- JWT_ACCESS_TOKEN_EXPIRY_MINUTES=${JWT_ACCESS_TOKEN_EXPIRY_MINUTES:-15}
- JWT_REFRESH_TOKEN_EXPIRY_DAYS=${JWT_REFRESH_TOKEN_EXPIRY_DAYS:-30}
env_file:
- config/.env
depends_on:
mongodb:
condition: service_healthy
networks:
- normogen-network
restart: unless-stopped
deploy:
resources:
limits:
cpus: '1.0' # 1 CPU core
memory: 1000M # 1GB RAM
reservations:
cpus: '0.25'
memory: 256M
check:
test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:8000/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
mongodb:
image: mongo:6.0
container_name: normogen-mongodb
ports:
- "27017:27017"
environment:
- MONGO_INITDB_DATABASE=${MONGO_DATABASE:-normogen}
volumes:
- mongodb_data:/data/db
- mongodb_config:/data/configdb
networks:
- normogen-network
restart: unless-stopped
deploy:
resources:
limits:
cpus: '0.5'
memory: 512M
reservations:
cpus: '0.25'
memory: 128M
check:
test: ["CMD", "mongosh", "--eval", "db.adminCommand('ping')"]
interval: 10s
timeout: 5s
retries: 5
start_period: 10s
volumes:
mongodb_data:
driver: local
mongodb_config:
driver: local
networks:
normogen-network:
driver: bridge
```
### Development Docker Compose (Hot Reload)
```yaml
# docker-compose.dev.yml (development)
version: '3.8'
services:
backend:
build:
context: .
dockerfile: docker/Dockerfile.dev
container_name: normogen-backend-dev
ports:
- "6000:8000"
volumes:
- ./src:/app/src # Hot reload: Mount source code
- ./Cargo.toml:/app/Cargo.toml
environment:
- RUST_LOG=debug
- SERVER_PORT=8000
- MONGODB_URI=mongodb://mongodb:27017/normogen
- MONGODB_DATABASE=normogen_dev
- JWT_SECRET=dev-secret-do-not-use-in-production
- CARGO_WATCH=1 # Enable hot reload
depends_on:
mongodb:
condition: service_healthy
networks:
- normogen-network
restart: unless-stopped
mongodb:
image: mongo:6.0
container_name: normogen-mongodb-dev
ports:
- "27017:27017"
environment:
- MONGO_INITDB_DATABASE=normogen_dev
volumes:
- mongodb_dev_data:/data/db
networks:
- normogen-network
check:
test: ["CMD", "mongosh", "--eval", "db.adminCommand('ping')"]
interval: 10s
timeout: 5s
retries: 5
volumes:
mongodb_dev_data:
driver: local
networks:
normogen-network:
driver: bridge
```
### Development Dockerfile (with cargo-watch)
```dockerfile
# docker/Dockerfile.dev
FROM rust:1.75-alpine
WORKDIR /app
# Install runtime and build dependencies
RUN apk add --no-cache \
musl-dev \
pkgconf \
openssl-dev \
curl \
wget \
git
# Install cargo-watch for hot reload
RUN cargo install cargo-watch
# Copy manifests
COPY Cargo.toml Cargo.lock ./
# Create dummy main.rs to build dependencies
RUN mkdir src && echo "fn main() {}" > src/main.rs
RUN cargo build && rm -rf src
# Copy source code (will be mounted as volume in dev)
COPY src ./src
# Expose port
EXPOSE 8000
# Run with hot reload
CMD ["cargo-watch", "-x", "run"]
```
### Forgejo CI/CD (Example)
```yaml
# .forgejo/workflows/backend-build.yml
name: Build and Test Backend
on:
push:
branches: [main, develop]
pull_request:
branches: [main, develop]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install Rust
uses: actions-rust-lang/setup-rust-toolchain@v1
- name: Cache cargo registry
uses: actions/cache@v3
with:
path: ~/.cargo/registry
key: ${{ runner.os }}-cargo-registry-${{ hashFiles('**/Cargo.lock') }}
- name: Build backend
working-directory: ./backend
run: cargo build --verbose
- name: Run tests
working-directory: ./backend
run: cargo test --verbose
- name: Build Docker image
working-directory: ./backend
run: docker build -f docker/Dockerfile -t normogen-backend:${{ github.sha }} .
deploy-homelab:
needs: build
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
steps:
- name: Deploy to homelab
run: |
echo "Deploy to homelab server"
# Add deployment script here
```
### CORS Configuration (for Reverse Proxy)
```rust
// src/config/cors.rs
use tower_http::cors::{CorsLayer, Any};
use axum::http::{HeaderValue, Method};
pub fn create_cors_layer(allowed_origins: &str) -> CorsLayer {
let origins: Vec<&str> = allowed_origins.split(',').map(|s| s.trim()).collect();
CorsLayer::new()
.allow_origin(origins.into_iter().map(|s| {
s.parse::<HeaderValue>().unwrap()
}).collect::<Vec<_>>())
.allow_methods([Method::GET, Method::POST, Method::PUT, Method::PATCH, Method::DELETE, Method::OPTIONS])
.allow_headers(Any)
.allow_credentials(true)
.max_age(3600)
}
```
## Updated Deployment Checklist
### Initial Setup
- [ ] Clone repository from Forgejo
- [ ] Copy `backend/config/.env.example` to `backend/config/.env`
- [ ] Configure environment variables
- [ ] Generate JWT secret: `openssl rand -base64 32`
- [ ] Set resource limits in docker-compose.yml
### Development Setup
- [ ] Run development: `docker compose -f docker-compose.dev.yml up -d`
- [ ] Check logs: `docker compose -f docker-compose.dev.yml logs -f backend`
- [ ] Test hot reload: Make changes to src/, verify auto-rebuild
### Production Deployment
- [ ] Run production: `docker compose up -d`
- [ ] Check logs: `docker compose logs -f backend`
- [ ] Verify resource limits: `docker stats`
- [ ] Configure reverse proxy (Nginx/Traefik/Caddy)
- [ ] Configure reverse proxy CORS headers
- [ ] Test API through reverse proxy