docs(docker): Add /var space issue documentation and monitoring scripts

2026-02-15 15:18:21 -03:00 · 2026-02-15 15:18:21 -03:00 · b0318430ad
commit b0318430ad
parent f0b5109f61
4 changed files with 347 additions and 0 deletions
--- a/backend/DAILY-SUMMARY.md
+++ b/backend/DAILY-SUMMARY.md
@ -0,0 +1,154 @@
+# Today's Docker Issues - Summary and Solutions
+
+## Date: 2026-02-15
+
+## Issues Fixed
+
+### 1. Edition 2024 Error
+**Problem:** Rust 1.83-alpine didn't support Edition 2024
+**Solution:** Updated Dockerfiles to use Rust 1.93-slim
+**Files Modified:**
+- `backend/docker/Dockerfile`
+- `backend/docker/Dockerfile.dev`
+**Documentation:** `backend/docker/EDITION2024-FIX.md`
+
+### 2. MongoDB Healthcheck Configuration
+**Problem:** Healthcheck timing out, complex command
+**Solution:** Simplified healthcheck with 60s startup grace period
+**Files Modified:**
+- `backend/docker-compose.dev.yml`
+**Documentation:** `backend/docker/MONGODB-TROUBLESHOOTING.md`
+
+### 3. MongoDB Disk Space Issue
+**Problem:** MongoDB crashing with "No space left on device" error
+**Root Cause:** `/var` filesystem was 100% full (not root `/` filesystem)
+**Solution:** Freed up space in `/var` filesystem
+**Key Insight:** Docker stores volumes in `/var/lib/docker/volumes/`, so `/var` space matters more than root space for MongoDB
+**Documentation:** `backend/docker/MONGODB-VAR-FULL-ISSUE.md`
+
+## Lessons Learned
+
+1. **Always check all filesystems** with `df -h`, not just root (`/`)
+2. **Docker data location matters** - `/var/lib/docker` by default
+3. **Separate mounts have different space** - `/var` can be full while `/` has space
+4. **Monitor Docker space usage** regularly with `docker system df`
+
+## Prevention Setup
+
+### Regular Monitoring
+Add to crontab:
+```bash
+# Check disk space every hour
+0 * * * * /path/to/normogen/backend/scripts/check-disk-space.sh
+
+# Clean Docker weekly
+0 2 * * 0 docker system prune -f --filter "until=168h"
+```
+
+### Manual Checks
+```bash
+# Quick space check
+df -h
+
+# Docker space usage
+docker system df
+
+# Verify stack is running
+./backend/scripts/verify-stack.sh
+```
+
+## Documentation Created
+
+1. `backend/docker/EDITION2024-FIX.md` - Edition 2024 fix
+2. `backend/docker/MONGODB-TROUBLESHOOTING.md` - MongoDB issues
+3. `backend/docker/MONGODB-PERMISSIONS-FIX.md` - Permissions guide
+4. `backend/docker/MONGODB-DISKSPACE-FIX.md` - Disk space guide
+5. `backend/docker/MONGODB-VAR-FULL-ISSUE.md` - /var space issue
+6. `backend/docker/DOCKER-COMMANDS.md` - Docker commands reference
+7. `backend/scripts/check-disk-space.sh` - Space monitoring
+8. `backend/scripts/verify-stack.sh` - Stack verification
+9. `backend/diagnose-mongodb.sh` - MongoDB diagnostics
+
+## Quick Reference
+
+### Start the Stack
+```bash
+cd backend
+docker compose -f docker-compose.dev.yml up -d
+```
+
+### Check Status
+```bash
+docker compose -f docker-compose.dev.yml ps
+docker ps | grep normogen
+```
+
+### View Logs
+```bash
+# All services
+docker compose -f docker-compose.dev.yml logs -f
+
+# MongoDB only
+docker logs -f normogen-mongodb-dev
+
+# Backend only
+docker logs -f normogen-backend-dev
+```
+
+### Stop the Stack
+```bash
+docker compose -f docker-compose.dev.yml down
+```
+
+### Clean Restart
+```bash
+docker compose -f docker-compose.dev.yml down -v
+docker compose -f docker-compose.dev.yml up -d
+```
+
+## Success Indicators
+
+When everything is working, you should see:
+
+1. **Containers running:**
+   ```
+   $ docker ps | grep normogen
+   normogen-mongodb-dev   Up X minutes (healthy)   0.0.0.0:27017->27017/tcp
+   normogen-backend-dev   Up X minutes            0.0.0.0:6800->8000/tcp
+   ```
+
+2. **MongoDB logs:**
+   ```
+   {"msg":"Waiting for connections on port 27017"}
+   ```
+
+3. **Backend logs:**
+   ```
+   Server is running on http://0.0.0.0:8000
+   ```
+
+4. **Healthcheck:**
+   ```
+   $ docker inspect normogen-mongodb-dev --format='{{.State.Health.Status}}'
+   healthy
+   ```
+
+## Git Commits
+
+1. `d63f160` - fix(docker): Update to Rust 1.93 to support Edition 2024
+2. `b218594` - fix(docker): Fix MongoDB healthcheck configuration
+3. `b068579` - fix(docker): Simplify MongoDB healthcheck and add troubleshooting
+4. `f0b5109` - fix(docker): Document MongoDB disk space issue and solutions
+
+## Next Steps
+
+1. Build completed successfully
+2. `/var` space issue resolved
+3. Verify stack is running with `./backend/scripts/verify-stack.sh`
+4. Test API endpoints
+5. Continue with Phase 2.4 development
+
+---
+
+**Last Updated:** 2026-02-15
+**Status:** All issues resolved