diff --git a/backend/docker-compose.dev.yml b/backend/docker-compose.dev.yml index 5a43056..7562e91 100644 --- a/backend/docker-compose.dev.yml +++ b/backend/docker-compose.dev.yml @@ -35,12 +35,11 @@ services: networks: - normogen-network healthcheck: - test: | - mongosh --eval "db.adminCommand('ping').ok" --quiet + test: echo 'db.runCommand("ping").ok' | mongosh localhost:27017/test --quiet interval: 10s timeout: 5s retries: 5 - start_period: 40s + start_period: 60s volumes: mongodb_dev_data: driver: local diff --git a/backend/docker/MONGODB-TROUBLESHOOTING.md b/backend/docker/MONGODB-TROUBLESHOOTING.md new file mode 100644 index 0000000..8e0248e --- /dev/null +++ b/backend/docker/MONGODB-TROUBLESHOOTING.md @@ -0,0 +1,137 @@ +# MongoDB Health Check Troubleshooting + +## Problem +MongoDB container failing health checks despite running properly. + +## Root Cause Analysis + +### Issue 1: Complex Healthcheck Command +The original healthcheck used shell script format which can be problematic: +```yaml +test: | + mongosh --eval "db.adminCommand('ping').ok" --quiet +``` + +### Issue 2: Insufficient Startup Time +Even with 40s start_period, MongoDB may need more time on: +- First run (data initialization) +- Slower systems +- Systems with high I/O wait + +### Issue 3: Log Format Issues +The logs show extreme verbosity and duplication, suggesting the container is outputting logs in an unusual format. + +## Solution: Simplified Healthcheck + +### Updated Configuration +```yaml +healthcheck: + test: echo 'db.runCommand("ping").ok' | mongosh localhost:27017/test --quiet + interval: 10s + timeout: 5s + retries: 5 + start_period: 60s # Increased from 40s to 60s +``` + +### Key Changes +1. **Piped command instead of --eval**: More reliable with mongosh +2. **Explicit localhost**: Avoids DNS resolution issues +3. **Simple test database**: Uses /test instead of admin +4. **Longer start_period**: 60s gives MongoDB plenty of time + +## Alternative: Disable Healthcheck for Development + +If healthchecks continue to cause issues, you can disable them for development: + +```yaml +healthcheck: + disable: true +``` + +Or remove the healthcheck entirely and use a simple dependency: + +```yaml +depends_on: + - mongodb + # Remove: condition: service_healthy +``` + +## How to Apply + +### Option 1: Pull and Restart (Recommended) +```bash +git pull origin main +docker compose -f docker-compose.dev.yml down -v +docker compose -f docker-compose.dev.yml up -d +docker compose -f docker-compose.dev.yml logs -f mongodb +``` + +### Option 2: Disable Healthcheck (Quick Fix) +Edit `docker-compose.dev.yml` and replace the healthcheck section with: +```yaml +healthcheck: + disable: true +``` + +Then restart: +```bash +docker compose -f docker-compose.dev.yml down -v +docker compose -f docker-compose.dev.yml up -d +``` + +## Verification + +### Check Container Status +```bash +docker ps --format "table {{.Names}} {{.Status}}" +``` + +### Check MongoDB Connection +```bash +docker exec normogen-mongodb-dev mongosh --eval "db.adminCommand('ping')" +``` + +### Check Health Status +```bash +docker inspect normogen-mongodb-dev --format='{{json .State.Health}}' | jq +``` + +## Common Issues and Fixes + +### Issue: Port Already in Use +```bash +# Check what's using port 27017 +sudo lsof -i :27017 + +# Kill the process if needed +sudo kill -9 +``` + +### Issue: Corrupted Volume +```bash +# Remove the volume and start fresh +docker compose -f docker-compose.dev.yml down -v +docker compose -f docker-compose.dev.yml up -d +``` + +### Issue: mongosh Not Found +This shouldn't happen with mongo:6.0, but if it does: +```bash +# Verify mongosh exists +docker exec normogen-mongodb-dev which mongosh + +# If not found, try using mongo (legacy) +docker exec normogen-mongodb-dev which mongo +``` + +## Development vs Production + +### Development (Current) +- Healthcheck enabled but with generous timeouts +- Focus on getting up and running quickly +- Can disable healthcheck if causing issues + +### Production +- Healthcheck is critical +- Must use proper healthcheck with monitoring +- Consider using orchestration tools (Kubernetes, etc.)