will/adopt-a-street
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: complete MongoDB to CouchDB migration

Browse Source 
- Migrate Report model to CouchDB with embedded street/user data
- Migrate UserBadge model to CouchDB with badge population
- Update all remaining routes (reports, users, badges, payments) to use CouchDB
- Add CouchDB health check and graceful shutdown to server.js
- Add missing methods to couchdbService (checkConnection, findWithPagination, etc.)
- Update Kubernetes deployment manifests for CouchDB support
- Add comprehensive CouchDB setup documentation

All core functionality now uses CouchDB as primary database while maintaining
MongoDB for backward compatibility during transition period.

🤖 Generated with [AI Assistant]

Co-Authored-By: AI Assistant <noreply@ai-assistant.com>
This commit is contained in:
William Valentin
2025-11-01 13:29:48 -07:00
parent 9ac21fca72
commit df94c17e1f
14 changed files with 684 additions and 155 deletions
Download Patch File Download Diff File Expand all files Collapse all files

66
deploy/README.md
View File

@@ -10,7 +10,8 @@ deploy/
│ ├── namespace.yaml # Namespace definition
│ ├── configmap.yaml # Environment configuration
│ ├── secrets.yaml.example # Secret template (COPY TO secrets.yaml)
│ ├── mongodb-statefulset.yaml # MongoDB StatefulSet with PVC
│ ├── couchdb-statefulset.yaml # CouchDB StatefulSet with PVC
│ ├── couchdb-configmap.yaml # CouchDB configuration
│ ├── backend-deployment.yaml # Backend Deployment + Service
│ ├── frontend-deployment.yaml # Frontend Deployment + Service
│ └── ingress.yaml # Ingress for routing
@@ -94,6 +95,19 @@ nano deploy/k8s/frontend-deployment.yaml
# Change: image: your-registry/adopt-a-street-frontend:latest
```
### 4. Configure CouchDB
```bash
# Apply CouchDB configuration
kubectl apply -f deploy/k8s/couchdb-configmap.yaml
# Deploy CouchDB
kubectl apply -f deploy/k8s/couchdb-statefulset.yaml
# Wait for CouchDB to be ready
kubectl wait --for=condition=ready pod -l app=couchdb -n adopt-a-street --timeout=120s
```
### 4. Update Domain Name
Update the ingress host:
@@ -116,11 +130,9 @@ kubectl apply -f deploy/k8s/secrets.yaml
# Create ConfigMap
kubectl apply -f deploy/k8s/configmap.yaml
# Deploy MongoDB
kubectl apply -f deploy/k8s/mongodb-statefulset.yaml
# Wait for MongoDB to be ready (this may take 1-2 minutes)
kubectl wait --for=condition=ready pod -l app=mongodb -n adopt-a-street --timeout=120s
# Deploy CouchDB (already done in step 4)
# Wait for CouchDB to be ready (this may take 1-2 minutes)
kubectl wait --for=condition=ready pod -l app=couchdb -n adopt-a-street --timeout=120s
# Deploy backend
kubectl apply -f deploy/k8s/backend-deployment.yaml
@@ -155,7 +167,7 @@ kubectl get pods -n adopt-a-street
# adopt-a-street-backend-xxxxxxxxxx-xxxxx 1/1 Running 0 5m
# adopt-a-street-frontend-xxxxxxxxx-xxxxx 1/1 Running 0 5m
# adopt-a-street-frontend-xxxxxxxxx-xxxxx 1/1 Running 0 5m
# adopt-a-street-mongodb-0 1/1 Running 0 10m
# adopt-a-street-couchdb-0 1/1 Running 0 10m
```
### Check Logs
@@ -167,8 +179,8 @@ kubectl logs -f deployment/adopt-a-street-backend -n adopt-a-street
# Frontend logs
kubectl logs -f deployment/adopt-a-street-frontend -n adopt-a-street
# MongoDB logs
kubectl logs -f adopt-a-street-mongodb-0 -n adopt-a-street
# CouchDB logs
kubectl logs -f adopt-a-street-couchdb-0 -n adopt-a-street
```
### Check Services
@@ -180,7 +192,7 @@ kubectl get svc -n adopt-a-street
# NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
# adopt-a-street-backend ClusterIP 10.43.x.x <none> 5000/TCP 5m
# adopt-a-street-frontend ClusterIP 10.43.x.x <none> 80/TCP 5m
# adopt-a-street-mongodb ClusterIP None <none> 27017/TCP 10m
# adopt-a-street-couchdb ClusterIP None <none> 5984/TCP 10m
```
### Check Ingress
@@ -205,10 +217,11 @@ kubectl port-forward svc/adopt-a-street-frontend 3000:80 -n adopt-a-street
The deployment is optimized for Raspberry Pi hardware:
### MongoDB (Pi 5 nodes only)
### CouchDB (Pi 5 nodes only)
- **Requests:** 512Mi RAM, 250m CPU
- **Limits:** 2Gi RAM, 1000m CPU
- **Storage:** 10Gi persistent volume
- **Additional:** 64Mi RAM, 50m CPU for metrics exporter
### Backend (prefers Pi 5 nodes)
- **Requests:** 256Mi RAM, 100m CPU
@@ -221,7 +234,7 @@ The deployment is optimized for Raspberry Pi hardware:
- **Replicas:** 2 pods
### Total Cluster Requirements
- **Minimum RAM:** ~3.5 GB (1.5GB MongoDB + 1GB backend + 200MB frontend + 800MB system)
- **Minimum RAM:** ~3.6 GB (1.5GB CouchDB + 1GB backend + 200MB frontend + 800MB system)
- **Recommended:** 2x Pi 5 (8GB each) handles this comfortably
## Scaling
@@ -236,7 +249,7 @@ kubectl scale deployment adopt-a-street-backend --replicas=3 -n adopt-a-street
kubectl scale deployment adopt-a-street-frontend --replicas=3 -n adopt-a-street
```
**Note:** MongoDB is a StatefulSet with 1 replica. Scaling MongoDB requires configuring replication.
**Note:** CouchDB is a StatefulSet with 1 replica. Scaling CouchDB requires configuring clustering.
## Updating
@@ -318,14 +331,17 @@ kubectl logs <pod-name> -n adopt-a-street --previous
- Verify cluster can access registry
- Check if imagePullSecrets are needed
### MongoDB Connection Issues
### CouchDB Connection Issues
```bash
# Shell into backend pod
kubectl exec -it <backend-pod-name> -n adopt-a-street -- sh
# Test MongoDB connection
wget -qO- http://adopt-a-street-mongodb:27017
# Test CouchDB connection
curl -f http://adopt-a-street-couchdb:5984/_up
# Test authentication
curl -u $COUCHDB_USER:$COUCHDB_PASSWORD http://adopt-a-street-couchdb:5984/_session
```
### Persistent Volume Issues
@@ -353,7 +369,8 @@ kubectl delete namespace adopt-a-street
kubectl delete -f deploy/k8s/ingress.yaml
kubectl delete -f deploy/k8s/frontend-deployment.yaml
kubectl delete -f deploy/k8s/backend-deployment.yaml
kubectl delete -f deploy/k8s/mongodb-statefulset.yaml
kubectl delete -f deploy/k8s/couchdb-statefulset.yaml
kubectl delete -f deploy/k8s/couchdb-configmap.yaml
kubectl delete -f deploy/k8s/configmap.yaml
kubectl delete -f deploy/k8s/secrets.yaml
kubectl delete -f deploy/k8s/namespace.yaml
@@ -365,20 +382,21 @@ kubectl delete -f deploy/k8s/namespace.yaml
1. **Never commit secrets.yaml** - Always use secrets.yaml.example
2. **Use strong JWT_SECRET** - Generate with: `openssl rand -base64 32`
3. **Enable TLS/HTTPS** - Uncomment TLS section in ingress.yaml and use cert-manager
4. **Restrict ingress** - Use network policies to limit pod communication
5. **Use image digests** - Pin images to specific SHA256 digests for production
6. **Enable RBAC** - Create service accounts with minimal permissions
7. **Scan images** - Use tools like Trivy to scan for vulnerabilities
3. **Use strong CouchDB passwords** - Generate with: `openssl rand -base64 32`
4. **Enable TLS/HTTPS** - Uncomment TLS section in ingress.yaml and use cert-manager
5. **Restrict ingress** - Use network policies to limit pod communication
6. **Use image digests** - Pin images to specific SHA256 digests for production
7. **Enable RBAC** - Create service accounts with minimal permissions
8. **Scan images** - Use tools like Trivy to scan for vulnerabilities
## Performance Optimization
1. **Use imagePullPolicy: IfNotPresent** - After initial deployment to save bandwidth
2. **Implement HPA** - Horizontal Pod Autoscaler for dynamic scaling
3. **Add Redis** - For caching to reduce MongoDB load
3. **Add Redis** - For caching to reduce CouchDB load
4. **Use CDN** - For frontend static assets
5. **Enable compression** - Nginx already configured with gzip
6. **Monitor resources** - Use Prometheus + Grafana for metrics
6. **Monitor resources** - Use Prometheus + Grafana for metrics (CouchDB exporter included)
## Additional Resources

11
deploy/k8s/backend-deployment.yaml
View File

@@ -54,6 +54,17 @@ spec:
name: adopt-a-street-config
- secretRef:
name: adopt-a-street-secrets
env:
- name: COUCHDB_USER
valueFrom:
secretKeyRef:
name: adopt-a-street-secrets
key: COUCHDB_USER
- name: COUCHDB_PASSWORD
valueFrom:
secretKeyRef:
name: adopt-a-street-secrets
key: COUCHDB_PASSWORD
resources:
requests:
memory: "256Mi"

5
deploy/k8s/configmap.yaml
View File

@@ -4,8 +4,9 @@ metadata:
name: adopt-a-street-config
namespace: adopt-a-street
data:
# MongoDB Connection
MONGO_URI: "mongodb://adopt-a-street-mongodb:27017/adopt-a-street"
# CouchDB Connection
COUCHDB_URL: "http://adopt-a-street-couchdb:5984"
COUCHDB_DB_NAME: "adopt-a-street"
# Backend Configuration
PORT: "5000"

23
deploy/k8s/couchdb-configmap.yaml Normal file
View File

@@ -0,0 +1,23 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: couchdb-config
namespace: adopt-a-street
data:
10-cluster.ini: |
[cluster]
n = 1
q = 8
; Enable cluster features
[chttpd]
bind_address = 0.0.0.0
port = 5984
[couchdb]
single_node = false
enable_cors = true
[cors]
origins = *
credentials = true
headers = accept, authorization, content-type, origin, referer, x-csrf-token
methods = GET, PUT, POST, HEAD, DELETE
max_age = 3600

146
deploy/k8s/couchdb-statefulset.yaml Normal file
View File

@@ -0,0 +1,146 @@
apiVersion: v1
kind: Service
metadata:
name: adopt-a-street-couchdb
namespace: adopt-a-street
labels:
app: couchdb
spec:
clusterIP: None # Headless service for StatefulSet
selector:
app: couchdb
ports:
- port: 5984
targetPort: 5984
name: couchdb
- port: 4369
targetPort: 4369
name: epmd
- port: 9100
targetPort: 9100
name: couchdb-exporter
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: adopt-a-street-couchdb
namespace: adopt-a-street
spec:
serviceName: adopt-a-street-couchdb
replicas: 1
selector:
matchLabels:
app: couchdb
template:
metadata:
labels:
app: couchdb
spec:
# Place CouchDB on Pi 5 nodes (more RAM)
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/arch
operator: In
values:
- arm64 # Pi 5 architecture
containers:
- name: couchdb
image: couchdb:3.3
ports:
- containerPort: 5984
name: couchdb
- containerPort: 4369
name: epmd
- containerPort: 9100
name: couchdb-exporter
env:
- name: COUCHDB_USER
valueFrom:
secretKeyRef:
name: adopt-a-street-secrets
key: COUCHDB_USER
- name: COUCHDB_PASSWORD
valueFrom:
secretKeyRef:
name: adopt-a-street-secrets
key: COUCHDB_PASSWORD
- name: COUCHDB_SECRET
valueFrom:
secretKeyRef:
name: adopt-a-street-secrets
key: COUCHDB_SECRET
- name: NODENAME
value: couchdb@0.adopt-a-street-couchdb.adopt-a-street
- name: ERL_FLAGS
value: "+K true +A 4"
resources:
requests:
memory: "512Mi"
cpu: "250m"
limits:
memory: "2Gi"
cpu: "1000m"
volumeMounts:
- name: couchdb-data
mountPath: /opt/couchdb/data
- name: couchdb-config
mountPath: /opt/couchdb/etc/local.d
livenessProbe:
httpGet:
path: /_up
port: 5984
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
readinessProbe:
httpGet:
path: /_up
port: 5984
initialDelaySeconds: 5
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
- name: couchdb-exporter
image: gesellix/couchdb-exporter:latest
ports:
- containerPort: 9100
name: metrics
env:
- name: COUCHDB_URL
value: "http://localhost:5984"
- name: COUCHDB_USER
valueFrom:
secretKeyRef:
name: adopt-a-street-secrets
key: COUCHDB_USER
- name: COUCHDB_PASSWORD
valueFrom:
secretKeyRef:
name: adopt-a-street-secrets
key: COUCHDB_PASSWORD
resources:
requests:
memory: "64Mi"
cpu: "50m"
limits:
memory: "128Mi"
cpu: "100m"
volumes:
- name: couchdb-config
configMap:
name: couchdb-config
volumeClaimTemplates:
- metadata:
name: couchdb-data
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 10Gi
# Uncomment and set your storage class if needed
# storageClassName: local-path

9
deploy/k8s/secrets.yaml.example
View File

@@ -8,6 +8,11 @@ stringData:
# JWT Secret - CHANGE THIS IN PRODUCTION!
JWT_SECRET: "your-super-secret-jwt-key-change-in-production"
# CouchDB Configuration
COUCHDB_USER: "admin" # Change this in production
COUCHDB_PASSWORD: "admin" # Change this in production
COUCHDB_SECRET: "some-random-secret-string" # Change this in production
# Cloudinary Configuration
CLOUDINARY_CLOUD_NAME: "your-cloudinary-cloud-name"
CLOUDINARY_API_KEY: "your-cloudinary-api-key"
@@ -16,9 +21,13 @@ stringData:
# Stripe Configuration (optional - currently mocked)
# STRIPE_SECRET_KEY: "your-stripe-secret-key"
# OpenAI Configuration (optional - for AI features)
# OPENAI_API_KEY: "your-openai-api-key"
---
# IMPORTANT:
# 1. Copy this file to secrets.yaml
# 2. Replace all placeholder values with real secrets
# 3. DO NOT commit secrets.yaml to version control
# 4. Add secrets.yaml to .gitignore
# 5. Generate strong passwords for CouchDB using: openssl rand -base64 32