flynn/docs/plans/2026-02-13-gmail-deployment-patterns.md

Gmail Push Notifications — Deployment Patterns

Architecture Overview

┌─────────────────────────────────────────────────────────────────────┐
│                         Google Cloud Platform                        │
│                                                                      │
│  ┌─────────────┐         ┌──────────────────────────────────────┐  │
│  │  Gmail API  │────────▶│  Pub/Sub Topic: gmail-push           │  │
│  └─────────────┘         └──────────────────────────────────────┘  │
│       │ New email                     │                             │
│       │ triggers                      ├──── Push Subscription       │
│       │ watch()                       │     (pushEndpoint)          │
│       │                               │                             │
│       │                               └──── Pull Subscription       │
│       │                                     (no endpoint)           │
│       │                                                             │
└───────┼─────────────────────────────────────────────────────────────┘
        │                                                      │
        │                                                      │
        │ History API                                          │
        │ (fallback poll)                                      │
        │                                                      │
        ▼                                                      ▼
┌────────────────────────────────────────────────────────────────────┐
│                              Flynn                                  │
│                                                                     │
│  ┌──────────────┐    ┌────────────────┐    ┌────────────────────┐ │
│  │ Push Handler │◀───│ Gateway Server │    │  Pull Subscriber   │ │
│  │              │    │                │    │                    │ │
│  │ POST /gmail/ │    │  Port 18800    │    │ Pull every 60s     │ │
│  │ push         │    └────────────────┘    └────────────────────┘ │
│  └──────────────┘                                  │               │
│         │                                          │               │
│         └──────────────┬───────────────────────────┘               │
│                        ▼                                           │
│              ┌──────────────────┐                                  │
│              │  GmailWatcher    │                                  │
│              │  (ChannelAdapter)│                                  │
│              └──────────────────┘                                  │
│                        │                                           │
│                        ▼                                           │
│              ┌──────────────────┐                                  │
│              │ AgentOrchestrator│                                  │
│              └──────────────────┘                                  │
│                        │                                           │
│                        ▼                                           │
│              ┌──────────────────┐                                  │
│              │ Output Channel   │                                  │
│              │ (Telegram/etc)   │                                  │
│              └──────────────────┘                                  │
└─────────────────────────────────────────────────────────────────────┘

---

Pattern 1: Push Only (Public Endpoint)

Config:

automation:
  gmail:
    pubsub_topic: projects/my-project/topics/gmail-push

Flow:

1. Gmail API sends notification to Pub/Sub topic
2. Push subscription forwards to POST https://flynn.example.com/gmail/push
3. Flynn processes immediately (~real-time)
4. History API polls every 5min (fallback)

Requirements:

• Public IP or domain
• DNS A/AAAA record
• SSL/TLS certificate
• Firewall allows inbound HTTP/HTTPS

Network:

Internet ──▶ Public IP ──▶ Flynn Gateway ──▶ GmailWatcher
             (Port 443)     (Port 18800)

---

Pattern 2: Push Only (Tailscale Funnel)

Config:

server:
  tailscale:
    serve: true
    funnel: true  # ← Required for Google to reach endpoint

automation:
  gmail:
    pubsub_topic: gmail-push

Flow:

1. Gmail API sends notification to Pub/Sub topic
2. Push subscription forwards to POST https://flynn.tailnet-name.ts.net/gmail/push
3. Tailscale funnel proxies to local Flynn
4. Flynn processes immediately (~real-time)

Requirements:

• Tailscale installed and authenticated
• Funnel enabled (exposes service to public internet)

Network:

Internet ──▶ Tailscale Funnel ──▶ Flynn Gateway ──▶ GmailWatcher
             (*.ts.net)             (Port 18800)

---

Pattern 3: Pull Only (Private Deployment)

Config:

automation:
  gmail:
    pubsub_subscription_id: projects/my-project/subscriptions/gmail-pull
    pubsub_pull_interval: "60s"

Flow:

1. Gmail API sends notification to Pub/Sub topic
2. Message sits in pull subscription queue
3. Flynn polls subscription every 60s
4. Flynn pulls and processes messages
5. History API polls every 5min (fallback)

Requirements:

• None (no inbound connections)
• Works behind NAT, firewall, VPN

Network:

Flynn ──(poll)──▶ GCP Pub/Sub API ──▶ Pull Subscription
                  (HTTPS outbound)

---

Pattern 4: Hybrid (Recommended)

Config:

automation:
  gmail:
    pubsub_topic: gmail-push
    pubsub_subscription_id: gmail-pull
    pubsub_pull_interval: "60s"
    poll_interval: "300s"

Flow:

1. Gmail API sends notification to Pub/Sub topic
2. Push subscription forwards to Flynn gateway (if reachable)
   • Processes immediately (~real-time)
3. Pull subscription also receives message (60s latency)
   • Deduplicates via historyId comparison
4. History API polls every 5min (tertiary fallback)

Requirements:

• Push subscription: Public endpoint OR Tailscale funnel
• Pull subscription: Always works (no inbound)

Network:

┌──────────────────────────────────────────────┐
│ Primary:   Internet ──▶ Flynn (push)         │  ~Real-time
├──────────────────────────────────────────────┤
│ Fallback:  Flynn ──(poll)──▶ GCP (pull)      │  ~60s latency
├──────────────────────────────────────────────┤
│ Tertiary:  Flynn ──(poll)──▶ Gmail History   │  ~300s latency
└──────────────────────────────────────────────┘

Benefits:

• ✅ Best latency when push is reachable
• ✅ Always reliable (pull fallback)
• ✅ Network-agnostic (works behind NAT/firewall)
• ✅ Self-healing (network changes don't break it)

---

Pattern 5: Polling Only (Development)

Config:

automation:
  gmail:
    poll_interval: "60s"  # No pubsub_topic or pubsub_subscription_id

Flow:

1. Flynn polls Gmail History API every 60s
2. Fetches new messages since last historyId
3. No GCP Pub/Sub setup required

Requirements:

• None (just OAuth2 credentials)

Network:

Flynn ──(poll)──▶ Gmail History API
                  (HTTPS outbound)

Use Case:

• Development/testing
• Quick setup without GCP project
• Low-volume inboxes (polling is fine)

---

Comparison Table

Pattern	Latency	Reliability	Network Req	GCP Setup	Recommended For
Push only (public)	~1s	High*	Public IP	Topic + push sub	Production with ingress
Push only (funnel)	~1s	High*	Tailscale funnel	Topic + push sub	Private with funnel
Pull only	~60s	High	Any	Topic + pull sub	Private behind NAT
Hybrid ⭐	~1s†	Highest	Any	Topic + both subs	All production
Polling only	~300s	Medium	Any	None	Development only

* Single point of failure (push endpoint unreachable = delayed notification)
† Falls back to pull (~60s) if push fails

---

GCP Setup Commands

1. Create Topic

gcloud pubsub topics create gmail-push --project=my-project

2. Grant Gmail API Permission

gcloud pubsub topics add-iam-policy-binding projects/my-project/topics/gmail-push \
  --member=serviceAccount:gmail-api-push@system.gserviceaccount.com \
  --role=roles/pubsub.publisher

3. Create Push Subscription (for push mode)

gcloud pubsub subscriptions create gmail-push-sub \
  --topic=gmail-push \
  --push-endpoint=https://flynn.example.com/gmail/push \
  --ack-deadline=60

4. Create Pull Subscription (for pull mode)

gcloud pubsub subscriptions create gmail-pull-sub \
  --topic=gmail-push \
  --ack-deadline=60

5. Verify Setup

# List subscriptions
gcloud pubsub subscriptions list --filter="topic:gmail-push"

# Test pull subscription
gcloud pubsub subscriptions pull gmail-pull-sub --limit=5

---

Troubleshooting Decision Tree

Start here: flynn doctor
    │
    ├─▶ ✓ Gmail configured (push + pull + poll-fallback)
    │   └─▶ ✅ Hybrid mode active (best config)
    │
    ├─▶ ✓ Gmail configured (push + poll-fallback)
    │   └─▶ ⚠️  Single point of failure (add pull subscription)
    │
    ├─▶ ⚠️  Gmail configured (push ⚠️ push requires public endpoint)
    │   └─▶ Enable Tailscale funnel OR add pull subscription
    │
    ├─▶ ✓ Gmail configured (pull + poll-fallback)
    │   └─▶ ✅ Reliable but slower (60s latency)
    │
    └─▶ ✓ Gmail configured (poll)
        └─▶ ⚠️  Slow (300s latency) — add push or pull

---

Migration Guide

From: Polling Only

# Before
automation:
  gmail:
    poll_interval: "60s"

To: Hybrid (Recommended)

# After
automation:
  gmail:
    pubsub_topic: gmail-push                   # Add push
    pubsub_subscription_id: gmail-pull-sub     # Add pull
    pubsub_pull_interval: "60s"
    poll_interval: "300s"                      # Keep as fallback

Steps:

1. Create GCP topic and subscriptions (see above)
2. Update config with new fields
3. Restart Flynn: systemctl restart flynn
4. Verify: flynn doctor shows "push + pull + poll-fallback"