Files
flynn/docs/plans/2026-02-13-gmail-push-revisit-implementation-plan.md
T

1014 lines
34 KiB
Markdown

# Gmail Push Notifications Revisit — Implementation Plan
> **Archived (2026-02-18):** Historical implementation checklist. Canonical status is tracked in `docs/plans/state.json`; unchecked boxes here are not active backlog unless explicitly re-opened.
**Date**: 2026-02-13
**Status**: Planned
**Goal**: Make Gmail notifications operationally reliable and well-documented
## Overview
The Gmail watcher currently supports two modes:
1. **Pub/Sub push notifications** (Google sends HTTP POST to `/gmail/push`)
2. **Polling fallback** (History API queries every N seconds)
Current implementation has operational ambiguity around:
- Deployment patterns (public webhook vs Pub/Sub pull vs hybrid)
- GCP resource setup (topic/subscription/IAM relationships)
- Network reachability constraints (Tailscale-only gateways)
- Configuration surface (implicit vs explicit controls)
- Error diagnostics (vague failure messages)
This plan addresses these gaps without changing core functionality.
---
## Problem Analysis
### Current Behavior
**Push setup flow** (`src/automation/gmail.ts:222-253`):
1. `resolvePubSubTopicName()` reads `pubsub_topic` config or `FLYNN_GMAIL_PUBSUB_TOPIC` env var
2. Calls `gmail.users.watch()` with the topic
3. If watch fails → logs warning, continues with polling only
4. If watch succeeds → schedules renewal every 6 days
**Issues**:
- **Topic ownership mismatch**: OAuth credentials' `project_id` may differ from the Pub/Sub topic's project
- **Network reachability**: Google cannot reach Tailscale-only endpoints (`server.tailscale_only: true`)
- **Implicit disable**: No explicit way to disable push without deleting `pubsub_topic` field
- **Poor error messages**: "Invalid topicName" error includes no actionable guidance
- **Undocumented subscription**: Users must manually create Pub/Sub subscription → push endpoint mapping
- **Doctor check gap**: `flynn doctor` validates OAuth but not Pub/Sub setup
### Deployment Patterns
| Pattern | Reachability | Flynn Config | GCP Setup | Use Case |
|---------|--------------|--------------|-----------|----------|
| **Public webhook** | Public IP + DNS | `pubsub_topic` set | Topic + push subscription → `POST https://example.com/gmail/push` | Production deployment with public ingress |
| **Tailscale Serve** | Tailscale funnel | `pubsub_topic` set, `server.tailscale.funnel: true` | Topic + push subscription → `POST https://flynn.ts.net/gmail/push` | Private deployment with Tailscale funnel enabled |
| **Pub/Sub pull** | Any (no inbound) | `pubsub_subscription_id` set | Topic + pull subscription | Private deployment, Flynn polls subscription |
| **Polling only** | Any (no inbound) | `pubsub_topic` omitted | None | Development, no GCP setup |
| **Hybrid (recommended)** | Any | `pubsub_topic` + `pubsub_subscription_id` | Topic + push subscription + pull subscription | Push when reachable, pull fallback, History API tertiary |
---
## Recommended Changes
### 1. Config Schema Additions
**File**: `src/config/schema.ts`
Add new fields to `gmailSchema`:
```typescript
const gmailSchema = z.object({
enabled: z.boolean().default(false),
credentials_file: z.string().optional(),
token_file: z.string().default('~/.config/flynn/gmail-token.json'),
// ── Push notifications ────────────────────────────────────────────────
/**
* Pub/Sub topic for Gmail push notifications (watch API).
* Format: projects/<project-id>/topics/<topic> or just <topic> (auto-prefixed).
* If omitted, push notifications are disabled (polling only).
*/
pubsub_topic: z.string().optional(),
/**
* Pub/Sub subscription ID for pull-based message retrieval.
* Format: projects/<project-id>/subscriptions/<subscription> or just <subscription>.
* When set, Flynn periodically pulls messages from this subscription in addition
* to waiting for push notifications at POST /gmail/push.
* Enables hybrid mode: push when reachable, pull fallback.
*/
pubsub_subscription_id: z.string().optional(),
/**
* Pull interval for Pub/Sub subscription polling (hybrid mode).
* Only used when pubsub_subscription_id is set. Default: 60s.
*/
pubsub_pull_interval: z.string().default('60s'),
/**
* Maximum messages to pull per request when using pull mode. Default: 10.
*/
pubsub_max_messages: z.number().min(1).max(100).default(10),
/**
* Disable watch API even if pubsub_topic is set.
* Useful for debugging or when push endpoint is temporarily unreachable.
*/
disable_push: z.boolean().default(false),
// ── Existing fields ────────────────────────────────────────────────────
watch_labels: z.array(z.string()).default(['INBOX']),
poll_interval: z.string().default('300s'), // History API fallback
history_start: z.string().optional(),
output: z.object({
channel: z.string().min(1),
peer: z.string().min(1),
}),
message: z.string().default('New email from {{from}}: {{subject}}\n\n{{snippet}}'),
}).optional();
```
**Defaults**:
- `pubsub_topic`: `undefined` (no push)
- `pubsub_subscription_id`: `undefined` (no pull)
- `pubsub_pull_interval`: `'60s'`
- `pubsub_max_messages`: `10`
- `disable_push`: `false`
- `poll_interval`: `'300s'` (History API fallback)
---
### 2. Implementation Changes
#### 2.1 Add Pub/Sub Pull Support
**File**: `src/automation/gmail.ts`
Add new method `setupPullSubscription()`:
```typescript
private pullSubscriptionTimer?: ReturnType<typeof setInterval>;
private subscriptionPath?: string;
/**
* Set up Pub/Sub pull subscription for periodic message retrieval.
* Alternative/fallback to push notifications when endpoint is unreachable.
*/
private async setupPullSubscription(): Promise<void> {
if (!this.config.pubsub_subscription_id) {
return; // Pull mode not configured
}
const subId = this.config.pubsub_subscription_id.trim();
// Allow shorthand: just the subscription id (e.g. "gmail-pull")
if (!subId.includes('/')) {
if (!this.googleProjectId) {
console.warn(
`GmailWatcher: pubsub_subscription_id '${subId}' must be fully qualified ` +
`(projects/<project-id>/subscriptions/<sub>) because project_id was not found in credentials`
);
return;
}
this.subscriptionPath = `projects/${this.googleProjectId}/subscriptions/${subId}`;
} else {
const isValid = /^projects\/[^/]+\/subscriptions\/[^/]+$/.test(subId);
if (!isValid) {
console.warn(
`GmailWatcher: Invalid pubsub_subscription_id '${subId}'. ` +
`Expected: projects/<project-id>/subscriptions/<sub>`
);
return;
}
this.subscriptionPath = subId;
}
console.log(`GmailWatcher: Pull subscription registered (${this.subscriptionPath})`);
// Start periodic pull
const pullMs = parseInterval(this.config.pubsub_pull_interval ?? '60s');
this.pullSubscriptionTimer = setInterval(() => {
this.pullSubscriptionMessages().catch((err) => {
console.error('GmailWatcher: Pull subscription error —', err instanceof Error ? err.message : err);
});
}, pullMs);
}
/**
* Pull messages from the Pub/Sub subscription and process them.
*/
private async pullSubscriptionMessages(): Promise<void> {
if (!this.subscriptionPath || !this.oauth2Client) {
return;
}
try {
const { PubSub } = await import('@google-cloud/pubsub');
const pubsub = new PubSub({
projectId: this.googleProjectId,
authClient: this.oauth2Client,
});
const subscription = pubsub.subscription(this.subscriptionPath);
const [messages] = await subscription.pull({
maxMessages: this.config.pubsub_max_messages ?? 10,
});
for (const message of messages) {
if (message.data) {
await this.handlePushNotification(message.data.toString('base64'));
}
// Acknowledge the message
await subscription.ack(message);
}
} catch (error) {
console.error(
'GmailWatcher: Failed to pull Pub/Sub messages —',
error instanceof Error ? error.message : error
);
}
}
```
Update `connect()` to call `setupPullSubscription()`:
```typescript
async connect(): Promise<void> {
this._status = 'connecting';
try {
this.oauth2Client = await this.authorize();
} catch (error) {
const errMsg = error instanceof Error ? error.message : 'Unknown error';
console.error(`GmailWatcher: Authorization failed — ${errMsg}`);
console.error('GmailWatcher: Run "flynn gmail-auth" to set up Gmail credentials.');
this._status = 'error';
return;
}
// Set up Gmail push watch (Pub/Sub) if not disabled
if (!this.config.disable_push) {
try {
await this.setupWatch();
} catch (error) {
const errMsg = error instanceof Error ? error.message : 'Unknown error';
const hint = this.buildWatchErrorHint(errMsg);
console.warn(`GmailWatcher: Watch setup failed (push disabled) — ${errMsg}${hint}`);
}
} else {
console.log('GmailWatcher: Push notifications disabled (disable_push: true)');
}
// Set up Pub/Sub pull subscription (hybrid mode)
await this.setupPullSubscription();
// Start History API polling fallback
const pollMs = parseInterval(this.config.poll_interval ?? '300s');
this.pollTimer = setInterval(() => {
this.pollForNewMessages().catch((err) => {
console.error('GmailWatcher: Poll error —', err instanceof Error ? err.message : err);
});
}, pollMs);
this._status = 'connected';
console.log(
`GmailWatcher: Connected (` +
`push=${!this.config.disable_push && !!this.config.pubsub_topic}, ` +
`pull=${!!this.config.pubsub_subscription_id}, ` +
`poll_interval=${this.config.poll_interval ?? '300s'})`
);
auditLogger?.systemStart('GmailWatcher', {
push: !this.config.disable_push && !!this.config.pubsub_topic,
pull: !!this.config.pubsub_subscription_id,
poll_interval: this.config.poll_interval,
});
}
```
Update `disconnect()` to clear pull timer:
```typescript
async disconnect(): Promise<void> {
if (this.pollTimer) {
clearInterval(this.pollTimer);
this.pollTimer = undefined;
}
if (this.watchTimer) {
clearTimeout(this.watchTimer);
this.watchTimer = undefined;
}
if (this.pullSubscriptionTimer) {
clearInterval(this.pullSubscriptionTimer);
this.pullSubscriptionTimer = undefined;
}
this.oauth2Client = undefined;
this._status = 'disconnected';
auditLogger?.systemStop('GmailWatcher');
}
```
#### 2.2 Improve Error Messages
**File**: `src/automation/gmail.ts`
Add helper method `buildWatchErrorHint()`:
```typescript
/**
* Build a helpful error hint based on the watch failure message.
*/
private buildWatchErrorHint(errMsg: string): string {
if (errMsg.includes('Invalid topicName')) {
const topicHint = this.googleProjectId
? `projects/${this.googleProjectId}/topics/gmail-push`
: '<project-id>/topics/gmail-push';
return `\n` +
` Tip: Set automation.gmail.pubsub_topic to "${topicHint}"\n` +
` Ensure the topic exists in GCP and the subscription is configured to POST to /gmail/push\n` +
` Or set automation.gmail.pubsub_subscription_id for pull-based mode (no webhook required)`;
}
if (errMsg.includes('Permission denied') || errMsg.includes('Forbidden')) {
return `\n` +
` Tip: Grant Gmail API permission to publish to the Pub/Sub topic:\n` +
` gcloud pubsub topics add-iam-policy-binding <topic> \\\n` +
` --member=serviceAccount:gmail-api-push@system.gserviceaccount.com \\\n` +
` --role=roles/pubsub.publisher`;
}
if (errMsg.includes('Not found') || errMsg.includes('404')) {
return `\n` +
` Tip: Create the Pub/Sub topic first:\n` +
` gcloud pubsub topics create gmail-push --project=<project-id>`;
}
return '';
}
```
Update `setupWatch()` error handling:
```typescript
try {
await this.setupWatch();
} catch (error) {
const errMsg = error instanceof Error ? error.message : 'Unknown error';
const hint = this.buildWatchErrorHint(errMsg);
console.warn(`GmailWatcher: Watch setup failed (push disabled) — ${errMsg}${hint}`);
}
```
---
### 3. Doctor Checks
**File**: `src/cli/doctor.ts`
Enhance `checkGmail` to validate Pub/Sub setup:
```typescript
const checkGmail: Check = async (ctx) => {
if (!ctx.config) {
return { status: 'skip', label: 'Gmail configured', detail: '(config invalid)' };
}
const gmail = ctx.config.automation.gmail;
if (!gmail?.enabled) {
return { status: 'skip', label: 'Gmail configured', detail: '(not enabled)' };
}
// 1. Check credentials file
const credentialsPath = expandPath(gmail.credentials_file ?? '~/.config/flynn/gmail-credentials.json');
if (!existsSync(credentialsPath)) {
return { status: 'fail', label: 'Gmail configured', detail: `credentials file not found: ${credentialsPath}` };
}
// 2. Check token file
const tokenPath = expandPath(gmail.token_file ?? '~/.config/flynn/gmail-token.json');
if (!existsSync(tokenPath)) {
return { status: 'warn', label: 'Gmail configured', detail: 'run `flynn gmail-auth` to authenticate' };
}
// 3. Parse credentials to get project_id
let projectId: string | undefined;
try {
const creds = JSON.parse(readFileSync(credentialsPath, 'utf-8'));
projectId = creds.installed?.project_id ?? creds.web?.project_id;
} catch {
// Ignore parse errors — already validated above
}
// 4. Check Pub/Sub configuration
const modes: string[] = [];
const warnings: string[] = [];
if (gmail.pubsub_topic && !gmail.disable_push) {
const topic = gmail.pubsub_topic.trim();
if (!topic.includes('/') && !projectId) {
warnings.push('pubsub_topic requires project_id in credentials');
} else {
modes.push('push');
// Warn if tailscale_only is enabled (Google cannot reach tailnet-only endpoints)
if (ctx.config.server?.tailscale_only && !ctx.config.server?.tailscale?.funnel) {
warnings.push('push requires public endpoint or Tailscale funnel');
}
}
}
if (gmail.pubsub_subscription_id) {
const sub = gmail.pubsub_subscription_id.trim();
if (!sub.includes('/') && !projectId) {
warnings.push('pubsub_subscription_id requires project_id in credentials');
} else {
modes.push('pull');
}
}
if (modes.length === 0) {
modes.push('poll');
} else {
modes.push('poll-fallback');
}
const modeStr = modes.join(' + ');
const detail = warnings.length > 0
? `${modeStr} (⚠️ ${warnings.join(', ')})`
: `${modeStr}${gmail.output.channel}/${gmail.output.peer}`;
return {
status: warnings.length > 0 ? 'warn' : 'pass',
label: 'Gmail configured',
detail,
};
};
```
---
### 4. Documentation Updates
#### 4.1 README.md
**File**: `README.md`
Update Gmail section (lines 461-512):
```markdown
### Gmail Watcher
Monitor a Gmail inbox for new messages and trigger agent responses. Flynn supports three retrieval modes:
1. **Push notifications** (Pub/Sub watch) — Google sends HTTP POST to `/gmail/push`
2. **Pull subscription** (Pub/Sub pull) — Flynn periodically pulls messages from a subscription
3. **History API polling** (fallback) — Flynn queries the History API every N seconds
**Recommended deployment**: Hybrid mode (push + pull + polling) for maximum reliability.
#### Prerequisites
1. **Google Cloud Project** with Gmail API and Pub/Sub API enabled
2. **OAuth2 credentials** (Desktop application) — download JSON from Cloud Console
3. **Pub/Sub topic** (for push or pull):
```bash
gcloud pubsub topics create gmail-push --project=your-project
```
4. **Grant Gmail API publish permission**:
```bash
gcloud pubsub topics add-iam-policy-binding projects/your-project/topics/gmail-push \
--member=serviceAccount:gmail-api-push@system.gserviceaccount.com \
--role=roles/pubsub.publisher
```
5. **Authenticate**:
```bash
flynn gmail-auth
```
#### Deployment Patterns
| Mode | Config | GCP Setup | Network | Use Case |
|------|--------|-----------|---------|----------|
| **Push only** | `pubsub_topic` | Topic + push subscription → `POST https://example.com/gmail/push` | Public IP or Tailscale funnel | Production with ingress |
| **Pull only** | `pubsub_subscription_id` | Topic + pull subscription | Any (no inbound) | Private deployment |
| **Hybrid** ⭐ | Both | Topic + push + pull subscriptions | Any | Maximum reliability |
| **Polling only** | Neither | None | Any | Development/testing |
#### Push Notifications Setup
When `pubsub_topic` is set, Flynn calls `gmail.users.watch()` to register for push notifications.
1. Create a **push subscription** in GCP:
```bash
gcloud pubsub subscriptions create gmail-push-sub \
--topic=gmail-push \
--push-endpoint=https://your-domain.com/gmail/push
```
2. If using **Tailscale Serve**, enable funnel:
```yaml
server:
tailscale:
serve: true
funnel: true # ← Required for Google to reach your endpoint
```
3. Set `pubsub_topic` in config:
```yaml
automation:
gmail:
enabled: true
pubsub_topic: projects/your-project/topics/gmail-push
```
**Important**: Push notifications require a publicly reachable endpoint. If `server.tailscale_only: true` and funnel is disabled, push will fail (Flynn falls back to pull/poll).
#### Pull Subscription Setup
When `pubsub_subscription_id` is set, Flynn periodically pulls messages from the subscription.
1. Create a **pull subscription** in GCP:
```bash
gcloud pubsub subscriptions create gmail-pull-sub \
--topic=gmail-push \
--ack-deadline=60
```
2. Set `pubsub_subscription_id` in config:
```yaml
automation:
gmail:
enabled: true
pubsub_subscription_id: projects/your-project/topics/gmail-pull-sub
pubsub_pull_interval: "60s"
```
**Note**: Pull mode works with any network setup (no inbound connections required).
#### Hybrid Mode (Recommended)
Combine push + pull for maximum reliability:
```yaml
automation:
gmail:
enabled: true
credentials_file: ~/.config/flynn/gmail-credentials.json
token_file: ~/.config/flynn/gmail-token.json
# Push notifications (when endpoint is reachable)
pubsub_topic: projects/your-project/topics/gmail-push
# Pull fallback (always works)
pubsub_subscription_id: projects/your-project/subscriptions/gmail-pull-sub
pubsub_pull_interval: "60s"
# History API tertiary fallback
poll_interval: "300s"
watch_labels: [INBOX]
message: "New email from {{from}}: {{subject}}\n\n{{snippet}}"
output:
channel: telegram
peer: "123456789"
```
#### Configuration Reference
| Field | Required | Default | Description |
|-------|----------|---------|-------------|
| `enabled` | no | `false` | Enable the Gmail watcher |
| `credentials_file` | yes | — | Path to OAuth2 credentials JSON |
| `token_file` | no | `~/.config/flynn/gmail-token.json` | Path to stored refresh token |
| `pubsub_topic` | no | — | Pub/Sub topic for push notifications (format: `projects/<project>/topics/<topic>`) |
| `pubsub_subscription_id` | no | — | Pub/Sub subscription for pull mode (format: `projects/<project>/subscriptions/<sub>`) |
| `pubsub_pull_interval` | no | `60s` | Pull interval when using subscription |
| `pubsub_max_messages` | no | `10` | Max messages per pull request (1-100) |
| `disable_push` | no | `false` | Disable watch API even if `pubsub_topic` is set |
| `watch_labels` | no | `[INBOX]` | Gmail labels to watch |
| `poll_interval` | no | `300s` | History API fallback interval |
| `history_start` | no | — | ISO date — only process emails after this date |
| `message` | no | `New email from {{from}}: {{subject}}\n\n{{snippet}}` | Template for agent message |
| `output.channel` | yes | — | Channel to route responses (e.g. `telegram`) |
| `output.peer` | yes | — | Peer/chat ID on output channel |
#### Template Variables
| Variable | Description |
|----------|-------------|
| `{{from}}` | Sender email address |
| `{{to}}` | Recipient email address |
| `{{subject}}` | Email subject line |
| `{{snippet}}` | Gmail-provided snippet (first ~200 chars) |
| `{{date}}` | Email date header |
| `{{id}}` | Gmail message ID |
| `{{labels}}` | Comma-separated label names |
#### Troubleshooting
**"Watch setup failed — Invalid topicName"**
- Ensure `pubsub_topic` format is correct: `projects/<project>/topics/<topic>`
- Verify the topic exists: `gcloud pubsub topics list`
- Check IAM permissions (Gmail API must be able to publish)
**"Permission denied" or "Forbidden"**
- Grant publish permission to Gmail API:
```bash
gcloud pubsub topics add-iam-policy-binding <topic> \
--member=serviceAccount:gmail-api-push@system.gserviceaccount.com \
--role=roles/pubsub.publisher
```
**Push notifications not arriving**
- Verify endpoint is reachable: `curl -X POST https://your-domain.com/gmail/push`
- Check subscription configuration: `gcloud pubsub subscriptions describe <sub>`
- Enable Tailscale funnel if using `tailscale_only: true`
- Fall back to pull or polling mode
**Check status**:
```bash
flynn doctor # Validates config and reports active modes (push/pull/poll)
```
```
#### 4.2 config/default.yaml
**File**: `config/default.yaml`
Update Gmail example (lines 128-136):
```yaml
# gmail:
# enabled: false
# credentials_file: ~/.config/flynn/gmail-credentials.json
# token_file: ~/.config/flynn/gmail-token.json
#
# # Push notifications (requires public endpoint or Tailscale funnel)
# pubsub_topic: projects/<your-project>/topics/gmail-push
# # pubsub_topic: gmail-push # Shorthand when project_id is in credentials
#
# # Pull subscription (works with any network setup)
# # pubsub_subscription_id: projects/<your-project>/subscriptions/gmail-pull-sub
# # pubsub_pull_interval: "60s"
# # pubsub_max_messages: 10
#
# # Disable push even if pubsub_topic is set (for debugging)
# # disable_push: false
#
# watch_labels: [INBOX]
# poll_interval: "60s" # History API fallback interval
# message: "New email from {{from}}: {{subject}}\n\n{{snippet}}"
# output:
# channel: telegram
# peer: "123456789"
```
---
### 5. Dependencies
**File**: `package.json`
Add `@google-cloud/pubsub` for pull subscription support:
```json
{
"dependencies": {
"@google-cloud/pubsub": "^4.0.0"
}
}
```
---
## Test Plan
### 5.1 Unit Tests
**File**: `src/automation/gmail.test.ts`
Add tests for new functionality:
```typescript
describe('GmailWatcher - Pub/Sub Pull', () => {
it('should set up pull subscription with full subscription path', async () => {
const config = {
...baseConfig,
pubsub_subscription_id: 'projects/test-project/subscriptions/gmail-pull',
pubsub_pull_interval: '30s',
};
const watcher = new GmailWatcher(config, channelLookup);
await watcher.connect();
// Verify subscriptionPath is set
expect(watcher['subscriptionPath']).toBe('projects/test-project/subscriptions/gmail-pull');
expect(watcher['pullSubscriptionTimer']).toBeDefined();
await watcher.disconnect();
expect(watcher['pullSubscriptionTimer']).toBeUndefined();
});
it('should auto-prefix subscription with project_id when shorthand is used', async () => {
const config = {
...baseConfig,
pubsub_subscription_id: 'gmail-pull',
};
const watcher = new GmailWatcher(config, channelLookup);
await watcher.connect();
expect(watcher['subscriptionPath']).toBe('projects/test-project/subscriptions/gmail-pull');
await watcher.disconnect();
});
it('should skip pull setup when pubsub_subscription_id is not set', async () => {
const watcher = new GmailWatcher(baseConfig, channelLookup);
await watcher.connect();
expect(watcher['subscriptionPath']).toBeUndefined();
expect(watcher['pullSubscriptionTimer']).toBeUndefined();
await watcher.disconnect();
});
it('should warn on invalid subscription path format', async () => {
const consoleSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
const config = {
...baseConfig,
pubsub_subscription_id: 'invalid-format',
credentials_file: '~/.config/flynn/gmail-credentials-no-project.json',
};
// Mock credentials without project_id
vi.mocked(readFileSync).mockImplementation((path: string) => {
if (path.includes('credentials')) {
return JSON.stringify({
installed: {
client_id: 'test-client-id',
client_secret: 'test-client-secret',
// No project_id
},
});
}
return JSON.stringify({ access_token: 'token' });
});
const watcher = new GmailWatcher(config, channelLookup);
await watcher.connect();
expect(consoleSpy).toHaveBeenCalledWith(
expect.stringContaining('pubsub_subscription_id \'invalid-format\' must be fully qualified')
);
await watcher.disconnect();
consoleSpy.mockRestore();
});
});
describe('GmailWatcher - disable_push', () => {
it('should skip watch setup when disable_push is true', async () => {
const config = {
...baseConfig,
pubsub_topic: 'projects/test-project/topics/gmail-push',
disable_push: true,
};
const watcher = new GmailWatcher(config, channelLookup);
await watcher.connect();
// Watch should not be called
const mockWatch = vi.mocked(google.gmail().users.watch);
expect(mockWatch).not.toHaveBeenCalled();
await watcher.disconnect();
});
it('should log push disabled message when disable_push is true', async () => {
const consoleSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
const config = {
...baseConfig,
pubsub_topic: 'gmail-push',
disable_push: true,
};
const watcher = new GmailWatcher(config, channelLookup);
await watcher.connect();
expect(consoleSpy).toHaveBeenCalledWith(
'GmailWatcher: Push notifications disabled (disable_push: true)'
);
await watcher.disconnect();
consoleSpy.mockRestore();
});
});
describe('GmailWatcher - buildWatchErrorHint', () => {
it('should return topic hint for Invalid topicName error', () => {
const watcher = new GmailWatcher(baseConfig, channelLookup);
const hint = watcher['buildWatchErrorHint']('Invalid topicName');
expect(hint).toContain('Set automation.gmail.pubsub_topic');
expect(hint).toContain('projects/test-project/topics/gmail-push');
});
it('should return IAM hint for Permission denied error', () => {
const watcher = new GmailWatcher(baseConfig, channelLookup);
const hint = watcher['buildWatchErrorHint']('Permission denied');
expect(hint).toContain('gcloud pubsub topics add-iam-policy-binding');
expect(hint).toContain('gmail-api-push@system.gserviceaccount.com');
});
it('should return creation hint for Not found error', () => {
const watcher = new GmailWatcher(baseConfig, channelLookup);
const hint = watcher['buildWatchErrorHint']('Not found');
expect(hint).toContain('gcloud pubsub topics create');
});
});
describe('GmailWatcher - connection logging', () => {
it('should log all active modes on connect', async () => {
const consoleSpy = vi.spyOn(console, 'log').mockImplementation(() => {});
const config = {
...baseConfig,
pubsub_topic: 'gmail-push',
pubsub_subscription_id: 'gmail-pull',
};
const watcher = new GmailWatcher(config, channelLookup);
await watcher.connect();
expect(consoleSpy).toHaveBeenCalledWith(
expect.stringContaining('push=true, pull=true, poll_interval=')
);
await watcher.disconnect();
consoleSpy.mockRestore();
});
});
```
### 5.2 Doctor Check Tests
**File**: `src/cli/doctor.test.ts`
Add tests for enhanced Gmail validation:
```typescript
describe('doctor - checkGmail enhanced', () => {
it('should detect push mode', async () => {
const config = {
...baseConfig,
automation: {
gmail: {
enabled: true,
credentials_file: '/mock/credentials.json',
token_file: '/mock/token.json',
pubsub_topic: 'projects/test/topics/gmail-push',
output: { channel: 'telegram', peer: '123' },
},
},
};
const result = await checkGmail({ config });
expect(result.status).toBe('pass');
expect(result.detail).toContain('push + poll-fallback');
});
it('should detect pull mode', async () => {
const config = {
...baseConfig,
automation: {
gmail: {
enabled: true,
credentials_file: '/mock/credentials.json',
token_file: '/mock/token.json',
pubsub_subscription_id: 'projects/test/subscriptions/gmail-pull',
output: { channel: 'telegram', peer: '123' },
},
},
};
const result = await checkGmail({ config });
expect(result.status).toBe('pass');
expect(result.detail).toContain('pull + poll-fallback');
});
it('should detect hybrid mode', async () => {
const config = {
...baseConfig,
automation: {
gmail: {
enabled: true,
credentials_file: '/mock/credentials.json',
token_file: '/mock/token.json',
pubsub_topic: 'gmail-push',
pubsub_subscription_id: 'gmail-pull',
output: { channel: 'telegram', peer: '123' },
},
},
};
const result = await checkGmail({ config });
expect(result.status).toBe('pass');
expect(result.detail).toContain('push + pull + poll-fallback');
});
it('should warn when push requires public endpoint but tailscale_only is enabled', async () => {
const config = {
...baseConfig,
server: {
tailscale_only: true,
},
automation: {
gmail: {
enabled: true,
credentials_file: '/mock/credentials.json',
token_file: '/mock/token.json',
pubsub_topic: 'gmail-push',
output: { channel: 'telegram', peer: '123' },
},
},
};
const result = await checkGmail({ config });
expect(result.status).toBe('warn');
expect(result.detail).toContain('push requires public endpoint or Tailscale funnel');
});
it('should pass when tailscale funnel is enabled', async () => {
const config = {
...baseConfig,
server: {
tailscale_only: true,
tailscale: { funnel: true },
},
automation: {
gmail: {
enabled: true,
credentials_file: '/mock/credentials.json',
token_file: '/mock/token.json',
pubsub_topic: 'gmail-push',
output: { channel: 'telegram', peer: '123' },
},
},
};
const result = await checkGmail({ config });
expect(result.status).toBe('pass');
expect(result.detail).not.toContain('⚠️');
});
});
```
### 5.3 Integration Tests
Manual testing checklist:
- [ ] **Push only**: Set `pubsub_topic`, start daemon, send test email, verify POST arrives at `/gmail/push`, verify agent response
- [ ] **Pull only**: Set `pubsub_subscription_id`, start daemon, send test email, verify pull retrieves message, verify agent response
- [ ] **Hybrid**: Set both, start daemon, send test email, verify either push or pull handles it (no duplicate processing)
- [ ] **Polling only**: Omit both, start daemon, send test email, verify History API poll finds it
- [ ] **Disable push**: Set `disable_push: true` with `pubsub_topic`, verify watch is not called
- [ ] **Doctor check**: Run `flynn doctor` with each mode, verify output shows correct modes
- [ ] **Error messages**: Trigger watch failure (invalid topic), verify hint contains actionable guidance
- [ ] **Tailscale warning**: Set `tailscale_only: true` without funnel, verify doctor warns about push
---
## Files to Modify
### Core Implementation
| File | Changes |
|------|---------|
| `src/config/schema.ts` | Add `pubsub_subscription_id`, `pubsub_pull_interval`, `pubsub_max_messages`, `disable_push` fields |
| `src/automation/gmail.ts` | Add `setupPullSubscription()`, `pullSubscriptionMessages()`, `buildWatchErrorHint()`, update `connect()` / `disconnect()` |
| `src/cli/doctor.ts` | Enhance `checkGmail` with Pub/Sub mode detection and Tailscale reachability check |
| `package.json` | Add `@google-cloud/pubsub` dependency |
### Documentation
| File | Changes |
|------|---------|
| `README.md` | Rewrite Gmail section with deployment patterns, setup instructions, troubleshooting |
| `config/default.yaml` | Update Gmail example with new fields and comments |
| `docs/plans/state.json` | Mark `gmail-push-revisit` as completed |
### Tests
| File | Changes |
|------|---------|
| `src/automation/gmail.test.ts` | Add tests for pull mode, `disable_push`, error hints, connection logging |
| `src/cli/doctor.test.ts` | Add tests for enhanced Gmail validation (mode detection, Tailscale warning) |
---
## Summary
This implementation plan makes Gmail notifications operationally reliable by:
1. **Supporting three deployment patterns**: Push (public endpoint), Pull (private), Hybrid (both)
2. **Clarifying config surface**: Explicit `pubsub_subscription_id`, `pubsub_pull_interval`, `disable_push` fields
3. **Improving error messages**: Actionable hints for common setup failures (invalid topic, missing IAM, etc.)
4. **Enhancing diagnostics**: `flynn doctor` reports active modes and warns about reachability issues
5. **Comprehensive documentation**: Setup guides for each pattern, GCP resource instructions, troubleshooting
**Recommended default**: Hybrid mode (push + pull + polling) for maximum reliability across all network configurations.
**Estimated effort**: 1-2 days (implementation + testing + docs)
**No breaking changes**: Existing `pubsub_topic`-only configs continue to work (push + polling)