Two Weeks Before Launch: Infrastructure, Billing, and a Category Pivot

The last post on this blog went up April 5 — AutoResearch, the Karpathy Loop with governance. Since then, the calendar says fourteen days, but anyone who's done a startup launch will tell you the internal clock runs ten times that.

This one is written in a slightly different voice than usual. Part retrospective, part operational notes for future-me. If you're not running a similar infrastructure, the specific file paths and commands won't help you — skip to "What we learned" for the general stuff. Everything below is pulled from actual session notes, not reconstructed from memory.

Splitting the VPS in two

On April 13 we split the infrastructure into two separate Hetzner boxes. Before: one VPS running the gateway, dashboard, blog, and the autopilot container pool. After: curateme-platform (10.0.1.1, public 178.105.8.25) runs platform services + the Docker registry + one CI runner. curateme-runners (10.0.1.2, public 178.105.1.95) runs the autopilot container pool + three CI runners. Private 10.0.1.0/24 network between them.

Why we did it: autopilot containers started eating resources the platform needed. A runaway autopilot loop on shared infrastructure meant the dashboard got slow. After the third time I heard a design partner say "was there an outage?" we split them.

The pattern I didn't appreciate until we were doing it: the platform VPS also becomes the Docker registry. All runner images get pushed here, then pulled across the private network to the runners VPS. Both boxes need /etc/docker/daemon.json with "insecure-registries": ["10.0.1.1:5000"]. The registry binds to three addresses: localhost:5000, [::1]:5000, and 10.0.1.1:5000 — the private one is how the runners VPS reaches it.

Autopilot dispatch across the network is handled by setting DOCKER_HOST=tcp://10.0.1.2:2375 on the gateway via an AUTOPILOT_DOCKER_HOST env var. The runner VPS's Docker TCP listener is configured via a systemd override at /etc/systemd/system/docker.service.d/override.conf. Not obvious, but if you're ever wondering "where did this runner container come from," that's the path.

The GitHub Actions runner labels matter and are worth getting right from the start: cm-deploy for the platform runner (can reach local services + registry, used by deploy workflows), cm-ci for the three runners on the runners VPS (pure CI — lint, test, build, e2e). Any workflow that needs to reach a running service at deploy time targets runs-on: [self-hosted, cm-deploy]. Everything else goes to cm-ci for three-way parallelism.

Useful commands that took me too long to write down:

# Pre-pull runner images to the runners VPS from the platform registry
DOCKER_HOST=tcp://10.0.1.2:2375 docker pull 10.0.1.1:5000/curate-me/openclaw-base:latest

# Check which services are running on each VPS
ssh curateme@178.105.8.25 'docker ps --format "table {{.Names}}\t{{.Status}}"'
ssh curateme@178.105.1.95 'docker ps --format "table {{.Names}}\t{{.Status}}"'

# Register a new self-hosted runner
gh api -X POST /repos/Curate-Me-ai/platform/actions/runners/registration-token
# then on the VPS:
./deploy/vps/install-github-runner.sh <index> <registration-token>

The runner installer script is idempotent — running it twice with the same index is safe. It also auto-configures Docker daemon + TCP listener on the runners VPS, which previously I'd been doing by hand each time.

A gotcha that cost me an hour: Redis service containers in CI jobs bind host ports. If two jobs on the same self-hosted runner both try to use :16379:6379, the second job fails at "Initialize containers" with "port is already allocated." Fixed by assigning ports per job explicitly: gateway-test → 16379, backend-test → 16380. If you run self-hosted CI, allocate ports up front.

The billing surgery

We'd wired up Stripe checkout a few weeks ago and claimed it was "done." On April 14, Amanda and I sat down to write the billing test suite before launch, because the one thing worse than not having paid users is having paid users whose webhooks silently fail.

Forty-six tests later, in tests/b2b/unit/billing/test_b2b_billing_flow.py, we'd caught a real bug. Nine test classes covering the webhook handler, checkout session creation, portal session, the Stripe billing service, each event handler, price resolution, usage sync, subscription status, and plan tiers.

The bug was better than a silent failure — it was a wrong-kind-of-loud failure. The /api/v1/webhooks/stripe-billing endpoint returned 401 Authorization header required for legitimate Stripe events. Root cause: TenantIsolationMiddleware had /api/v1/webhook/ (singular) in its public-prefixes list, but the route is /api/v1/webhooks/ (plural). One letter. The middleware was correctly requiring auth on what it thought was a customer endpoint — and Stripe doesn't send JWTs.

Fix: add /api/v1/webhooks/ to PUBLIC_PREFIXES in src/middleware/tenant_isolation.py. Verification: POST to the webhook URL now returns 400 Missing Stripe-Signature header (correct — it's public, signature verification happens inside the handler). Stripe sends events to https://api-admin.curate-me.ai/api/v1/webhooks/stripe-billing with a valid Stripe-Signature header and they now reach the handler.

If you've shipped Stripe integrations before this will sound familiar. If you haven't: run your full webhook path through middleware manually. The middleware is where these bugs hide, not in the handler itself.

We also verified the whole flow end-to-end. Navigated to /settings/billing as a test partner org, clicked "Select Plan" on Starter ($49/mo), and watched Stripe Checkout open with four line items correctly populated: the $49 base subscription, gateway overage at $0.10 per 1K requests, token overage at $1.00 per 1M tokens, and runner-hours overage at $0.25 per hour. Did not complete payment (live mode, test only). This whole path used to involve five separate manual tests with four coffee breaks. Now it's in scripts/gateway-smoke-test.sh --production and runs in ~40 seconds.

Finding the scale ceiling

On April 14 (same day, different session) we pointed a k6 load test at the gateway and pushed until something broke. File: tests/load/gateway-load-test.js. Four scenarios mixed to reflect real traffic: health checks at 30% weight, governance chain at 40%, usage endpoint at 15%, PII scanning at 15%. Thresholds: P50 under 200ms, P95 under 500ms, P99 under 1s, success rate over 95%.

Results from a 15-second smoke against production: governance latency P50 at 148ms, P95 at 346ms. Both inside thresholds. Zero 429s in the smoke window. Good signal but not representative of a sustained launch — we ran a 10-minute test next, and that's what broke.

What broke: MongoDB index contention on the gateway_traces collection. The _id index was the only index on the whole collection. The dashboard's real-time trace query scans by org and orders by time. At a few thousand records, fine; at a few million, painful.

The fix was four indexes, all added live in production via mongosh:

db.gateway_traces.createIndex({ trace_id: 1 });
db.gateway_traces.createIndex({ org_id: 1, created_at: -1 }, { name: "org_timestamp" });
db.gateway_traces.createIndex({ request_id: 1 });
db.gateway_traces.createIndex({ created_at: 1 }, { expireAfterSeconds: 7776000 }); // 90-day TTL

Latency on the trace aggregation endpoint dropped from ~1.8s to 19ms after the compound index landed. Worth flagging: the gateway_usage collection already had proper indexes and a 90-day TTL — whoever set that one up six months ago deserves a quiet thank-you.

At the same time we added daily MongoDB backups. Script at deploy/vps/mongo-backup.sh — mongodump + tar.gz, 7-day retention. Cron 0 2 * * *. First test run produced an 11MB compressed backup. Manual restore tested once; documented in the runbook.

Also did a Redis audit. Turns out we had three separate Redis connection pools across different modules, and one of them was leaking connections on long-running SSE streams. Fixed by moving everyone onto the unified cache layer in src/utils/cache.py. Connection count dropped from "worrying" to "flat line." Redis itself had more headroom than I expected: 384MB maxmemory with volatile-lru policy, current usage 2.25MB of 384MB, 186 keys total. We are not Redis-bound.

The meta-lesson is not subtle: load test before launch, not after. We could have shipped without this work and discovered the MongoDB index issue in production with our first twenty real customers. We got to discover it in a controlled environment with synthetic traffic and fix it before anyone noticed.

Microsoft Teams came up the way Slack used to

On April 9 we shipped the full Microsoft Teams integration — Bot Framework, Adaptive Cards for HITL approvals, Entra ID SSO, tab integration in the Teams client. This was a design-partner-driven decision. Two of our late-March design partners use Teams as their primary messaging layer; one uses Slack for engineering and Teams for everything else. We'd built Slack first because the engineering teams we initially talked to used Slack.

The honest take: we should have gone Teams-first for enterprise design partners from the start. Slack-first is correct for indie devs and small startups, wrong for hospitals, finance, and anyone with an M365 subscription.

On April 14 (a week later) we did the more important work: we unified the dispatcher. Instead of two separate integrations with duplicated alerting logic, we built src/gateway/unified_alerts.py — a single entry point that takes a logical event (HITL approval request, cost threshold hit, runner status change) and fans out concurrently to whichever channels the org has configured, via asyncio.gather().

Seven alert points in governance.py moved from slack_alerts.fire_* to unified_alerts.fire_*. The channel enum in alerting.py gained a TEAMS value. Two new Celery tasks mirrored existing Slack ones: src/tasks/teams_scheduled_reports.py (hourly daily-digest cards) and src/tasks/teams_status_bar.py (60-second runner-status cards). Both registered in celery_app.py via include and beat_schedule.

Adding a third channel — we'll probably do Discord next — now means adding one adapter, not rebuilding the integration.

The short version: Bot Framework + Adaptive Cards is the Teams equivalent of Block Kit, and if you've built Slack HITL flows with buttons, the concepts translate almost directly. The painful part is SSO — Entra ID is more finicky than we expected and the per-tenant onboarding story is still rough. A dedicated Teams integration post is on the list for the next retrospective.

The signup flow, end to end

Also on April 14 (long day), we tested the signup flow end-to-end for the first time as a real user, not as a dev with cached state.

Created a test org "Design Partner Test Co" (org_08ed91bc311e45e6979307e3). Generated an API key, free tier (cm_sk_534530...). Verified in MongoDB: user record, org record, API key record, default governance policy, default alert rules — all five created correctly in the right collections, all org-scoped.

Confirmed governance defaults on the free tier: 10 RPM, $10/day budget, PII scan enabled, block action on policy violations. Made a request against a premium model (stepfun/step-3.5-flash) and confirmed it was correctly rejected with a clear error message citing the plan limit. Made a request with a placeholder provider key and confirmed it was rejected with remediation instructions — not a cryptic 500, a specific "add your OpenAI key here" message.

Multi-org isolation confirmed by comparing: the new org saw exactly one usage record (the request we just made), the main org saw 756 (unchanged). Zero leakage. Fifteen orgs total in production.

Two friction fixes surfaced that we shipped in the same session:

1. Duplicate "provider keys" links on Step 6 of the onboarding wizard. The wizard was rendering two buttons pointing at the same place. Fixed by replacing the second link with "Read the documentation" pointing at docs.curate-me.ai.

2. Login redirect after the welcome wizard. We had a bug where completing onboarding successfully would drop you at a "please log in" screen, because the NextAuth session wasn't being established before the redirect to /welcome. Fixed by calling await login(email, password) right after successful onboarding so the session is established before the navigation. "Go to Dashboard" now works without a re-login.

Both fixes would have been the first thing a design partner hit. Both would have been the thing they told us about instead of using the product. This is why you test the signup flow from cold state before launch.

Test infrastructure debt

On April 14 we also cleared the gateway test suite from 19 failures down to 0. The gateway has about 8,870 tests now. Getting to 100% green wasn't about fixing bugs — the tests themselves had drifted from the code.

A sample of the fixes, because this list is useful for future-me:

• test_hitl_timeout (9 tests) — the tests mocked _db directly, but check_timeouts() now goes through _get_collection(). Switched the mock to the right code path. All 9 passed.
• test_provider_router (1) — a regression claimed anthropic/* should route to OpenRouter. It correctly routes to the direct Anthropic provider. Fixed the assertion.
• test_prompt_caching (1) — the fixture mock_resilient_post was missing a circuit_breaker param the function signature now requires. One-line fix.
• test_runner_attestation (1) — same org_secret should produce the same HMAC regardless of org_id. The test was asserting the opposite. Fixed the assertion.
• test_runner_health_routes (2) — mock was missing gateway_state_size_bytes and gateway_pending_work_count fields the route now returns. Added both.
• test_slack_interactive_coverage (4) — the test used module-level imports that broke under pytest-xdist parallelism. Wrapped with patch.dict(sys.modules) for xdist compatibility.
• test_e2e_smoke (1) — imported main_b2b which fails in xdist. Added graceful skip.
• test_coordinator_synthesizer — imported the now-deleted TaskPhase class. Removed the import.

The pattern across almost all of these: the tests were written right, the code was written right, they drifted apart. Keep old test names when you refactor mocks. Changing mocks to match the new code path is fine; renaming a test because "the old name doesn't fit" is how you lose the thread of what was originally being tested.

Separately: the Playwright E2E config had an ESM issue. import.meta.url doesn't resolve in the CommonJS context the tests run under locally. Replaced with path.resolve(__dirname) — works in both contexts, no config changes downstream.

The celery worker on VPS 2 had been reporting unhealthy for days. Docker Compose was resolving $HOSTNAME at compose-time (empty string), so the healthcheck was looking for a worker with no name. Simplified to celery -A src.tasks.celery_app inspect ping without targeting a specific host. Worker now reports healthy.

Gateway test count after this session: 8,870 passed, 0 failed, 12 errors (pre-existing cloud runner routes that depend on missing env vars).

The harness pivot

The biggest non-code change of the last two weeks is a category pivot. Product identity stays "Agent Governance Platform." Marketing tagline becomes "the harness for production AI agents." SEO targets the word "agent harness" via content; the product owns "agent governance." Details in Framework vs Harness vs Gateway — the companion post to this one.

The reason this retrospective mentions it: category pivots are as big a ship as code. Updating the launch playbook, HN post, content calendar, dashboard copy, and marketing pages took about the same time as a medium-sized feature. But the payoff is that every future post, every launch asset, every conversation with a design partner now has one consistent frame.

Files touched during the pivot: 14 of them. Marketing hero copy, dashboard metadata, docs headers, the "Governance Score" feature renamed to "Harness Score" (with backward-compat aliases so existing API calls don't break), a new endpoint at /gateway/admin/harness-score, a new SVG badge, a new GitHub Action output string. All shipped in four commits.

For future-me looking back at this post: the positioning decision is in docs/strategy/AGENT_HARNESS_PIVOT.md in the platform monorepo. The decision was hybrid (product name kept, marketing hook uses "harness"). If you're reading this and wondering why we didn't go further, the doc explains. Every option was evaluated.

Dogfooding: our own agents use our own gateway

The thing I'm most proud of from the last two weeks isn't on this list yet. On April 13 we finished wiring our internal autopilot system — the containerized workers that write code for us — to route their LLM calls through the Curate-Me gateway.

Setup: each autopilot container gets three env vars at startup — OPENAI_API_KEY, CM_GATEWAY_API_KEY, and CM_API_KEY — all set to a gateway API key stored in MongoDB api_keys (B2B database) with a bcrypt hash and a key_prefix for lookup. OPENAI_BASE_URL points at the gateway. Default model is routed through OpenRouter upstream. The runner-agent executor rebuilds auth-profiles.json using CM_GATEWAY_API_KEY — not OPENAI_API_KEY directly — so the containers see the same governance envelope customers do.

Payoff: when we find a gateway bug, we find it in our own dogfood before customers do. The April 14 webhook fix was partially surfaced by autopilot runs failing in ways that pointed at the middleware stack. We would have found it later, during a customer onboarding, if we weren't consuming our own stack.

Secondary benefit: the cost tracking on the dashboard shows autopilot spend alongside design-partner spend. I can see at a glance how much our internal R&D is costing, which is a useful number to have when someone asks "how much do your agents cost?"

Gotchas worth remembering

The specific stuff that took me an afternoon each to figure out:

• Route ordering in the gateway admin routes matters. gateway_runner_admin.py has a /{runner_id} catch-all. If you add a new router with static paths under /gateway/admin/runners/ — say, /reviews — it has to be registered BEFORE the admin router in gateway/app.py, or the catch-all eats it. Symptom: 404 on a route you swear you just added. Fix: registration order.

• Gateway JWTs require type: access in claims. Without it, _decode_dashboard_jwt() silently returns None and the request looks unauthenticated. aud claim is optional — the gateway catches both InvalidAudienceError and MissingRequiredClaimError. If your test JWT isn't working, check type.

• OpenClaw gateway resource optimization. Without skip flags, openclaw gateway spawns 19+ worker processes, each 200-350MB. Inject OPENCLAW_SKIP_CRON=1, OPENCLAW_SKIP_BROWSER_CONTROL_SERVER=1, OPENCLAW_SKIP_CANVAS_HOST=1, OPENCLAW_SKIP_GMAIL_WATCHER=1, OPENCLAW_SKIP_PROVIDERS=1, OPENCLAW_NO_RESPAWN=1, OPENCLAW_DISABLE_BONJOUR=1, OPENCLAW_HIDE_BANNER=1, OPENCLAW_LOG_LEVEL=warn plus --max-old-space-size=768 for the Node heap. Result: ~450-600MB per container at steady state instead of 5.7GB. Do NOT set OPENCLAW_SKIP_CHANNELS — channels are needed for Slack integration.

• The gateway_traces collection was missing indexes from its initial migration. If you're copying this schema elsewhere, make sure trace_id, (org_id, created_at desc), request_id, and a TTL on created_at are all there from day one, not added later under load pressure.

• Stripe webhook URL plural vs singular. The route is /api/v1/webhooks/stripe-billing (plural). The middleware public-prefix list had /api/v1/webhook/ (singular). This pattern of off-by-one-letter middleware bypass is worth a dedicated grep pass before any launch: grep -r "PUBLIC_PREFIX\|public_prefix" --include="*.py" and verify every entry matches an actual route prefix.

What we learned

The stuff I want in my head for the next two weeks, and any future session that reads this post from memory:

1. When you split infrastructure, the registry lives on the side that doesn't scale up and down. Platform VPS is fixed. Runners VPS scales. Registry goes on platform.

2. Webhook handlers fail under concurrent load before they fail anywhere else. Add a Redis-backed idempotency lock keyed on event ID with a 5-minute TTL as the default pattern, before you ship any webhook handler.

3. MongoDB index on (org_id, <time_field> desc) solves 80% of multi-tenant dashboard performance. If you're scanning by org and ordering by time, that compound index is the first thing to try.

4. Three Redis connection pools is two too many. Unify on a single cache layer. It will leak connections exactly when you can't afford it to.

5. Teams-first is correct for enterprise design partners. Don't build Slack and retrofit Teams — build a NotificationDispatcher abstraction from the start, then add channels as adapters.

6. k6 loaded against the gateway with realistic payload sizes is the single highest-ROI pre-launch test. We found two production-blocking issues in an afternoon.

7. Category positioning is a ship. Not one you can skip. Budget 1-2 days for updating all the assets (playbook, HN post, hero copy, content calendar, dashboard copy, blog posts).

8. Design partners drive the roadmap whether you like it or not. Teams came from design partners. The Hospital CFO fleet template came from a design partner. The templates page exists because five design partners independently asked for the same pattern.

9. The Stripe smoke test takes 40 seconds to run. It used to take 15 minutes of manual clicking. Everything that used to be "let me just verify manually" is now a script. If something breaks on launch day it will be something that has no script.

10. Memory files are infrastructure. The stuff we wrote down during each of the April 14 sessions — and there were six of them — is why this post exists. If I'd tried to reconstruct these two weeks from commit messages alone, I'd miss half of it.

11. Port collisions on self-hosted CI runners need explicit allocation. gateway-test → 16379, backend-test → 16380. Write down the port ↔ job mapping somewhere findable.

12. Route ordering in FastAPI matters when you have catch-all path params. Register static-path routers BEFORE routers with /{something} catches-alls, or the catch-all eats the new route.

13. Plural vs singular in middleware public-prefix lists is the bug you will ship. Every new public route should be spot-checked against the middleware pattern before it leaves review.

14. Dogfooding the gateway from your own agents catches bugs customers would find first. The autopilot containers route through the gateway, so when the gateway breaks, our own CI breaks before a customer hits the bug.

15. Keep test names when you refactor mocks. Changing a mock to match a new code path is normal work. Renaming the test because "the old name doesn't fit" is how you lose the thread of what was originally being tested.

What's next

Launch day is May 12 — 23 days out. Between now and then:

• Land five design partners on production traffic (currently at three).
• Ship two design-partner case studies as blog posts (one drafted, one waiting on permission).
• Pre-brief fifteen launch-day mutuals for engagement.
• Run the gateway end-to-end smoke test from an IP we haven't tested from.
• Stand up status.curate-me.ai with real uptime history.
• Launch assets: OG images, 60-second demo GIF, Product Hunt gallery.

I'll post one more retrospective the week before launch, and then the launch-day recap a few days after. If something ships between now and then that's interesting on its own, it'll get its own post. If the next two weeks feel like the last two, we'll have six more posts to write.

Your harness makes the agent work. The infrastructure, the tests, the notifications, the dogfooding, and the category pivot are what make "we make it safe" true in production.

See you at launch.