Case study · production agentic, owned end to end

Flowstate AI

A production, multi-tenant Customer Success Intelligence platform I built and shipped for a high-value agency client. It unifies Asana and Klaviyo into one command centre and answers natural-language questions over live operational data. Around 10,000 customer interactions a month.

591
commits
~33k
lines of code
~10k
interactions / mo
100%
agent eval
The problem

Marketing agencies run client work across Asana (projects and tasks) and Klaviyo (email and SMS), with no unified view of client health. Churn arrives as a surprise. The job was to turn scattered operational data into one place a team can ask questions of, and into an early warning system for accounts going quiet.

Architecture
Flowstate, end to end
surface
Next.js 15 + React 19
dashboards, WebSocket streaming
agents
Asana intelligence agent
Klaviyo marketing agent
Agent manager
orchestration + per-agent knowledge base
AI + retrieval
Provider router
OpenAI → Anthropic → OpenRouter auto-fallback
Pinecone
vector RAG
data + integrations
Supabase / Postgres
row-level security, 4 roles
Asana / Klaviyo / Slack / Google
one-way read-only sync
Redis
cache

Multi-tenant isolation is enforced at the database with Supabase row-level security and four roles (super-admin, org-admin, CSM, viewer), not just in application code. Each AI agent has a dedicated knowledge base; a provider router gives every call automatic failover across OpenAI, Anthropic and OpenRouter. Integrations sync one-way and read-only by design, and a versioned API (with dual session-or-cron auth) sits in front. Heavy syncs run in a dedicated worker.

Decisions I made, and why
Multi-provider failover over a single model
Resilience and cost control mattered more than betting on one vendor. The router degrades gracefully and picks the right model per task.
Row-level security for multi-tenancy
Tenant isolation belongs in the database, not just the app layer. Per-request clients, never a shared singleton, keep RLS correct under concurrency.
One-way, read-only integration sync
The platform reads the client's Asana and Klaviyo, it never writes back. That removed a whole class of risk from a system touching live client tooling.
RAG over fine-tuning
Grounding answers in the client's own current data with retrieval beat training a model that would go stale the moment their projects changed.
Evaluation: how I made the agents trustworthy

The first version of the Asana intelligence agent was only about 65% correct on real business questions, which is useless when a team is making calls off the answers. So I built a ground-truth evaluation harness: a set of real questions with SQL-verified expected answers, run on every change.

Then I iterated the system prompt and the tools against it until the agent passed 17 of 17, a 100% pass rate on the ground-truth set. That harness is the thing that turned a demo into something a paying client could rely on, and it is the discipline I would bring to correctness, cost and safety evaluation on any agent in production.

What broke, and what I'd do differently
brokeDashboards silently rendered $0 revenue.
fixedSupabase's automatic joins didn't cover the query shapes, so metrics came back empty instead of erroring. I replaced them with explicit manual joins and locked each API response to the frontend's contract. A silent $0 is worse than a 500, so next time I'd put contract tests between the API and the UI from day one.
brokeThe agent was only ~65% correct at first.
fixedI stood up the ground-truth eval harness and drove the Asana agent to 100% (17 of 17). The lesson I banked: build the eval harness with real ground truth before you ship, not after you notice wrong answers.
brokeA charting library threw on undefined values and misaligned on real data.
fixedI ripped out the fancy chart component and used plain, correct tables. For a client dashboard, boring and right beats clever and broken. It's the same instinct behind the inline-SVG charts in my design system.
brokeComplex dashboard queries 500'd under real multi-tenant joins.
fixedManual array joins and tighter response shapes. The takeaway was to model the exact query the UI needs, not to trust an ORM's convenience joins on a row-level-secured schema.
Outcome

A production platform a paying client relies on, handling around 10,000 customer interactions a month, with per-org isolation, dual AI agents, and a unified client-health view that turns a churn surprise into an early signal. Roughly 33,000 lines, 591 commits, shipped end to end.