AI phone system
GENIUS
How we designed and built GENIUS — an AI phone system that turns the call itself into finished work, across web, native iOS and desktop.
- Year
- 2026
- What we did
- Web · mobile · desktop, telephony, POS, payroll & AI
The problem
Small businesses lose more revenue at the phone than almost anywhere else — and the tools to fix it were built for everyone but them.
The best person can’t answer every call. When they do, half of what’s said lands on a sticky note — address right, callback number wrong — and the job that was almost booked quietly slips away.
Meanwhile, national chains run round-the-clock phone teams with scripts, systems, and people whose only job is the next call. A local shop competes against that with a flip phone and a good memory.
GENIUS came to us with the problem and the conviction. Our job was to build something that closed the gap without asking a small business to change how it works, buy new hardware, or give up its number.
What makes it different
The build decisions that made it work.
GENIUS only delivers if the technology disappears into a normal phone call. These are the choices we made to get there.
Native iOS, written in Swift
Owners and staff live on their phones, so the iOS app had to feel like it belonged there. We built it natively in Swift rather than wrapping a web view — real performance, real platform behavior, a call experience that holds up in the field.
A real-time media gateway
Live assistance during a call is only possible if audio moves in real time. We built a media gateway that bridges the carrier to streaming speech-to-text, so the transcript and the next-question prompt arrive while the customer is still talking — not after.
Capture the call as data, not notes
Instead of trusting a human to remember and transcribe, the system treats the conversation as structured input. By the time the call ends, the job, work order or case is already drafted from what was actually said.
One shared core across every surface
Web, iOS and desktop run on the same underlying model and customer memory. A change made on the phone is the same change everywhere — no syncing seams, no app that knows less than the others.
Sits on the line you already have
We built GENIUS to layer onto the business’s existing phone service over Twilio and Plivo. Keep the number, keep the workflow, switch on in about five minutes — adoption was a design constraint, not an afterthought.
Change history on every record
Every field carries its own history — who changed it and when, including edits the assistant makes mid-call. The result is a system staff can trust precisely because nothing happens invisibly.
The goal wasn’t a smarter phone. It was a finished job by the time you hang up.
Under the hood
A small-business operating system, specified.
A Next.js 16 / React 19 monorepo that spans a phone network, a point of sale and a payroll engine — with one core feeding web, mobile and desktop.
Core platform
- Framework
- Next.js 16 (App Router) · React 19 · TypeScript in a Turborepo, with shared packages for AI, telephony, payments and customer context.
- Surfaces
- Web, a native iOS app (Swift) and a desktop app (Electron) — built on one shared core so a field added once appears everywhere.
- Data
- Postgres (Neon) with Prisma; Upstash Redis for fast state; Inngest for durable background jobs like call summaries and recovery.
- Identity
- Clerk for auth and organizations, scoping every business to its own workspace.
The phone network
- Voice
- Programmable voice over Twilio and Plivo, with a media-gateway bridging live carrier audio to the AI in real time.
- Transcripts
- Streaming speech-to-text produces live, searchable transcripts; realtime signals (Ably) push call state to every screen as it happens.
- AI attendant
- A voice-core AI (the InstantExpert / attendant) detects workflows, answers overflow and after-hours calls, and drafts the record from the conversation.
- Desk phones
- SIP desk-phone provisioning (Poly Lens) for shops that want physical handsets alongside the software.
The rest of the counter
- Point of sale
- A bright, fast POS terminal and admin — clock-in, menu, cart, live orders, reporting, employees and a shared memory layer.
- Payroll & books
- Embedded payroll and bookkeeping, white-labeled into the suite so the business never leaves GENIUS to run the back office.
- Knowledge
- A Workipedia knowledge layer the AI draws on to answer questions in the business’s own voice.
Operations
- Records & audit
- Field-level change history on customer records — every edit shows who changed it and when, including AI corrections made mid-call.
- Observability
- Sentry for errors and an ops pipeline that turns incidents into tracked, fixable work.
- Delivery
- Deployed across modern edge/container infrastructure (Railway, Vercel, Cloudflare) for the realtime telephony path.
What it took to build
From the App Store to the desk phone.
GENIUS runs on more than the web. From one core we shipped native iPhone and iPad apps and a desktop app — and wrote the provisioning software that configures physical Poly desk phones over the air, so a handset boots straight into GENIUS. Native mobile, desktop and specialty device provisioning are all things we build in-house.
- iPhone app
- iPad app
- Desktop app
- Desk-phone provisioning · Poly Lens
The build, on screen
Selected screens.
Technology
What it’s built on.
- TypeScript
- Swift
TypeScript
Swift- Next.js 16
- React 19
- Turborepo
- Electron
Next.js 16
React 19
Turborepo
Electron- Neon (Postgres)
- Prisma
Neon (Postgres)
Prisma- Clerk
Clerk- Anthropic
- OpenAI
Anthropic
OpenAI
Google- Inngest
- Ably
- Upstash Redis
Inngest
Ably
Upstash Redis- Stripe
Stripe- Vercel Blob
Vercel Blob- Twilio
- Plivo
- Media gateway (npm)
- Streaming STT
- Poly Lens
- Native iOS (Swift)
Twilio
Plivo
Poly Lens- Railway
- Vercel
- Cloudflare
- Sentry
Railway
Cloudflare
SentryOur approach
How we approached it.
We treated GENIUS as one product with many surfaces, and built outward from the moment that matters most — the live call.
- 01
Start at the phone call.
We anchored the whole system on the call because that’s where the revenue is won or lost. Everything else — the suite, the records, the apps — was built to serve what happens in those few minutes on the line.
- 02
Make the assistant a teammate, not a takeover.
Staff answer normally. The assistant listens, recognizes the workflow, and surfaces the single next question that matters. We kept the human in the lead and used the technology to remove the dropped-detail failure mode.
- 03
Build the plumbing for real time.
Telephony over Twilio and Plivo, a media gateway in the middle, and streaming speech-to-text on top. This is the unglamorous infrastructure that makes ‘live’ actually live — and it’s where most of the engineering went.
- 04
Extend past the phone to the whole counter.
Once the call was solved, we built outward: point of sale, payroll, books, and a shared customer memory — plus over-the-air provisioning for physical desk phones — so the phone connects to the rest of the business instead of standing apart from it.
Next case study
We designed and built SmartComply — the system of record that runs a town’s entire stormwater permit program from day one.