I Built An AI Chat App with realtime voice chat in a Day with 0 Infrastructure (and So Can You!)

We just released Rizz.ai, a full-fledged AI companion app with realtime AI voice and text chat. But that's not the cool part of this blog post. The part is that we did it with 0 infrastructure.

No database. No servers. Nothing. You need a Stripe account, a Vercel account, and Gabber account. That's it.

To paint a picture of what this looked like before this blog post and this new no-infrastructure Rizz, we previously built the same app using an AI voice provider (Vapi) + OpenAI's GPT-4 + Deepgram + LiveKit. It took us 2 months, was a monster of infrastructure and coordination between various service providers, and it required a ton of middleware and user management. and it required a ton of middleware and user management.

This time was different though. It took 1 day. Everything from user management to realtime AI voice and text, to creating personas and scenarios for end users to interact with, was all handled by Gabber. I'll attach the full open-source repo at the bottom of the post, as well as a technical writeup for this blog post.

For those curious, Gabber is an end-to-end AI brain that creates human-like AI voices and text experiences. It can be integrated with with most apps and platforms. You can think of it as a voice and and text engine, an AI persona creator, a credits and user management system, and a memory and engagement system that works cross-platform.

The magic behind Gabber is it's user-centric, meaning not only is the tech easy to integrate for developers, but it also creates a really great user experience for users.

The Goal: Build an AI Chat App

The mission? In this case it's to create a functional browser-based AI-powered chat app using nothing but the Gabber's SDK, Stripe for payments, and OAuth for user login. I also wanted to add features like:

• Live Chat: Real-time conversations with AI personas. It should have texting and voice chat. This is table stakes.
• Scoring System: Analyze chat interactions for humor, creativity, and more. This is where the real value is and gives people a way to quantify their social skills.
• A Variety of Personas: Set unique styles for AI interactions. I didn't want 5 boring personas. There are 40, including aliens, a certain Hylian princess Zelda, and many more fun personas.
• Easy Deployment: Get it live and accessible in minutes.

Step 1: Setting Up the Project

The foundation of the project is Next.js with TypeScript and Tailwind CSS for styling. I chose Next.js because it's a great framework for building modern web apps, and it's easy to deploy to Vercel.

This project's package.json is pretty thin. Gabber will handle handle the heavy lifting, so you don't need to worry about installing a ton of ton of dependencies. Google, Stripe, and a few other libraries will handle the rest.

Step 2: Getting Gabber Setup

Gabber's SDK made it shockingly easy to integrate AI capabilities. Here's how I set it up:

1. Install the SDK:
   npm install gabber-client-react
2. Get an API Key:
   • Head to the Gabber Dashboard.
   • Create a project and generate an API key.
   • Add it to .env.local:
   GABBER_API_KEY=<your-api-key>

3. Configure the SDK: I wrapped my app with Gabber's SessionProvider to manage sessions and from there it mostly just works.

Step 3: Adding Payments with Stripe

Stripe integration ensures the app can monetize features without me having to worry about really anything. I used Stripe to let users purchase credit packages for chat interactions:

1. Create a Stripe account and set up products with a metadata field credit_amount. Gabber will use this to value to update the user's credit balance in the app and Gabber credit system.
2. Add API keys to .env.local:

Use Stripe's APIs to handle payments in the app.

Step 4: Setting Up Google OAuth

For user authentication, Google OAuth was a no-brainer. With basically 0 work you get a secure auth flow. Here's how I added it:

1. Create a Google Cloud Project:
   • Enable the OAuth consent screen and create an OAuth client ID.
   • Set up authorized origins and redirect URIs.
2. Add to .env.local:

Configure the OAuth flow in Next.js, and just like that, users could log in with Google!

Step 5: Creating Personas and Scenarios

Obviously I need AI personas to interact with. I created 40+ personas using Gabber's persona creator

This was the most fun part of the project. I used lexica.art to generate images for each persona, and got creative with the personalities and voices. I even cloned some voices so the characters felt more realistic.

I also created a few scenarios for each persona to interact with. This app was supposed to be a social training tool for people to practice social skills, so the scenarios were designed to be realistic and engaging.

Step 6: Implementing Live Chat

This was the fun part. Using Gabber's APIs, I created a dynamic chat interface:

• Chat History: Displayed messages with timestamps.
• Real-Time Updates: Managed live sessions with Gabber's SessionProvider.
• Input Bar: Allowed users to send messages and receive AI-generated replies.

With Gabber handling the heavy lifting, I could focus on making the chat interface look and feel intuitive without worrying about the underlying infrastructure or AI voice.

Step 7: Adding Scoring and Customization

What's an AI app without some (sassy) interactivity? I used Gabber's APIs to:

• Score Conversations: Analyzed chat attributes like creativity and confidence.
• Customize Personas: Defined unique communication styles and scenarios to match user needs.

This added a layer of fun and made the app more engaging.

Step 8: Viewing Past Transcripts

Viewing a score is cool, but what if users wanted to review the tape per se? Reading past sessions, whether to check rizz, or analyze and extract insights from a conversation, is useful, so I built an easy way to view past sessions.

I used the Gabber API to not only pull the transcript, but also get stats about the session like how much time the user vs the AI spent talking.

Step 9: Creating a Credits System

Obviously, you need a way to buy credits and charge your users. I used Gabber to create a credits system. Essentially, I created a Stripe product with a metadata field credit_amount. Gabber will use this to value to update the user's credit balance in the app and Gabber credit system.

I then made a function that would take the credit amount and create a Stripe checkout session. Once that was done, I would redirect the user to to the Stripe checkout page.

When users complete the purchase, I update the user's credit balance usign the Gabber API. Gabber is obviously the driver of usage here, but it's designed to be platform agnostic (i.e. you can use Gabber's credit system with any app).

So instead of Gabber directly deducting credits for the specific user, Gabber would call a webhook set up in the app to update credits realtime in the app so that users hit a usage wall when they run out of credits. Then I would update the user's credit balance in Gabber via that webhook.

Step 10: Deploying to Vercel

The final step was deployment. Thanks to Vercel, it took just a few minutes:

1. Configure next.config.js for production.
2. Add environment variables to Vercel.
3. Deploy with a single command:
   vercel --prod

And that's it! The app was live and ready for users.

The Result

In one day, I had a fully functional AI-powered chat app that could:

• Handle live AI interactions.
• Score conversations.
• Support multiple personas.
• Process payments seamlessly.

Whether you're building a toy, a coach, or a companion app, Gabber makes it simple. You can check out the Rizz.ai GitHub repo for the full code.