React Native AI Captions Editor with Convex, ElevenLabs, Clerk, Sentry and Expo

This is a React Native AI Captions Editor using Convex for real-time data synchronization, ElevenLabs for text-to-speech, Clerk for user authentication and Sentry for error tracking.

Additional features:

• Expo Router file-based navigation
• Convex Database for data storage
• Convex File Storage for file storage
• Sentry for error tracking
• Clerk Passkeys for passwordless authentication
• Haptics for haptic feedback
• Jotai for state management
• NativeWind for styling
• Expo Secure Store for secure storage
• Expo Image Picker for image picking
• Expo Video for video playback
• Expo Audio for audio playback

🎥 Follow the video tutorial

Watch and build this AI Captions Editor step by step:

TODO: Add video

Setup

Environment Setup

Make sure you have the Expo CLI installed.

For the best development experience, you should have Android Studio and Xcode (Mac only) installed. For more information on setting up your development environment, refer to the Expo documentation for Android Studio and the React Native documentation for Xcode.

App Setup

To build the app, follow these steps:

1. Clone the repository
2. Run npm install
3. Run npx expo prebuild
4. Run npx expo run:ios or npx expo run:android

Convex Setup

1. Create an account on Convex
2. Run bunx convex dev to start the development server

Microservice Setup

To export the video with burned in captions we need an environment with ffmpeg installed (you can do this locally).

1. Go into the /microservice folder
2. Run npm install to install the dependencies
3. Run npm run dev to start the microservice

You then have to add the URL to the Convex environment variable MICROSERVICE_URL. I recommend using something like ngrok to test the microservice locally.

npx convex env set MICROSERVICE_URL <your-microservice-url>/transcode

Clerk Setup

Make sure to add your EXPO_PUBLIC_CLERK_PUBLISHABLE_KEY to the .env file.

Authentication Setup

Create a Clerk account and project, then update the convex/auth.config.js file with your domain and applicationID.

export default {
  providers: [
    {
      domain: 'https://your-clerk-domain.clerk.accounts.dev',
      applicationID: 'your-clerk-application-id',
    },
  ],
};

You also need to connect Convex and Clerk with a JWT template. For this, cehck out the video and Convex docs.

Webhook Setup

You need to set up a webhook in Clerk to handle the user creation and update events.

1. Go to Clerk and select your project
2. Go to API and select Webhooks
3. Add the following webhook, which should point to your Convex instance and include the user.created and user.deleted events:

ElevenLabs Setup

1. Create an account on ElevenLabs
2. Get an API key from ElevenLabs
3. Add the key to Convex by running bunx convex env set ELEVENLABS_API_KEY=<your-api-key>

Sentry Setup

1. Create a new project on Sentry
2. Use the npx @sentry/wizard@latest -s -i reactNative command to setup Sentry for your project

App Screenshots

Demo

Sentry Screenshots

Convex Screenshots

🚀 More

Take a shortcut from web developer to mobile development fluency with guided learning Enjoyed this project? Learn to use React Native to build production-ready, native mobile apps for both iOS and Android based on your existing web development skills.