Skip to content
/ Healthcare-AI-Voice-agent Public

A comprehensive full-stack healthcare application that leverages AI-powered voice interactions for patient triage, symptom analysis, specialist mapping, and appointment booking. It is built with modern technologies including FastAPI, PostgreSQL, React, and integrated with leading AI models like Gemini and GPT-4.

License

MIT license
40 stars 13 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

danieladdisonorg/Healthcare-AI-Voice-agent

BranchesTags

Repository files navigation

🩺 AI Healthcare Voice Assistant

A comprehensive full-stack healthcare application that leverages AI-powered voice interactions for patient triage, symptom analysis, specialist mapping, and appointment booking. Built with modern technologies including FastAPI, PostgreSQL, React, and integrated with leading AI models.

🔄 System Architecture & User Flow

┌─────────────────────────────────────────────────────────────────────────────────────┐
│                           🎤 VOICE-ENABLED USER INTERFACE                           │
│                                  (React Frontend)                                   │
└─────────────────────┬───────────────────────────────────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│                              🔐 USER AUTHENTICATION                                 │
│  ┌─────────────────┐    ┌──────────────────┐    ┌─────────────────────────────────┐ │
│  │   User Login    │───▶│  FastAPI Backend │───▶│     PostgreSQL Database         │ │
│  │ (Email/Password)│    │   sp_login_user  │    │      (users table)             │ │
│  └─────────────────┘    └──────────────────┘    └─────────────────────────────────┘ │
└─────────────────────┬───────────────────────────────────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│                          🎙️ VOICE SYMPTOM COLLECTION                               │
│  ┌─────────────────┐    ┌──────────────────┐    ┌─────────────────────────────────┐ │
│  │  Web Speech API │───▶│ Speech-to-Text   │───▶│    Symptom Phrases Array       │ │
│  │  (Microphone)   │    │   Conversion     │    │   ["headache", "fever", ...]   │ │
│  └─────────────────┘    └──────────────────┘    └─────────────────────────────────┘ │
└─────────────────────┬───────────────────────────────────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│                        🤖 AI-POWERED SYMPTOM ANALYSIS                              │
│                              (LangGraph Workflow)                                  │
│  ┌─────────────────┐    ┌──────────────────┐    ┌─────────────────────────────────┐ │
│  │   Raw Symptoms  │───▶│  Gemini/GPT-4    │───▶│   Normalized Symptoms           │ │
│  │   Processing    │    │   AI Analysis    │    │ ["migraine", "pyrexia", ...]    │ │
│  └─────────────────┘    └──────────────────┘    └─────────────────────────────────┘ │
│                                  │                                                  │
│                                  ▼                                                  │
│  ┌─────────────────┐    ┌──────────────────┐    ┌─────────────────────────────────┐ │
│  │   Specialist    │◀───│   AI Mapping     │◀───│     Symptom Analysis            │ │
│  │ Recommendation  │    │    Algorithm     │    │      & Classification          │ │
│  └─────────────────┘    └──────────────────┘    └─────────────────────────────────┘ │
└─────────────────────┬───────────────────────────────────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│                          🏥 DOCTOR MATCHING & RETRIEVAL                            │
│  ┌─────────────────┐    ┌──────────────────┐    ┌─────────────────────────────────┐ │
│  │   Specialist    │───▶│     FastAPI      │───▶│      PostgreSQL Query          │ │
│  │     Types       │    │sp_get_doctors_by │    │    (doctors table filter)      │ │
│  │ ["cardiology",  │    │   _specialist    │    │                                 │ │
│  │ "neurology"]    │    │                  │    │                                 │ │
│  └─────────────────┘    └──────────────────┘    └─────────────────────────────────┘ │
│                                  │                                                  │
│                                  ▼                                                  │
│  ┌─────────────────┐    ┌──────────────────┐    ┌─────────────────────────────────┐ │
│  │  Available      │◀───│   Doctor List    │◀───│     Available Time Slots       │ │
│  │   Doctors       │    │   with Slots     │    │        & Schedules             │ │
│  └─────────────────┘    └──────────────────┘    └─────────────────────────────────┘ │
└─────────────────────┬───────────────────────────────────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│                           📅 APPOINTMENT BOOKING                                   │
│  ┌─────────────────┐    ┌──────────────────┐    ┌─────────────────────────────────┐ │
│  │   User Selects  │───▶│     Patient      │───▶│       Appointment               │ │
│  │ Doctor & Slot   │    │   Information    │    │      Confirmation               │ │
│  │                 │    │sp_get_patient_   │    │  sp_create_appointment          │ │
│  │                 │    │    details       │    │                                 │ │
│  └─────────────────┘    └──────────────────┘    └─────────────────────────────────┘ │
└─────────────────────┬───────────────────────────────────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│                            💳 PAYMENT PROCESSING                                   │
│  ┌─────────────────┐    ┌──────────────────┐    ┌─────────────────────────────────┐ │
│  │   Razorpay      │───▶│   Secure Payment │───▶│      Booking Confirmation       │ │
│  │   Gateway       │    │    Processing    │    │       & Notification            │ │
│  └─────────────────┘    └──────────────────┘    └─────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────────┘

                                        ▼

┌─────────────────────────────────────────────────────────────────────────────────────┐
│                          ✅ COMPLETE HEALTHCARE JOURNEY                            │
│                                                                                     │
│  🎤 Voice Input → 🤖 AI Analysis → 🏥 Doctor Match → 📅 Booking → 💳 Payment      │
└─────────────────────────────────────────────────────────────────────────────────────┘

🔍 Detailed Component Interaction Flow

1. User Authentication Flow

User Input (Email/Password) → FastAPI Login Endpoint → PostgreSQL sp_login_user → JWT Token/Session

2. Voice Processing Pipeline

Microphone → Web Speech API → Text Conversion → Phrase Array → LangGraph Processing

3. AI Analysis Workflow

Raw Symptoms → Gemini/GPT-4 → Normalized Symptoms → Specialist Mapping → Doctor Recommendations

4. Database Integration Pattern

FastAPI Endpoints → PostgreSQL Stored Procedures → Data Retrieval → JSON Response → Frontend Display

5. Appointment Booking Chain

Doctor Selection → Patient Details Fetch → Slot Validation → Appointment Creation → Payment Processing

🏗️ Technical Architecture Overview

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│                 │    │                 │    │                 │    │                 │
│   FRONTEND      │◀──▶│    BACKEND      │◀──▶│    DATABASE     │◀──▶│   EXTERNAL      │
│                 │    │                 │    │                 │    │   SERVICES      │
│  • React        │    │  • FastAPI      │    │  • PostgreSQL   │    │  • Gemini API   │
│  • Web Speech   │    │  • LangGraph    │    │  • Stored Procs │    │  • OpenAI API   │
│  • JavaScript   │    │  • Python       │    │  • Functions    │    │  • Razorpay     │
│                 │    │                 │    │                 │    │                 │
└─────────────────┘    └─────────────────┘    └─────────────────┘    └─────────────────┘
        │                        │                        │                        │
        └────────────────────────┼────────────────────────┼────────────────────────┘
                                 │                        │
                            HTTP/REST API            SQL Queries
                         (JSON Data Exchange)    (Stored Procedures)

🌟 Overview

This healthcare AI assistant streamlines the patient care journey by providing:

  • Intelligent Voice Triage: Natural language symptom collection and analysis
  • AI-Powered Diagnosis: Advanced symptom normalization and specialist recommendation
  • Smart Doctor Matching: Automated healthcare provider lookup based on specialization
  • Seamless Booking: Integrated appointment scheduling with payment processing
  • Conversational Interface: Intuitive voice-enabled user experience

✨ Key Features

🎤 Voice-Enabled Interaction

  • Real-time speech-to-text conversion using Web Speech API
  • Natural language processing for symptom collection
  • Voice-guided patient triage workflow

🤖 AI-Powered Medical Intelligence

  • Integration with Gemini and GPT-4 for symptom analysis
  • LangGraph-based symptom normalization
  • Intelligent specialist mapping and recommendations

🏥 Healthcare Management

  • Comprehensive doctor database with specialization filtering
  • PostgreSQL-powered efficient data retrieval
  • Automated appointment scheduling system

💳 Payment Integration

  • Secure payment processing via Razorpay
  • Test mode support for development
  • Transaction management and tracking

🛠️ Technology Stack

Component Technology Purpose
Frontend React, CSS, JavaScript User interface and voice interactions
Backend FastAPI, Python API services and business logic
AI/ML Gemini, GPT-4, LangGraph Natural language processing and AI agents
Database PostgreSQL Data persistence and stored procedures
Voice Web Speech API Speech recognition and synthesis
Payments Razorpay Payment processing and gateway
Deployment Uvicorn, Vite Development and production servers

📂 Project Architecture

healthcare-ai-assistant/
├── backend/                    # FastAPI backend services
│   ├── main.py                # Application entry point
│   ├── agents/                # LangGraph AI agents
│   ├── db/                    # Database connections and queries
│   ├── models.py              # Data models and schemas
│   ├── config.py              # Configuration management
│   └── .env                   # Environment variables (excluded from git)
│
├── frontend/                   # React frontend application
│   ├── components/            # Reusable UI components
│   ├── pages/                 # Application pages/views
│   └── .env.local             # Frontend environment variables
│
├── sql/                       # Database schema and functions
│   ├── schema.sql             # Database table definitions
│   └── functions/             # PostgreSQL stored procedures
│       ├── create_login_function.sql
│       ├── create_appointment_function.sql
│       ├── get_patient_details.sql
│       └── get_doctors_by_specialist.sql
│
├── requirements.txt           # Python dependencies
├── package.json              # Node.js dependencies
├── .gitignore                # Git ignore rules
└── README.md                 # Project documentation

🚀 Quick Start Guide

Prerequisites

  • Python 3.8+
  • Node.js 16+
  • PostgreSQL 12+
  • Git

1. 🗄️ Database Setup

Create Database and User

psql -U postgres
CREATE DATABASE healthcare;
CREATE USER fastapi_user WITH PASSWORD 'your_secure_password';
GRANT ALL PRIVILEGES ON DATABASE healthcare TO fastapi_user;
\q

Initialize Schema and Functions

psql -U fastapi_user -d healthcare -f sql/schema.sql
psql -U fastapi_user -d healthcare -f sql/functions/create_login_function.sql
psql -U fastapi_user -d healthcare -f sql/functions/create_appointment_function.sql
psql -U fastapi_user -d healthcare -f sql/functions/get_patient_details.sql
psql -U fastapi_user -d healthcare -f sql/functions/get_doctors_by_specialist.sql

2. 🔧 Backend Configuration

Setup Virtual Environment

cd backend
python -m venv venv

Windows:

venv\Scripts\activate

macOS/Linux:

source venv/bin/activate

Install Dependencies

pip install -r requirements.txt

Environment Configuration

Create .env file in the backend/ directory:

# Database Configuration
DB_HOST=localhost
DB_PORT=5432
DB_USER=fastapi_user
DB_PASSWORD=your_secure_password
DB_NAME=healthcare

# CORS Configuration
FRONTEND_ORIGIN=http://localhost:5173

# AI API Keys
OPENAI_API_KEY=your_openai_api_key_here
GEMINI_API_KEY=your_gemini_api_key_here

Start Backend Server

uvicorn main:app --reload --host 0.0.0.0 --port 8000

3. 🎨 Frontend Setup

Install Dependencies

npm install

Environment Configuration

Create .env.local file in the root directory:

# API Configuration
VITE_BACKEND_URL=http://localhost:8000
VITE_GEMINI_API_KEY=your_gemini_api_key_here

# Payment Configuration
VITE_RAZORPAY_KEY_ID=your_razorpay_key_id_here

Start Development Server

npm run dev

4. 💳 Payment Setup (Optional)

  1. Create a Razorpay account
  2. Navigate to API Keys section in dashboard
  3. Copy the Key ID and Key Secret
  4. Update the payment configuration in frontend/components/Recommendation.jsx

🔑 API Keys Setup

OpenAI API Key

  1. Visit OpenAI Platform
  2. Create an account and navigate to API Keys
  3. Generate a new secret key
  4. Add to backend .env file

Google Gemini API Key

  1. Go to Google AI Studio
  2. Create a new project or select existing
  3. Generate API key
  4. Add to both backend .env and frontend .env.local

Razorpay Configuration

  1. Sign up at Razorpay Dashboard
  2. Switch to Test Mode for development
  3. Copy API keys from Settings > API Keys
  4. Configure in frontend environment

🏃‍♂️ Running the Application

  1. Start PostgreSQL service
  2. Launch Backend: uvicorn main:app --reload (from backend/ directory)
  3. Launch Frontend: npm run dev (from root directory)
  4. Access Application: Navigate to http://localhost:5173

🧪 Testing

Backend API Testing

curl http://localhost:8000/health

Database Connection Testing

python -c "from backend.db.connection import get_db_connection; print('DB Connected!' if get_db_connection() else 'DB Connection Failed!')"

🚀 Deployment

Backend Deployment

  • Configure production database credentials
  • Set up environment variables on hosting platform
  • Deploy using platforms like Heroku, Railway, or DigitalOcean

Frontend Deployment

  • Build production bundle: npm run build
  • Deploy to Vercel, Netlify, or similar platforms
  • Update CORS settings in backend for production domain

🤝 Contributing

  1. Fork the repository
  2. Create a feature branch: git checkout -b feature/amazing-feature
  3. Commit changes: git commit -m 'Add amazing feature'
  4. Push to branch: git push origin feature/amazing-feature
  5. Open a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🆘 Support

For support and questions:

  • Create an issue in the GitHub repository
  • Check existing documentation and FAQs
  • Review the troubleshooting section below

🔧 Troubleshooting

Common Issues

Database Connection Error

  • Verify PostgreSQL is running
  • Check database credentials in .env
  • Ensure database and user exist

Voice Recognition Not Working

  • Use HTTPS or localhost only
  • Check browser microphone permissions
  • Verify Web Speech API support

API Key Errors

  • Validate API keys are correctly set
  • Check for trailing spaces or quotes
  • Verify API key permissions and quotas

🔮 Future Enhancements

  • Multi-language support
  • Mobile application development
  • Advanced AI model integration
  • Telemedicine video consultation
  • Electronic health records integration
  • Real-time chat support
  • Advanced analytics dashboard

Built with ❤️ for better healthcare accessibility

About

A comprehensive full-stack healthcare application that leverages AI-powered voice interactions for patient triage, symptom analysis, specialist mapping, and appointment booking. It is built with modern technologies including FastAPI, PostgreSQL, React, and integrated with leading AI models like Gemini and GPT-4.

Topics

react python gemini razorpay postgreql uvicorn fastapi gpt-4 langgraph

Resources

Readme

License

MIT license
Activity

Stars

40 stars

Watchers

11 watching

Forks

13 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages