-->

⏱️ Estimated Reading Time: 15 minutes

1. What is Chat-Ollama?

Chat-Ollama is an open-source AI chatbot platform that prioritizes data privacy and security while providing powerful capabilities of cutting-edge language models. With 3.3k+ stars on GitHub and active development, this project offers a complete solution for secure AI conversations in local environments.

Key Features

  • 🔒 Privacy First: Local data processing with minimal external server dependencies
  • 🤖 Multiple AI Model Support: Ollama, OpenAI, Anthropic, Google AI, Groq, and more
  • 🔧 MCP Integration: External tool connectivity via Model Context Protocol
  • 🎤 Real-time Voice Chat: Voice conversations with Gemini 2.0 Flash model
  • 📚 RAG Support: Document upload and knowledge base conversations
  • 🌐 Multi-language Support: Global interface for international users

2. Technology Stack & Architecture

Frontend

  • Nuxt 3: Modern Vue.js framework
  • Vue 3: Composition API-based reactive UI
  • Tailwind CSS: Utility-first CSS framework
  • Nuxt UI: Consistent component system

Backend

  • Nitro: Nuxt server engine
  • Prisma ORM: Type-safe database access
  • SQLite/PostgreSQL: Development/production databases

AI & Vector Processing

  • LangChain: AI model abstraction layer
  • ChromaDB/Milvus: Vector databases
  • Multiple AI Providers: OpenAI, Anthropic, Google, Ollama, etc.

3. macOS Environment Setup & Installation

3.1 Prerequisites

# Check Node.js installation (v18+ recommended)
node --version

# Install pnpm
npm install -g pnpm

# Check Git installation
git --version

# Docker installation (optional, for easy deployment)
docker --version
docker-compose --version

3.2 Project Clone & Initial Setup

# Clone the project
git clone https://github.com/sugarforever/chat-ollama.git
cd chat-ollama

# Install dependencies
pnpm install

# Setup environment variables
cp .env.example .env

3.3 Environment Variables Configuration

Edit the .env file to add necessary configurations:

# Database settings
DATABASE_URL="file:../../chatollama.sqlite"

# Server settings
PORT=3000
HOST=

# Vector database settings
VECTOR_STORE=chroma
CHROMADB_URL=http://localhost:8000

# AI model API keys (optional)
OPENAI_API_KEY=your_openai_key
ANTHROPIC_API_KEY=your_anthropic_key
GOOGLE_API_KEY=your_gemini_key
GROQ_API_KEY=your_groq_key

# Feature flags
MCP_ENABLED=true
KNOWLEDGE_BASE_ENABLED=true
REALTIME_CHAT_ENABLED=true
MODELS_MANAGEMENT_ENABLED=true

3.4 Database Initialization

# Generate Prisma client
pnpm prisma-generate

# Run database migrations
pnpm prisma-migrate

3.5 ChromaDB Setup (Vector Database)

# Run ChromaDB Docker container
docker run -d -p 8000:8000 --name chromadb chromadb/chroma

# Verify container is running
curl http://localhost:8000/api/v1/version

3.6 Development Server Launch

# Start server in development mode
pnpm dev

# Access http://localhost:3000 in browser

4. Easy Deployment with Docker

Docker allows you to run Chat-Ollama without complex configuration.

4.1 Docker Compose Deployment

# Run from project directory
docker-compose up -d

# Check service status
docker-compose ps

# View logs
docker-compose logs chatollama

4.2 Docker Environment Variables

Configure environment variables in the docker-compose.yml file:

services:
  chatollama:
    environment:
      - NUXT_MCP_ENABLED=true
      - NUXT_KNOWLEDGE_BASE_ENABLED=true
      - NUXT_REALTIME_CHAT_ENABLED=true
      - NUXT_MODELS_MANAGEMENT_ENABLED=true
      - OPENAI_API_KEY=your_key_here
      - ANTHROPIC_API_KEY=your_key_here

4.3 Data Persistence

In Docker deployment, data is stored as follows:

  • Vector Data: Docker volume (chromadb_volume)
  • Relational Data: SQLite file (~/.chatollama/chatollama.sqlite)
  • Session Data: Redis container

5. Ollama Model Setup

5.1 Ollama Installation & Configuration

# Install Ollama on macOS
curl -fsSL https://ollama.com/install.sh | sh

# Or use Homebrew
brew install ollama

# Start Ollama service
ollama serve

5.2 Model Download & Execution

# Install popular models
ollama pull llama3.1:8b
ollama pull codellama:13b
ollama pull mistral:7b
ollama pull qwen2.5:14b

# Check installed models
ollama list

# Test a model
ollama run llama3.1:8b "Hello, please respond in English"

5.3 Model Configuration in Chat-Ollama

  1. Access Web Interface: http://localhost:3000
  2. Click Settings menu
  3. Add Ollama model in Models tab:
    • Model Name: llama3.1:8b
    • API Base URL: http://localhost:11434
    • Model Type: Chat

6. MCP (Model Context Protocol) Integration

MCP enables AI models to access external tools and data sources.

6.1 MCP Server Configuration

Navigate to Settings → MCP in the web interface to add servers:

Filesystem Tools Setup

Name: Filesystem Tools
Transport: stdio
Command: uvx
Args: mcp-server-filesystem
Environment Variables:
  PATH: ${PATH}

Git Tools Setup

Name: Git Tools
Transport: stdio
Command: uvx  
Args: mcp-server-git
Environment Variables:
  PATH: ${PATH}

# File system manipulation
uvx mcp-server-filesystem

# Git repository management
uvx mcp-server-git

# SQLite database queries
uvx mcp-server-sqlite

# Web search (Brave Search)
uvx mcp-server-brave-search

6.3 MCP Functionality Testing

Once MCP is activated, AI models can automatically use tools during conversations:

  • “Show me the file list in the current directory” → Calls filesystem tool
  • “Check the Git commit history for this project” → Calls Git tool
  • “Search for the latest AI news” → Calls web search tool

7. Knowledge Base & RAG Implementation

7.1 Knowledge Base Creation

  1. Click Knowledge Bases menu
  2. Click “Create Knowledge Base” button
  3. Basic Configuration:
    • Name: Company Documents
    • Chunk Size: 1000
    • Chunk Overlap: 200

7.2 Document Upload

Supported file formats:

  • PDF: Text extraction and vectorization
  • DOCX: Microsoft Word documents
  • TXT: Plain text files
# Create sample document (for testing)
echo "Chat-Ollama is an open-source AI chatbot platform. 
Built on Nuxt 3 and Vue 3, 
it supports various AI models." > sample_doc.txt

When uploading documents through the web interface, it automatically:

  1. Extracts text
  2. Splits into chunks
  3. Generates embedding vectors
  4. Stores in ChromaDB

7.3 RAG-based Conversations

Once knowledge bases are created, conversations reference relevant document content:

User: Please explain Chat-Ollama's technology stack.
AI: Based on the uploaded documents, Chat-Ollama is built on Nuxt 3 and Vue 3... [Document-based response]

8. Real-time Voice Chat

8.1 Gemini API Configuration

# Add Google API key to .env file
GOOGLE_API_KEY=your_google_api_key_here

8.2 Voice Chat Activation

  1. Enable Realtime Chat in Settings → General
  2. Verify Google API key configuration
  3. Access /realtime page
  4. Allow microphone permissions

8.3 Voice Chat Features

  • Real-time Speech Recognition: Voice to text conversion
  • Natural Voice Response: High-quality TTS from Gemini 2.0 Flash
  • Pause & Resume: Interrupt/resume conversation anytime

9. Advanced Configuration & Customization

9.1 Proxy Configuration

For specific network environments requiring proxy:

# Proxy settings in .env file
NUXT_PUBLIC_MODEL_PROXY_ENABLED=true
NUXT_MODEL_PROXY_URL=http://127.0.0.1:1080

9.2 Cohere Reranking Setup

Cohere Rerank API for improved search result accuracy:

# Add Cohere API key to .env file
COHERE_API_KEY=your_cohere_key

9.3 Feature Toggles

Selectively enable specific features:

# Development environment (.env)
MCP_ENABLED=true
KNOWLEDGE_BASE_ENABLED=true  
REALTIME_CHAT_ENABLED=false
MODELS_MANAGEMENT_ENABLED=true

# Docker environment (docker-compose.yml)
NUXT_MCP_ENABLED=true
NUXT_KNOWLEDGE_BASE_ENABLED=true
NUXT_REALTIME_CHAT_ENABLED=false
NUXT_MODELS_MANAGEMENT_ENABLED=true

10. Production Deployment Guide

10.1 PostgreSQL Migration

PostgreSQL is recommended for production environments:

# Install PostgreSQL (macOS)
brew install postgresql
brew services start postgresql

# Create database and user
psql postgres
CREATE DATABASE chatollama;
CREATE USER chatollama WITH PASSWORD 'secure_password';
GRANT ALL PRIVILEGES ON DATABASE chatollama TO chatollama;
\q

# Update .env file
DATABASE_URL="postgresql://chatollama:secure_password@localhost:5432/chatollama"

# Run migrations
pnpm prisma migrate deploy

10.2 SQLite to PostgreSQL Data Migration

# Backup existing SQLite data
cp chatollama.sqlite chatollama.sqlite.backup

# Run migration
pnpm migrate:sqlite-to-postgres

10.3 Production Build

# Build for production
pnpm build

# Test in preview mode
pnpm preview

10.4 System Service Registration (macOS)

# Create plist file for LaunchDaemon
sudo tee /Library/LaunchDaemons/com.chatollama.plist << EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.chatollama</string>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/local/bin/node</string>
        <string>/path/to/chat-ollama/.output/server/index.mjs</string>
    </array>
    <key>WorkingDirectory</key>
    <string>/path/to/chat-ollama</string>
    <key>KeepAlive</key>
    <true/>
    <key>StandardOutPath</key>
    <string>/var/log/chatollama.log</string>
    <key>StandardErrorPath</key>
    <string>/var/log/chatollama.error.log</string>
</dict>
</plist>
EOF

# Register and start service
sudo launchctl load /Library/LaunchDaemons/com.chatollama.plist
sudo launchctl start com.chatollama

11. Troubleshooting

11.1 Common Issues

Port Conflicts

# Check processes using port
lsof -i :3000

# Run on different port
PORT=3001 pnpm dev

ChromaDB Connection Failure

# Check ChromaDB container status
docker ps | grep chroma

# Restart container
docker restart chromadb

# Manually run container
docker run -d -p 8000:8000 --name chromadb chromadb/chroma

Database Migration Failure

# Reset database
rm chatollama.sqlite
pnpm prisma migrate reset

# Create new migration
pnpm prisma migrate dev --name init

11.2 Performance Optimization

Memory Usage Optimization

# Set Node.js memory limit
NODE_OPTIONS="--max_old_space_size=4096" pnpm dev

ChromaDB Performance Tuning

# Run ChromaDB with optimized settings
docker run -d -p 8000:8000 \
  -e CHROMA_SERVER_HOST=0.0.0.0 \
  -e CHROMA_SERVER_HTTP_PORT=8000 \
  -v chroma-data:/chroma/chroma \
  --name chromadb chromadb/chroma

12. Security Considerations

12.1 API Key Security

# Set .env file permissions
chmod 600 .env

# Manage sensitive information via environment variables
export OPENAI_API_KEY="your-secret-key"
export ANTHROPIC_API_KEY="your-secret-key"

12.2 Network Security

# Allow local access only
HOST=127.0.0.1 pnpm dev

# HTTPS configuration (production)
NUXT_PUBLIC_SITE_URL=https://your-domain.com

12.3 Data Backup

# Regular backup script
#!/bin/bash
BACKUP_DIR="/path/to/backups"
DATE=$(date +%Y%m%d_%H%M%S)

# SQLite backup
cp chatollama.sqlite "$BACKUP_DIR/chatollama_$DATE.sqlite"

# ChromaDB volume backup
docker run --rm -v chromadb_volume:/data -v $BACKUP_DIR:/backup busybox tar czf /backup/chromadb_$DATE.tar.gz /data

13. Community & Support

13.1 Official Resources

13.2 Contributing

  • Issue Reporting: Report bugs via GitHub Issues
  • Feature Requests: Use Feature Request template
  • Code Contributions: Submit improvements via Pull Requests

14. Conclusion

Chat-Ollama provides a complete solution that prioritizes privacy while offering powerful AI capabilities. This guide has covered everything from installation to advanced feature utilization, providing sufficient information to customize according to your environment and requirements.

Key Advantages Summary

  • 🔒 Complete Data Privacy: All processing in local environment
  • 🤖 Multiple Model Support: Integration with Ollama, OpenAI, Anthropic, etc.
  • 🔧 Extensibility: Unlimited functionality expansion through MCP
  • 📚 Knowledge Base: RAG-based document search and conversation
  • 🎤 Voice Support: Real-time voice chat capabilities

Build a secure and powerful AI assistant using Chat-Ollama. For questions or issues, please utilize GitHub Issues or the Discord community.

Reference Links:

참고