Skip to content

Latest commit

 

History

History
81 lines (62 loc) · 3.18 KB

README.md

File metadata and controls

81 lines (62 loc) · 3.18 KB

Private LLMs

Private AI Backend Service

Topos

A personal intelligence service, using your own computer to power private conversations with friends, family, and coworkers, collect/store your own private data, and use AI privately. Runs great with the chat arena app available on desktop and mobile.

Tech: ollama, postgres, FastAPI, nixOS, huggingface-transformers

(MacOS) Easy Install With .dmg

(Experimental): This is new, and should work on most MacOS machines! Simply double click the topos.dmg file, and drag the app into your Applications directory. You should be able to launch the Topos service anywhere from your machine.

Install with nix (Recommended)

If nix is not installed:

  1. Install nix:
    macos/linux: sh <(curl -L https://nixos.org/nix/install)
    windows: sh <(curl -L https://nixos.org/nix/install) --daemon Run the software with nix:
  2. Download this repo git clone https://github.com/jonnyjohnson1/topos-cli
  3. cd topos-cli
  4. build the backend service (only run the topos set --spacy trf line if it is your first time setting up)
nix-shell
topos set --spacy trf
topos run

Install Instructions

requires brew install just requires brew install poetry requires brew install python-tk

Graph Database - Install Neo4j

Install Neo4j Database on Osx

brew install neo4j brew services start neo4j

Topos

Step 1: Install Topos

install the topos package with the command just build

Step 2: Set the Spacy Model Size

Set the size of the spacy model you wish to use on your system. There are 'small', 'med', 'large', and 'trf'.

Use the tag like this. topos set --spacy small topos set --spacy trf

Step 3: Start Topos on your local machine

topos run topos run --local

Step 4a (zrok): Set up web proxy

We are going to expose our backend service to a public network so our phone/tablet can use it. In this case, we use zrok. Below is are the guides to set up ngrok.

zrok is opensourced and free. ngrok has a gated requests/month under its free tier, then requires you pay for it.

  1. Be sure you have the topos server running already in another terminal.
  2. Install zrok command
  3. zrok enable <given_key>
  4. zrok status should show you information
  5. Route local path through zrok: zrok share public http://0.0.0.0:13341 This will take you to a new screen with an https:// at the top. Insert this url into the field under settings-> "Api Endpoints" -> "Custom API"
  6. After you've insert it into the field, press the test button, and "hello world" should appear next to the button.

[ ] Enable permanent sharing of zrok url docs (requires Docker)

Step 4b (ngrok): Set up web proxy