Private AI Backend Service
A personal intelligence service, using your own computer to power private conversations with friends, family, and coworkers, collect/store your own private data, and use AI privately. Runs great with the chat arena app available on desktop and mobile.
Tech: ollama, postgres, FastAPI, nixOS, huggingface-transformers
(Experimental): This is new, and should work on most MacOS machines! Simply double click the topos.dmg file, and drag the app into your Applications directory. You should be able to launch the Topos service anywhere from your machine.
If nix is not installed:
- Install nix:
macos/linux:sh <(curl -L https://nixos.org/nix/install)
windows:sh <(curl -L https://nixos.org/nix/install) --daemon
Run the software with nix: - Download this repo
git clone https://github.com/jonnyjohnson1/topos-cli
cd topos-cli
- build the backend service (only run the topos set --spacy trf line if it is your first time setting up)
nix-shell
topos set --spacy trf
topos run
requires brew install just
requires brew install poetry
requires brew install python-tk
brew install neo4j brew services start neo4j
install the topos package with the command just build
Set the size of the spacy model you wish to use on your system. There are 'small', 'med', 'large', and 'trf'.
Use the tag like this.
topos set --spacy small
topos set --spacy trf
topos run
topos run --local
We are going to expose our backend service to a public network so our phone/tablet can use it. In this case, we use zrok. Below is are the guides to set up ngrok.
zrok is opensourced and free. ngrok has a gated requests/month under its free tier, then requires you pay for it.
- Be sure you have the
topos
server running already in another terminal. - Install zrok command
zrok enable <given_key>
zrok status
should show you information- Route local path through zrok:
zrok share public http://0.0.0.0:13341
This will take you to a new screen with an https:// at the top. Insert this url into the field under settings-> "Api Endpoints" -> "Custom API" - After you've insert it into the field, press the test button, and "hello world" should appear next to the button.
[ ] Enable permanent sharing of zrok url docs (requires Docker)