alex/LocalAGI
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update README

Browse Source
This commit is contained in:
mudler
2023-08-05 22:42:52 +02:00
parent be331d22cd
commit 5b28a34e5b
1 changed files with 16 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
README.md
View File

@@ -17,7 +17,7 @@ The goal is:
- Run with Docker/Podman/Containers - Run with Docker/Podman/Containers
- Rather than trying to do everything, provide a good starting point for other projects - Rather than trying to do everything, provide a good starting point for other projects
Note: this is a fun project, not a serious one. Be warned! It was hacked in a weekend, and it's just an experiment to see what can be done with local LLMs. Note: Be warned! It was hacked in a weekend, and it's just an experiment to see what can be done with local LLMs.
## Demo ## Demo
@@ -29,6 +29,8 @@ Plan a road trip (batch mode)
https://github.com/mudler/LocalAGI/assets/2420543/19f936ea-9fa5-4d91-9cdf-006a42dd9a0c https://github.com/mudler/LocalAGI/assets/2420543/19f936ea-9fa5-4d91-9cdf-006a42dd9a0c
> Note: The demo is with a GPU and `30b` models size
## 🚀 Features ## 🚀 Features
- 🧠 LLM for intent detection - 🧠 LLM for intent detection
@@ -156,9 +158,10 @@ The intention of this project is to keep the agent minimal, so can be built on t
- [x] Make the virtual assistant do functions like store long-term memory and autonomously search between them when needed - [x] Make the virtual assistant do functions like store long-term memory and autonomously search between them when needed
- [x] Create the assistant avatar with Stable Diffusion - [x] Create the assistant avatar with Stable Diffusion
- [x] Give it a voice - [x] Give it a voice
- [] Get voice input (push to talk or wakeword) - [ ] Use weaviate instead of Chroma
- [] Make a REST API (OpenAI compliant?) so can be plugged by e.g. a third party service - [ ] Get voice input (push to talk or wakeword)
- [] Take a system prompt so can act with a "character" (e.g. "answer in rick and morty style") - [ ] Make a REST API (OpenAI compliant?) so can be plugged by e.g. a third party service
- [x] Take a system prompt so can act with a "character" (e.g. "answer in rick and morty style")
## Development ## Development
@@ -167,3 +170,12 @@ Run docker-compose with main.py checked-out:
```bash ```bash
docker-compose run -v main.py:/app/main.py -i --rm localagi docker-compose run -v main.py:/app/main.py -i --rm localagi
``` ```
## Notes
- a 13b model is enough for doing contextualized research and search/retrieve memory
- a 30b model is enough to generate a roadmap trip plan ( so cool! )
- With superhot models looses its magic, but maybe suitable for search
- Context size is your enemy. `--postprocess` some times helps, but not always
- It can be silly!
- It is slow on CPU, don't expect `7b` models to perform good, and `13b` models perform better but on CPU are quite slow.