LocalAI/README.md

LocalAI

💡 Get help - ❓FAQ 💭Discussions 💬 Discord 📖 Documentation website

💻 Quickstart 📣 News 🛫 Examples 🖼️ Models 🚀 Roadmap

LocalAI is the free, Open Source OpenAI alternative. LocalAI act as a drop-in replacement REST API that’s compatible with OpenAI (Elevenlabs, Anthropic... ) API specifications for local AI inferencing. It allows you to run LLMs, generate images, audio (and not only) locally or on-prem with consumer grade hardware, supporting multiple model families. Does not require GPU.

🔥🔥 Hot topics / Roadmap

Roadmap

• Openvino support: https://github.com/mudler/LocalAI/pull/1892
• Vector store: https://github.com/mudler/LocalAI/pull/1795
• All-in-one container image: https://github.com/mudler/LocalAI/issues/1855
• Parallel function calling: https://github.com/mudler/LocalAI/pull/1726 / Tools API support: https://github.com/mudler/LocalAI/pull/1715
• Upload file API: https://github.com/mudler/LocalAI/pull/1703
• ROCm container images: https://github.com/mudler/LocalAI/pull/1595 / Intel GPU support (sycl, transformers, diffusers): https://github.com/mudler/LocalAI/issues/1653
• Mamba support: https://github.com/mudler/LocalAI/pull/1589
• Start and share models with config file: https://github.com/mudler/LocalAI/pull/1522
• 🐸 Coqui: https://github.com/mudler/LocalAI/pull/1489
• Img2vid https://github.com/mudler/LocalAI/pull/1442

Hot topics (looking for contributors):

• Backends v2: https://github.com/mudler/LocalAI/issues/1126
• Improving UX v2: https://github.com/mudler/LocalAI/issues/1373
• Assistant API: https://github.com/mudler/LocalAI/issues/1273
• Moderation endpoint: https://github.com/mudler/LocalAI/issues/999
• Vulkan: https://github.com/mudler/LocalAI/issues/1647

If you want to help and contribute, issues up for grabs: https://github.com/mudler/LocalAI/issues?q=is%3Aissue+is%3Aopen+label%3A%22up+for+grabs%22

💻 Getting started

For a detailed step-by-step introduction, refer to the Getting Started guide.

For those in a hurry, here's a straightforward one-liner to launch a LocalAI AIO(All-in-one) Image using docker:

docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-aio-cpu
# or, if you have an Nvidia GPU:
# docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-aio-gpu-nvidia-cuda-12

🚀 Features

• 📖 Text generation with GPTs (llama.cpp, gpt4all.cpp, ... 📖 and more)
• 🗣 Text to Audio
• 🔈 Audio to Text (Audio transcription with whisper.cpp)
• 🎨 Image generation with stable diffusion
• 🔥 OpenAI functions 🆕
• 🧠 Embeddings generation for vector databases
• ✍️ Constrained grammars
• 🖼️ Download Models directly from Huggingface
• 🆕 Vision API

💻 Usage

Check out the Getting started section in our documentation.

🔗 Community and integrations

Build and deploy custom containers:

• https://github.com/sozercan/aikit

WebUIs:

• https://github.com/Jirubizu/localai-admin
• https://github.com/go-skynet/LocalAI-frontend

Model galleries

• https://github.com/go-skynet/model-gallery

Other:

• Helm chart https://github.com/go-skynet/helm-charts
• VSCode extension https://github.com/badgooooor/localai-vscode-plugin
• Local Smart assistant https://github.com/mudler/LocalAGI
• Home Assistant https://github.com/sammcj/homeassistant-localai / https://github.com/drndos/hass-openai-custom-conversation
• Discord bot https://github.com/mudler/LocalAGI/tree/main/examples/discord
• Slack bot https://github.com/mudler/LocalAGI/tree/main/examples/slack
• Telegram bot https://github.com/mudler/LocalAI/tree/master/examples/telegram-bot
• Examples: https://github.com/mudler/LocalAI/tree/master/examples/

🔗 Resources

• 🆕 New! LLM finetuning guide
• How to build locally
• How to install in Kubernetes
• Projects integrating LocalAI
• How tos section (curated by our community)

📖 🎥 Media, Blogs, Social

• Run LocalAI on AWS EKS with Pulumi
• Run LocalAI on AWS
• Create a slackbot for teams and OSS projects that answer to documentation
• LocalAI meets k8sgpt
• Question Answering on Documents locally with LangChain, LocalAI, Chroma, and GPT4All
• Tutorial to use k8sgpt with LocalAI

Citation

If you utilize this repository, data in a downstream project, please consider citing it with:

@misc{localai,
  author = {Ettore Di Giacinto},
  title = {LocalAI: The free, Open source OpenAI alternative},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/go-skynet/LocalAI}},

❤️ Sponsors

Do you find LocalAI useful?

Support the project by becoming a backer or sponsor. Your logo will show up here with a link to your website.

A huge thank you to our generous sponsors who support this project:

Spectro Cloud
Spectro Cloud kindly supports LocalAI by providing GPU and computing resources to run tests on lamdalabs!

And a huge shout-out to individuals sponsoring the project by donating hardware or backing the project.

• Sponsor list
• JDAM00 (donating HW for the CI)

🌟 Star history

📖 License

LocalAI is a community-driven project created by Ettore Di Giacinto.

MIT - Author Ettore Di Giacinto

🙇 Acknowledgements

LocalAI couldn't have been built without the help of great software already available from the community. Thank you!

• llama.cpp
• https://github.com/tatsu-lab/stanford_alpaca
• https://github.com/cornelk/llama-go for the initial ideas
• https://github.com/antimatter15/alpaca.cpp
• https://github.com/EdVince/Stable-Diffusion-NCNN
• https://github.com/ggerganov/whisper.cpp
• https://github.com/saharNooby/rwkv.cpp
• https://github.com/rhasspy/piper

🤗 Contributors

This is a community project, a special thanks to our contributors! 🤗