mirror of
https://github.com/mudler/LocalAI.git
synced 2025-04-24 21:09:41 +00:00
4f239bac89
* wip Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Update lotusdocs and hugo Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * rephrasing Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fixups Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Latest fixups Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Adjust readme section Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
290 lines
15 KiB
Markdown
290 lines
15 KiB
Markdown
<h1 align="center">
|
|
<br>
|
|
<img height="300" src="./core/http/static/logo.png"> <br>
|
|
<br>
|
|
</h1>
|
|
|
|
<p align="center">
|
|
<a href="https://github.com/go-skynet/LocalAI/fork" target="blank">
|
|
<img src="https://img.shields.io/github/forks/go-skynet/LocalAI?style=for-the-badge" alt="LocalAI forks"/>
|
|
</a>
|
|
<a href="https://github.com/go-skynet/LocalAI/stargazers" target="blank">
|
|
<img src="https://img.shields.io/github/stars/go-skynet/LocalAI?style=for-the-badge" alt="LocalAI stars"/>
|
|
</a>
|
|
<a href="https://github.com/go-skynet/LocalAI/pulls" target="blank">
|
|
<img src="https://img.shields.io/github/issues-pr/go-skynet/LocalAI?style=for-the-badge" alt="LocalAI pull-requests"/>
|
|
</a>
|
|
<a href='https://github.com/go-skynet/LocalAI/releases'>
|
|
<img src='https://img.shields.io/github/release/go-skynet/LocalAI?&label=Latest&style=for-the-badge'>
|
|
</a>
|
|
</p>
|
|
|
|
<p align="center">
|
|
<a href="https://hub.docker.com/r/localai/localai" target="blank">
|
|
<img src="https://img.shields.io/badge/dockerhub-images-important.svg?logo=Docker" alt="LocalAI Docker hub"/>
|
|
</a>
|
|
<a href="https://quay.io/repository/go-skynet/local-ai?tab=tags&tag=latest" target="blank">
|
|
<img src="https://img.shields.io/badge/quay.io-images-important.svg?" alt="LocalAI Quay.io"/>
|
|
</a>
|
|
</p>
|
|
|
|
<p align="center">
|
|
<a href="https://twitter.com/LocalAI_API" target="blank">
|
|
<img src="https://img.shields.io/twitter/follow/LocalAI_API?label=Follow: LocalAI_API&style=social" alt="Follow LocalAI_API"/>
|
|
</a>
|
|
<a href="https://discord.gg/uJAeKSAGDy" target="blank">
|
|
<img src="https://dcbadge.vercel.app/api/server/uJAeKSAGDy?style=flat-square&theme=default-inverted" alt="Join LocalAI Discord Community"/>
|
|
</a>
|
|
</p>
|
|
|
|
<p align="center">
|
|
<a href="https://trendshift.io/repositories/5539" target="_blank"><img src="https://trendshift.io/api/badge/repositories/5539" alt="mudler%2FLocalAI | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
|
|
</p>
|
|
|
|
> :bulb: Get help - [❓FAQ](https://localai.io/faq/) [💭Discussions](https://github.com/go-skynet/LocalAI/discussions) [:speech_balloon: Discord](https://discord.gg/uJAeKSAGDy) [:book: Documentation website](https://localai.io/)
|
|
>
|
|
> [💻 Quickstart](https://localai.io/basics/getting_started/) [🖼️ Models](https://models.localai.io/) [🚀 Roadmap](https://github.com/mudler/LocalAI/issues?q=is%3Aissue+is%3Aopen+label%3Aroadmap) [🥽 Demo](https://demo.localai.io) [🌍 Explorer](https://explorer.localai.io) [🛫 Examples](https://github.com/mudler/LocalAI-examples)
|
|
|
|
[](https://github.com/go-skynet/LocalAI/actions/workflows/test.yml)[](https://github.com/go-skynet/LocalAI/actions/workflows/release.yaml)[](https://github.com/go-skynet/LocalAI/actions/workflows/image.yml)[](https://github.com/go-skynet/LocalAI/actions/workflows/bump_deps.yaml)[](https://artifacthub.io/packages/search?repo=localai)
|
|
|
|
**LocalAI** is the free, Open Source OpenAI alternative. LocalAI act as a drop-in replacement REST API that's compatible with OpenAI (Elevenlabs, Anthropic... ) API specifications for local AI inferencing. It allows you to run LLMs, generate images, audio (and not only) locally or on-prem with consumer grade hardware, supporting multiple model families. Does not require GPU. It is created and maintained by [Ettore Di Giacinto](https://github.com/mudler).
|
|
|
|
|
|
## 📚🆕 Local Stack Family
|
|
|
|
🆕 LocalAI is now part of a comprehensive suite of AI tools designed to work together:
|
|
|
|
<table>
|
|
<tr>
|
|
<td width="50%" valign="top">
|
|
<a href="https://github.com/mudler/LocalAGI">
|
|
<img src="https://raw.githubusercontent.com/mudler/LocalAGI/refs/heads/main/webui/react-ui/public/logo_2.png" width="300" alt="LocalAGI Logo">
|
|
</a>
|
|
</td>
|
|
<td width="50%" valign="top">
|
|
<h3><a href="https://github.com/mudler/LocalAGI">LocalAGI</a></h3>
|
|
<p>A powerful Local AI agent management platform that serves as a drop-in replacement for OpenAI's Responses API, enhanced with advanced agentic capabilities.</p>
|
|
</td>
|
|
</tr>
|
|
<tr>
|
|
<td width="50%" valign="top">
|
|
<a href="https://github.com/mudler/LocalRecall">
|
|
<img src="https://raw.githubusercontent.com/mudler/LocalRecall/refs/heads/main/static/localrecall_horizontal.png" width="300" alt="LocalRecall Logo">
|
|
</a>
|
|
</td>
|
|
<td width="50%" valign="top">
|
|
<h3><a href="https://github.com/mudler/LocalRecall">LocalRecall</a></h3>
|
|
<p>A REST-ful API and knowledge base management system that provides persistent memory and storage capabilities for AI agents.</p>
|
|
</td>
|
|
</tr>
|
|
</table>
|
|
|
|
## Screenshots
|
|
|
|
|
|
| Talk Interface | Generate Audio |
|
|
| --- | --- |
|
|
|  |  |
|
|
|
|
| Models Overview | Generate Images |
|
|
| --- | --- |
|
|
|  |  |
|
|
|
|
| Chat Interface | Home |
|
|
| --- | --- |
|
|
|  |  |
|
|
|
|
| Login | Swarm |
|
|
| --- | --- |
|
|
| |  |
|
|
|
|
## 💻 Quickstart
|
|
|
|
Run the installer script:
|
|
|
|
```bash
|
|
curl https://localai.io/install.sh | sh
|
|
```
|
|
|
|
Or run with docker:
|
|
|
|
### CPU only image:
|
|
```bash
|
|
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-cpu
|
|
```
|
|
### Nvidia GPU:
|
|
```bash
|
|
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12
|
|
```
|
|
### CPU and GPU image (bigger size):
|
|
```bash
|
|
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest
|
|
```
|
|
### AIO images (it will pre-download a set of models ready for use, see https://localai.io/basics/container/)
|
|
```bash
|
|
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-aio-cpu
|
|
```
|
|
|
|
To load models:
|
|
|
|
```bash
|
|
# From the model gallery (see available models with `local-ai models list`, in the WebUI from the model tab, or visiting https://models.localai.io)
|
|
local-ai run llama-3.2-1b-instruct:q4_k_m
|
|
# Start LocalAI with the phi-2 model directly from huggingface
|
|
local-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf
|
|
# Install and run a model from the Ollama OCI registry
|
|
local-ai run ollama://gemma:2b
|
|
# Run a model from a configuration file
|
|
local-ai run https://gist.githubusercontent.com/.../phi-2.yaml
|
|
# Install and run a model from a standard OCI registry (e.g., Docker Hub)
|
|
local-ai run oci://localai/phi-2:latest
|
|
```
|
|
|
|
For more information, see [💻 Getting started](https://localai.io/basics/getting_started/index.html)
|
|
|
|
## 📰 Latest project news
|
|
|
|
- Apr 2025: [LocalAGI](https://github.com/mudler/LocalAGI) and [LocalRecall](https://github.com/mudler/LocalRecall) join the LocalAI family stack.
|
|
- Apr 2025: WebUI overhaul, AIO images updates
|
|
- Feb 2025: Backend cleanup, Breaking changes, new backends (kokoro, OutelTTS, faster-whisper), Nvidia L4T images
|
|
- Jan 2025: LocalAI model release: https://huggingface.co/mudler/LocalAI-functioncall-phi-4-v0.3, SANA support in diffusers: https://github.com/mudler/LocalAI/pull/4603
|
|
- Dec 2024: stablediffusion.cpp backend (ggml) added ( https://github.com/mudler/LocalAI/pull/4289 )
|
|
- Nov 2024: Bark.cpp backend added ( https://github.com/mudler/LocalAI/pull/4287 )
|
|
- Nov 2024: Voice activity detection models (**VAD**) added to the API: https://github.com/mudler/LocalAI/pull/4204
|
|
- Oct 2024: examples moved to [LocalAI-examples](https://github.com/mudler/LocalAI-examples)
|
|
- Aug 2024: 🆕 FLUX-1, [P2P Explorer](https://explorer.localai.io)
|
|
- July 2024: 🔥🔥 🆕 P2P Dashboard, LocalAI Federated mode and AI Swarms: https://github.com/mudler/LocalAI/pull/2723. P2P Global community pools: https://github.com/mudler/LocalAI/issues/3113
|
|
- May 2024: 🔥🔥 Decentralized P2P llama.cpp: https://github.com/mudler/LocalAI/pull/2343 (peer2peer llama.cpp!) 👉 Docs https://localai.io/features/distribute/
|
|
- May 2024: 🔥🔥 Distributed inferencing: https://github.com/mudler/LocalAI/pull/2324
|
|
- April 2024: Reranker API: https://github.com/mudler/LocalAI/pull/2121
|
|
|
|
Roadmap items: [List of issues](https://github.com/mudler/LocalAI/issues?q=is%3Aissue+is%3Aopen+label%3Aroadmap)
|
|
|
|
## 🚀 [Features](https://localai.io/features/)
|
|
|
|
- 📖 [Text generation with GPTs](https://localai.io/features/text-generation/) (`llama.cpp`, `transformers`, `vllm` ... [:book: and more](https://localai.io/model-compatibility/index.html#model-compatibility-table))
|
|
- 🗣 [Text to Audio](https://localai.io/features/text-to-audio/)
|
|
- 🔈 [Audio to Text](https://localai.io/features/audio-to-text/) (Audio transcription with `whisper.cpp`)
|
|
- 🎨 [Image generation](https://localai.io/features/image-generation)
|
|
- 🔥 [OpenAI-alike tools API](https://localai.io/features/openai-functions/)
|
|
- 🧠 [Embeddings generation for vector databases](https://localai.io/features/embeddings/)
|
|
- ✍️ [Constrained grammars](https://localai.io/features/constrained_grammars/)
|
|
- 🖼️ [Download Models directly from Huggingface ](https://localai.io/models/)
|
|
- 🥽 [Vision API](https://localai.io/features/gpt-vision/)
|
|
- 📈 [Reranker API](https://localai.io/features/reranker/)
|
|
- 🆕🖧 [P2P Inferencing](https://localai.io/features/distribute/)
|
|
- [Agentic capabilities](https://github.com/mudler/LocalAGI)
|
|
- 🔊 Voice activity detection (Silero-VAD support)
|
|
- 🌍 Integrated WebUI!
|
|
|
|
|
|
### 🔗 Community and integrations
|
|
|
|
Build and deploy custom containers:
|
|
- https://github.com/sozercan/aikit
|
|
|
|
WebUIs:
|
|
- https://github.com/Jirubizu/localai-admin
|
|
- https://github.com/go-skynet/LocalAI-frontend
|
|
- QA-Pilot(An interactive chat project that leverages LocalAI LLMs for rapid understanding and navigation of GitHub code repository) https://github.com/reid41/QA-Pilot
|
|
|
|
Model galleries
|
|
- https://github.com/go-skynet/model-gallery
|
|
|
|
Other:
|
|
- Helm chart https://github.com/go-skynet/helm-charts
|
|
- VSCode extension https://github.com/badgooooor/localai-vscode-plugin
|
|
- Langchain: https://python.langchain.com/docs/integrations/providers/localai/
|
|
- Terminal utility https://github.com/djcopley/ShellOracle
|
|
- Local Smart assistant https://github.com/mudler/LocalAGI
|
|
- Home Assistant https://github.com/sammcj/homeassistant-localai / https://github.com/drndos/hass-openai-custom-conversation / https://github.com/valentinfrlch/ha-gpt4vision
|
|
- Discord bot https://github.com/mudler/LocalAGI/tree/main/examples/discord
|
|
- Slack bot https://github.com/mudler/LocalAGI/tree/main/examples/slack
|
|
- Shell-Pilot(Interact with LLM using LocalAI models via pure shell scripts on your Linux or MacOS system) https://github.com/reid41/shell-pilot
|
|
- Telegram bot https://github.com/mudler/LocalAI/tree/master/examples/telegram-bot
|
|
- Another Telegram Bot https://github.com/JackBekket/Hellper
|
|
- Auto-documentation https://github.com/JackBekket/Reflexia
|
|
- Github bot which answer on issues, with code and documentation as context https://github.com/JackBekket/GitHelper
|
|
- Github Actions: https://github.com/marketplace/actions/start-localai
|
|
- Examples: https://github.com/mudler/LocalAI/tree/master/examples/
|
|
|
|
|
|
### 🔗 Resources
|
|
|
|
- [LLM finetuning guide](https://localai.io/docs/advanced/fine-tuning/)
|
|
- [How to build locally](https://localai.io/basics/build/index.html)
|
|
- [How to install in Kubernetes](https://localai.io/basics/getting_started/index.html#run-localai-in-kubernetes)
|
|
- [Projects integrating LocalAI](https://localai.io/docs/integrations/)
|
|
- [How tos section](https://io.midori-ai.xyz/howtos/) (curated by our community)
|
|
|
|
## :book: 🎥 [Media, Blogs, Social](https://localai.io/basics/news/#media-blogs-social)
|
|
|
|
- [Run Visual studio code with LocalAI (SUSE)](https://www.suse.com/c/running-ai-locally/)
|
|
- 🆕 [Run LocalAI on Jetson Nano Devkit](https://mudler.pm/posts/local-ai-jetson-nano-devkit/)
|
|
- [Run LocalAI on AWS EKS with Pulumi](https://www.pulumi.com/blog/low-code-llm-apps-with-local-ai-flowise-and-pulumi/)
|
|
- [Run LocalAI on AWS](https://staleks.hashnode.dev/installing-localai-on-aws-ec2-instance)
|
|
- [Create a slackbot for teams and OSS projects that answer to documentation](https://mudler.pm/posts/smart-slackbot-for-teams/)
|
|
- [LocalAI meets k8sgpt](https://www.youtube.com/watch?v=PKrDNuJ_dfE)
|
|
- [Question Answering on Documents locally with LangChain, LocalAI, Chroma, and GPT4All](https://mudler.pm/posts/localai-question-answering/)
|
|
- [Tutorial to use k8sgpt with LocalAI](https://medium.com/@tyler_97636/k8sgpt-localai-unlock-kubernetes-superpowers-for-free-584790de9b65)
|
|
|
|
## Citation
|
|
|
|
If you utilize this repository, data in a downstream project, please consider citing it with:
|
|
|
|
```
|
|
@misc{localai,
|
|
author = {Ettore Di Giacinto},
|
|
title = {LocalAI: The free, Open source OpenAI alternative},
|
|
year = {2023},
|
|
publisher = {GitHub},
|
|
journal = {GitHub repository},
|
|
howpublished = {\url{https://github.com/go-skynet/LocalAI}},
|
|
```
|
|
|
|
## ❤️ Sponsors
|
|
|
|
> Do you find LocalAI useful?
|
|
|
|
Support the project by becoming [a backer or sponsor](https://github.com/sponsors/mudler). Your logo will show up here with a link to your website.
|
|
|
|
A huge thank you to our generous sponsors who support this project covering CI expenses, and our [Sponsor list](https://github.com/sponsors/mudler):
|
|
|
|
<p align="center">
|
|
<a href="https://www.spectrocloud.com/" target="blank">
|
|
<img height="200" src="https://github.com/user-attachments/assets/72eab1dd-8b93-4fc0-9ade-84db49f24962">
|
|
</a>
|
|
<a href="https://www.premai.io/" target="blank">
|
|
<img height="200" src="https://github.com/mudler/LocalAI/assets/2420543/42e4ca83-661e-4f79-8e46-ae43689683d6"> <br>
|
|
</a>
|
|
</p>
|
|
|
|
## 🌟 Star history
|
|
|
|
[](https://star-history.com/#go-skynet/LocalAI&Date)
|
|
|
|
## 📖 License
|
|
|
|
LocalAI is a community-driven project created by [Ettore Di Giacinto](https://github.com/mudler/).
|
|
|
|
MIT - Author Ettore Di Giacinto <mudler@localai.io>
|
|
|
|
## 🙇 Acknowledgements
|
|
|
|
LocalAI couldn't have been built without the help of great software already available from the community. Thank you!
|
|
|
|
- [llama.cpp](https://github.com/ggerganov/llama.cpp)
|
|
- https://github.com/tatsu-lab/stanford_alpaca
|
|
- https://github.com/cornelk/llama-go for the initial ideas
|
|
- https://github.com/antimatter15/alpaca.cpp
|
|
- https://github.com/EdVince/Stable-Diffusion-NCNN
|
|
- https://github.com/ggerganov/whisper.cpp
|
|
- https://github.com/rhasspy/piper
|
|
|
|
## 🤗 Contributors
|
|
|
|
This is a community project, a special thanks to our contributors! 🤗
|
|
<a href="https://github.com/go-skynet/LocalAI/graphs/contributors">
|
|
<img src="https://contrib.rocks/image?repo=go-skynet/LocalAI" />
|
|
</a>
|