🤖 The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed, P2P inference
Go to file
Ettore Di Giacinto 3aff87a5cf
chore(model gallery): add qwen2.5-math-14b-instruct ()
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-12-10 09:42:24 +01:00
.bruno/LocalAI Test Requests chore: drop examples folder now that LocalAI-examples has been created () 2024-10-30 09:10:33 +01:00
.devcontainer feat: devcontainer part 4 () 2024-08-20 19:25:22 +02:00
.devcontainer-scripts test: preliminary tests and merge fix for authv2 () 2024-09-24 09:32:48 +02:00
.github chore(ci): set auto-labeler for dependencies 2024-12-04 18:35:54 +01:00
.vscode feat: Initial Version of vscode DevContainer () 2024-08-14 09:06:41 +02:00
aio feat(backends): Drop bert.cpp () 2024-11-27 16:34:28 +01:00
backend feat(llama.cpp): expose cache_type_k and cache_type_v for quant of kv cache () 2024-12-06 10:23:59 +01:00
configuration refactor: move remaining api packages to core () 2024-03-01 16:19:53 +01:00
core Revert "feat: include tokens usage for streamed output" () 2024-12-08 17:53:36 +01:00
custom-ca-certs feat(certificates): add support for custom CA certificates () 2023-11-01 20:10:14 +01:00
docs chore(deps): Bump docs/themes/hugo-theme-relearn from be85052 to bd1f3d3 () 2024-12-09 20:09:26 +00:00
embedded feat(backends): Drop bert.cpp () 2024-11-27 16:34:28 +01:00
examples chore: create examples/README to redirect to the new repository 2024-10-30 09:11:32 +01:00
gallery chore(model gallery): add qwen2.5-math-14b-instruct () 2024-12-10 09:42:24 +01:00
internal feat: cleanups, small enhancements 2023-07-04 18:58:19 +02:00
models Add docker-compose 2023-04-13 01:13:14 +02:00
pkg Revert "feat: include tokens usage for streamed output" () 2024-12-08 17:53:36 +01:00
prompt-templates Requested Changes from GPT4ALL to Luna-AI-Llama2 () 2023-09-22 11:22:17 +02:00
scripts chore(scripts): handle summarization errors () 2024-11-26 14:51:55 +01:00
swagger feat(swagger): update swagger () 2024-11-20 23:10:51 +01:00
tests fix(rwkv model): add stoptoken () 2024-11-28 09:34:35 +01:00
.dockerignore feat: Initial Version of vscode DevContainer () 2024-08-14 09:06:41 +02:00
.editorconfig feat(stores): Vector store backend () 2024-03-22 21:14:04 +01:00
.env feat: Initial Version of vscode DevContainer () 2024-08-14 09:06:41 +02:00
.gitattributes chore(linguist): add *.hpp files to linguist-vendored () 2024-11-14 14:12:16 +01:00
.gitignore feat(bark-cpp): add new bark.cpp backend () 2024-11-28 22:16:44 +01:00
.gitmodules docs/examples: enhancements () 2024-01-18 19:41:08 +01:00
.yamllint fix: yamlint warnings and errors () 2024-04-25 17:25:56 +00:00
assets.go feat: Update gpt4all, support multiple implementations in runtime () 2023-06-01 23:38:52 +02:00
CONTRIBUTING.md Update CONTRIBUTING.md () 2024-10-03 20:03:35 +02:00
docker-compose.yaml feat: Initial Version of vscode DevContainer () 2024-08-14 09:06:41 +02:00
Dockerfile fix(container-images): install uv as system package () 2024-11-08 11:47:43 +01:00
Dockerfile.aio feat(aio): entrypoint, update workflows () 2024-03-21 22:09:04 +01:00
Earthfile Rename project to LocalAI () 2023-04-19 18:43:10 +02:00
Entitlements.plist Feat: OSX Local Codesigning () 2023-11-23 15:22:54 +01:00
entrypoint.sh deps(llama.cpp): bump to latest, update build variables () 2024-06-27 23:10:04 +02:00
go.mod feat(template): read jinja templates from gguf files () 2024-12-08 13:50:33 +01:00
go.sum feat(template): read jinja templates from gguf files () 2024-12-08 13:50:33 +01:00
LICENSE docs/examples: enhancements () 2024-01-18 19:41:08 +01:00
main.go chore: fix go.mod module () 2024-06-23 08:24:36 +00:00
Makefile chore: ⬆️ Update ggerganov/llama.cpp to 26a8406ba9198eb6fdd8329fa717555b4f77f05f () 2024-12-10 09:10:58 +01:00
README.md Update README.md 2024-12-04 11:31:08 +01:00
renovate.json ci: manually update deps 2023-05-04 15:01:29 +02:00
SECURITY.md Create SECURITY.md 2024-02-29 19:53:04 +01:00



LocalAI

LocalAI forks LocalAI stars LocalAI pull-requests

LocalAI Docker hub LocalAI Quay.io

Follow LocalAI_API Join LocalAI Discord Community

go-skynet%2FLocalAI | Trendshift

💡 Get help - FAQ 💭Discussions 💬 Discord 📖 Documentation website

💻 Quickstart 🖼️ Models 🚀 Roadmap 🥽 Demo 🌍 Explorer 🛫 Examples

testsBuild and Releasebuild container imagesBump dependenciesArtifact Hub

LocalAI is the free, Open Source OpenAI alternative. LocalAI act as a drop-in replacement REST API thats compatible with OpenAI (Elevenlabs, Anthropic... ) API specifications for local AI inferencing. It allows you to run LLMs, generate images, audio (and not only) locally or on-prem with consumer grade hardware, supporting multiple model families. Does not require GPU. It is created and maintained by Ettore Di Giacinto.

screen

Run the installer script:

curl https://localai.io/install.sh | sh

Or run with docker:

# CPU only image:
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-cpu

# Nvidia GPU:
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12

# CPU and GPU image (bigger size):
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest

# AIO images (it will pre-download a set of models ready for use, see https://localai.io/basics/container/)
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-aio-cpu

To load models:

# From the model gallery (see available models with `local-ai models list`, in the WebUI from the model tab, or visiting https://models.localai.io)
local-ai run llama-3.2-1b-instruct:q4_k_m
# Start LocalAI with the phi-2 model directly from huggingface
local-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf
# Install and run a model from the Ollama OCI registry
local-ai run ollama://gemma:2b
# Run a model from a configuration file
local-ai run https://gist.githubusercontent.com/.../phi-2.yaml
# Install and run a model from a standard OCI registry (e.g., Docker Hub)
local-ai run oci://localai/phi-2:latest

💻 Getting started

📰 Latest project news

Roadmap items: List of issues

🔥🔥 Hot topics (looking for help):

If you want to help and contribute, issues up for grabs: https://github.com/mudler/LocalAI/issues?q=is%3Aissue+is%3Aopen+label%3A%22up+for+grabs%22

🚀 Features

💻 Usage

Check out the Getting started section in our documentation.

🔗 Community and integrations

Build and deploy custom containers:

WebUIs:

Model galleries

Other:

🔗 Resources

📖 🎥 Media, Blogs, Social

Citation

If you utilize this repository, data in a downstream project, please consider citing it with:

@misc{localai,
  author = {Ettore Di Giacinto},
  title = {LocalAI: The free, Open source OpenAI alternative},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/go-skynet/LocalAI}},

❤️ Sponsors

Do you find LocalAI useful?

Support the project by becoming a backer or sponsor. Your logo will show up here with a link to your website.

A huge thank you to our generous sponsors who support this project covering CI expenses, and our Sponsor list:


🌟 Star history

LocalAI Star history Chart

📖 License

LocalAI is a community-driven project created by Ettore Di Giacinto.

MIT - Author Ettore Di Giacinto mudler@localai.io

🙇 Acknowledgements

LocalAI couldn't have been built without the help of great software already available from the community. Thank you!

🤗 Contributors

This is a community project, a special thanks to our contributors! 🤗