🤖 The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed, P2P inference
Go to file
Ettore Di Giacinto 31ed13094b
Update README.md
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
2023-09-16 23:00:42 +02:00
.github chore(deps): update docker/metadata-action action to v5 () 2023-09-14 10:40:51 +02:00
.vscode feat: Add more test-cases and remove dev container () 2023-05-30 13:01:55 +02:00
api feat(speculative-sampling): allow to specify a draft model in the model config () 2023-09-14 17:44:16 +02:00
cmd/grpc feat: add llama-stable backend () 2023-08-20 16:35:42 +02:00
examples Update README to reflect changes in Continue's config file () 2023-09-07 16:29:07 +02:00
extra feat(speculative-sampling): allow to specify a draft model in the model config () 2023-09-14 17:44:16 +02:00
internal feat: cleanups, small enhancements 2023-07-04 18:58:19 +02:00
models Add docker-compose 2023-04-13 01:13:14 +02:00
pkg feat(speculative-sampling): allow to specify a draft model in the model config () 2023-09-14 17:44:16 +02:00
prompt-templates feat(llama2): add template for chat messages () 2023-07-22 11:31:39 -04:00
tests feat(llama2): add template for chat messages () 2023-07-22 11:31:39 -04:00
.dockerignore Remove .git from .dockerignore 2023-07-06 21:25:10 +02:00
.env feat(llama.cpp): update () 2023-09-08 18:38:22 +02:00
.gitattributes Create .gitattributes to force git clone to keep the LF line endings on .sh files () 2023-07-30 15:27:43 +02:00
.gitignore Feat: rwkv improvements: () 2023-08-22 18:48:06 +02:00
assets.go feat: Update gpt4all, support multiple implementations in runtime () 2023-06-01 23:38:52 +02:00
docker-compose.yaml images: cleanup, drop .dev Dockerfile () 2023-05-30 15:58:10 +02:00
Dockerfile feat(vllm): Initial vllm backend implementation 2023-09-09 17:03:23 +02:00
Earthfile Rename project to LocalAI () 2023-04-19 18:43:10 +02:00
entrypoint.sh Added CPU information to entrypoint.sh () 2023-07-23 19:27:55 +00:00
go.mod fix(deps): update github.com/nomic-ai/gpt4all/gpt4all-bindings/golang digest to cf4eb53 () 2023-09-14 10:41:07 +02:00
go.sum fix(deps): update github.com/nomic-ai/gpt4all/gpt4all-bindings/golang digest to cf4eb53 () 2023-09-14 10:41:07 +02:00
LICENSE docs: update docs/license(clarification) and point to new website () 2023-05-29 23:09:19 +02:00
main.go feat: add --single-active-backend to allow only one backend active at the time () 2023-08-19 01:49:33 +02:00
Makefile ⬆️ Update go-skynet/go-llama.cpp () 2023-09-14 10:40:36 +02:00
README.md Update README.md 2023-09-16 23:00:42 +02:00
renovate.json ci: manually update deps 2023-05-04 15:01:29 +02:00



LocalAI

LocalAI forks LocalAI stars LocalAI pull-requests

💡 Get help - FAQ 💭Discussions 💬 Discord 📖 Documentation website

💻 Quickstart 📣 News 🛫 Examples 🖼️ Models

testsBuild and Releasebuild container imagesBump dependenciesArtifact Hub

LocalAI is a drop-in replacement REST API that's compatible with OpenAI API specifications for local inferencing. It allows you to run LLMs (and not only) locally or on-prem with consumer grade hardware, supporting multiple model families that are compatible with the ggml format, pytorch and more. Does not require GPU.

Follow LocalAI

Follow LocalAI_API Join LocalAI Discord Community

Connect with the Creator

Follow mudler_it Follow on Github

Share LocalAI Repository

Follow _LocalAI Share on Telegram Share on Reddit Buy Me A Coffee


In a nutshell:

  • Local, OpenAI drop-in alternative REST API. You own your data.
  • NO GPU required. NO Internet access is required either
    • Optional, GPU Acceleration is available in llama.cpp-compatible LLMs. See also the build section.
  • Supports multiple models
  • 🏃 Once loaded the first time, it keep models loaded in memory for faster inference
  • Doesn't shell-out, but uses C++ bindings for a faster inference and better performance.

LocalAI was created by Ettore Di Giacinto and is a community-driven project, focused on making the AI accessible to anyone. Any contribution, feedback and PR is welcome!

Note that this started just as a fun weekend project in order to try to create the necessary pieces for a full AI assistant like ChatGPT: the community is growing fast and we are working hard to make it better and more stable. If you want to help, please consider contributing (see below)!

🔥🔥 Hot topics / Roadmap

🚀 Features

📖 🎥 Media, Blogs, Social

💻 Usage

Check out the Getting started section in our documentation.

💡 Example: Use GPT4ALL-J model

See the documentation

🔗 Resources

Citation

If you utilize this repository, data in a downstream project, please consider citing it with:

@misc{localai,
  author = {Ettore Di Giacinto},
  title = {LocalAI: The free, Open source OpenAI alternative},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/go-skynet/LocalAI}},

❤️ Sponsors

Do you find LocalAI useful?

Support the project by becoming a backer or sponsor. Your logo will show up here with a link to your website.

A huge thank you to our generous sponsors who support this project:

Spectro Cloud logo_600x600px_transparent bg
Spectro Cloud
Spectro Cloud kindly supports LocalAI by providing GPU and computing resources to run tests on lamdalabs!

And a huge shout-out to individuals sponsoring the project by donating hardware or backing the project.

🌟 Star history

LocalAI Star history Chart

📖 License

LocalAI is a community-driven project created by Ettore Di Giacinto.

MIT - Author Ettore Di Giacinto

🙇 Acknowledgements

LocalAI couldn't have been built without the help of great software already available from the community. Thank you!

🤗 Contributors

This is a community project, a special thanks to our contributors! 🤗