mirror of
https://github.com/mudler/LocalAI.git
synced 2025-04-09 12:21:31 +00:00
docs: Add llava, update hot topics (#1322)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
parent
69f53211a1
commit
ba5ab26f2e
11
README.md
11
README.md
@ -22,7 +22,7 @@
|
||||
|
||||
> :bulb: Get help - [❓FAQ](https://localai.io/faq/) [💭Discussions](https://github.com/go-skynet/LocalAI/discussions) [:speech_balloon: Discord](https://discord.gg/uJAeKSAGDy) [:book: Documentation website](https://localai.io/)
|
||||
>
|
||||
> [💻 Quickstart](https://localai.io/basics/getting_started/) [📣 News](https://localai.io/basics/news/) [ 🛫 Examples ](https://github.com/go-skynet/LocalAI/tree/master/examples/) [ 🖼️ Models ](https://localai.io/models/)
|
||||
> [💻 Quickstart](https://localai.io/basics/getting_started/) [📣 News](https://localai.io/basics/news/) [ 🛫 Examples ](https://github.com/go-skynet/LocalAI/tree/master/examples/) [ 🖼️ Models ](https://localai.io/models/) [ 🚀 Roadmap ](https://github.com/mudler/LocalAI/issues?q=is%3Aissue+is%3Aopen+label%3Aroadmap)
|
||||
|
||||
|
||||
[](https://github.com/go-skynet/LocalAI/actions/workflows/test.yml)[](https://github.com/go-skynet/LocalAI/actions/workflows/release.yaml)[](https://github.com/go-skynet/LocalAI/actions/workflows/image.yml)[](https://github.com/go-skynet/LocalAI/actions/workflows/bump_deps.yaml)[](https://artifacthub.io/packages/search?repo=localai)
|
||||
@ -79,7 +79,12 @@ LocalAI was created by [Ettore Di Giacinto](https://github.com/mudler/) and is a
|
||||
|
||||
Note that this started just as a [fun weekend project](https://localai.io/#backstory) in order to try to create the necessary pieces for a full AI assistant like `ChatGPT`: the community is growing fast and we are working hard to make it better and more stable. If you want to help, please consider contributing (see below)!
|
||||
|
||||
## 🔥🔥 [Hot topics / Roadmap](https://localai.io/#-hot-topics--roadmap)
|
||||
## 🔥🔥 Hot topics / Roadmap
|
||||
|
||||
- [Roadmap](https://github.com/mudler/LocalAI/issues?q=is%3Aissue+is%3Aopen+label%3Aroadmap)
|
||||
|
||||
Hot topics:
|
||||
- https://github.com/mudler/LocalAI/issues/1126
|
||||
|
||||
## 🚀 [Features](https://localai.io/features/)
|
||||
|
||||
@ -91,7 +96,7 @@ Note that this started just as a [fun weekend project](https://localai.io/#backs
|
||||
- 🧠 [Embeddings generation for vector databases](https://localai.io/features/embeddings/)
|
||||
- ✍️ [Constrained grammars](https://localai.io/features/constrained_grammars/)
|
||||
- 🖼️ [Download Models directly from Huggingface ](https://localai.io/models/)
|
||||
|
||||
- 🆕 [Vision API](https://localai.io/features/gpt-vision/)
|
||||
|
||||
## :book: 🎥 [Media, Blogs, Social](https://localai.io/basics/news/#media-blogs-social)
|
||||
|
||||
|
@ -20,7 +20,7 @@ title = "LocalAI"
|
||||
|
||||
> 💡 Get help - [❓FAQ](https://localai.io/faq/) [❓How tos](https://localai.io/howtos/) [💭Discussions](https://github.com/go-skynet/LocalAI/discussions) [💭Discord](https://discord.gg/uJAeKSAGDy)
|
||||
>
|
||||
> [💻 Quickstart](https://localai.io/basics/getting_started/) [📣 News](https://localai.io/basics/news/) [ 🛫 Examples ](https://github.com/go-skynet/LocalAI/tree/master/examples/) [ 🖼️ Models ](https://localai.io/models/)
|
||||
> [💻 Quickstart](https://localai.io/basics/getting_started/) [📣 News](https://localai.io/basics/news/) [ 🛫 Examples ](https://github.com/go-skynet/LocalAI/tree/master/examples/) [ 🖼️ Models ](https://localai.io/models/) [ 🚀 Roadmap ](https://github.com/mudler/LocalAI/issues?q=is%3Aissue+is%3Aopen+label%3Aroadmap)
|
||||
|
||||
**LocalAI** is the free, Open Source OpenAI alternative. LocalAI act as a drop-in replacement REST API that's compatible with OpenAI API specifications for local inferencing. It allows you to run LLMs, generate images, audio (and not only) locally or on-prem with consumer grade hardware, supporting multiple model families that are compatible with the ggml format. Does not require GPU. It is maintained by [mudler](https://github.com/mudler).
|
||||
|
||||
@ -84,37 +84,15 @@ Note that this started just as a [fun weekend project](https://localai.io/#backs
|
||||
- 🧠 [Embeddings generation for vector databases](https://localai.io/features/embeddings/)
|
||||
- ✍️ [Constrained grammars](https://localai.io/features/constrained_grammars/)
|
||||
- 🖼️ [Download Models directly from Huggingface ](https://localai.io/models/)
|
||||
- 🆕 [Vision API](https://localai.io/features/gpt-vision/)
|
||||
|
||||
|
||||
## 🔥🔥 Hot topics / Roadmap
|
||||
|
||||
- Backend rework for LocalAI v2 https://github.com/go-skynet/LocalAI/issues/1126
|
||||
- Roadmap: https://github.com/go-skynet/LocalAI/issues?q=is%3Aissue+is%3Aopen+label%3Aroadmap
|
||||
|
||||
Some items:
|
||||
|
||||
- [ ] Enable gallery management directly from the webui. https://github.com/go-skynet/LocalAI/issues/918
|
||||
- [x] llama.cpp lora adapters: https://github.com/go-skynet/LocalAI/issues/919
|
||||
- [ ] image variants and edits: https://github.com/go-skynet/LocalAI/issues/921
|
||||
- [ ] esrgan for diffusers: https://github.com/go-skynet/LocalAI/issues/917
|
||||
- [ ] ggml-stablediffusion: https://github.com/go-skynet/LocalAI/issues/916
|
||||
- [ ] SAM: https://github.com/go-skynet/LocalAI/issues/915
|
||||
- [x] diffusers lora adapters: https://github.com/go-skynet/LocalAI/issues/914
|
||||
- [ ] resource management and control: https://github.com/go-skynet/LocalAI/issues/912
|
||||
- [ ] ChatGLM: https://github.com/go-skynet/LocalAI/issues/754
|
||||
- [ ] text-to-video : https://github.com/go-skynet/LocalAI/issues/933
|
||||
- [ ] rustformers: https://github.com/go-skynet/LocalAI/issues/939
|
||||
- [x] Vall-e: https://github.com/go-skynet/LocalAI/issues/985
|
||||
- [ ] Speculative sampling: https://github.com/go-skynet/LocalAI/issues/1013
|
||||
- [ ] Falcon/GPTNeoX on llama.cpp: https://github.com/go-skynet/LocalAI/issues/1009
|
||||
- [ ] transformers/vllm: https://github.com/go-skynet/LocalAI/issues/1015
|
||||
- [ ] TortoiseTTS: https://github.com/go-skynet/LocalAI/issues/1016
|
||||
- [ ] Exllama2: https://github.com/go-skynet/LocalAI/issues/1053
|
||||
- [ ] ctransformers: https://github.com/go-skynet/LocalAI/issues/1056
|
||||
- [ ] GPTQ for LLama: https://github.com/go-skynet/LocalAI/issues/1055
|
||||
- [ ] LLaVA and miniGPT-4: https://github.com/go-skynet/LocalAI/issues/1054
|
||||
- [ ] Test generation inference: https://github.com/go-skynet/LocalAI/issues/1042
|
||||
- [ ] docs - extending LocalAI with external backends: https://github.com/go-skynet/LocalAI/issues/1057
|
||||
- [Roadmap](https://github.com/mudler/LocalAI/issues?q=is%3Aissue+is%3Aopen+label%3Aroadmap)
|
||||
|
||||
Hot topics:
|
||||
- https://github.com/mudler/LocalAI/issues/1126
|
||||
|
||||
## How does it work?
|
||||
|
||||
|
@ -13,4 +13,5 @@ This section contains the documentation for the features supported by LocalAI.
|
||||
- [🎨 Image generation]({{%relref "features/image-generation" %}})
|
||||
- [🧠 Embeddings]({{%relref "features/embeddings" %}})
|
||||
- [🔥 OpenAI functions]({{%relref "features/openai-functions" %}})
|
||||
- [🆕 GPT Vision API]({{%relref "features/gpt-vision" %}})
|
||||
- [✍️ Constrained grammars]({{%relref "features/constrained_grammars" %}})
|
||||
|
30
docs/content/features/gpt-vision.md
Normal file
30
docs/content/features/gpt-vision.md
Normal file
@ -0,0 +1,30 @@
|
||||
|
||||
+++
|
||||
disableToc = false
|
||||
title = "🆕 GPT Vision"
|
||||
weight = 2
|
||||
+++
|
||||
|
||||
{{% notice note %}}
|
||||
Available only on `master` builds
|
||||
{{% /notice %}}
|
||||
|
||||
LocalAI supports understanding images by using [LLaVA](https://llava.hliu.cc/), and implements the [GPT Vision API](https://platform.openai.com/docs/guides/vision) from OpenAI.
|
||||
|
||||

|
||||
|
||||
## Usage
|
||||
|
||||
OpenAI docs: https://platform.openai.com/docs/guides/vision
|
||||
|
||||
To let LocalAI understand and reply with what sees in the image, use the `/v1/chat/completions` endpoint, for example with curl:
|
||||
|
||||
```bash
|
||||
curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
|
||||
"model": "llava",
|
||||
"messages": [{"role": "user", "content": [{"type":"text", "text": "What is in the image?"}, {"type": "image_url", "image_url": {"url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg" }}], "temperature": 0.9}]}'
|
||||
```
|
||||
|
||||
### Setup
|
||||
|
||||
To setup the LLaVa models, follow the full example in the [configuration examples](https://github.com/mudler/LocalAI/blob/master/examples/configurations/README.md#llava).
|
@ -1,7 +1,7 @@
|
||||
|
||||
+++
|
||||
disableToc = false
|
||||
title = "🆕 Vall-E-X"
|
||||
title = "Vall-E-X"
|
||||
weight = 4
|
||||
+++
|
||||
|
||||
|
@ -1,7 +1,7 @@
|
||||
|
||||
+++
|
||||
disableToc = false
|
||||
title = "🆕 vLLM"
|
||||
title = "vLLM"
|
||||
weight = 4
|
||||
+++
|
||||
|
||||
|
Loading…
x
Reference in New Issue
Block a user