mirror of
https://github.com/mudler/LocalAI.git
synced 2024-12-30 09:28:51 +00:00
57 lines
2.4 KiB
Markdown
57 lines
2.4 KiB
Markdown
# Continue
|
|
|
|
|
|
|
|
This document presents an example of integration with [continuedev/continue](https://github.com/continuedev/continue).
|
|
|
|
|
|
|
|
For a live demonstration, please click on the link below:
|
|
|
|
- [How it works (Video demonstration)](https://www.youtube.com/watch?v=3Ocrc-WX4iQ)
|
|
|
|
## Integration Setup Walkthrough
|
|
|
|
1. [As outlined in `continue`'s documentation](https://continue.dev/docs/getting-started), install the [Visual Studio Code extension from the marketplace](https://marketplace.visualstudio.com/items?itemName=Continue.continue) and open it.
|
|
2. In this example, LocalAI will download the gpt4all model and set it up as "gpt-3.5-turbo". Refer to the `docker-compose.yaml` file for details.
|
|
|
|
```bash
|
|
# Clone LocalAI
|
|
git clone https://github.com/go-skynet/LocalAI
|
|
|
|
cd LocalAI/examples/continue
|
|
|
|
# Start with docker-compose
|
|
docker-compose up --build -d
|
|
```
|
|
|
|
3. Type `/config` within Continue's VSCode extension, or edit the file located at `~/.continue/config.py` on your system with the following configuration:
|
|
|
|
```py
|
|
from continuedev.src.continuedev.libs.llm.openai import OpenAI, OpenAIServerInfo
|
|
|
|
config = ContinueConfig(
|
|
...
|
|
models=Models(
|
|
default=OpenAI(
|
|
api_key="my-api-key",
|
|
model="gpt-3.5-turbo",
|
|
openai_server_info=OpenAIServerInfo(
|
|
api_base="http://localhost:8080",
|
|
model="gpt-3.5-turbo"
|
|
)
|
|
)
|
|
),
|
|
)
|
|
```
|
|
|
|
This setup enables you to make queries directly to your model running in the Docker container. Note that the `api_key` does not need to be properly set up; it is included here as a placeholder.
|
|
|
|
If editing the configuration seems confusing, you may copy and paste the provided default `config.py` file over the existing one in `~/.continue/config.py` after initializing the extension in the VSCode IDE.
|
|
|
|
## Additional Resources
|
|
|
|
- [Official Continue documentation](https://continue.dev/docs/intro)
|
|
- [Documentation page on using self-hosted models](https://continue.dev/docs/customization#self-hosting-an-open-source-model)
|
|
- [Official extension link](https://marketplace.visualstudio.com/items?itemName=Continue.continue)
|