ExternalVendorCode/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2025-01-17 18:30:07 +00:00
Code Issues Actions 16 Packages Projects Releases Wiki Activity

Update _index.en.md

Browse Source 
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
This commit is contained in:
Ettore Di Giacinto 2023-12-12 00:37:01 +01:00 committed by GitHub
parent 718a5d4a9e
commit e80cbca6b0
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 57 additions and 59 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

116
docs/content/getting_started/_index.en.md
View File

@ -115,10 +115,66 @@ helm install local-ai go-skynet/local-ai -f values.yaml
{{< /tabs >}}
### Container images
LocalAI has a set of images to support CUDA, ffmpeg and 'vanilla' (CPU-only). The image list is on [quay](https://quay.io/repository/go-skynet/local-ai?tab=tags):
{{< tabs >}}
{{% tab name="Vanilla / CPU Images" %}}
- `master`
- `latest`
- `v2.0.0`
- `v2.0.0-ffmpeg`
- `v2.0.0-ffmpeg-core`
Core Images - Smaller images without predownload python dependencies
{{% /tab %}}
{{% tab name="GPU Images CUDA 11" %}}
- `master-cublas-cuda11`
- `master-cublas-cuda11-core`
- `v2.0.0-cublas-cuda11`
- `v2.0.0-cublas-cuda11-core`
- `v2.0.0-cublas-cuda11-ffmpeg`
- `v2.0.0-cublas-cuda11-ffmpeg-core`
Core Images - Smaller images without predownload python dependencies
{{% /tab %}}
{{% tab name="GPU Images CUDA 12" %}}
- `master-cublas-cuda12`
- `master-cublas-cuda12-core`
- `v2.0.0-cublas-cuda12`
- `v2.0.0-cublas-cuda12-core`
- `v2.0.0-cublas-cuda12-ffmpeg`
- `v2.0.0-cublas-cuda12-ffmpeg-core`
Core Images - Smaller images without predownload python dependencies
{{% /tab %}}
{{< /tabs >}}
Example:
- Standard (GPT + `stablediffusion`): `quay.io/go-skynet/local-ai:latest`
- FFmpeg: `quay.io/go-skynet/local-ai:v2.0.0-ffmpeg`
- CUDA 11+FFmpeg: `quay.io/go-skynet/local-ai:v2.0.0-cublas-cuda11-ffmpeg`
- CUDA 12+FFmpeg: `quay.io/go-skynet/local-ai:v2.0.0-cublas-cuda12-ffmpeg`
{{% notice note %}}
Note: the binary inside the image is pre-compiled, and might not suite all CPUs.
To enable CPU optimizations for the execution environment,
the default behavior is to rebuild when starting the container.
To disable this auto-rebuild behavior,
set the environment variable `REBUILD` to `false`.
See [docs on all environment variables]({{%relref "advanced#environment-variables" %}})
for more info.
{{% /notice %}}
### Example: Use luna-ai-llama2 model with `docker`
```bash
mkdir models
@ -180,64 +236,6 @@ You can control LocalAI with command line arguments, to specify a binding addres
| --preload-backend-only | $PRELOAD_BACKEND_ONLY | false | If set, the api is NOT launched, and only the preloaded models / backends are started. This is intended for multi-node setups. |
| --external-grpc-backends | EXTERNAL_GRPC_BACKENDS | none | Comma separated list of external gRPC backends to use. Format: `name:host:port` or `name:/path/to/file` |
### Container images
LocalAI has a set of images to support CUDA, ffmpeg and 'vanilla' (CPU-only). The image list is on [quay](https://quay.io/repository/go-skynet/local-ai?tab=tags):
{{< tabs >}}
{{% tab name="Vanilla / CPU Images" %}}
- `master`
- `latest`
- `v2.0.0`
- `v2.0.0-ffmpeg`
- `v2.0.0-ffmpeg-core`
Core Images - Smaller images without predownload python dependencies
{{% /tab %}}
{{% tab name="GPU Images CUDA 11" %}}
- `master-cublas-cuda11`
- `master-cublas-cuda11-core`
- `v2.0.0-cublas-cuda11`
- `v2.0.0-cublas-cuda11-core`
- `v2.0.0-cublas-cuda11-ffmpeg`
- `v2.0.0-cublas-cuda11-ffmpeg-core`
Core Images - Smaller images without predownload python dependencies
{{% /tab %}}
{{% tab name="GPU Images CUDA 12" %}}
- `master-cublas-cuda12`
- `master-cublas-cuda12-core`
- `v2.0.0-cublas-cuda12`
- `v2.0.0-cublas-cuda12-core`
- `v2.0.0-cublas-cuda12-ffmpeg`
- `v2.0.0-cublas-cuda12-ffmpeg-core`
Core Images - Smaller images without predownload python dependencies
{{% /tab %}}
{{< /tabs >}}
Example:
- Standard (GPT + `stablediffusion`): `quay.io/go-skynet/local-ai:latest`
- FFmpeg: `quay.io/go-skynet/local-ai:v2.0.0-ffmpeg`
- CUDA 11+FFmpeg: `quay.io/go-skynet/local-ai:v2.0.0-cublas-cuda11-ffmpeg`
- CUDA 12+FFmpeg: `quay.io/go-skynet/local-ai:v2.0.0-cublas-cuda12-ffmpeg`
{{% notice note %}}
Note: the binary inside the image is pre-compiled, and might not suite all CPUs.
To enable CPU optimizations for the execution environment,
the default behavior is to rebuild when starting the container.
To disable this auto-rebuild behavior,
set the environment variable `REBUILD` to `false`.
See [docs on all environment variables]({{%relref "advanced#environment-variables" %}})
for more info.
{{% /notice %}}
### Run LocalAI in Kubernetes
LocalAI can be installed inside Kubernetes with helm.