LocalAI/pkg
Sertaç Özercan a670318a9f
feat: auto select llama-cpp cuda runtime (#2306)
* auto select cpu variant

Signed-off-by: Sertac Ozercan <sozercan@gmail.com>

* remove cuda target for now

Signed-off-by: Sertac Ozercan <sozercan@gmail.com>

* fix metal

Signed-off-by: Sertac Ozercan <sozercan@gmail.com>

* fix path

Signed-off-by: Sertac Ozercan <sozercan@gmail.com>

* cuda

Signed-off-by: Sertac Ozercan <sozercan@gmail.com>

* auto select cuda

Signed-off-by: Sertac Ozercan <sozercan@gmail.com>

* update test

Signed-off-by: Sertac Ozercan <sozercan@gmail.com>

* select CUDA backend only if present

Signed-off-by: mudler <mudler@localai.io>

* ci: keep cuda bin in path

Signed-off-by: mudler <mudler@localai.io>

* Makefile: make dist now builds also cuda

Signed-off-by: mudler <mudler@localai.io>

* Keep pushing fallback in case auto-flagset/nvidia fails

There could be other reasons for which the default binary may fail. For example we might have detected an Nvidia GPU,
however the user might not have the drivers/cuda libraries installed in the system, and so it would fail to start.

We keep the fallback of llama.cpp at the end of the llama.cpp backends to try to fallback loading in case things go wrong

Signed-off-by: mudler <mudler@localai.io>

* Do not build cuda on MacOS

Signed-off-by: mudler <mudler@localai.io>

* cleanup

Signed-off-by: Sertac Ozercan <sozercan@gmail.com>

* Apply suggestions from code review

Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>

---------

Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
Signed-off-by: mudler <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
Co-authored-by: mudler <mudler@localai.io>
2024-05-14 19:40:18 +02:00
..
assets fix: security scanner warning noise: error handlers part 1 (#2141) 2024-04-26 10:34:31 +02:00
downloader fix: reduce chmod permissions for created files and directories (#2137) 2024-04-26 00:47:06 +02:00
functions feat(functions): allow to set JSON matcher (#2319) 2024-05-14 09:39:20 +02:00
gallery feat(ui): prompt for chat, support vision, enhancements (#2259) 2024-05-08 00:42:34 +02:00
grpc refactor(application): introduce application global state (#2072) 2024-04-29 17:42:37 +00:00
langchain feat(llama.cpp): do not specify backends to autoload and add llama.cpp variants (#2232) 2024-05-04 17:56:12 +02:00
model feat: auto select llama-cpp cuda runtime (#2306) 2024-05-14 19:40:18 +02:00
stablediffusion feat: support upscaled image generation with esrgan (#509) 2023-06-05 17:21:38 +02:00
startup feat: Galleries UI (#2104) 2024-04-23 09:22:58 +02:00
store feat(stores): Vector store backend (#1795) 2024-03-22 21:14:04 +01:00
templates fix: reduce chmod permissions for created files and directories (#2137) 2024-04-26 00:47:06 +02:00
tinydream feat: add tiny dream stable diffusion support (#1283) 2023-12-24 19:27:24 +00:00
utils refactor(application): introduce application global state (#2072) 2024-04-29 17:42:37 +00:00
xsync feat(ui): prompt for chat, support vision, enhancements (#2259) 2024-05-08 00:42:34 +02:00
xsysinfo feat(startup): show CPU/GPU information with --debug (#2241) 2024-05-05 09:10:23 +02:00