The binary grew up to 1.8GB quickly - rocm at least raises +800MB by
itself - so we might just want to manage the GPU libs separately.
Adds a comment to list all the libraries found so far that we are
depending on, but will likely follow up in a way to bundle these
separately.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This PR bundles further libs into the arm64 and x86_64 binaries
This can be improved by a lot - it's far from perfect, however in this PR I wanted to collect the required libs, and give a simple baseline to improve later upon. It is quite challenging to do this exercise with CI only - but it's the fastest way I see now.
I hope that after the list is initially built we can further improve this down the line and remove some of the technical debt left here to speedup things and do not get stuck in the middle of CI cycles.
In this PR:
- The x86_64 binary now bundles hipblas, nvidia and intel libraries too to avoid any dependency to be installed in the host
- Similarly, for the arm64 we now bundle all the required assets
## What's left
We should be also able to cross-compile Nvidia for arm64 - however I didn't succeed so far so I've left that open. Similarly I might have missed some libraries, but we will see with bug reports and testing around with the new binaries. I've tested on my arm64 board and I could finally start things up.
An open point still is shipping libraries for e.g. tts and stablediffusion. this is not done yet, however with the same methodology we should be able to extend support also for these two backends in the binary.
When offering fallback libs, use the proper env var for darwin
Note: this does not include the libraries itself, but only sets the
proper env var for the libs to be picked up on darwin.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* WIP
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* gen a static page instead (we force DNS redirects to it)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(gallery): install models from CLI, unify install
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Uniform graphic of model page
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Makefile: update targets
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Slightly enhance gallery view
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* ci: try to build for arm64
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Allow to skip hipblas on make dist
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* use arm64 cross compiler
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* correctly target go arm64
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* create a separate target
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* cross-compile grpc
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Add Protobuf include dirs
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* temp disable CUDA build
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* aarch64 builds: Reduce backends
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Even less backends
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Even less backends
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(startup): allow to load libs from extracted assets
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* makefile: set arch
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(ui): add page to talk with voice, transcription, and tts
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Enhance graphics and status reporting
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Better UX by blocking unvalid actions
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* wip: guess informations from gguf file
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* update go mod
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Small fixups
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Identify llama3
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Do not try to guess the name, as reading gguf files can be expensive
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Allow to disable guessing
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
