352 Commits

Author	SHA1	Message	Date
Ettore Di Giacinto	21bdfe5fa4	fix: use rice when embedding large binaries (#5309) ... * fix(embed): use go-rice for large backend assets Golang embed FS has a hard limit that we might exceed when providing many binary alternatives. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * simplify golang deps Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(tests): switch to testcontainers and print logs Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(tests): do not build a test binary Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * small fixup Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-05-04 16:42:42 +02:00
Ettore Di Giacinto	5c6cd50ed6	feat(llama.cpp): estimate vram usage (#5299) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-05-02 17:40:26 +02:00
Ettore Di Giacinto	6424f0666d	chore(deps): Bump edgevpn to v0.30.1 (#4840) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-02-17 16:51:22 +01:00
Dave	3cddf24747	feat: Centralized Request Processing middleware (#3847) ... * squash past, centralize request middleware PR Signed-off-by: Dave Lee <dave@gray101.com> * migrate bruno request files to examples repo Signed-off-by: Dave Lee <dave@gray101.com> * fix Signed-off-by: Dave Lee <dave@gray101.com> * Update tests/e2e-aio/e2e_test.go Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com> --------- Signed-off-by: Dave Lee <dave@gray101.com> Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com> Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>	2025-02-10 12:06:16 +01:00
Gianluca Boiano	032a33de49	chore: remove deprecated tinydream backend (#4631) ... Signed-off-by: Gianluca Boiano <morf3089@gmail.com>	2025-01-18 18:35:30 +01:00
Ettore Di Giacinto	4426efab05	chore(deps): bump edgevpn to v0.29.0 (#4564) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-01-09 09:33:22 +01:00
Ettore Di Giacinto	cea5a0ea42	feat(template): read jinja templates from gguf files (#4332) ... * Read jinja templates as fallback Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Move templating out of model loader Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Test TemplateMessages Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Set role and content from transformers Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Tests: be more flexible Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * More jinja Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Small refactoring and adaptations Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-12-08 13:50:33 +01:00
Ettore Di Giacinto	2b62260b6d	feat(models): use rwkv from llama.cpp (#4264) ... feat(rwkv): use rwkv from llama.cpp Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-11-26 14:22:55 +01:00
Ettore Di Giacinto	76d813ed1c	chore(go.mod): tidy (#4209) ... chore(go.mod): tidy Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-11-20 18:13:42 +01:00
Ettore Di Giacinto	b1ea9318e6	feat(silero): add Silero-vad backend (#4204) ... * feat(vad): add silero-vad backend (WIP) Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vad): add API endpoint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(vad): correctly place the onnxruntime libs Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(vad): hook silero-vad to binary and container builds Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gRPC): register VAD Server Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(Makefile): consume ONNX_OS consistently Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(Makefile): handle macOS Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>	2024-11-20 14:48:40 +01:00
mintyleaf	9892d7d584	feat(p2p): add support for configuration of edgevpn listen_maddrs, dht_announce_maddrs and bootstrap_peers (#4200) ... * add support for edgevpn listen_maddrs, dht_announce_maddrs, dht_bootstrap_peers * upd docs for libp2p loglevel	2024-11-20 14:18:52 +01:00
Dave	6fd0341eca	chore: update go-piper to latest (#3939) ... Signed-off-by: Dave Lee <dave@gray101.com>	2024-10-23 11:16:38 +02:00
Dave	90cacb9692	test: preliminary tests and merge fix for authv2 (#3584) ... * add api key to existing app tests, add preliminary auth test Signed-off-by: Dave Lee <dave@gray101.com> * small fix, run test Signed-off-by: Dave Lee <dave@gray101.com> * status on non-opaque Signed-off-by: Dave Lee <dave@gray101.com> * tweak auth error Signed-off-by: Dave Lee <dave@gray101.com> * exp Signed-off-by: Dave Lee <dave@gray101.com> * quick fix on real laptop Signed-off-by: Dave Lee <dave@gray101.com> * add downloader version that allows providing an auth header Signed-off-by: Dave Lee <dave@gray101.com> * stash some devcontainer fixes during testing Signed-off-by: Dave Lee <dave@gray101.com> * s2 Signed-off-by: Dave Lee <dave@gray101.com> * s Signed-off-by: Dave Lee <dave@gray101.com> * done with experiment Signed-off-by: Dave Lee <dave@gray101.com> * done with experiment Signed-off-by: Dave Lee <dave@gray101.com> * after merge fix Signed-off-by: Dave Lee <dave@gray101.com> * rename and fix Signed-off-by: Dave Lee <dave@gray101.com> --------- Signed-off-by: Dave Lee <dave@gray101.com> Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>	2024-09-24 09:32:48 +02:00
Dave	db1159b651	feat: auth v2 - supersedes #2894 (#3476) ... feat: auth v2 - supercedes #2894, metrics to follow later Signed-off-by: Dave Lee <dave@gray101.com>	2024-09-16 23:29:07 -04:00
Ettore Di Giacinto	6a6094a58d	chore(deps): update edgevpn to v0.28.3	2024-08-27 17:29:32 +02:00
Ettore Di Giacinto	8369614b6e	chore(deps): update edgevpn to v0.28.2	2024-08-27 13:03:16 +02:00
Ettore Di Giacinto	cac472d4a1	chore(deps): update edgevpn to v0.28 (#3412) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-08-27 10:48:55 +02:00
Ettore Di Giacinto	18dddc1ae0	chore(deps): update edgevpn (#3385) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-08-26 20:19:27 +02:00
Dave	ac5f6f210b	feat: external backend launching log improvements and relative path support (#3348) ... * specify workdir when launching external backend for safety / relative paths, bump version, logs Signed-off-by: Dave Lee <dave@gray101.com> * sneak in a devcontainer fix Signed-off-by: Dave Lee <dave@gray101.com> --------- Signed-off-by: Dave Lee <dave@gray101.com>	2024-08-24 00:27:14 +02:00
Ettore Di Giacinto	70e53bc191	chore(deps): update edgevpn ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-08-21 10:18:43 +02:00
Ettore Di Giacinto	16f7140461	chore(deps): update edgevpn (#3346) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-08-20 22:54:16 +02:00
Ettore Di Giacinto	a28b3771a7	chore(deps): update edgevpn (#3340) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-08-20 19:17:35 +02:00
Ettore Di Giacinto	9729d2ae37	feat(explorer): make possible to run sync in a separate process (#3224) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-08-12 19:25:44 +02:00
Ettore Di Giacinto	c4534cd908	chore(deps): update edgevpn (#3214) ... * chore(deps): update edgevpn Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix: initialize failure map Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-08-11 10:46:17 +02:00
Dave	1f7cedf5ee	build: fix go.mod - don't import ourself (#2896) ... * minor cleanup to go.mod - importing ourself? Signed-off-by: Dave Lee <dave@gray101.com> * figured out why we were importing ourself and fixed it Signed-off-by: Dave Lee <dave@gray101.com> * set pull_request_target Signed-off-by: Dave Lee <dave@gray101.com> --------- Signed-off-by: Dave Lee <dave@gray101.com> Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>	2024-07-16 22:49:43 +02:00
Ettore Di Giacinto	fc60031ac1	chore: update edgevpn dependency (#2855) ... deps: update edgevpn dependency Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-07-13 23:26:17 +00:00
Rene Leonhardt	fc87507012	chore(deps): Update Dependencies (#2538) ... * chore(deps): Update dependencies Signed-off-by: Rene Leonhardt <65483435+reneleonhardt@users.noreply.github.com> * chore(deps): Upgrade github.com/imdario/mergo to dario.cat/mergo Signed-off-by: Rene Leonhardt <65483435+reneleonhardt@users.noreply.github.com> * remove version identifiers for MeloTTS Signed-off-by: Rene Leonhardt <65483435+reneleonhardt@users.noreply.github.com> --------- Signed-off-by: Rene Leonhardt <65483435+reneleonhardt@users.noreply.github.com> Signed-off-by: Dave <dave@gray101.com> Co-authored-by: Dave <dave@gray101.com>	2024-07-12 19:54:08 +00:00
Sertaç Özercan	5866fc8ded	chore: fix go.mod module (#2635) ... Signed-off-by: Sertac Ozercan <sozercan@gmail.com>	2024-06-23 08:24:36 +00:00
Ettore Di Giacinto	f569237a50	feat(oci): support OCI images and Ollama models (#2628) ... * Support specifying oci:// and ollama:// for model URLs Fixes: https://github.com/mudler/LocalAI/issues/2527 Fixes: https://github.com/mudler/LocalAI/issues/1028 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Lower watcher warnings Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Allow to install ollama models from CLI Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fixup tests Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Do not keep file ownership Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Skip test on darwin Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-06-22 08:17:41 +02:00
Ettore Di Giacinto	aae7ad9d73	feat(llama.cpp): guess model defaults from file (#2522) ... * wip: guess informations from gguf file Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * update go mod Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Small fixups Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Identify llama3 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Do not try to guess the name, as reading gguf files can be expensive Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Allow to disable guessing Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-06-08 22:13:02 +02:00
Dave	4e1463fec2	feat: fiber CSRF (#2482) ... new config option - enables or disables the fiber csrf middleware Signed-off-by: Dave Lee <dave@gray101.com>	2024-06-04 19:43:46 +00:00
Dave	34ab442ce9	toil: bump grpc version (#2480) ... bump the grpc package version --------- Signed-off-by: Dave Lee <dave@gray101.com>	2024-06-04 08:39:19 +02:00
Ettore Di Giacinto	fdb45153fe	feat(llama.cpp): Totally decentralized, private, distributed, p2p inference (#2343) ... * feat(llama.cpp): Enable decentralized, distributed inference As https://github.com/mudler/LocalAI/pull/2324 introduced distributed inferencing thanks to @rgerganov implementation in https://github.com/ggerganov/llama.cpp/pull/6829 in upstream llama.cpp, now it is possible to distribute the workload to remote llama.cpp gRPC server. This changeset now uses mudler/edgevpn to establish a secure, distributed network between the nodes using a shared token. The token is generated automatically when starting the server with the `--p2p` flag, and can be used by starting the workers with `local-ai worker p2p-llama-cpp-rpc` by passing the token via environment variable (TOKEN) or with args (--token). As per how mudler/edgevpn works, a network is established between the server and the workers with dht and mdns discovery protocols, the llama.cpp rpc server is automatically started and exposed to the underlying p2p network so the API server can connect on. When the HTTP server is started, it will discover the workers in the network and automatically create the port-forwards to the service locally. Then llama.cpp is configured to use the services. This feature is behind the "p2p" GO_FLAGS Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * go mod tidy Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * ci: add p2p tag Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * better message Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-05-20 19:17:59 +02:00
Sertaç Özercan	e2c3ffb09b	feat: auto select llama-cpp cpu variant (#2305) ... * auto select cpu variant Signed-off-by: Sertac Ozercan <sozercan@gmail.com> * remove cuda target for now Signed-off-by: Sertac Ozercan <sozercan@gmail.com> * fix metal Signed-off-by: Sertac Ozercan <sozercan@gmail.com> * fix path Signed-off-by: Sertac Ozercan <sozercan@gmail.com> --------- Signed-off-by: Sertac Ozercan <sozercan@gmail.com>	2024-05-13 11:37:52 +02:00
Ettore Di Giacinto	b69ff46c7e	feat(startup): show CPU/GPU information with --debug (#2241) ... Signed-off-by: mudler <mudler@localai.io>	2024-05-05 09:10:23 +02:00
Dave	006306b183	fix: use bluemonday as recommended by blackfriday (#2142) ... use bluemonday as recommended by blackfriday Signed-off-by: Dave Lee <dave@gray101.com>	2024-04-26 10:34:50 +02:00
Ettore Di Giacinto	0d8bf91699	feat: Galleries UI (#2104) ... * WIP: add models to webui Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Register routes Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix: don't cache models Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * small fixups Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix: fixup multiple installs (strings.Clone) Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-04-23 09:22:58 +02:00
cryptk	8d30b39811	feat: fiber logs with zerlog and add trace level (#2082) ... Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>	2024-04-20 10:43:37 +02:00
dependabot[bot]	da82ce81b5	build(deps): bump github.com/opencontainers/runc from 1.1.5 to 1.1.12 (#2000) ... Bumps [github.com/opencontainers/runc](https://github.com/opencontainers/runc) from 1.1.5 to 1.1.12. - [Release notes](https://github.com/opencontainers/runc/releases) - [Changelog](https://github.com/opencontainers/runc/blob/main/CHANGELOG.md) - [Commits](https://github.com/opencontainers/runc/compare/v1.1.5...v1.1.12) --- updated-dependencies: - dependency-name: github.com/opencontainers/runc dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-04-11 18:57:33 +00:00
dependabot[bot]	cbda06fb96	build(deps): bump github.com/gofiber/fiber/v2 from 2.52.0 to 2.52.4 (#2008) ... Bumps [github.com/gofiber/fiber/v2](https://github.com/gofiber/fiber) from 2.52.0 to 2.52.4. - [Release notes](https://github.com/gofiber/fiber/releases) - [Commits](https://github.com/gofiber/fiber/compare/v2.52.0...v2.52.4) --- updated-dependencies: - dependency-name: github.com/gofiber/fiber/v2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-04-11 16:52:54 +00:00
dependabot[bot]	fce606fc0f	build(deps): bump github.com/charmbracelet/glamour from 0.6.0 to 0.7.0 (#2004) ... Bumps [github.com/charmbracelet/glamour](https://github.com/charmbracelet/glamour) from 0.6.0 to 0.7.0. - [Release notes](https://github.com/charmbracelet/glamour/releases) - [Commits](https://github.com/charmbracelet/glamour/compare/v0.6.0...v0.7.0) --- updated-dependencies: - dependency-name: github.com/charmbracelet/glamour dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-04-11 15:41:58 +00:00
dependabot[bot]	fdfd868953	build(deps): bump github.com/gofiber/fiber/v2 from 2.52.0 to 2.52.1 (#2001) ... Bumps [github.com/gofiber/fiber/v2](https://github.com/gofiber/fiber) from 2.52.0 to 2.52.1. - [Release notes](https://github.com/gofiber/fiber/releases) - [Commits](https://github.com/gofiber/fiber/compare/v2.52.0...v2.52.1) --- updated-dependencies: - dependency-name: github.com/gofiber/fiber/v2 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-04-11 12:21:52 +00:00
dependabot[bot]	0795975486	build(deps): bump github.com/docker/docker from 20.10.7+incompatible to 24.0.9+incompatible (#1999) ... build(deps): bump github.com/docker/docker Bumps [github.com/docker/docker](https://github.com/docker/docker) from 20.10.7+incompatible to 24.0.9+incompatible. - [Release notes](https://github.com/docker/docker/releases) - [Commits](https://github.com/docker/docker/compare/v20.10.7...v24.0.9) --- updated-dependencies: - dependency-name: github.com/docker/docker dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-04-11 11:44:34 +00:00
dependabot[bot]	a49248d29f	build(deps): bump google.golang.org/protobuf from 1.31.0 to 1.33.0 (#1998) ... Bumps google.golang.org/protobuf from 1.31.0 to 1.33.0. --- updated-dependencies: - dependency-name: google.golang.org/protobuf dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-04-11 11:07:45 +00:00
cryptk	24d7dadfed	feat: kong cli refactor fixes #1955 (#1974) ... * feat: migrate to alecthomas/kong for CLI Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat: bring in new flag for granular log levels Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * chore: go mod tidy Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat: allow loading cli flag values from ["./localai.yaml", "~/.config/localai.yaml", "/etc/localai.yaml"] in that order Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat: load from .env file instead of a yaml file Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat: better loading for environment files Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat(doc): add initial documentation about configuration Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * fix: remove test log lines Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat: integrate new documentation into existing pages Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat: add documentation on .env files Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * fix: cleanup some documentation table errors Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat: refactor CLI logic out to it's own package under core/cli Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> --------- Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>	2024-04-11 09:19:24 +02:00
Ettore Di Giacinto	123a5a2e16	feat(swagger): Add swagger API doc (#1926) ... * makefile(build): add minimal and api build target * feat(swagger): Add swagger	2024-03-29 22:29:33 +01:00
Ettore Di Giacinto	bf65ed6eb8	feat(webui): add partials, show backends associated to models (#1922) ... * feat(webui): add partials, show backends associated to models * fix(auth): put assistant and backend under auth	2024-03-28 21:52:52 +01:00
Ettore Di Giacinto	66ee4afb95	feat(welcome): add simple welcome page (#1912) ... * feat(welcome): add simple welcome page * feat(api): add 404 handling	2024-03-27 21:10:58 +01:00
Ettore Di Giacinto	4b1ee0c170	feat(aio): add tests, update model definitions (#1880)	2024-03-22 21:13:11 +01:00
Ettore Di Giacinto	e533dcf506	feat(functions/aio): all-in-one images, function template enhancements (#1862) ... * feat(startup): allow to specify models from local files * feat(aio): add Dockerfile, make targets, aio profiles * feat(template): add Function and LastMessage * add hermes2-pro-mistral * update hermes2 definition * feat(template): add sprig * feat(template): expose FunctionCall * feat(aio): switch llm for text	2024-03-21 01:12:20 +01:00