248 Commits

Author	SHA1	Message	Date
Ettore Di Giacinto	1d6afbd65d	feat(llama.cpp): Add support to grammar triggers (#4733) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-02-02 13:25:03 +01:00
Ettore Di Giacinto	af41436f1b	fix(tests): pin to branch for config used in tests (#4721) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-01-31 09:57:58 +01:00
Ettore Di Giacinto	72e52c4f6a	chore: drop embedded models (#4715) ... Since the remote gallery was introduced this is now completely superseded by it. In order to keep the code clean and remove redudant parts let's simplify the usage. Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-01-30 00:03:01 +01:00
Peter Cover	a05737c7e4	chore: fix some function names in comment (#4665) ... Signed-off-by: petercover <raowanxiang@outlook.com>	2025-01-22 19:35:53 +01:00
Ettore Di Giacinto	e15d29aba2	chore(stablediffusion-ncn): drop in favor of ggml implementation (#4652) ... * chore(stablediffusion-ncn): drop in favor of ggml implementation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(ci): drop stablediffusion build Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(tests): add Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(tests): try to fixup current tests Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Try to fix tests Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Tests improvements Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(tests): use quality to specify step Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(tests): switch to sd-1.5 also increase prep time for downloading models Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-01-22 19:34:16 +01:00
Gianluca Boiano	032a33de49	chore: remove deprecated tinydream backend (#4631) ... Signed-off-by: Gianluca Boiano <morf3089@gmail.com>	2025-01-18 18:35:30 +01:00
Ettore Di Giacinto	1e9bf19c8d	feat(transformers): merge sentencetransformers backend (#4624) ... * merge sentencetransformers Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add alias to silently redirect sentencetransformers to transformers Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add alias also for transformers-musicgen Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Drop from makefile Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Move tests from sentencetransformers Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Remove sentencetransformers Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Remove tests from CI (part of transformers) Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Do not always try to load the tokenizer Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Adapt tests Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Fix typo Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Tiny adjustments Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-01-18 18:30:30 +01:00
mintyleaf	96306a39a0	chore(docs): extra-Usage and Machine-Tag docs (#4627) ... Rename LocalAI-Extra-Usage -> Extra-Usage, add MACHINE_TAG as cli flag option, add docs about extra-usage and machine-tag Signed-off-by: mintyleaf <mintyleafdev@gmail.com>	2025-01-18 08:58:38 +01:00
mintyleaf	96f8ec0402	feat: add machine tag and inference timings (#4577) ... * Add machine tag option, add extraUsage option, grpc-server -> proto -> endpoint extraUsage data is broken for now Signed-off-by: mintyleaf <mintyleafdev@gmail.com> * remove redurant timing fields, fix not working timings output Signed-off-by: mintyleaf <mintyleafdev@gmail.com> * use middleware for Machine-Tag only if tag is specified Signed-off-by: mintyleaf <mintyleafdev@gmail.com> --------- Signed-off-by: mintyleaf <mintyleafdev@gmail.com>	2025-01-17 17:05:58 +01:00
Ettore Di Giacinto	7d0ac1ea3f	chore(vall-e-x): Drop backend (#4619) ... There are many new architectures that are SOTA and replaces vall-e-x nowadays. Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-01-17 09:35:10 +01:00
Max Goltzsche	8cc2d01caa	feat(ui): path prefix support via HTTP header (#4497) ... Makes the web app honour the `X-Forwarded-Prefix` HTTP request header that may be sent by a reverse-proxy in order to inform the app that its public routes contain a path prefix. For instance this allows to serve the webapp via a reverse-proxy/ingress controller under a path prefix/sub path such as e.g. `/localai/` while still being able to use the regular LocalAI routes/paths without prefix when directly connecting to the LocalAI server. Changes: * Add new `StripPathPrefix` middleware to strip the path prefix (provided with the `X-Forwarded-Prefix` HTTP request header) from the request path prior to matching the HTTP route. * Add a `BaseURL` utility function to build the base URL, honouring the `X-Forwarded-Prefix` HTTP request header. * Generate the derived base URL into the HTML (`head.html` template) as `<base/>` tag. * Make all webapp-internal URLs (within HTML+JS) relative in order to make the browser resolve them against the `<base/>` URL specified within each HTML page's header. * Make font URLs within the CSS files relative to the CSS file. * Generate redirect location URLs using the new `BaseURL` function. * Use the new `BaseURL` function to generate absolute URLs within gallery JSON responses. Closes #3095 TL;DR: The header-based approach allows to move the path prefix configuration concern completely to the reverse-proxy/ingress as opposed to having to align the path prefix configuration between LocalAI, the reverse-proxy and potentially other internal LocalAI clients. The gofiber swagger handler already supports path prefixes this way, see e2d9e9916d/swagger.go (L79) Signed-off-by: Max Goltzsche <max.goltzsche@gmail.com>	2025-01-07 17:18:21 +01:00
mintyleaf	2bc4b56a79	feat: stream tokens usage (#4415) ... * Use pb.Reply instead of []byte with Reply.GetMessage() in llama grpc to get the proper usage data in reply streaming mode at the last [DONE] frame * Fix 'hang' on empty message from the start Seems like that empty message marker trick was unnecessary --------- Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>	2024-12-18 09:48:50 +01:00
Ettore Di Giacinto	24abf568cb	chore(tests): stabilize tts test (#4417) ... chore(tests): stabilize test Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-12-17 00:46:48 +01:00
Ettore Di Giacinto	f943c4b803	Revert "feat: include tokens usage for streamed output" (#4336) ... Revert "feat: include tokens usage for streamed output (#4282)" This reverts commit 0d6c3a7d57.	2024-12-08 17:53:36 +01:00
Ettore Di Giacinto	cea5a0ea42	feat(template): read jinja templates from gguf files (#4332) ... * Read jinja templates as fallback Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Move templating out of model loader Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Test TemplateMessages Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Set role and content from transformers Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Tests: be more flexible Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * More jinja Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Small refactoring and adaptations Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-12-08 13:50:33 +01:00
Ettore Di Giacinto	d4c1746c7d	feat(llama.cpp): expose cache_type_k and cache_type_v for quant of kv cache (#4329) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-12-06 10:23:59 +01:00
Ettore Di Giacinto	44a5dac312	feat(backend): add stablediffusion-ggml (#4289) ... * feat(backend): add stablediffusion-ggml Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(ci): track stablediffusion-ggml Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fixups Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Use default scheduler and sampler if not specified Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fixups Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Move cfg scale out of diffusers block Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Make it working Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix: set free_params_immediately to false to call the model in sequence https://github.com/leejet/stable-diffusion.cpp/issues/366 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-12-03 22:41:22 +01:00
Ettore Di Giacinto	58ff47de26	feat(bark-cpp): add new bark.cpp backend (#4287) ... * feat(bark-cpp): add new bark.cpp backend Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * build on linux only for now Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * track bark.cpp in CI bumps Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Drop old entries from bumper Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * No need to test rwkv specifically, now part of llama.cpp Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-11-28 22:16:44 +01:00
mintyleaf	0d6c3a7d57	feat: include tokens usage for streamed output (#4282) ... Use pb.Reply instead of []byte with Reply.GetMessage() in llama grpc to get the proper usage data in reply streaming mode at the last [DONE] frame Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>	2024-11-28 14:47:56 +01:00
Ettore Di Giacinto	3c3050f68e	feat(backends): Drop bert.cpp (#4272) ... * feat(backends): Drop bert.cpp use llama.cpp 3.2 as a drop-in replacement for bert.cpp Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(tests): make test more robust Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-11-27 16:34:28 +01:00
Ettore Di Giacinto	f028ee8a26	fix(p2p): parse correctly ExtraLLamaCPPArgs (#4220) ... Previously we were sensible when args aren't defined and we would clash parsing extra args. Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-11-21 15:17:48 +01:00
Ettore Di Giacinto	47dc4337ba	fix(p2p): parse maddr correctly (#4219) ... Previously in case of not specifying a value it would pass a slice of 1 empty element Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-11-21 14:06:49 +01:00
Ettore Di Giacinto	4f1ab2366d	chore(refactor): imply modelpath (#4208) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-11-20 18:06:35 +01:00
Ettore Di Giacinto	b1ea9318e6	feat(silero): add Silero-vad backend (#4204) ... * feat(vad): add silero-vad backend (WIP) Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vad): add API endpoint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(vad): correctly place the onnxruntime libs Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(vad): hook silero-vad to binary and container builds Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gRPC): register VAD Server Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(Makefile): consume ONNX_OS consistently Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(Makefile): handle macOS Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>	2024-11-20 14:48:40 +01:00
mintyleaf	9892d7d584	feat(p2p): add support for configuration of edgevpn listen_maddrs, dht_announce_maddrs and bootstrap_peers (#4200) ... * add support for edgevpn listen_maddrs, dht_announce_maddrs, dht_bootstrap_peers * upd docs for libp2p loglevel	2024-11-20 14:18:52 +01:00
mintyleaf	de148cb2ad	feat: add WebUI API token authorization (#4197) ... * return 401 instead of 403, provide www-authenticate header, redirect to the login page, add cookie token support * set cookies completely through js in auth page	2024-11-19 18:43:02 +01:00
Ettore Di Giacinto	1770b92fb6	chore(api): return values from schema (#4153) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-11-14 14:12:29 +01:00
Ettore Di Giacinto	6daef00d30	chore(refactor): drop unnecessary code in loader (#4096) ... * chore: simplify passing options to ModelOptions Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(refactor): do not expose internal backend Loader Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-11-08 21:54:25 +01:00
Ettore Di Giacinto	e2a8dd64db	fix(tts): correctly pass backend config when generating model options (#4091) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-11-07 18:30:22 +01:00
Ettore Di Giacinto	20a5b20b59	chore(p2p): enhance logging (#4090) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-11-07 18:09:33 +01:00
Ettore Di Giacinto	2c041a2077	feat(ui): move model detailed info to a modal (#4086) ... * feat(ui): move model detailed info to a modal Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore: add static asset Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-11-06 18:25:59 +01:00
Ettore Di Giacinto	b425a870b0	fix(diffusers): correctly parse height and width request without parametrization (#4082) ... * fix(diffusers): allow to specify width and height without enable-parameters Let's simplify usage by not gating width and height by parameters Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore: use sane defaults Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-11-06 08:53:02 +01:00
Ettore Di Giacinto	947224b952	feat(diffusers): allow multiple lora adapters (#4081) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-11-05 15:14:33 +01:00
Arnaud A	65c3df392c	feat(tts): Implement naive response_format for tts endpoint (#4035) ... Signed-off-by: n-Arno <arnaud.alcabas@gmail.com>	2024-11-02 19:13:35 +00:00
Ettore Di Giacinto	8f7045cfa6	chore(tests): bump timeouts (#4024) ... To avoid flaky runs Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>	2024-10-31 15:40:43 +01:00
Ettore Di Giacinto	88edb1e2af	chore(tests): expand timeout (#4019) ... Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>	2024-10-30 15:34:44 +01:00
Ettore Di Giacinto	546dce68a6	chore: change url to github repository (#3972) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-10-26 14:50:18 +02:00
Ettore Di Giacinto	ae1ec4e096	feat(vllm): expose 'load_format' (#3943) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-10-23 15:34:57 +02:00
Ettore Di Giacinto	8737a65760	feat: allow to disable '/metrics' endpoints for local stats (#3945) ... Seem the "/metrics" endpoint that is source of confusion as people tends to believe we collect telemetry data just because we import "opentelemetry", however it is still a good idea to allow to disable even local metrics if not really required. See also: https://github.com/mudler/LocalAI/issues/3942 Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-10-23 15:34:32 +02:00
Ettore Di Giacinto	ccc7cb0287	feat(templates): use a single template for multimodals messages (#3892) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-10-22 09:34:05 +02:00
Ettore Di Giacinto	d5da8c3509	feat(templates): extract text from multimodal requests (#3866) ... When offloading template construction to the backend, we want to keep text around in case of multimodal requests. Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-10-17 17:33:50 +02:00
Dave	a1634b219a	fix: roll out bluemonday Sanitize more widely (#3794) ... * initial pass: roll out bluemonday sanitization more widely Signed-off-by: Dave Lee <dave@gray101.com> * add one additional sanitize - the overall modelslist used by the docs site Signed-off-by: Dave Lee <dave@gray101.com> --------- Signed-off-by: Dave Lee <dave@gray101.com>	2024-10-12 09:45:47 +02:00
Dave	65ca754166	Fix: listmodelservice / welcome endpoint use LOOSE_ONLY (#3791) ... * fix list model service and welcome Signed-off-by: Dave Lee <dave@gray101.com> * comment Signed-off-by: Dave Lee <dave@gray101.com> --------- Signed-off-by: Dave Lee <dave@gray101.com>	2024-10-11 23:49:00 +02:00
Ettore Di Giacinto	a0f0505f0d	fix(welcome): do not list model twice if we have a config (#3790) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-10-11 17:30:14 +02:00
Ettore Di Giacinto	d9b63fae7c	chore(tests): improve rwkv tests and consume TEST_FLAKES (#3765) ... chores(tests): improve rwkv tests and consume TEST_FLAKES consistently use TEST_FLAKES and reduce flakiness of rwkv tests by being case insensitive Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-10-08 09:24:19 +02:00
Ettore Di Giacinto	648ffdf449	feat(multimodal): allow to template placeholders (#3728) ... feat(multimodal): allow to template image placeholders Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-10-04 18:32:29 +02:00
Ettore Di Giacinto	d06a052d54	chore(federated): display a message when nodes are not available (#3721) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-10-03 18:38:41 +02:00
Ettore Di Giacinto	e5586e8781	chore: get model also from query (#3716) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-10-02 20:20:50 +02:00
Ettore Di Giacinto	3acd767ac4	chore: simplify model loading (#3715) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-10-02 08:59:06 +02:00
Shraddha	5488fc3bc1	feat: tokenization endpoint (#3710) ... endpoint to access the tokenizer Signed-off-by: shraddhazpy <shraddha@shraddhafive.in> Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com> Co-authored-by: Dave <dave@gray101.com>	2024-10-02 08:56:18 +02:00