28 Commits

Author	SHA1	Message	Date
Brandon Beiler	6a6e1a0ea9	feat(vllm): Additional vLLM config options (Disable logging, dtype, and Per-Prompt media limits) (#4855) ... * Adding the following vLLM config options: disable_log_status, dtype, limit_mm_per_prompt Signed-off-by: TheDropZone <brandonbeiler@gmail.com> * using " marks in the config.yaml file Signed-off-by: TheDropZone <brandonbeiler@gmail.com> * adding in missing colon Signed-off-by: TheDropZone <brandonbeiler@gmail.com> --------- Signed-off-by: TheDropZone <brandonbeiler@gmail.com>	2025-02-18 19:27:58 +01:00
Ettore Di Giacinto	1d6afbd65d	feat(llama.cpp): Add support to grammar triggers (#4733) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-02-02 13:25:03 +01:00
mintyleaf	96f8ec0402	feat: add machine tag and inference timings (#4577) ... * Add machine tag option, add extraUsage option, grpc-server -> proto -> endpoint extraUsage data is broken for now Signed-off-by: mintyleaf <mintyleafdev@gmail.com> * remove redurant timing fields, fix not working timings output Signed-off-by: mintyleaf <mintyleafdev@gmail.com> * use middleware for Machine-Tag only if tag is specified Signed-off-by: mintyleaf <mintyleafdev@gmail.com> --------- Signed-off-by: mintyleaf <mintyleafdev@gmail.com>	2025-01-17 17:05:58 +01:00
Ettore Di Giacinto	d4c1746c7d	feat(llama.cpp): expose cache_type_k and cache_type_v for quant of kv cache (#4329) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-12-06 10:23:59 +01:00
Ettore Di Giacinto	44a5dac312	feat(backend): add stablediffusion-ggml (#4289) ... * feat(backend): add stablediffusion-ggml Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(ci): track stablediffusion-ggml Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fixups Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Use default scheduler and sampler if not specified Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fixups Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Move cfg scale out of diffusers block Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Make it working Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix: set free_params_immediately to false to call the model in sequence https://github.com/leejet/stable-diffusion.cpp/issues/366 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-12-03 22:41:22 +01:00
Ettore Di Giacinto	b1ea9318e6	feat(silero): add Silero-vad backend (#4204) ... * feat(vad): add silero-vad backend (WIP) Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vad): add API endpoint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(vad): correctly place the onnxruntime libs Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(vad): hook silero-vad to binary and container builds Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gRPC): register VAD Server Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(Makefile): consume ONNX_OS consistently Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(Makefile): handle macOS Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>	2024-11-20 14:48:40 +01:00
Ettore Di Giacinto	947224b952	feat(diffusers): allow multiple lora adapters (#4081) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-11-05 15:14:33 +01:00
Ettore Di Giacinto	61c964dce7	fix(grpc): pass by modelpath (#4023) ... Instead of trying to derive it from the model file. In backends that specify HF url this results in a fragile logic. Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-10-31 12:12:22 +01:00
Ettore Di Giacinto	835932e95e	feat: update proto file ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-10-23 15:46:06 +02:00
siddimore	f84b55d1ef	feat: Add Get Token Metrics to GRPC server (#3687) ... * Add Get Token Metrics to GRPC server Signed-off-by: Siddharth More <siddimore@gmail.com> * Expose LocalAI endpoint Signed-off-by: Siddharth More <siddimore@gmail.com> --------- Signed-off-by: Siddharth More <siddimore@gmail.com>	2024-10-01 14:41:20 +02:00
siddimore	50a3b54e34	feat(api): add correlationID to Track Chat requests (#3668) ... * Add CorrelationID to chat request Signed-off-by: Siddharth More <siddimore@gmail.com> * remove get_token_metrics Signed-off-by: Siddharth More <siddimore@gmail.com> * Add CorrelationID to proto Signed-off-by: Siddharth More <siddimore@gmail.com> * fix correlation method name Signed-off-by: Siddharth More <siddimore@gmail.com> * Update core/http/endpoints/openai/chat.go Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com> Signed-off-by: Siddharth More <siddimore@gmail.com> * Update core/http/endpoints/openai/chat.go Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com> Signed-off-by: Siddharth More <siddimore@gmail.com> --------- Signed-off-by: Siddharth More <siddimore@gmail.com> Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com> Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>	2024-09-28 17:23:56 +02:00
Ettore Di Giacinto	191bc2e50a	feat(api): allow to pass audios to backends (#3603) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-09-19 12:26:53 +02:00
Ettore Di Giacinto	fbb9facda4	feat(api): allow to pass videos to backends (#3601) ... This prepares the API to receive videos as well for video understanding. It works similarly to images, where the request should be in the form: { "type": "video_url", "video_url": { "url": "url or base64 data" } } Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-09-19 11:21:59 +02:00
Dave	81ae92f017	feat: elevenlabs sound-generation api (#3355) ... * initial version of elevenlabs compatible soundgeneration api and cli command Signed-off-by: Dave Lee <dave@gray101.com> * minor cleanup Signed-off-by: Dave Lee <dave@gray101.com> * restore TTS, add test Signed-off-by: Dave Lee <dave@gray101.com> * remove stray s Signed-off-by: Dave Lee <dave@gray101.com> * fix Signed-off-by: Dave Lee <dave@gray101.com> --------- Signed-off-by: Dave Lee <dave@gray101.com> Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com> Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>	2024-08-24 00:20:28 +00:00
Ettore Di Giacinto	03b1cf51fd	feat(whisper): add translate option (#2649) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-06-24 19:21:22 +02:00
Chakib Benziane	b99182c8d4	TTS API improvements (#2308) ... * update doc on COQUI_LANGUAGE env variable Signed-off-by: blob42 <contact@blob42.xyz> * return errors from tts gRPC backend Signed-off-by: blob42 <contact@blob42.xyz> * handle speaker_id and language in coqui TTS backend Signed-off-by: blob42 <contact@blob42.xyz> * TTS endpoint: add optional language paramter Signed-off-by: blob42 <contact@blob42.xyz> * tts fix: empty language string breaks non-multilingual models Signed-off-by: blob42 <contact@blob42.xyz> * allow tts param definition in config file - consolidate TTS options under `tts` config entry Signed-off-by: blob42 <contact@blob42.xyz> * tts: update doc Signed-off-by: blob42 <contact@blob42.xyz> --------- Signed-off-by: blob42 <contact@blob42.xyz> Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>	2024-06-01 18:26:27 +00:00
Ettore Di Giacinto	e49ea0123b	feat(llama.cpp): add flash_attention and no_kv_offloading (#2310) ... feat(llama.cpp): add flash_attn and no_kv_offload Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-05-13 19:07:51 +02:00
Ettore Di Giacinto	b664edde29	feat(rerankers): Add new backend, support jina rerankers API (#2121) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-04-25 00:19:02 +02:00
Taikono-Himazin	03adc1f60d	Add tensor_parallel_size setting to vllm setting items (#2085) ... Signed-off-by: Taikono-Himazin <kazu@po.harenet.ne.jp>	2024-04-20 14:37:02 +00:00
Ettore Di Giacinto	e843d7df0e	feat(grpc): return consumed token count and update response accordingly (#2035) ... Fixes: #1920	2024-04-15 19:47:11 +02:00
Ludovic Leroux	12c0d9443e	feat: use tokenizer.apply_chat_template() in vLLM (#1990) ... Use tokenizer.apply_chat_template() in vLLM Signed-off-by: Ludovic LEROUX <ludovic@inpher.io>	2024-04-11 19:20:22 +02:00
Richard Palethorpe	643d85d2cc	feat(stores): Vector store backend (#1795) ... Add simple vector store backend Signed-off-by: Richard Palethorpe <io@richiejp.com>	2024-03-22 21:14:04 +01:00
Ettore Di Giacinto	20136ca8b7	feat(tts): add Elevenlabs and OpenAI TTS compatibility layer (#1834) ... * feat(elevenlabs): map elevenlabs API support to TTS This allows elevenlabs Clients to work automatically with LocalAI by supporting the elevenlabs API. The elevenlabs server endpoint is implemented such as it is wired to the TTS endpoints. Fixes: https://github.com/mudler/LocalAI/issues/1809 * feat(openai/tts): compat layer with openai tts Fixes: #1276 * fix: adapt tts CLI	2024-03-14 23:08:34 +01:00
Ludovic Leroux	939411300a	Bump vLLM version + more options when loading models in vLLM (#1782) ... * Bump vLLM version to 0.3.2 * Add vLLM model loading options * Remove transformers-exllama * Fix install exllama	2024-03-01 22:48:53 +01:00
Ettore Di Giacinto	cb7512734d	transformers: correctly load automodels (#1643) ... * backends(transformers): use AutoModel with LLM types * examples: animagine-xl * Add codellama examples	2024-01-26 00:13:21 +01:00
Ettore Di Giacinto	9e653d6abe	feat: 🐍 add mamba support (#1589) ... feat(mamba): Initial import This is a first iteration of the mamba backend, loosely based on mamba-chat(https://github.com/havenhq/mamba-chat).	2024-01-19 23:42:50 +01:00
Ettore Di Giacinto	7641f92cde	feat(diffusers): update, add autopipeline, controlnet (#1432) ... * feat(diffusers): update, add autopipeline, controlenet * tests with AutoPipeline * simplify logic	2023-12-13 19:20:22 +01:00
Ettore Di Giacinto	ad0e30bca5	refactor: move backends into the backends directory (#1279) ... * refactor: move backends into the backends directory Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor: move main close to implementation for every backend Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2023-11-13 22:40:16 +01:00