Ettore Di Giacinto
cea5a0ea42
feat(template): read jinja templates from gguf files ( #4332 )
...
* Read jinja templates as fallback
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Move templating out of model loader
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Test TemplateMessages
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Set role and content from transformers
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Tests: be more flexible
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* More jinja
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Small refactoring and adaptations
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-12-08 13:50:33 +01:00
LocalAI [bot]
f5e1527a5a
chore: ⬆️ Update ggerganov/llama.cpp to 3573fa8e7b7f0865638b52b4e9b4d2006f0558a2 ( #4335 )
...
⬆️ Update ggerganov/llama.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
2024-12-07 21:51:45 +00:00
Ettore Di Giacinto
7184ca546f
chore(model gallery): add llama-3.3-70b-instruct ( #4333 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-12-07 10:39:20 +01:00
LocalAI [bot]
5592f5e820
chore: ⬆️ Update ggerganov/llama.cpp to c5ede3849fc021174862f9c0bf8273808d8f0d39 ( #4330 )
...
⬆️ Update ggerganov/llama.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
2024-12-06 21:46:51 +00:00
Ettore Di Giacinto
d4c1746c7d
feat(llama.cpp): expose cache_type_k and cache_type_v for quant of kv cache ( #4329 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-12-06 10:23:59 +01:00
LocalAI [bot]
88737e1d76
chore: ⬆️ Update ggerganov/llama.cpp to c9c6e01daedac542b174c235872569fce5385982 ( #4328 )
...
⬆️ Update ggerganov/llama.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
2024-12-06 09:15:21 +01:00
LocalAI [bot]
ba225f660b
docs: ⬆️ update docs version mudler/LocalAI ( #4327 )
...
⬆️ Update docs version mudler/LocalAI
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
2024-12-05 21:54:00 +00:00
Ettore Di Giacinto
3127cd1352
chore(docs): update available backends ( #4325 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-12-05 16:57:56 +01:00
PetrFlegr
b90d78d9f6
Updated links of yamls ( #4324 )
...
Updated links
Links to deplyment*.yaml was changed
Signed-off-by: PetrFlegr <ptrflegr@gmail.com>
2024-12-05 16:06:51 +01:00
Ettore Di Giacinto
b86a3e4fa6
chore(model gallery): add math-iio-7b-instruct ( #4323 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-12-05 10:05:35 +01:00
Ettore Di Giacinto
be907d993f
chore(model gallery): add loki-v2.6-8b-1024k ( #4321 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-12-05 10:02:02 +01:00
Ettore Di Giacinto
ab0f8648a3
chore(model gallery): add rp-naughty-v1.0c-8b ( #4322 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-12-05 10:01:49 +01:00
LocalAI [bot]
c226149503
chore: ⬆️ Update leejet/stable-diffusion.cpp to 9578fdcc4632dc3de5565f28e2fb16b7c18f8d48 ( #4320 )
...
⬆️ Update leejet/stable-diffusion.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
2024-12-05 09:09:11 +01:00
LocalAI [bot]
4a079f893c
chore: ⬆️ Update ggerganov/llama.cpp to 59f4db10883a4f3e855cffbf2c3ab68430e95272 ( #4319 )
...
⬆️ Update ggerganov/llama.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
2024-12-04 22:19:35 +00:00
Ettore Di Giacinto
87b7648591
chore(ci): set auto-labeler for dependencies
...
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
2024-12-04 18:35:54 +01:00
Ettore Di Giacinto
cf4f024420
Update README.md
...
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
2024-12-04 11:31:08 +01:00
Ettore Di Giacinto
3c0ac49d90
chore(model gallery): add bark-cpp-small ( #4318 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-12-04 09:32:42 +01:00
Ettore Di Giacinto
4307ae5d52
chore(model gallery): add flux.1-dev-ggml ( #4317 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-12-04 09:26:51 +01:00
Ettore Di Giacinto
50f71f73d7
chore(model gallery): add homercreativeanvita-mix-qw7b ( #4316 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-12-04 09:17:23 +01:00
Ettore Di Giacinto
dc04a43868
chore(model gallery): add chatty-harry_v3.0 ( #4315 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-12-04 09:15:47 +01:00
Ettore Di Giacinto
cc04b62d3a
chore(model gallery): add cybercore-qwen-2.1-7b ( #4314 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-12-04 09:15:35 +01:00
LocalAI [bot]
feb54e65c2
chore: ⬆️ Update ggerganov/llama.cpp to cc98896db858df7aa40d0e16a505883ef196a482 ( #4312 )
...
⬆️ Update ggerganov/llama.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
2024-12-03 23:53:11 +00:00
Ettore Di Giacinto
44a5dac312
feat(backend): add stablediffusion-ggml ( #4289 )
...
* feat(backend): add stablediffusion-ggml
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* chore(ci): track stablediffusion-ggml
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fixups
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Use default scheduler and sampler if not specified
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fixups
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Move cfg scale out of diffusers block
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Make it working
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix: set free_params_immediately to false to call the model in sequence
https://github.com/leejet/stable-diffusion.cpp/issues/366
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-12-03 22:41:22 +01:00
Ettore Di Giacinto
074b52bbfe
chore(model gallery): add bggpt-gemma-2-2.6b-it-v1.0 ( #4311 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-12-03 18:56:53 +01:00
Ettore Di Giacinto
236a60bab8
chore(model gallery): add qwen2.5-7b-homercreative-mix ( #4310 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-12-03 18:56:03 +01:00
Ettore Di Giacinto
7b70f0543b
chore(model gallery): add sparse-llama-3.1-8b-2of4 ( #4309 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-12-03 18:55:49 +01:00
LocalAI [bot]
5f33962932
chore: ⬆️ Update ggerganov/llama.cpp to 8648c521010620c2daccfa1d26015c668ba2c717 ( #4307 )
...
⬆️ Update ggerganov/llama.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
2024-12-03 09:20:59 +01:00
dependabot[bot]
45b91d501e
chore(deps): Bump docs/themes/hugo-theme-relearn from 28fce6b to be85052 ( #4305 )
...
chore(deps): Bump docs/themes/hugo-theme-relearn
Bumps [docs/themes/hugo-theme-relearn](https://github.com/McShelby/hugo-theme-relearn ) from `28fce6b` to `be85052`.
- [Release notes](https://github.com/McShelby/hugo-theme-relearn/releases )
- [Commits](28fce6b04c...be85052efe
2024-12-02 22:47:37 +00:00
Ettore Di Giacinto
e51792784a
chore(deps): bump grpcio to 1.68.1 ( #4301 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-12-02 19:13:26 +01:00
LocalAI [bot]
28594336e9
chore: ⬆️ Update ggerganov/llama.cpp to 5e1ed95583ca552a98d8528b73e1ff81249c2bf9 ( #4299 )
...
⬆️ Update ggerganov/llama.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
2024-12-01 21:44:59 +00:00
Ettore Di Giacinto
9c9359fc96
chore(model gallery): add teleut-7b ( #4298 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-12-01 10:12:48 +01:00
Ettore Di Giacinto
bc5d1f255b
chore(model gallery): add skywork-o1-open-llama-3.1-8b ( #4297 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-12-01 10:12:35 +01:00
Ettore Di Giacinto
0fcefbc168
chore(model gallery): add volare-i1 ( #4296 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-12-01 10:12:17 +01:00
LocalAI [bot]
9044b17e4d
chore: ⬆️ Update ggerganov/llama.cpp to 0c39f44d70d058940fe2afe50cfc789e3e44d756 ( #4295 )
...
⬆️ Update ggerganov/llama.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
2024-11-30 21:46:07 +00:00
Ettore Di Giacinto
ad31daf03b
chore(model gallery): add qwestion-24b ( #4294 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-11-30 11:18:25 +01:00
Ettore Di Giacinto
1167487f5e
chore(model gallery): add freyja-v4.95-maldv-7b-non-fiction-i1 ( #4293 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-11-30 11:15:05 +01:00
Ettore Di Giacinto
61358e4d35
chore(model gallery): add q2.5-32b-slush-i1 ( #4292 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-11-30 11:02:41 +01:00
LocalAI [bot]
2c8a87b1e4
chore: ⬆️ Update ggerganov/llama.cpp to 3a8e9af402f7893423bdab444aa16c5d9a2d429a ( #4290 )
...
⬆️ Update ggerganov/llama.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
2024-11-29 21:58:24 +00:00
LocalAI [bot]
55aad5f525
chore: ⬆️ Update ggerganov/llama.cpp to dc22344088a7ee81a1e4f096459b03a72f24ccdc ( #4288 )
...
⬆️ Update ggerganov/llama.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
2024-11-28 23:04:31 +00:00
Ettore Di Giacinto
58ff47de26
feat(bark-cpp): add new bark.cpp backend ( #4287 )
...
* feat(bark-cpp): add new bark.cpp backend
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* build on linux only for now
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* track bark.cpp in CI bumps
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Drop old entries from bumper
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* No need to test rwkv specifically, now part of llama.cpp
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-11-28 22:16:44 +01:00
mintyleaf
0d6c3a7d57
feat: include tokens usage for streamed output ( #4282 )
...
Use pb.Reply instead of []byte with Reply.GetMessage() in llama grpc to get the proper usage data in reply streaming mode at the last [DONE] frame
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
2024-11-28 14:47:56 +01:00
Ettore Di Giacinto
e001fada6c
chore(model gallery): add llama-smoltalk-3.2-1b-instruct ( #4285 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-11-28 09:35:38 +01:00
Ettore Di Giacinto
f4547fcf8a
chore(model gallery): add qwq-32b-preview ( #4284 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-11-28 09:34:44 +01:00
Ettore Di Giacinto
7b75e9de2d
fix(rwkv model): add stoptoken ( #4283 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-11-28 09:34:35 +01:00
Ettore Di Giacinto
cbedf2f428
fix(llama.cpp): embed metal file into result binary for darwin ( #4279 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-11-28 04:17:00 +00:00
LocalAI [bot]
0597f3b9e9
chore: ⬆️ Update ggerganov/llama.cpp to 3ad5451f3b75809e3033e4e577b9f60bcaf6676a ( #4280 )
...
⬆️ Update ggerganov/llama.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
2024-11-27 22:24:47 +00:00
Ettore Di Giacinto
5f688d7a8d
chore(model): add eva-qwen2.5-72b-v0.2 to the gallery ( #4278 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-11-27 19:02:05 +01:00
Ettore Di Giacinto
fa20628b3a
chore(model): add llama-3.1-tulu-3-8b-sft to the gallery ( #4277 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-11-27 19:01:54 +01:00
Ettore Di Giacinto
13bf048cfc
chore(model): add llama-3.1-tulu-3-70b-dpo model config ( #4276 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-11-27 18:44:53 +01:00
Ettore Di Giacinto
bdd6920910
chore(model): add q2.5-ms-mistoria-72b-v2 model config ( #4275 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2024-11-27 16:56:55 +01:00
