595a73fce4
⬆️ Update ggerganov/llama.cpp ( #1813 )
...
Signed-off-by: GitHub <noreply@github.com >
Co-authored-by: mudler <mudler@users.noreply.github.com >
2024-03-09 09:27:06 +01:00
dc919e08e8
⬆️ Update ggerganov/llama.cpp ( #1811 )
...
Signed-off-by: GitHub <noreply@github.com >
Co-authored-by: mudler <mudler@users.noreply.github.com >
2024-03-08 08:21:25 +01:00
5d1018495f
feat(intel): add diffusers/transformers support ( #1746 )
...
* feat(intel): add diffusers support
* try to consume upstream container image
* Debug
* Manually install deps
* Map transformers/hf cache dir to modelpath if not specified
* fix(compel): update initialization, pass by all gRPC options
* fix: add dependencies, implement transformers for xpu
* base it from the oneapi image
* Add pillow
* set threads if specified when launching the API
* Skip conda install if intel
* defaults to non-intel
* ci: add to pipelines
* prepare compel only if enabled
* Skip conda install if intel
* fix cleanup
* Disable compel by default
* Install torch 2.1.0 with Intel
* Skip conda on some setups
* Detect python
* Quiet output
* Do not override system python with conda
* Prefer python3
* Fixups
* exllama2: do not install without conda (overrides pytorch version)
* exllama/exllama2: do not install if not using cuda
* Add missing dataset dependency
* Small fixups, symlink to python, add requirements
* Add neural_speed to the deps
* correctly handle model offloading
* fix: device_map == xpu
* go back at calling python, fixed at dockerfile level
* Exllama2 restricted to only nvidia gpus
* Tokenizer to xpu
2024-03-07 14:37:45 +01:00
ad6fd7a991
⬆️ Update ggerganov/llama.cpp ( #1805 )
...
Signed-off-by: GitHub <noreply@github.com >
Co-authored-by: mudler <mudler@users.noreply.github.com >
2024-03-06 23:28:31 +01:00
e022b5959e
⬆️ Update mudler/go-stable-diffusion ( #1802 )
...
Signed-off-by: GitHub <noreply@github.com >
Co-authored-by: mudler <mudler@users.noreply.github.com >
2024-03-05 23:39:57 +00:00
db7f4955a1
⬆️ Update ggerganov/llama.cpp ( #1801 )
...
Signed-off-by: GitHub <noreply@github.com >
Co-authored-by: mudler <mudler@users.noreply.github.com >
2024-03-05 21:50:27 +00:00
5c69dd155f
feat(autogpt/transformers): consume trust_remote_code ( #1799 )
...
trusting remote code by default is a danger to our users
2024-03-05 19:47:15 +01:00
504f2e8bf4
Update Backend Dependancies ( #1797 )
...
* Update transformers.yml
Signed-off-by: TwinFin <57421631+TwinFinz@users.noreply.github.com >
* Update transformers-rocm.yml
Signed-off-by: TwinFin <57421631+TwinFinz@users.noreply.github.com >
* Update transformers-nvidia.yml
Signed-off-by: TwinFin <57421631+TwinFinz@users.noreply.github.com >
---------
Signed-off-by: TwinFin <57421631+TwinFinz@users.noreply.github.com >
2024-03-05 10:10:00 +00:00
e586dc2924
Edit links in readme and integrations page ( #1796 )
...
* Update integrations.md
Signed-off-by: Luna Midori <118759930+lunamidori5@users.noreply.github.com >
* Update README.md
Signed-off-by: Luna Midori <118759930+lunamidori5@users.noreply.github.com >
* Update README.md
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
Signed-off-by: Luna Midori <118759930+lunamidori5@users.noreply.github.com >
* Update README.md
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
Signed-off-by: Luna Midori <118759930+lunamidori5@users.noreply.github.com >
---------
Signed-off-by: Luna Midori <118759930+lunamidori5@users.noreply.github.com >
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
2024-03-05 10:14:30 +01:00
333f918005
Update integrations.md
...
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
2024-03-05 09:45:54 +01:00
c8e29033c2
⬆️ Update ggerganov/llama.cpp ( #1794 )
...
Signed-off-by: GitHub <noreply@github.com >
Co-authored-by: mudler <mudler@users.noreply.github.com >
2024-03-05 08:59:09 +01:00
d0bd961bde
⬆️ Update ggerganov/llama.cpp ( #1791 )
...
Signed-off-by: GitHub <noreply@github.com >
Co-authored-by: mudler <mudler@users.noreply.github.com >
2024-03-04 09:44:21 +01:00
006511ee25
Revert "feat(assistant): Initial implementation of assistants api" ( #1790 )
...
Revert "feat(assistant): Initial implementation of assistants api (#1761 )"
This reverts commit 4ab72146cd
2024-03-03 10:31:06 +01:00
4ab72146cd
feat(assistant): Initial implementation of assistants api ( #1761 )
...
Initial implementation of assistants api
2024-03-03 08:50:43 +01:00
b60a3fc879
⬆️ Update ggerganov/llama.cpp ( #1789 )
...
Signed-off-by: GitHub <noreply@github.com >
Co-authored-by: mudler <mudler@users.noreply.github.com >
2024-03-03 08:49:23 +01:00
a0eeb74957
Update hot topics/roadmap
...
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
2024-03-02 09:35:40 +01:00
daa0b8741c
⬆️ Update ggerganov/llama.cpp ( #1785 )
...
Signed-off-by: GitHub <noreply@github.com >
Co-authored-by: mudler <mudler@users.noreply.github.com >
2024-03-01 22:38:24 +00:00
939411300a
Bump vLLM version + more options when loading models in vLLM ( #1782 )
...
* Bump vLLM version to 0.3.2
* Add vLLM model loading options
* Remove transformers-exllama
* Fix install exllama
2024-03-01 22:48:53 +01:00
1c312685aa
refactor: move remaining api packages to core ( #1731 )
...
* core 1
* api/openai/files fix
* core 2 - core/config
* move over core api.go and tests to the start of core/http
* move over localai specific endpoints to core/http, begin the service/endpoint split there
* refactor big chunk on the plane
* refactor chunk 2 on plane, next step: port and modify changes to request.go
* easy fixes for request.go, major changes not done yet
* lintfix
* json tag lintfix?
* gitignore and .keep files
* strange fix attempt: rename the config dir?
2024-03-01 16:19:53 +01:00
316de82f51
⬆️ Update ggerganov/llama.cpp ( #1779 )
...
Signed-off-by: GitHub <noreply@github.com >
Co-authored-by: mudler <mudler@users.noreply.github.com >
2024-02-29 22:33:30 +00:00
9068bc5271
Create SECURITY.md
...
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
2024-02-29 19:53:04 +01:00
31a4c9c9d3
Fix Command Injection Vulnerability ( #1778 )
...
* Added fix for command injection
* changed function name from sh to runCommand
2024-02-29 18:32:29 +00:00
c1966af2cf
ci: reduce stress on self-hosted runners ( #1776 )
...
Split jobs by self-hosted and free public runner provided by Github
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
2024-02-29 11:40:08 +01:00
c665898652
⬆️ Update donomii/go-rwkv.cpp ( #1771 )
...
Signed-off-by: GitHub <noreply@github.com >
Co-authored-by: mudler <mudler@users.noreply.github.com >
2024-02-28 23:50:27 +00:00
f651a660aa
⬆️ Update ggerganov/llama.cpp ( #1772 )
...
Signed-off-by: GitHub <noreply@github.com >
Co-authored-by: mudler <mudler@users.noreply.github.com >
2024-02-28 23:02:30 +01:00
ba672b51da
Update README.md
...
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
2024-02-28 16:03:38 +01:00
be498c5dd9
Update openai-functions.md
...
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
2024-02-28 15:58:31 +01:00
6e95beccb9
Update overview.md
...
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
2024-02-28 15:24:08 +01:00
c8be839481
Update openai-functions.md
...
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
2024-02-27 23:24:46 +01:00
c7e08813a5
⬆️ Update ggerganov/llama.cpp ( #1767 )
...
Signed-off-by: GitHub <noreply@github.com >
Co-authored-by: mudler <mudler@users.noreply.github.com >
2024-02-27 23:12:51 +01:00
d21a6b33ab
⬆️ Update ggerganov/llama.cpp ( #1756 )
...
Signed-off-by: GitHub <noreply@github.com >
Co-authored-by: mudler <mudler@users.noreply.github.com >
2024-02-27 18:07:51 +00:00
9112cf153e
Update integrations.md ( #1765 )
...
Added Jetbrains compatible plugin for LocalAI
Signed-off-by: Joshua Waring <Joshhua5@users.noreply.github.com >
2024-02-27 17:35:59 +01:00
3868ac8402
Update README.md
...
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
2024-02-27 15:44:15 +01:00
3f09010227
Update README.md
...
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
2024-02-27 15:43:15 +01:00
d6cf82aba3
fix(tests): re-enable tests after code move ( #1764 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-02-27 15:04:19 +01:00
dfe54639b1
Update README.md
...
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
2024-02-27 10:37:56 +01:00
bc5f5aa538
deps(llama.cpp): update ( #1759 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-02-26 13:18:44 +01:00
05818e0425
fix(functions): handle correctly when there are no results ( #1758 )
2024-02-26 08:38:23 +01:00
7f72a61104
ci: add stablediffusion to release ( #1757 )
...
Signed-off-by: Sertac Ozercan <sozercan@gmail.com >
2024-02-25 23:06:18 +00:00
8e45d47740
⬆️ Update ggerganov/llama.cpp ( #1753 )
...
Signed-off-by: GitHub <noreply@github.com >
Co-authored-by: mudler <mudler@users.noreply.github.com >
2024-02-25 10:03:19 +01:00
71771d1e9b
⬆️ Update docs version mudler/LocalAI ( #1752 )
...
Signed-off-by: GitHub <noreply@github.com >
Co-authored-by: mudler <mudler@users.noreply.github.com >
2024-02-25 10:02:52 +01:00
aa098e4d0b
fix(sse): do not omit empty finish_reason ( #1745 )
...
Fixes https://github.com/mudler/LocalAI/issues/1744
2024-02-24 11:51:59 +01:00
0135e1e3b9
fix: vllm - use AsyncLLMEngine to allow true streaming mode ( #1749 )
...
* fix: use vllm AsyncLLMEngine to bring true stream
Current vLLM implementation uses the LLMEngine, which was designed for offline batch inference, which results in the streaming mode outputing all blobs at once at the end of the inference.
This PR reworks the gRPC server to use asyncio and gRPC.aio, in combination with vLLM's AsyncLLMEngine to bring true stream mode.
This PR also passes more parameters to vLLM during inference (presence_penalty, frequency_penalty, stop, ignore_eos, seed, ...).
* Remove unused import
2024-02-24 11:48:45 +01:00
ff88c390bb
⬆️ Update ggerganov/llama.cpp ( #1750 )
...
Signed-off-by: GitHub <noreply@github.com >
Co-authored-by: mudler <mudler@users.noreply.github.com >
2024-02-24 00:06:46 +01:00
d825821a22
⬆️ Update ggerganov/llama.cpp ( #1740 )
...
Signed-off-by: GitHub <noreply@github.com >
Co-authored-by: mudler <mudler@users.noreply.github.com >
2024-02-23 00:07:15 +01:00
cbed6ab1bb
Update README.md ( #1739 )
...
* Update README.md
Signed-off-by: Luna Midori <118759930+lunamidori5@users.noreply.github.com >
* Update README.md
Signed-off-by: Luna Midori <118759930+lunamidori5@users.noreply.github.com >
---------
Signed-off-by: Luna Midori <118759930+lunamidori5@users.noreply.github.com >
2024-02-22 16:35:06 +01:00
6fc122fa1a
⬆️ Update ggerganov/llama.cpp ( #1705 )
...
Signed-off-by: GitHub <noreply@github.com >
Co-authored-by: mudler <mudler@users.noreply.github.com >
2024-02-22 09:33:23 +00:00
feba38be36
examples(mistral-openorca): add stopword
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-02-22 00:15:08 +01:00
ba85d0bcad
feat(upload-api): do not display error if uploadedFiles.json is not present
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-02-22 00:15:08 +01:00
ad3623dd8d
examples(phi-2): strip newline at the end of the prompt template
...
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
2024-02-21 23:17:51 +01:00
