530bec9c64
feat(llama.cpp): do not specify backends to autoload and add llama.cpp variants ( #2232 )
...
* feat(initializer): do not specify backends to autoload
We can simply try to autoload the backends extracted in the asset dir.
This will allow to build variants of the same backend (for e.g. with different instructions sets),
so to have a single binary for all the variants.
Signed-off-by: mudler <mudler@localai.io >
* refactor(prepare): refactor out llama.cpp prepare steps
Make it so are idempotent and that we can re-build
Signed-off-by: mudler <mudler@localai.io >
* [TEST] feat(build): build noavx version along
Signed-off-by: mudler <mudler@localai.io >
* build: make build parallel
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* build: do not override CMAKE_ARGS
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* build: add fallback variant
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* Fixups
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* fix(huggingface-langchain): fail if no token is set
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* fix(huggingface-langchain): rename
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* fix: do not autoload local-store
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* fix: give priority between the listed backends
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
---------
Signed-off-by: mudler <mudler@localai.io >
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-05-04 17:56:12 +02:00
54faaa87ea
fix(webui): correct documentation URL for text2img ( #2233 )
...
Signed-off-by: mudler <mudler@localai.io >
Co-authored-by: Dave <dave@gray101.com >
2024-05-04 00:25:13 +00:00
b58274b8a2
feat(ui): support multilineand style ul ( #2226 )
...
* feat(ui/chat): handle multiline in the input field
Signed-off-by: mudler <mudler@localai.io >
* feat(ui/chat): correctly display multiline messages
Signed-off-by: mudler <mudler@localai.io >
* feat(ui/chat): add list style
Signed-off-by: mudler <mudler@localai.io >
---------
Signed-off-by: mudler <mudler@localai.io >
2024-05-03 00:43:02 +02:00
2c5a46bc34
feat(ux): Add chat, tts, and image-gen pages to the WebUI ( #2222 )
...
* feat(webui): Add chat page
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* feat(webui): Add image-gen page
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* feat(webui): Add tts page
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-05-02 21:14:10 +02:00
c4f958e11b
refactor(application): introduce application global state ( #2072 )
...
* start breaking up the giant channel refactor now that it's better understood - easier to merge bites
Signed-off-by: Dave Lee <dave@gray101.com >
* add concurrency and base64 back in, along with new base64 tests.
Signed-off-by: Dave Lee <dave@gray101.com >
* Automatic rename of whisper.go's Result to TranscriptResult
Signed-off-by: Dave Lee <dave@gray101.com >
* remove pkg/concurrency - significant changes coming in split 2
Signed-off-by: Dave Lee <dave@gray101.com >
* fix comments
Signed-off-by: Dave Lee <dave@gray101.com >
* add list_model service as another low-risk service to get it out of the way
Signed-off-by: Dave Lee <dave@gray101.com >
* split backend config loader into seperate file from the actual config struct. No changes yet, just reduce cognative load with smaller files of logical blocks
Signed-off-by: Dave Lee <dave@gray101.com >
* rename state.go ==> application.go
Signed-off-by: Dave Lee <dave@gray101.com >
* fix lost import?
Signed-off-by: Dave Lee <dave@gray101.com >
---------
Signed-off-by: Dave Lee <dave@gray101.com >
2024-04-29 17:42:37 +00:00
11c48a0004
fix: security scanner warning noise: error handlers part 2 ( #2145 )
...
check off a few more error handlers
Signed-off-by: Dave Lee <dave@gray101.com >
2024-04-29 15:11:42 +02:00
e8d44447ad
feat(gallery): support model deletion ( #2173 )
...
* feat(gallery): op now supports deletion of models
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* Wire things with WebUI(WIP)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* minor improvements
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-04-28 23:42:46 +02:00
fb2a05ff43
feat(gallery): display job status also during navigation ( #2151 )
...
* feat(gallery): keep showing progress also when refreshing
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* fix(intel-gpu): better defaults
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* feat: make it thread-safe
Signed-off-by: mudler <mudler@localai.io >
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
Signed-off-by: mudler <mudler@localai.io >
2024-04-27 09:08:33 +02:00
2dc1fa2474
fix: config_file_watcher.go - root all file reads for safety ( #2144 )
...
callHandler() now has all file access rooted within DynamicConfigDir
Signed-off-by: Dave Lee <dave@gray101.com >
2024-04-26 16:46:35 +00:00
006306b183
fix: use bluemonday as recommended by blackfriday ( #2142 )
...
use bluemonday as recommended by blackfriday
Signed-off-by: Dave Lee <dave@gray101.com >
2024-04-26 10:34:50 +02:00
2cd4936c99
fix: security scanner warning noise: error handlers part 1 ( #2141 )
...
first group of error handlers to reduce security scanner warning noise level
Signed-off-by: Dave Lee <dave@gray101.com >
2024-04-26 10:34:31 +02:00
c8dd8e5ef4
fix: reduce chmod permissions for created files and directories ( #2137 )
...
quiet more security scanner issues: pass one of chmod restriction to remove group and other permissions
Signed-off-by: Dave Lee <dave@gray101.com >
2024-04-26 00:47:06 +02:00
d98063e80e
fix: api key polling was not using correct filepath ( #2132 )
...
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
2024-04-25 20:06:22 +00:00
b664edde29
feat(rerankers): Add new backend, support jina rerankers API ( #2121 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-04-25 00:19:02 +02:00
4fffc47e77
deps(llama.cpp): update, use better model for function call tests ( #2119 )
...
deps(llama.cpp): update
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-04-24 18:44:04 +02:00
ac56ac2b2d
fix(gallery): show a fake image if no there is no icon ( #2111 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-04-23 20:10:58 +02:00
a09fe1b9ba
fix(gallery): set margin for images
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-04-23 20:00:20 +02:00
d344daf129
feat(models-ui): minor visual enhancements ( #2109 )
...
Show image if present, URL, tags, and better display buttons
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-04-23 18:43:25 +02:00
3411e072ca
Fix cleanup sonarqube findings ( #2106 )
...
* fix: update dockerignore and gitignore to exclude sonarqube work dir
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
* fix: remove useless equality check
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
* fix: use sonarqube Dockerfile recommendations
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
---------
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
2024-04-23 18:43:00 +02:00
0d8bf91699
feat: Galleries UI ( #2104 )
...
* WIP: add models to webui
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* Register routes
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* fix: don't cache models
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* small fixups
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* fix: fixup multiple installs (strings.Clone)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-04-23 09:22:58 +02:00
220958a87c
fix: typo in models.go ( #2099 )
2024-04-22 04:34:59 +00:00
f3f6535aad
fix: rename fiber entrypoint from http/api to http/app ( #2096 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
Co-authored-by: Dave <dave@gray101.com >
2024-04-21 22:39:28 +02:00
180cd4ccda
fix(llama.cpp-ggml): fixup max_tokens for old backend ( #2094 )
...
fix(llama.cpp-ggml): set 0 as default for `max_tokens`
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-04-21 16:34:00 +02:00
284ad026b1
refactor(routes): split routes registration ( #2077 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-04-21 01:19:57 +02:00
afa1bca1e3
fix(llama.cpp): set -1 as default for max tokens ( #2087 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-04-20 20:20:10 +02:00
03adc1f60d
Add tensor_parallel_size setting to vllm setting items ( #2085 )
...
Signed-off-by: Taikono-Himazin <kazu@po.harenet.ne.jp >
2024-04-20 14:37:02 +00:00
b319ed58b0
models(gallery): add gallery ( #2078 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-04-20 15:22:54 +02:00
8d30b39811
feat: fiber logs with zerlog and add trace level ( #2082 )
...
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
2024-04-20 10:43:37 +02:00
b9e7708643
feat: enable polling configs for systems with broken fsnotify (docker volumes on windows) ( #2081 )
...
* feat: enable polling configs for systems with broken fsnotify (docker volumes on windows)
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
* fix: update logging to make it clear that the config file is being polled
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
---------
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
2024-04-19 19:31:15 -05:00
bbea62b907
feat(functions): support models with no grammar, add tests ( #2068 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-04-18 22:43:12 +02:00
502c1eedaa
feat: refactor the dynamic json configs for api_keys and external_backends ( #2055 )
...
* feat: refactor the dynamic json configs for api_keys and external_backends
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
* fix: remove commented code
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
---------
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com >
2024-04-18 03:21:55 +00:00
af9e5a2d05
Revert #1963 ( #2056 )
...
* Revert "fix(fncall): fix regression introduced in #1963 (#2048 )"
This reverts commit 6b06d4e0af#2038 )"
This reverts commit fdec8a9d00#2035 )"
This reverts commit e843d7df0e#1963 )"
This reverts commit eed5706994Fixes : #1920
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-04-17 23:33:49 +02:00
6b06d4e0af
fix(fncall): fix regression introduced in #1963 ( #2048 )
...
Signed-off-by: Dave <dave@gray101.com >
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
Signed-off-by: Dave <dave@gray101.com >
Co-authored-by: Dave <dave@gray101.com >
2024-04-16 21:20:11 +00:00
33c78d2228
feat(store): add Golang client ( #1977 )
...
This adds a basic store client for Go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-04-16 15:54:14 +02:00
fdec8a9d00
fix: action-tmate back to upstream, dead code removal ( #2038 )
...
cleanup: upstream action-tmate has taken my PR, drop master reference. Also remove dead code from api.go
Signed-off-by: Dave Lee <dave@gray101.com >
2024-04-16 01:46:36 +00:00
538a086309
fix: previous CLI rework broke debug logging ( #2036 )
...
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
Co-authored-by: Dave <dave@gray101.com >
2024-04-15 22:13:59 +02:00
e843d7df0e
feat(grpc): return consumed token count and update response accordingly ( #2035 )
...
Fixes : #1920
2024-04-15 19:47:11 +02:00
eed5706994
refactor: backend/service split, channel-based llm flow ( #1963 )
...
Refactor: channel based llm flow and services split
---------
Signed-off-by: Dave Lee <dave@gray101.com >
2024-04-13 09:45:34 +02:00
b2785ff06e
feat(gallery): support ConfigURLs ( #2012 )
...
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-04-12 00:49:23 +02:00
12c0d9443e
feat: use tokenizer.apply_chat_template() in vLLM ( #1990 )
...
Use tokenizer.apply_chat_template() in vLLM
Signed-off-by: Ludovic LEROUX <ludovic@inpher.io >
2024-04-11 19:20:22 +02:00
24d7dadfed
feat: kong cli refactor fixes #1955 ( #1974 )
...
* feat: migrate to alecthomas/kong for CLI
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
* feat: bring in new flag for granular log levels
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
* chore: go mod tidy
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
* feat: allow loading cli flag values from ["./localai.yaml", "~/.config/localai.yaml", "/etc/localai.yaml"] in that order
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
* feat: load from .env file instead of a yaml file
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
* feat: better loading for environment files
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
* feat(doc): add initial documentation about configuration
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
* fix: remove test log lines
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
* feat: integrate new documentation into existing pages
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
* feat: add documentation on .env files
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
* fix: cleanup some documentation table errors
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
* feat: refactor CLI logic out to it's own package under core/cli
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
---------
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
2024-04-11 09:19:24 +02:00
8342553214
fix(llama.cpp): set better defaults for llama.cpp ( #1961 )
...
fix(defaults): set better defaults for llama.cpp
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-04-06 22:56:45 +02:00
b85dad0286
feat: first pass at improving logging ( #1956 )
...
Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com >
2024-04-04 09:24:22 +02:00
ff77d3bc22
fix(seed): generate random seed per-request if -1 is set ( #1952 )
...
* fix(seed): generate random seed per-request if -1 is set
Also update ci with new workflows and allow the aio tests to run with an
api key
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
* docs(openvino): Add OpenVINO example
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io >
2024-04-03 22:25:47 +02:00
9bc209ba73
fix(welcome): stable model list ( #1949 )
2024-04-02 19:25:32 +02:00
e8f02c083f
fix(functions): respect when selected from string ( #1940 )
...
* fix(functions): respect when selected from string
* fix(toolschoice): decode both string and objects
2024-04-01 19:39:54 +02:00
35290e146b
fix(grammar): respect JSONmode and grammar from user input ( #1935 )
...
* fix(grammar): Fix JSON mode and custom grammar
* tests(aio): add jsonmode test
* tests(aio): add functioncall test
* fix(aio): use hermes-2-pro-mistral as llm for CPU profile
* add phi-2-orange
2024-03-31 13:04:09 +02:00
957f428fd5
fix(tools): correctly render tools response in templates ( #1932 )
...
* fix(tools): allow to correctly display both Functions and Tools
* models(hermes-2-pro): correctly display function results
2024-03-30 19:02:07 +01:00
61e5e6bc36
fix(swagger): do not specify a host ( #1930 )
...
In this way the requests are redirected to the host used by the client
to perform the request.
2024-03-30 12:04:41 +01:00
123a5a2e16
feat(swagger): Add swagger API doc ( #1926 )
...
* makefile(build): add minimal and api build target
* feat(swagger): Add swagger
2024-03-29 22:29:33 +01:00
