* Add more debug messages
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat: allow to disable gallery endpoints
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* improve p2p messaging
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* improve error handling
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Make sure to close the listening socket when context is exhausted
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* WIP
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Fixups
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Wire up a simple explorer DB
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* wip
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* WIP
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* refactor: group services id so can be identified easily in the ledger table
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(discovery): discovery service now gather worker informations correctly
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(explorer): display network token
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(explorer): display form to add new networks
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(explorer): stop from overwriting networks
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(explorer): display only networks with active workers
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(explorer): list only clusters in a network if it has online workers
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* remove invalid and inactive networks
if networks have no workers delete them from the database, similarly,
if invalid.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* ci: add workflow to deploy new explorer versions automatically
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* build-api: build with p2p tag
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Allow to specify a connection timeout
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* logging
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Better p2p defaults
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Set loglevel
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Fix dht enable
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Default to info for loglevel
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Add navbar
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Slightly improve rendering
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Allow to copy the token easily
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* ci fixups
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
chore: drop gpt4all
gpt4all is already supported in llama.cpp - the backend was kept for
keeping compatibility with old gpt4all models (prior to gguf format).
It is good time now to clean up and remove it to slim the compilation
process.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
feat(p2p): allow to run multiple clusters in the same network
Allow to specify a network ID via CLI which allows to run multiple
clusters, logically separated within the same network (by using the same
shared token).
Note: This segregation is not "secure" by any means, anyone having the
network token can see the services available in all the network,
however, this provides a way to separate the inference endpoints.
This allows for instance to have a node which is both federated and
having attached a set of llama.cpp workers.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(openai): add json_schema and strict mode
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* handle err vs _
security scanners prefer if we put these branches in, and I tend to agree.
Signed-off-by: Dave <dave@gray101.com>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: Dave <dave@gray101.com>
Co-authored-by: Dave <dave@gray101.com>
* fix(downloader): be consistent in downloading files
This PR puts some order in the downloader such as functions are re-used
across several places.
This fixes an issue with having uri's inside the model YAML file, it
would resolve to MD5 rather then using the filename
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(scanner): do raise error only if unsafeFiles are found
Fixes: https://github.com/mudler/LocalAI/issues/3114
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* wip
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* get rid of panics
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* expose it properly from the config
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Simplify
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* forgot to commit
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Remove focus on test
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Small fixups
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* refactor: break down json grammar parser in different files
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix: patch to `refactor_grammars` - propagate errors (#3006)
propagate errors around
Signed-off-by: Dave Lee <dave@gray101.com>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: Dave Lee <dave@gray101.com>
Co-authored-by: Dave <dave@gray101.com>
* feat(functions): enhance parsing with broken JSON when we parse the raw results
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* breaking: make function name by default
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(grammar): dynamically generate grammars with mutating keys
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* refactor: simplify condition
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Update docs
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* docs(swagger): finish convering gallery section
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* docs: add section to explain how to install models with local-ai run
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Minor docs adjustments
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* docs(swagger): core more localai/openai endpoints
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix swagger descriptions for backend_monitor.go
Signed-off-by: Dave <dave@gray101.com>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: Dave <dave@gray101.com>
Co-authored-by: Dave <dave@gray101.com>
fix(model-list): be consistent, skip known files from listing
This changeset does two things:
- Removes the dependency of listing models from the OpenAI schema.
- Tries to reduce confusion between ListModels() in model loader and in
the service - now there is only one ListModels which is in services
and does not depend anymore on the OpenAI schema
- The OpenAI-schema functions were moved nearby the OpenAI specific
endpoints that needs the schema
- Drops the ListModel Service structure as there was no real need for
it.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Wip p2p enhancements
* get online state
* Pass-by token to show in the dashboard
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Style
* Minor fixups
* parametrize SearchID
* Refactoring
* Allow to expose/bind more services
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Add federation
* Display federated mode in the WebUI
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Small fixups
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* make federated nodes visible from the WebUI
* Fix version display
* improve web page
* live page update
* visual enhancements
* enhancements
* visual enhancements
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This fixes a breakage in rendering the template. Now the models passed
by to the renderer have the ID field rather then Name
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* refactor(gallery): move under core/
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(unarchive): do not allow symlinks
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
contains simple fixes to warnings and errors, removes a broken / outdated test, runs go mod tidy, and as the actual change, centralizes base64 image handling
Signed-off-by: Dave Lee <dave@gray101.com>
* Support specifying oci:// and ollama:// for model URLs
Fixes: https://github.com/mudler/LocalAI/issues/2527
Fixes: https://github.com/mudler/LocalAI/issues/1028
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Lower watcher warnings
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Allow to install ollama models from CLI
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fixup tests
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Do not keep file ownership
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Skip test on darwin
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(ui): add page to talk with voice, transcription, and tts
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Enhance graphics and status reporting
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Better UX by blocking unvalid actions
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* wip: guess informations from gguf file
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* update go mod
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Small fixups
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Identify llama3
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Do not try to guess the name, as reading gguf files can be expensive
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Allow to disable guessing
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* pass basePath down to pkg/downloader
Signed-off-by: Dave Lee <dave@gray101.com>
* enforce
Signed-off-by: Dave Lee <dave@gray101.com>
---------
Signed-off-by: Dave Lee <dave@gray101.com>
* feat(functions): relax mixedgrammars
Extend even more the functionalities and when mixed mode is enabled,
tolerate also both strings and JSON in the result - in this case we make
sure that the JSON can be correctly parsed.
This also updates the examples and the gallery model to configure the
grammar.
The changeset also breaks current function/grammar configuration as it
reserves now a stanza in the YAML config.
For example:
```yaml
function:
grammar:
# This allows the grammar to also return messages
mixed_mode: true
# Suffix to add to the grammar
# prefix: '<tool_call>\n'
# Force parallel calls in the grammar
# parallel_calls: true
```
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* refactor, add a way to disable mixed json and freestring
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Fix linting issues
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
feat(functions): support mixed JSON BNF grammar
This PR provides new options to control how functions are extracted from
the LLM, and also provides more control on how JSON grammars can be used
(also in conjunction).
New YAML settings introduced:
- `grammar_message`: when enabled, the generated grammar can also decide
to push strings and not only JSON objects. This allows the LLM to pick
to either respond freely or using JSON.
- `grammar_prefix`: Allows to prefix a string to the JSON grammar
definition.
- `replace_results`: Is a map that allows to replace strings in the LLM
result.
As an example, consider the following settings for Hermes-2-Pro-Mistral,
which allow extracting both JSON results coming from the model, and the
ones coming from the grammar:
```yaml
function:
# disable injecting the "answer" tool
disable_no_action: true
# This allows the grammar to also return messages
grammar_message: true
# Suffix to add to the grammar
grammar_prefix: '<tool_call>\n'
return_name_in_function_response: true
# Without grammar uncomment the lines below
# Warning: this is relying only on the capability of the
# LLM model to generate the correct function call.
# no_grammar: true
# json_regex_match: "(?s)<tool_call>(.*?)</tool_call>"
replace_results:
"<tool_call>": ""
"\'": "\""
```
Note: To disable entirely grammars usage in the example above, uncomment the
`no_grammar` and `json_regex_match`.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
