From 7ec02babd514da597bb493092d95f38d29d76714 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Wed, 21 Aug 2024 13:09:12 +0200 Subject: [PATCH 1/4] fix(parler-tts): pin numba Signed-off-by: Ettore Di Giacinto --- backend/python/parler-tts/requirements-after.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/backend/python/parler-tts/requirements-after.txt b/backend/python/parler-tts/requirements-after.txt index 086e3e98..09811bf4 100644 --- a/backend/python/parler-tts/requirements-after.txt +++ b/backend/python/parler-tts/requirements-after.txt @@ -1,2 +1,3 @@ git+https://github.com/huggingface/parler-tts.git@8e465f1b5fcd223478e07175cb40494d19ffbe17 llvmlite==0.43.0 +numba==0.60.0 From 2a3427e533d7c8d0347e572744633eb2b78ea9fc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E5=9B=9B=E5=B0=91=E7=88=B7?= Date: Wed, 21 Aug 2024 19:09:26 +0800 Subject: [PATCH 2/4] =?UTF-8?q?fix(docs):=20Refer=20to=20the=20OpenAI=20do?= =?UTF-8?q?cumentation=20to=20update=20the=20openai-functions=20docu?= =?UTF-8?q?=E2=80=A6=20(#3317)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Refer to the OpenAI documentation to update the openai-functions documentation I saw the openai official website, apIn the description: The parameters `function_call` and `functions` have been replaced by `tool_choice` and `tools`.So I submitted this update;But I haven't read the code of localai, so I'm not sure if it also applies to localai. Signed-off-by: 四少爷 * Update Usage Example The original usage example was too outdated, and calling with the new version of the openai python package would result in errors. Therefore, the curl example was rewritten (as curl examples are also used elsewhere). Signed-off-by: 四少爷 * add python example Signed-off-by: 四少爷 --------- Signed-off-by: 四少爷 --- .../content/docs/features/openai-functions.md | 126 ++++++++++++++---- 1 file changed, 102 insertions(+), 24 deletions(-) diff --git a/docs/content/docs/features/openai-functions.md b/docs/content/docs/features/openai-functions.md index cb667815..5d43ece0 100644 --- a/docs/content/docs/features/openai-functions.md +++ b/docs/content/docs/features/openai-functions.md @@ -40,43 +40,121 @@ parameters: To use the functions with the OpenAI client in python: ```python -import openai +from openai import OpenAI + # ... # Send the conversation and available functions to GPT -messages = [{"role": "user", "content": "What's the weather like in Boston?"}] -functions = [ +messages = [{"role": "user", "content": "What is the weather like in Beijing now?"}] +tools = [ { - "name": "get_current_weather", - "description": "Get the current weather in a given location", - "parameters": { - "type": "object", - "properties": { - "location": { - "type": "string", - "description": "The city and state, e.g. San Francisco, CA", + "type": "function", + "function": { + "name": "get_current_weather", + "description": "Return the temperature of the specified region specified by the user", + "parameters": { + "type": "object", + "properties": { + "location": { + "type": "string", + "description": "User specified region", + }, + "unit": { + "type": "string", + "enum": ["celsius", "fahrenheit"], + "description": "temperature unit" + }, }, - "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}, + "required": ["location"], }, - "required": ["location"], }, } ] -response = openai.ChatCompletion.create( - model="gpt-3.5-turbo", - messages=messages, - functions=functions, - function_call="auto", + +client = OpenAI( + # This is the default and can be omitted + api_key="test", + base_url="http://localhost:8080/v1/" ) -# ... + +response =client.chat.completions.create( + messages=messages, + tools=tools, + tool_choice ="auto", + model="gpt-4", +) +#... ``` -{{% alert note %}} -When running the python script, be sure to: +For example, with curl: -- Set `OPENAI_API_KEY` environment variable to a random string (the OpenAI api key is NOT required!) -- Set `OPENAI_API_BASE` to point to your LocalAI service, for example `OPENAI_API_BASE=http://localhost:8080` +```bash +curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{ + "model": "gpt-4", + "messages": [{"role": "user", "content": "What is the weather like in Beijing now?"}], + "tools": [ + { + "type": "function", + "function": { + "name": "get_current_weather", + "description": "Return the temperature of the specified region specified by the user", + "parameters": { + "type": "object", + "properties": { + "location": { + "type": "string", + "description": "User specified region" + }, + "unit": { + "type": "string", + "enum": ["celsius", "fahrenheit"], + "description": "temperature unit" + } + }, + "required": ["location"] + } + } + } + ], + "tool_choice":"auto" +}' +``` -{{% /alert %}} +Return data： + +```json +{ + "created": 1724210813, + "object": "chat.completion", + "id": "16b57014-477c-4e6b-8d25-aad028a5625e", + "model": "gpt-4", + "choices": [ + { + "index": 0, + "finish_reason": "tool_calls", + "message": { + "role": "assistant", + "content": "", + "tool_calls": [ + { + "index": 0, + "id": "16b57014-477c-4e6b-8d25-aad028a5625e", + "type": "function", + "function": { + "name": "get_current_weather", + "arguments": "{\"location\":\"Beijing\",\"unit\":\"celsius\"}" + } + } + ] + } + } + ], + "usage": { + "prompt_tokens": 221, + "completion_tokens": 26, + "total_tokens": 247 + } +} +``` ## Advanced From d3a217c2540ffa8882ad8f185017e0554344b557 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Wed, 21 Aug 2024 13:09:57 +0200 Subject: [PATCH 3/4] chore(docs): update p2p env var documentation (#3350) Signed-off-by: Ettore Di Giacinto --- docs/content/docs/features/distributed_inferencing.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/content/docs/features/distributed_inferencing.md b/docs/content/docs/features/distributed_inferencing.md index 2de7ae3c..c8c60f8d 100644 --- a/docs/content/docs/features/distributed_inferencing.md +++ b/docs/content/docs/features/distributed_inferencing.md @@ -130,8 +130,10 @@ There are options that can be tweaked or parameters that can be set using enviro | Environment Variable | Description | |----------------------|-------------| | **LOCALAI_P2P_DISABLE_DHT** | Set to "true" to disable DHT and enable p2p layer to be local only (mDNS) | -| **LOCALAI_P2P_DISABLE_LIMITS** | Set to "true" to disable connection limits and resources management | +| **LOCALAI_P2P_ENABLE_LIMITS** | Set to "true" to enable connection limits and resources management (useful when running with poor connectivity or want to limit resources consumption) | | **LOCALAI_P2P_TOKEN** | Set the token for the p2p network | +| **LOCALAI_P2P_LOGLEVEL** | Set the loglevel for the LocalAI p2p stack (default: info) | +| **LOCALAI_LIBP2P_LOGLEVEL** | Set the loglevel for the underlying libp2p stack (default: fatal) | ## Architecture From b5103523939b6fd2159ed13223e657db62adc0d2 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Wed, 21 Aug 2024 13:10:09 +0200 Subject: [PATCH 4/4] chore(anime.js): drop unused (#3351) * fix(anime.js): correctly set the static path Signed-off-by: Ettore Di Giacinto * Drop anime.js (unused) Signed-off-by: Ettore Di Giacinto --------- Signed-off-by: Ettore Di Giacinto --- core/http/views/partials/head.html | 6 +----- embedded/webui_static.yaml | 5 +---- 2 files changed, 2 insertions(+), 9 deletions(-) diff --git a/core/http/views/partials/head.html b/core/http/views/partials/head.html index 8d072093..5c119fba 100644 --- a/core/http/views/partials/head.html +++ b/core/http/views/partials/head.html @@ -6,11 +6,7 @@ rel="stylesheet" href="/static/assets/highlightjs.css" /> - - +