ExternalVendorCode/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2024-12-19 20:57:54 +00:00
Code Issues Actions 13 Packages Projects Releases Wiki Activity

fix(tools): correctly render tools response in templates (#1932)

Browse Source 
* fix(tools): allow to correctly display both Functions and Tools

* models(hermes-2-pro): correctly display function results
This commit is contained in:
Ettore Di Giacinto 2024-03-30 19:02:07 +01:00 committed by GitHub
parent 61e5e6bc36
commit 957f428fd5
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 64 additions and 54 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

21
aio/gpu-8g/text-to-text.yaml
View File

@ -3,30 +3,27 @@ mmap: true
parameters: parameters:
model: huggingface://NousResearch/Hermes-2-Pro-Mistral-7B-GGUF/Hermes-2-Pro-Mistral-7B.Q6_K.gguf model: huggingface://NousResearch/Hermes-2-Pro-Mistral-7B-GGUF/Hermes-2-Pro-Mistral-7B.Q6_K.gguf
roles:
assistant_function_call: assistant
function: tool
template: template:
chat_message: | chat_message: |
<|im_start|>{{if eq .RoleName "assistant"}}assistant{{else if eq .RoleName "system"}}system{{else if eq .RoleName "function"}}{{.Role}}{{else if eq .RoleName "user"}}user{{end}} <|im_start|>{{if eq .RoleName "assistant"}}assistant{{else if eq .RoleName "system"}}system{{else if eq .RoleName "tool"}}tool{{else if eq .RoleName "user"}}user{{end}}
{{ if eq .RoleName "assistant_function_call" }}<tool_call>{{end}} {{ if .FunctionCall }}<tool_call>{{end}}
{{ if eq .RoleName "function" }}<tool_result>{{end}} {{ if eq .RoleName "tool" }}<tool_result>{{end}}
{{if .Content}}{{.Content}}{{end}} {{if .Content}}{{.Content}}{{end}}
{{if .FunctionCall}}{{toJson .FunctionCall}}{{end}} {{if .FunctionCall}}{{toJson .FunctionCall}}{{end}}
{{ if eq .RoleName "assistant_function_call" }}</tool_call>{{end}} {{ if .FunctionCall }}</tool_call>{{end}}
{{ if eq .RoleName "function" }}</tool_result>{{end}} {{ if eq .RoleName "tool" }}</tool_result>{{end}}
<|im_end|> <|im_end|>
# https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-GGUF#prompt-format-for-function-calling # https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-GGUF#prompt-format-for-function-calling
function: | function: |
<|im_start|>system <|im_start|>system
You are a function calling AI model. You are provided with function signatures within <tools></tools> XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions. Here are the available tools: You are a function calling AI model. You are provided with function signatures within <tools></tools> XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions. Here are the available tools:
<tools> <tools>
{{range .Functions}} {{range .Functions}}
{'type': 'function', 'function': {'name': '{{.Name}}', 'description': '{{.Description}}', 'parameters': {{toJson .Parameters}} }} {'type': 'function', 'function': {'name': '{{.Name}}', 'description': '{{.Description}}', 'parameters': {{toJson .Parameters}} }}
{{end}} {{end}}
</tools> </tools>
Use the following pydantic model json schema for each tool call you will make: Use the following pydantic model json schema for each tool call you will make:
{'title': 'FunctionCall', 'type': 'object', 'properties': {'arguments': {'title': 'Arguments', 'type': 'object'}, 'name': {'title': 'Name', 'type': 'string'}}, 'required': ['arguments', 'name']} {'title': 'FunctionCall', 'type': 'object', 'properties': {'arguments': {'title': 'Arguments', 'type': 'object'}, 'name': {'title': 'Name', 'type': 'string'}}, 'required': ['arguments', 'name']}
For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows: For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows:
<tool_call> <tool_call>
{'arguments': <args-dict>, 'name': <function-name>} {'arguments': <args-dict>, 'name': <function-name>}

21
aio/intel/text-to-text.yaml
View File

@ -4,30 +4,27 @@ f16: false
parameters: parameters:
model: huggingface://NousResearch/Hermes-2-Pro-Mistral-7B-GGUF/Hermes-2-Pro-Mistral-7B.Q6_K.gguf model: huggingface://NousResearch/Hermes-2-Pro-Mistral-7B-GGUF/Hermes-2-Pro-Mistral-7B.Q6_K.gguf
roles:
assistant_function_call: assistant
function: tool
template: template:
chat_message: | chat_message: |
<|im_start|>{{if eq .RoleName "assistant"}}assistant{{else if eq .RoleName "system"}}system{{else if eq .RoleName "function"}}{{.Role}}{{else if eq .RoleName "user"}}user{{end}} <|im_start|>{{if eq .RoleName "assistant"}}assistant{{else if eq .RoleName "system"}}system{{else if eq .RoleName "tool"}}tool{{else if eq .RoleName "user"}}user{{end}}
{{ if eq .RoleName "assistant_function_call" }}<tool_call>{{end}} {{ if .FunctionCall }}<tool_call>{{end}}
{{ if eq .RoleName "function" }}<tool_result>{{end}} {{ if eq .RoleName "tool" }}<tool_result>{{end}}
{{if .Content}}{{.Content}}{{end}} {{if .Content}}{{.Content}}{{end}}
{{if .FunctionCall}}{{toJson .FunctionCall}}{{end}} {{if .FunctionCall}}{{toJson .FunctionCall}}{{end}}
{{ if eq .RoleName "assistant_function_call" }}</tool_call>{{end}} {{ if .FunctionCall }}</tool_call>{{end}}
{{ if eq .RoleName "function" }}</tool_result>{{end}} {{ if eq .RoleName "tool" }}</tool_result>{{end}}
<|im_end|> <|im_end|>
# https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-GGUF#prompt-format-for-function-calling # https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-GGUF#prompt-format-for-function-calling
function: | function: |
<|im_start|>system <|im_start|>system
You are a function calling AI model. You are provided with function signatures within <tools></tools> XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions. Here are the available tools: You are a function calling AI model. You are provided with function signatures within <tools></tools> XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions. Here are the available tools:
<tools> <tools>
{{range .Functions}} {{range .Functions}}
{'type': 'function', 'function': {'name': '{{.Name}}', 'description': '{{.Description}}', 'parameters': {{toJson .Parameters}} }} {'type': 'function', 'function': {'name': '{{.Name}}', 'description': '{{.Description}}', 'parameters': {{toJson .Parameters}} }}
{{end}} {{end}}
</tools> </tools>
Use the following pydantic model json schema for each tool call you will make: Use the following pydantic model json schema for each tool call you will make:
{'title': 'FunctionCall', 'type': 'object', 'properties': {'arguments': {'title': 'Arguments', 'type': 'object'}, 'name': {'title': 'Name', 'type': 'string'}}, 'required': ['arguments', 'name']} {'title': 'FunctionCall', 'type': 'object', 'properties': {'arguments': {'title': 'Arguments', 'type': 'object'}, 'name': {'title': 'Name', 'type': 'string'}}, 'required': ['arguments', 'name']}
For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows: For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows:
<tool_call> <tool_call>
{'arguments': <args-dict>, 'name': <function-name>} {'arguments': <args-dict>, 'name': <function-name>}

55
core/http/endpoints/openai/chat.go
View File

@ -236,7 +236,7 @@ func ChatEndpoint(cl *config.BackendConfigLoader, ml *model.ModelLoader, startup
// if function call, we might want to customize the role so we can display better that the "assistant called a json action" // if function call, we might want to customize the role so we can display better that the "assistant called a json action"
// if an "assistant_function_call" role is defined, we use it, otherwise we use the role that is passed by in the request // if an "assistant_function_call" role is defined, we use it, otherwise we use the role that is passed by in the request
if i.FunctionCall != nil && i.Role == "assistant" { if (i.FunctionCall != nil || i.ToolCalls != nil) && i.Role == "assistant" {
roleFn := "assistant_function_call" roleFn := "assistant_function_call"
r := config.Roles[roleFn] r := config.Roles[roleFn]
if r != "" { if r != "" {
@ -246,6 +246,11 @@ func ChatEndpoint(cl *config.BackendConfigLoader, ml *model.ModelLoader, startup
r := config.Roles[role] r := config.Roles[role]
contentExists := i.Content != nil && i.StringContent != "" contentExists := i.Content != nil && i.StringContent != ""
fcall := i.FunctionCall
if len(i.ToolCalls) > 0 {
fcall = i.ToolCalls
}
// First attempt to populate content via a chat message specific template // First attempt to populate content via a chat message specific template
if config.TemplateConfig.ChatMessage != "" { if config.TemplateConfig.ChatMessage != "" {
chatMessageData := model.ChatMessageTemplateData{ chatMessageData := model.ChatMessageTemplateData{
@ -253,7 +258,7 @@ func ChatEndpoint(cl *config.BackendConfigLoader, ml *model.ModelLoader, startup
Role: r, Role: r,
RoleName: role, RoleName: role,
Content: i.StringContent, Content: i.StringContent,
FunctionCall: i.FunctionCall, FunctionCall: fcall,
FunctionName: i.Name, FunctionName: i.Name,
LastMessage: messageIndex == (len(input.Messages) - 1), LastMessage: messageIndex == (len(input.Messages) - 1),
Function: config.Grammar != "" && (messageIndex == (len(input.Messages) - 1)), Function: config.Grammar != "" && (messageIndex == (len(input.Messages) - 1)),
@ -271,35 +276,49 @@ func ChatEndpoint(cl *config.BackendConfigLoader, ml *model.ModelLoader, startup
content = templatedChatMessage content = templatedChatMessage
} }
} }
marshalAnyRole := func(f any) {
j, err := json.Marshal(f)
if err == nil {
if contentExists {
content += "\n" + fmt.Sprint(r, " ", string(j))
} else {
content = fmt.Sprint(r, " ", string(j))
}
}
}
marshalAny := func(f any) {
j, err := json.Marshal(f)
if err == nil {
if contentExists {
content += "\n" + string(j)
} else {
content = string(j)
}
}
}
// If this model doesn't have such a template, or if that template fails to return a value, template at the message level. // If this model doesn't have such a template, or if that template fails to return a value, template at the message level.
if content == "" { if content == "" {
if r != "" { if r != "" {
if contentExists { if contentExists {
content = fmt.Sprint(r, i.StringContent) content = fmt.Sprint(r, i.StringContent)
} }
if i.FunctionCall != nil { if i.FunctionCall != nil {
j, err := json.Marshal(i.FunctionCall) marshalAnyRole(i.FunctionCall)
if err == nil { }
if contentExists { if i.ToolCalls != nil {
content += "\n" + fmt.Sprint(r, " ", string(j)) marshalAnyRole(i.ToolCalls)
} else {
content = fmt.Sprint(r, " ", string(j))
}
}
} }
} else { } else {
if contentExists { if contentExists {
content = fmt.Sprint(i.StringContent) content = fmt.Sprint(i.StringContent)
} }
if i.FunctionCall != nil { if i.FunctionCall != nil {
j, err := json.Marshal(i.FunctionCall) marshalAny(i.FunctionCall)
if err == nil { }
if contentExists { if i.ToolCalls != nil {
content += "\n" + string(j) marshalAny(i.ToolCalls)
} else {
content = string(j)
}
}
} }
} }
// Special Handling: System. We care if it was printed at all, not the r branch, so check seperately // Special Handling: System. We care if it was printed at all, not the r branch, so check seperately

21
embedded/models/hermes-2-pro-mistral.yaml
View File

@ -3,30 +3,27 @@ mmap: true
parameters: parameters:
model: huggingface://NousResearch/Hermes-2-Pro-Mistral-7B-GGUF/Hermes-2-Pro-Mistral-7B.Q6_K.gguf model: huggingface://NousResearch/Hermes-2-Pro-Mistral-7B-GGUF/Hermes-2-Pro-Mistral-7B.Q6_K.gguf
roles:
assistant_function_call: assistant
function: tool
template: template:
chat_message: | chat_message: |
<|im_start|>{{if eq .RoleName "assistant"}}assistant{{else if eq .RoleName "system"}}system{{else if eq .RoleName "function"}}{{.Role}}{{else if eq .RoleName "user"}}user{{end}} <|im_start|>{{if eq .RoleName "assistant"}}assistant{{else if eq .RoleName "system"}}system{{else if eq .RoleName "tool"}}tool{{else if eq .RoleName "user"}}user{{end}}
{{ if eq .RoleName "assistant_function_call" }}<tool_call>{{end}} {{ if .FunctionCall }}<tool_call>{{end}}
{{ if eq .RoleName "function" }}<tool_result>{{end}} {{ if eq .RoleName "tool" }}<tool_result>{{end}}
{{if .Content}}{{.Content}}{{end}} {{if .Content}}{{.Content}}{{end}}
{{if .FunctionCall}}{{toJson .FunctionCall}}{{end}} {{if .FunctionCall}}{{toJson .FunctionCall}}{{end}}
{{ if eq .RoleName "assistant_function_call" }}</tool_call>{{end}} {{ if .FunctionCall }}</tool_call>{{end}}
{{ if eq .RoleName "function" }}</tool_result>{{end}} {{ if eq .RoleName "tool" }}</tool_result>{{end}}
<|im_end|> <|im_end|>
# https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-GGUF#prompt-format-for-function-calling # https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-GGUF#prompt-format-for-function-calling
function: | function: |
<|im_start|>system <|im_start|>system
You are a function calling AI model. You are provided with function signatures within <tools></tools> XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions. Here are the available tools: You are a function calling AI model. You are provided with function signatures within <tools></tools> XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions. Here are the available tools:
<tools> <tools>
{{range .Functions}} {{range .Functions}}
{'type': 'function', 'function': {'name': '{{.Name}}', 'description': '{{.Description}}', 'parameters': {{toJson .Parameters}} }} {'type': 'function', 'function': {'name': '{{.Name}}', 'description': '{{.Description}}', 'parameters': {{toJson .Parameters}} }}
{{end}} {{end}}
</tools> </tools>
Use the following pydantic model json schema for each tool call you will make: Use the following pydantic model json schema for each tool call you will make:
{'title': 'FunctionCall', 'type': 'object', 'properties': {'arguments': {'title': 'Arguments', 'type': 'object'}, 'name': {'title': 'Name', 'type': 'string'}}, 'required': ['arguments', 'name']} {'title': 'FunctionCall', 'type': 'object', 'properties': {'arguments': {'title': 'Arguments', 'type': 'object'}, 'name': {'title': 'Name', 'type': 'string'}}, 'required': ['arguments', 'name']}
For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows: For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows:
<tool_call> <tool_call>
{'arguments': <args-dict>, 'name': <function-name>} {'arguments': <args-dict>, 'name': <function-name>}