--- name: "chatml" config_file: | mmap: true function: disable_no_action: true grammar: mixed_mode: false disable: true parallel_calls: true expect_strings_after_json: true json_regex_match: - "(?s)<tool_call>(.*?)</tool_call>" - "(?s)<tool_call>(.*)" capture_llm_results: - (?s)<scratchpad>(.*?)</scratchpad> replace_llm_results: - key: (?s)<scratchpad>(.*?)</scratchpad> value: "" template: chat_message: | <|im_start|>{{ .RoleName }} {{ if .FunctionCall -}} Function call: {{ else if eq .RoleName "tool" -}} Function response: {{ end -}} {{ if .Content -}} {{.Content }} {{ end -}} {{ if .FunctionCall -}} {{toJson .FunctionCall}} {{ end -}}<|im_end|> function: | <|im_start|>system # Tools You may call one or more functions to assist with the user query. You are provided with function signatures within <tools></tools> XML tags: <tools> {{range .Functions}} {'type': 'function', 'function': {'name': '{{.Name}}', 'description': '{{.Description}}', 'parameters': {{toJson .Parameters}} }} {{end}} </tools> For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags: <tool_call> {"name": <function-name>, "arguments": <args-json-object>} </tool_call> <|im_end|> {{.Input -}} <|im_start|>assistant chat: | {{.Input -}} <|im_start|>assistant completion: | {{.Input}} context_size: 4096 f16: true stopwords: - '<|im_end|>' - '<dummy32000>' - '</s>' - "<|eot_id|>" - "<|end_of_text|>"