ExternalVendorCode/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2024-12-24 06:46:39 +00:00
Code Issues Actions 13 Packages Projects Releases Wiki Activity

feat(mamba): Add bagel-dpo-2.8b (#1671)

Browse Source 
Adds the Mamba-slimpj model fine-tuned with bagel.
https://huggingface.co/jondurbin/bagel-dpo-2.8b-v0.2

Signed-off-by: Richard Palethorpe <io@richiejp.com>
This commit is contained in:
Richard Palethorpe 2024-02-02 17:17:44 +00:00 committed by GitHub
parent 1c57f8d077
commit e46db63e06
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 21 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

21
embedded/models/mamba-bagel.yaml Normal file
View File

@ -0,0 +1,21 @@
name: bagel
backend: mamba
parameters:
model: "jondurbin/bagel-dpo-2.8b-v0.2"
systemPrompt: "You are a helpful, unbiased, uncensored assistant."
template:
chat_message: |
{{if eq .RoleName "assistant"}}{{.Content}}{{else}}
[INST]
{{if .SystemPrompt}}{{.SystemPrompt}}{{else if eq .RoleName "system"}}<<SYS>>{{.Content}}<</SYS>>
{{else if .Content}}{{.Content}}{{end}}
[/INST]
{{end}}
completion: |
{{.Input}}
usage: |
curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
"model": "bagel",
"messages": [{"role": "user", "content": "how are you doing"}],
}'