ExternalVendorCode/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2024-12-18 20:27:57 +00:00
Code Issues Actions 12 Packages Projects Releases Wiki Activity

models(gallery): add thebeagle-v2beta-32b-mgs (#3975)

Browse Source 
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
Ettore Di Giacinto 2024-10-26 14:56:41 +02:00 committed by GitHub
parent 175ae751ba
commit 4528e969c9
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 26 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

26
gallery/index.yaml
View File

@ -675,6 +675,32 @@
- filename: Meissa-Qwen2.5-7B-Instruct.Q4_K_M.gguf
sha256: 632b10d5c0e98bc8d53295886da2d57772a54bb6f6fa01d458e9e8c7fa9c905a
uri: huggingface://QuantFactory/Meissa-Qwen2.5-7B-Instruct-GGUF/Meissa-Qwen2.5-7B-Instruct.Q4_K_M.gguf
- !!merge <<: *qwen25
name: "thebeagle-v2beta-32b-mgs"
urls:
- https://huggingface.co/fblgit/TheBeagle-v2beta-32B-MGS
- https://huggingface.co/bartowski/TheBeagle-v2beta-32B-MGS-GGUF
description: |
This model is an experimental version of our latest innovation: MGS. Its up to you to figure out what does it means, but its very explicit. We didn't applied our known UNA algorithm to the forward pass, but they are entirely compatible and operates in different parts of the neural network and in different ways, tho they both can be seen as a regularization technique.
Updated tokenizer_config.json (from the base_model)
Regenerated Quants (being uploaded)
Re-submitted Leaderboard Evaluation, MATH & IFeval have relevant updates
Aligned LICENSE with Qwen terms.
MGS stands for... Many-Geeks-Searching... and thats it. Hint: 1+1 is 2, and 1+1 is not 3
We still believe on 1-Epoch should be enough, so we just did 1 Epoch only.
Dataset
Used here the first decent (corpora & size) dataset on the hub: Magpie-Align/Magpie-Pro-300K-Filtered Kudos to the Magpie team to contribute with some decent stuff that I personally think is very good to ablate.
It achieves the following results on the evaluation set:
Loss: 0.5378 (1 Epoch), outperforming the baseline model.
overrides:
parameters:
model: TheBeagle-v2beta-32B-MGS-Q4_K_M.gguf
files:
- filename: TheBeagle-v2beta-32B-MGS-Q4_K_M.gguf
sha256: db0d3b3c5341d2d51115794bf5da6552b5c0714b041de9b82065cc0c982dd4f7
uri: huggingface://bartowski/TheBeagle-v2beta-32B-MGS-GGUF/TheBeagle-v2beta-32B-MGS-Q4_K_M.gguf
- &archfunct
license: apache-2.0
tags: