fix(llama.cpp): disable mirostat as default (#2911)

Even if increasing the quality of the output, it has shown to have performance drawbacks to be so noticeable that the confuses users about speed of LocalAI ( see also https://github.com/mudler/LocalAI/issues/2780 ). This changeset disables Mirostat by default (which can be still enabled manually). Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Dave <dave@gray101.com>
2025-03-23 04:25:32 +00:00 · 2025-02-06 19:39:59 +01:00 · 2025-02-06 19:39:59 +01:00 · cc1f6f913f
commit cc1f6f913f
parent 7f90ff7aec
1 changed files with 2 additions and 1 deletions
--- a/core/config/backend_config.go
+++ b/core/config/backend_config.go
@ -287,7 +287,8 @@ func (cfg *BackendConfig) SetDefaults(opts ...ConfigLoaderOption) {
 	defaultTopP := 0.95
 	defaultTopK := 40
 	defaultTemp := 0.9
-	defaultMirostat := 2
+	// https://github.com/mudler/LocalAI/issues/2780
+	defaultMirostat := 0
 	defaultMirostatTAU := 5.0
 	defaultMirostatETA := 0.1
 	defaultTypicalP := 1.0