LocalAI/examples/configurations/llava/llava.yaml

backend: llama-cpp
context_size: 4096
f16: true
threads: 11
gpu_layers: 90
mmap: true
name: llava
roles:
  user: "USER:"
  assistant: "ASSISTANT:"
  system: "SYSTEM:"
parameters:
  model: ggml-model-q4_k.gguf
  temperature: 0.2
  top_k: 40
  top_p: 0.95
template:
  chat: chat-simple
mmproj: mmproj-model-f16.gguf
llava.yaml (yaml format standardization) (#1303) Signed-off-by: lunamidori5 <118759930+lunamidori5@users.noreply.github.com> 2023-11-18 13:48:54 +00:00			`backend: llama-cpp`
:fire: add LaVA support and GPT vision API, Multiple requests for llama.cpp, return JSON types (#1254) * wip * wip * Make it functional Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * wip * Small fixups * do not inject space on role encoding, encode img at beginning of messages Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add examples/config defaults * Add include dir of current source dir * cleanup * fixes Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fixups * Revert "fixups" This reverts commit f1a4731ccadf7226c6589d6d39131376f0811625. * fixes Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> 2023-11-11 12:14:59 +00:00			`context_size: 4096`
			`f16: true`
			`threads: 11`
			`gpu_layers: 90`
			`mmap: true`
llava.yaml (yaml format standardization) (#1303) Signed-off-by: lunamidori5 <118759930+lunamidori5@users.noreply.github.com> 2023-11-18 13:48:54 +00:00			`name: llava`
:fire: add LaVA support and GPT vision API, Multiple requests for llama.cpp, return JSON types (#1254) * wip * wip * Make it functional Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * wip * Small fixups * do not inject space on role encoding, encode img at beginning of messages Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add examples/config defaults * Add include dir of current source dir * cleanup * fixes Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fixups * Revert "fixups" This reverts commit f1a4731ccadf7226c6589d6d39131376f0811625. * fixes Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> 2023-11-11 12:14:59 +00:00			`roles:`
			`user: "USER:"`
			`assistant: "ASSISTANT:"`
			`system: "SYSTEM:"`
			`parameters:`
			`model: ggml-model-q4_k.gguf`
			`temperature: 0.2`
			`top_k: 40`
			`top_p: 0.95`
			`template:`
			`chat: chat-simple`
llava.yaml (yaml format standardization) (#1303) Signed-off-by: lunamidori5 <118759930+lunamidori5@users.noreply.github.com> 2023-11-18 13:48:54 +00:00			`mmproj: mmproj-model-f16.gguf`