HimariO e22d38e4f2 llama : add Qwen2VL support + multimodal RoPE (llama/10361) ... * Barebone Qwen2VL LLM convertor * Add Qwen2VL cli entrypoint * [WIP] add qwen2vl arch * Verify m-rope output * Add vl-rope/2d-rope support for qwen2vl ViT * update qwen2vl cli tool * update 5D tensor op workaround * [WIP] qwen2vl vision model * make batch and clip utils compatible with qwen2vl * [WIP] create inference workflow, gguf convert script but fix * correcting vision-rope behavior, add the missing last layer back to ViT * add arg parser to qwen2vl_surgery * replace variable size array with vector * cuda-gdb cmake preset * add fp32 mrope, vision rope kernel * add fp16 support for qwen2vl and m-rope * add `GGML_ROPE_TYPE_MROPE`, `GGML_ROPE_TYPE_VISION` * fix rope op mode switching, out dated func args * update `llama_hparams` * update to keep up stream changes * resolve linter, test errors * add makefile entry, update speical image padding token * add mrope unit test, fix few compiler warnings * rename `mrope` related function, params * minor updates on debug util, bug fixs * add `m-rope` testcase to `test-backend-ops` * Apply suggestions from code review Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> * fix traililng whitespce * store `llama_hparams.rope_sections` with fixed size array * update position id tensor size check in GGML_OP_ROPE * minor updates * update `ggml_backend_*_supports_op` of unsupported backends * remote old `rope_section` compare operator --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>		2024-12-18 12:52:16 +02:00
..
ggml-alloc.h	ggml : fix typo in example usage ggml_gallocr_new (ggml/984)	2024-10-05 15:23:51 +03:00
ggml-backend.h	ggml: load all backends from a user-provided search path (llama/10699)	2024-12-18 12:52:16 +02:00
ggml-blas.h	ggml : build backends as libraries (llama/10256)	2024-11-20 21:00:08 +02:00
ggml-cann.h	ggml : build backends as libraries (llama/10256)	2024-11-20 21:00:08 +02:00
ggml-cpp.h	llama : use smart pointers for ggml resources (llama/10117)	2024-11-15 15:21:04 +02:00
ggml-cpu.h	ggml : refactor online repacking (llama/10446)	2024-12-18 12:52:16 +02:00
ggml-cuda.h	ggml : build backends as libraries (llama/10256)	2024-11-20 21:00:08 +02:00
ggml-kompute.h	ggml : build backends as libraries (llama/10256)	2024-11-20 21:00:08 +02:00
ggml-metal.h	ggml : build backends as libraries (llama/10256)	2024-11-20 21:00:08 +02:00
ggml-opencl.h	Introducing experimental OpenCL backend with support for Qualcomm Adreno GPUs (llama/10693)	2024-12-18 12:52:16 +02:00
ggml-opt.h	ggml: new optimization interface (ggml/988)	2024-11-20 21:00:08 +02:00
ggml-rpc.h	ggml : build backends as libraries (llama/10256)	2024-11-20 21:00:08 +02:00
ggml-sycl.h	ggml : build backends as libraries (llama/10256)	2024-11-20 21:00:08 +02:00
ggml-vulkan.h	ggml : build backends as libraries (llama/10256)	2024-11-20 21:00:08 +02:00
ggml.h	llama : add Qwen2VL support + multimodal RoPE (llama/10361)	2024-12-18 12:52:16 +02:00