2005 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Samuel Durante	fb36a1538a	readme : fix real-time audio input example build instructions (#2692)	2025-01-02 12:05:38 +02:00
Alter	c81b8b910b	objc : rename ggml-cpu-aarch64.c to .cpp (#2687)	2025-01-02 12:05:09 +02:00
Konosuke Sakai	85b60f31d0	docs : replace Core ML with OpenVINO (#2686)	2025-01-02 12:03:02 +02:00
Georgi Gerganov	227b5ffa36	make : fix "main" -> "whisper-cli"	2024-12-31 11:46:17 +02:00
Nikolaj Olsson	36a64a253f	ci : re-enable Windows cublas build (#2676) ... * Enable Windows cublas build * Re-add v12 cuda	2024-12-31 11:11:42 +02:00
KITAITI Makoto	c84b83c370	ruby : Fix of C++ header guard name, model URI support, type signature and more (#2683) ... * Add test to make Whisper::Context.new accept URI string * Add test to make Whisper::Context.new accept URI * Make Whisper::Context.new accept URI string and URI * Update README Revert "Fix argument of rb_undefine_finalizer" * Fix typos * Add type signature file * Assign literarl to const variable * Load Whisper::Model::URI from Init_whisper * Simplify .gitignore * Don't load whisper.so from whisper/model/uri.rb * Use each_with_object instead of each * Add Development section to README * Rename header guard to conform to C++ naming convention	2024-12-30 14:26:35 +02:00
Georgi Gerganov	5136fd92c2	examples : handle "main.exe" deprecation	2024-12-30 13:00:18 +02:00
Andreas Lubbe	7d55637f0b	cli : add --suppress_nst support (#2664)	2024-12-24 09:30:07 +02:00
Andreas Lubbe	0994506054	cli : add no_speech_thold (#2663)	2024-12-24 09:29:19 +02:00
Georgi Gerganov	53c9a3a984	cmake : remove hardcoded install rpath v1.7.4-pre-1	2024-12-23 21:22:10 +02:00
Georgi Gerganov	ed09075ca0	server : fix help print	2024-12-22 15:32:05 +02:00
KITAITI Makoto	f07a81aa9f	ruby : bug fix on callbacks and no_speech_prob (#2656) ... * Don't generate documentation on test * Move .startup to TestBase class * Extract new_segment_callback as a function * Extract progress_callback as a function * Extract abort_callback as a function * Extract register_callbacks as a function * Call callbacks in Whiser::Context#full and #full_parallel * Fix README * Care about the cases content-size is nil and TTY is not available * Add tests for no_speech_prob * Add Whisper::Context#full_get_segment_no_speech_prob and Whisper::Segment#no_speech_prob	2024-12-21 21:52:06 +02:00
Sacha Arbonel	4183517076	server : add no-speech threshold parameter and functionality (#2654)	2024-12-21 17:00:08 +02:00
Georgi Gerganov	f4668169a0	whisper : rename suppress_non_speech_tokens to suppress_nst (#2653)	2024-12-21 12:54:35 +02:00
Sacha Arbonel	944ce49439	server : add option to suppress non-speech tokens (#2649) ... * The parameter will suppress non-speech tokens like [LAUGH], [SIGH], etc. from the output when enabled. * add to whisper_params_parse * add missing param	2024-12-21 12:05:05 +02:00
Georgi Gerganov	2e59dced12	whisper : rename binaries + fix install (#2648) ... * whisper : rename binaries + fix install * cont : try to fix ci * cont : fix emscripten builds v1.7.4-pre-0	2024-12-21 09:43:49 +02:00
Georgi Gerganov	e4e05981d6	ruby : update gem version to v1.3.1	2024-12-20 11:53:27 +02:00
Georgi Gerganov	3de9deead5	release : v1.7.3 v1.7.3	2024-12-18 18:12:40 +02:00
Georgi Gerganov	47f989f9b3	ci : msys enable SDL2 build (#2635)	2024-12-18 12:52:41 +02:00
KITAITI Makoto	acc4e13dee	ruby : sync ggml (#2643)	2024-12-18 12:52:16 +02:00
Georgi Gerganov	ba6c2a8fd9	android : try to fix build	2024-12-18 12:52:16 +02:00
Georgi Gerganov	6576af00d7	files : remove old sources	2024-12-18 12:52:16 +02:00
Georgi Gerganov	8ac5db0169	sync : ggml	2024-12-18 12:52:16 +02:00
Georgi Gerganov	61edb117a0	talk-llama : sync llama.cpp	2024-12-18 12:52:16 +02:00
Georgi Gerganov	eb97b257eb	sync : ggml	2024-12-18 12:52:16 +02:00
Georgi Gerganov	479499dc0e	ggml : update ggml_backend_cpu_device_supports_op (llama/10867) ... * ggml : fix cpy op for IQ-quants to use reference impl ggml-ci * ggml : disable tests involving i-matrix quantization * ggml : update ggml_backend_cpu_device_supports_op ggml-ci	2024-12-18 12:52:16 +02:00
Eve	d420a759c5	vulkan: bugfixes for small subgroup size systems + llvmpipe test (llama/10809) ... * ensure mul mat shaders work on systems with subgroup size less than 32 more fixes add test * only s_warptile_mmq needs to be run with 32 threads or more	2024-12-18 12:52:16 +02:00
Zhiyuan Li	a1ab9b5e91	rwkv6: add wkv6 support for Vulkan backend (llama/10829) ... * rwkv_wkv6 vulkan shader * RWKV_WKV6 Vulkan op tests passed Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * Apply code format changes Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * add [[unroll]] and remove unnecessary conditions * add uma support * fix erros in EditorConfig Checker --------- Signed-off-by: Molly Sophia <mollysophia379@gmail.com> Co-authored-by: Molly Sophia <mollysophia379@gmail.com>	2024-12-18 12:52:16 +02:00
HimariO	e22d38e4f2	llama : add Qwen2VL support + multimodal RoPE (llama/10361) ... * Barebone Qwen2VL LLM convertor * Add Qwen2VL cli entrypoint * [WIP] add qwen2vl arch * Verify m-rope output * Add vl-rope/2d-rope support for qwen2vl ViT * update qwen2vl cli tool * update 5D tensor op workaround * [WIP] qwen2vl vision model * make batch and clip utils compatible with qwen2vl * [WIP] create inference workflow, gguf convert script but fix * correcting vision-rope behavior, add the missing last layer back to ViT * add arg parser to qwen2vl_surgery * replace variable size array with vector * cuda-gdb cmake preset * add fp32 mrope, vision rope kernel * add fp16 support for qwen2vl and m-rope * add `GGML_ROPE_TYPE_MROPE`, `GGML_ROPE_TYPE_VISION` * fix rope op mode switching, out dated func args * update `llama_hparams` * update to keep up stream changes * resolve linter, test errors * add makefile entry, update speical image padding token * add mrope unit test, fix few compiler warnings * rename `mrope` related function, params * minor updates on debug util, bug fixs * add `m-rope` testcase to `test-backend-ops` * Apply suggestions from code review Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> * fix traililng whitespce * store `llama_hparams.rope_sections` with fixed size array * update position id tensor size check in GGML_OP_ROPE * minor updates * update `ggml_backend_*_supports_op` of unsupported backends * remote old `rope_section` compare operator --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2024-12-18 12:52:16 +02:00
lhez	856fbaa92f	Introducing experimental OpenCL backend with support for Qualcomm Adreno GPUs (llama/10693) ... * [cl][adreno] Add Adreno GPU support Add new OpenCL backend to support Adreno GPUs --------- Co-authored-by: Skyler Szot <quic_sszot@quicinc.com> Co-authored-by: Shangqing Gu <quic_shawngu@quicinc.com> Co-authored-by: Alexander Angus <quic_aangus@quicinc.com> Co-authored-by: Hongqiang Wang <quic_wangh@quicinc.com> Co-authored-by: Max Krasnyansky <quic_maxk@quicinc.com> * [cl][ci] Add workflow for CL * [cl][adreno] Fix memory leak for non SMALL_ALLOC path * opencl: integrate backend dyn.load interface and fix compiler and format warnings * opencl: remove small-alloc support and fix build errors for non-opencl platforms * opencl: fixed merge conflict (MUSA added twice in cmake) * opencl-ci: use RUNNER_TEMP instead of github.workspace * opencl: fix embed tool invocation with python3 * opencl: CI workflow fixes * opencl: Clean up small-alloc in CMake files * opencl: cleanup ggml-opencl2 header file * opencl: use ulong for offsets and strides in ADD kernel * opencl: use cl_ulong for all offsets * opencl: use cl_ulong for sizes and strides * opencl: use `GGML_LOG_xxx` instead of `fprintf(stderr, ...)` * opencl: rename backend `opencl2` -> `opencl` * opencl: rename kernel files `ggml-opencl2` -> `ggml-opencl` * opencl: make OpenCL required, remove redundant lib and inc directories * `ggml-base`, `..` and `.` are added by `ggml_add_backend_library` * opencl: rename backend - funcs, structs, etc `opencl2` -> `opencl` * opencl: remove copyright marker since main license already covers * opencl: replace some more OPENCL2 leftovers * opencl: remove limits on `tensor_extra` * opencl: use pools for `tensor_extra` * opencl: fix compiler warnings with GCC and Clang Still getting the warning about clCreateCmdQueue being obsolete. Will fix that separately. * opencl: fail gracefully if opencl devices are not available Also for unsupported GPUs. * opencl: fix MSVC builds (string length error) * opencl: check for various requirements, allow deprecated API * opencl: update log message for unsupported GPUs --------- Co-authored-by: Skyler Szot <quic_sszot@quicinc.com> Co-authored-by: Shangqing Gu <quic_shawngu@quicinc.com> Co-authored-by: Alexander Angus <quic_aangus@quicinc.com> Co-authored-by: Hongqiang Wang <quic_wangh@quicinc.com> Co-authored-by: Max Krasnyansky <quic_maxk@quicinc.com>	2024-12-18 12:52:16 +02:00
谢乃闻	2c05efa4b1	Fix crash caused by ggml_backend_load_all when launching on Android Activity (llama/10812) ... * Fix crash caused by ggml_backend_load_all when launching on AndroidActivity. Details: Calling ggml_backend_load_all during initialization in the AndroidActivity project leads to a crash with the error: terminating with uncaught exception of type std::__ndk1::__fs::filesystem::filesystem_error: filesystem error: in directory_iterator::directory_iterator(...): Permission denied [./]. This issue occurs because AndroidActivity restricts file access due to sandboxing. Reproduction: In the example folder, the LlamaAndroid project can reproduce the crash by calling ggml_backend_load_all first in Java_android_llama_cpp_LLamaAndroid_backend_1init. * Update ggml/src/ggml-backend-reg.cpp --------- Co-authored-by: Diego Devesa <slarengh@gmail.com>	2024-12-18 12:52:16 +02:00
Eve	c21fb10b28	vulkan: small mul_mat_vec optimizations (llama/10665) ... * double the number of rows per workgroup * Update ggml-vulkan.cpp * Vulkan: Add VK_EXT_subgroup_size_control support to ensure full subgroups for coopmats * only increase the number of rows for amd and subgroup size 64 * fix missing NUM_ROWS for mul_mat_vec_iq4_nl_f16_f32, untested * use subgroup min and max to check for gcn (requires https://github.com/ggerganov/llama.cpp/pull/10721) * manual merge ggml-vulkan.cpp * set min and max subgroup size in any case * Also double the number of rows for Intel GPUs	2024-12-18 12:52:16 +02:00
Akarshan Biswas	26c9fd0cdc	SYCL: Reduce most of the compiler warnings (llama/10748) ... * Try to reduce some unused and typecast warnings * Reduce compiler warnings step 2 * add a newline at the end of the file * Initialize nreduce as size_t * [SYCL] Remove pragma directives from mmq.cpp * SYCL: mmq add condition to prevent blocks_per_tile_x_row variable from becoming 0 * SYCL softmax: Initialize nreduce as size_t * ggml-sycl.cpp: fix some trailing whitespaces * SYCL: remove the unused variables instead of commenting it out * SYCL poo2d kernel: set NAN for invalid pooling op * SYCL gemm.hpp: remove pragma directives * SYCL gemm.hpp: use const cast to properly support dnnl::memory * SYCL: wkv6 remove a comment * SYCL: clean comments step 2 * SYCL: clean comments and variables step 3 * SYCL: Use GGML_UNUSED for unused variables * SYCL: remove extra empty lines and a comment * Remove TODO * cleanup spaces * add a stdout for unsupported op * use sycl printf over fprintf * remove prints for CI * SYCL ggml-sycl: pool2D use sycl::nan and remove if-else block --------- Co-authored-by: Abhilash Majumder <30946547+abhilash1910@users.noreply.github.com>	2024-12-18 12:52:16 +02:00
Karol Kontny	e6eed605cf	ggml : Fix compilation issues on ARM platform when building without fp16 (llama/10811)	2024-12-18 12:52:16 +02:00
a3sh	abe3102cb7	CUDA: faster non-contiguous concat (llama/10760) ... * faster uncontiguous concat * Use a lambda to avoid code duplication Co-authored-by: Diego Devesa <slarengh@gmail.com> * Update ggml/src/ggml-cuda/concat.cu * add constexpr and static assert --------- Co-authored-by: Diego Devesa <slarengh@gmail.com>	2024-12-18 12:52:16 +02:00
Diego Devesa	1193e494a9	remove CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS (llama/10797) ... other windows build fixes	2024-12-18 12:52:16 +02:00
0cc4m	e5e951672e	Vulkan: Use improved q4_k and q5_k dequant code in dequant shaders (llama/10798)	2024-12-18 12:52:16 +02:00
0cc4m	0e24559ad9	Vulkan: Add VK_EXT_subgroup_size_control support to ensure full subgroups for coopmats (llama/10721) ... * Vulkan: Add VK_EXT_subgroup_size_control support to ensure full subgroups for coopmats * Fix subgroup size control extension support check Add accf32 and accf16 checks for coopmats * Also disable coopmats on amdvlk	2024-12-18 12:52:16 +02:00
Gilad S	527ac800cf	ggml: load all backends from a user-provided search path (llama/10699) ... * feat: load all backends from a user-provided search path * fix: Windows search path * refactor: rename `ggml_backend_load_all_in_search_path` to `ggml_backend_load_all_from_path` * refactor: rename `search_path` to `dir_path` * fix: change `NULL` to `nullptr` Co-authored-by: Diego Devesa <slarengh@gmail.com> * fix: change `NULL` to `nullptr` --------- Co-authored-by: Diego Devesa <slarengh@gmail.com>	2024-12-18 12:52:16 +02:00
Jeff Bolz	479bd77169	vulkan: request round-to-even for fp16 in im2col/rope_head (llama/10767) ... Vulkan doesn't mandate a specific rounding mode, but the shader_float_controls feature allows rounding mode to be requested if the implementation supports it.	2024-12-18 12:52:16 +02:00
Eve	d8bf63a41b	vulkan: dynamic subgroup size for the remaining k quants (llama/10745) ... * q5_k q4_k q3_k q2_k q6_k multi row example * revert as multi row isnt faster for k quants	2024-12-18 12:52:16 +02:00
Andreas Kieslinger	b82c8d76dc	CUDA: rename macros to avoid conflicts with WinAPI (llama/10736) ... * Renames NVIDIA GPU-architecture flags to avoid name clashes with WinAPI. (e.g. CC_PASCAL, GPU architecture or WinAPI pascal compiler flag?) * Reverts erroneous rename in SYCL-code. * Renames GGML_CUDA_MIN_CC_DP4A to GGML_CUDA_CC_DP4A. * Renames the rest of the compute capability macros for consistency.	2024-12-18 12:52:16 +02:00
Jeff Bolz	86346f811e	vulkan: disable spirv-opt for coopmat shaders (llama/10763) ... There are some bugs in the 1.3.296 SDK, so disable this. It isn't strictly necessary anyway. Add missing dependency on vulkan-shaders-gen, so shaders get recompiled when it changes. Fix coopmat support reporting when glslc doesn't support NV_coopmat2.	2024-12-18 12:52:16 +02:00
Daniel Bevenius	c635f40a34	ggml : remove return from ggml_gallocr_allocate_node (ggml/1048) ... This commit removes the return statement from ggml_gallocr_allocate_node function. The motivation behind this change is to make the code more readable and consistent.	2024-12-18 12:52:16 +02:00
Daniel Bevenius	e0be0de1ee	ggml : add check for grad_accs (ggml/1046) ... * ggml : add check for grad_accs This commit adds a check for grad_accs in ggml_graph_get_grad and ggml_graph_get_grad_acc functions. This is necessary to avoid segfaults when grad_accs is not initialized. The motivation for this change is that I find it nice to be able to print out a computation graph using ggml_graph_print but this function segfaults when grad_accs is not initialized: ```console (gdb) p g1 $2 = (ggml_cgraph *) 0x7ffff66004b0 (gdb) p *g1 $3 = {size = 2048, n_nodes = 1, n_leafs = 2, nodes = 0x7ffff6600500, grads = 0x0, grad_accs = 0x0, leafs = 0x7ffff6604500, visited_hash_set = {size = 4099, used = 0x7ffff6610518, keys = 0x7ffff6608500}, order = GGML_CGRAPH_EVAL_ORDER_LEFT_TO_RIGHT} (gdb) p ggml_graph_print(g1) === GRAPH === n_nodes = 1 Program received signal SIGSEGV, Segmentation fault. 0x0000555555579775 in ggml_graph_get_grad (cgraph=0x7ffff66004b0,node=0x7ffff6600340) at /ggml/ggml/src/ggml.c:5990 5990 return igrad != GGML_HASHSET_FULL && ggml_bitset_get(cgraph->visited_hash_set.used, igrad) ? cgraph->grads[igrad] : NULL; ``` * squash! ggml : add check for grad_accs Fix the check in ggml_graph_get_grad. The check was incorrectly using cgraph->grad_accs instead of cgraph->grads.	2024-12-18 12:52:16 +02:00
Georgi Gerganov	60dc6d003f	common : remove old types ... ggml-ci	2024-12-18 12:52:16 +02:00
Johannes Gäßler	eb27e0d834	CUDA: fix shared memory access condition for mmv (llama/10740)	2024-12-18 12:52:16 +02:00
Jeff Bolz	a682fdce0c	vulkan: fix compile warnings (llama/10731)	2024-12-18 12:52:16 +02:00
stduhpf	9ffbd3d969	Vulkan: fix NaN in tanh.comp with AMD proprietary driver on Windows (llama/10723) ... * Vulkan: fix NaN in tanh.comp * Faster NaN-free tanh	2024-12-18 12:52:16 +02:00
Jeff Bolz	6585a890b4	vulkan: compile a test shader in cmake to check for coopmat2 support (llama/10713)	2024-12-18 12:52:16 +02:00