2067 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
KITAITI Makoto	7ffcd05267	ruby : Make context accept initial parameters, API to retrieve a segment and more (#2749) ... * Fix type signature for Whisper.log_set * Use cache file for model when offline * Extract ruby_whisper_transcribe() into a file * Extract Whisper::Error * Use FileList for ext/*.{c,cpp,h} * Extract Whisper::Segment * Extract Whisper::Model * Extract Whisper::Params * Extract Whisper::Context * Extract log_callback function * Write base code in C rather than C++ * Use chdir instead of Dir.chdir in Rakefile * Define alloc func for Whisper::Model * Define Whisper::Params' calback and user data reader * Add test for Whisper::Params.new with keyword arguments * Make Whisper::Params.new accept keyword arguments * Update type signatures * Update README * Update CLEAN targets * Fix document comment for Whisper::Params#new_segment_callback= * Use macro to define params * Fix dependency of build task * Set Whisper.finalize_log_callback visibility to private * Make Whisper::Context#full and full_parallel return self * Add test for Whisper::Context#full_get_segment * Add Whisper::Context#full_get_segment * Update signatures * Update README * Fix signature * Resplace #initialize with .new in signature file [skip ci] * Fix potential overflow	2025-01-21 09:39:54 +02:00
Corey Earwood	7a423f1c00	whisper.objc : fix build and CI	2025-01-18 12:06:06 +02:00
Georgi Gerganov	99b011a9f5	talk-llama : sync llama.cpp	2025-01-14 10:38:01 +02:00
Georgi Gerganov	19d95f9f9a	sync : ggml	2025-01-14 10:38:01 +02:00
Johannes Gäßler	d5ef1737d8	GGUF: C++ refactor, backend support, misc fixes (skip) (llama/11030) ... ggml-ci	2025-01-14 10:38:01 +02:00
lhez	1deb41f0e7	ggml : add opencl backend (skip) (llama/10693) ... --------- Co-authored-by: Skyler Szot <quic_sszot@quicinc.com> Co-authored-by: Shangqing Gu <quic_shawngu@quicinc.com> Co-authored-by: Alexander Angus <quic_aangus@quicinc.com> Co-authored-by: Hongqiang Wang <quic_wangh@quicinc.com> Co-authored-by: Max Krasnyansky <quic_maxk@quicinc.com>	2025-01-14 10:38:01 +02:00
Andreas Kieslinger	2425caf4fd	cuda : CUDA Graph Compute Function Refactor (precursor for performance improvements) (llama/11042) ... * Refactor: Moves cuda graph executable update step to separate function. * Refactor: Moves cuda graph update check to separate function. * Refactor: Moves cuda graph maintenance (update or adjusting copy parameters) to separate function for improved readability. * Fix: Adds missing reference to maintain_cuda_graph() definition. * Refactor: Improves structure and abstractions by moving CUDA graph evaluation and capture to its own function. * Refactor: Moves node graph checks and copy ops into individual function for improved readability. * Refactor: Removes code permanently excluded from compilation to increase readability. * Style: Adds missing newline * Style: Consolidates several neighboring '#ifdef USE_CUDA_GRAPH' into a single one * Refactor: Makes 'cuda_graph_update_required' a local variable * remove double lines between functions --------- Co-authored-by: slaren <slarengh@gmail.com>	2025-01-14 10:38:01 +02:00
Radoslav Gerganov	a4b00bcaaf	ggml : do not define GGML_USE_CUDA when building with GGML_BACKEND_DL (llama/11211) ... Build fails when using HIP and GGML_BACKEND_DL: ``` /usr/bin/ld: ../ggml/src/libggml.so: undefined reference to `ggml_backend_cuda_reg' collect2: error: ld returned 1 exit status ``` This patch fixes this.	2025-01-14 10:38:01 +02:00
0cc4m	cdb8aa2f2e	Vulkan: Fix float16 use on devices without float16 support + fix subgroup_size_control validation error (llama/11161) ... * Vulkan: Remove float16 use in shaders * Fix validation error about subgroup_size_control extension	2025-01-14 10:38:01 +02:00
Molly Sophia	06209f6683	llama: add support for QRWKV6 model architecture (llama/11001) ... llama: add support for QRWKV6 model architecture (llama/11001) * WIP: Add support for RWKV6Qwen2 Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * RWKV: Some graph simplification Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * Add support for RWKV6Qwen2 with cpu and cuda GLA Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * RWKV6[QWEN2]: Concat lerp weights together to reduce cpu overhead Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * Fix some typos Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * code format changes Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * Fix wkv test & add gla test Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * Fix cuda warning Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * Update README.md Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * Update ggml/src/ggml-cuda/gla.cu Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> * Fix fused lerp weights loading with RWKV6 Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * better sanity check skipping for QRWKV6 in llama-quant thanks @compilade Signed-off-by: Molly Sophia <mollysophia379@gmail.com> Co-authored-by: compilade <git@compilade.net> --------- Signed-off-by: Molly Sophia <mollysophia379@gmail.com> Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> Co-authored-by: compilade <git@compilade.net>	2025-01-14 10:38:01 +02:00
Akarshan Biswas	c3235bd81e	SYCL: Refactor ggml_sycl_compute_forward (llama/11121) ... * SYCL: refactor ggml_sycl_compute_forward * SYCL: add back GGML_USED(dst) to ggml_sycl_cpy * SYCL: add function name to noop debug * SYCL: Some device info print refactoring and add details of XMX availability	2025-01-14 10:38:01 +02:00
hydai	262d0abc87	fix: add missing msg in static_assert (llama/11143) ... Signed-off-by: hydai <z54981220@gmail.com>	2025-01-14 10:38:01 +02:00
amritahs-ibm	124eec1664	llamafile : ppc64le MMA INT8 implementation (llama/10912) ... This change upstreams llamafile's cpu matrix multiplication kernels for ppc64le using MMA builtins for quantised int8 datatype. This change results in 10% - 70% improvement in total speed(ie all tokens/total time), across various batch sizes. The patch is tested with Meta-Lllama-3-8B, Mistral-7B, Llama-2-7B-chat-hf models on a IBM POWER10 machine. Signed-off-by: Amrita H S <amritahs@linux.vnet.ibm.com>	2025-01-14 10:38:01 +02:00
Mathieu Baudier	b08c3a88c8	Disable GL_KHR_cooperative_matrix Vulkan extension if not available. (llama/11117) ... * Disable GL_KHR_cooperative_matrix Vulkan extension if not available. * Perform Vulkan extensions checks in a more sensible order * Remove unnecessary #ifdef directive	2025-01-14 10:38:01 +02:00
ag2s20150909	0afce25a69	fix: Vulkan shader gen binary path when Cross-compiling (llama/11096) ... * fix: Vulkan shader gen binary path when cross compiling	2025-01-14 10:38:01 +02:00
Johannes Gäßler	acdbe58631	GGUF: C++ refactor, backend support, misc fixes (llama/11030) ... * GGUF: C++ refactor, backend support, misc fixes remove ggml_tensor.backend update CODEOWNERS [no ci] remove gguf_get_data from API revise GGUF API data types	2025-01-14 10:38:01 +02:00
Diego Devesa	09fabffdf5	ggml-backend : only offload from host buffers (fix) (llama/11124)	2025-01-14 10:38:01 +02:00
Diego Devesa	3988d6396b	ggml-backend : only offload from host buffers (llama/11120)	2025-01-14 10:38:01 +02:00
Radoslav Gerganov	c8c63eeec0	rpc : code cleanup (llama/11107) ... Remove duplicated macros, use GGML_LOG_ERROR for errors	2025-01-14 10:38:01 +02:00
Akarshan Biswas	abf7f24410	SYCL: Use get_multi_ptr instead of deprecated get_pointer in wkv6 (llama/11087) ... * SYCL: Use get_multi_ptr instead of deprecated get_pointer in wkv6 * Revert "SYCL: Use get_multi_ptr instead of deprecated get_pointer in wkv6" This reverts commit f62dc45f318e48d375e7734b34cbddee81deed52. * Reland: Use get_multi_ptr instead of deprecated get_pointer in wkv6	2025-01-14 10:38:01 +02:00
Johannes Gäßler	341f5c28e6	CUDA: add BF16 support (llama/11093) ... * CUDA: add BF16 support	2025-01-14 10:38:01 +02:00
0cc4m	5377099524	Vulkan: Add device-specific blacklist for coopmat for the AMD proprietary driver (llama/11074) ... * Vulkan: Add device-specific blacklist for coopmat for the AMD proprietary driver * Add (TM) to AMD name check	2025-01-14 10:38:01 +02:00
matt23654	dcbb375779	Support for models with non-512-aligned tensors over RPC. (llama/11047) ... * Added init tensor calling code * Added get_alloc_size forwarding * Cleaned up and improved type/error handling. * fix: remove trailing whitespaces. * Cleanup and use GGML error logging functions. * Handle potentially dangerous edge cases. * Apply suggestions from code review Co-authored-by: Diego Devesa <slarengh@gmail.com> --------- Co-authored-by: Diego Devesa <slarengh@gmail.com>	2025-01-14 10:38:01 +02:00
Gilad S.	4334c71aed	fix: Vulkan shader gen binary path (llama/11037)	2025-01-14 10:38:01 +02:00
Radoslav Gerganov	e875a82473	ggml : allow loading backend with env variable (ggml/1059) ... ref: #1058	2025-01-14 10:38:01 +02:00
Georgi Gerganov	507e230f1e	scripts : sync opencl, gguf	2025-01-14 09:42:16 +02:00
Georgi Gerganov	eb68324c86	whisper : fix gpu device selection (#2728)	2025-01-13 13:11:37 +02:00
Georgi Gerganov	e940fbf283	server : fix build (#2718)	2025-01-13 08:57:33 +02:00
Georgi Gerganov	35d0e02c72	talk-llama : sync llama.cpp (#2709)	2025-01-13 08:55:48 +02:00
NETZkultur GmbH	45d3faf961	server : generate unique tmp filenames (#2718) ... #Summary This Merge Request adds a mechanism to generate unique filenames for FFmpeg conversions in whisper_server.cpp. Previously, a single fixed filename was used (e.g., whisper-server-tmp.wav), which could result in unexpected file overwrites under certain circumstances. By generating a unique filename per request, any risk of overwriting temporary files is eliminated. #Background / Motivation • Problem: Relying on a static filename for temporary audio files may lead to overwrites if multiple operations occur simultaneously or if the same file name is reused. • Goal: Dynamically generate unique filenames, ensuring each request or operation uses an isolated temporary file.	2025-01-13 08:55:21 +02:00
Sandro Hanea	2ab2eb5110	whisper : add whisper_full_get_segment_no_speech_prob_from_state (#2716)	2025-01-09 16:21:07 +02:00
Jayant	b82d305282	readme : add docker instructions (#2711) ... I found the docker instructions to be useful in the README.md and the differences in docker variants such as ffmpeg and cuda support. However, this section was removed in v1.7.4 and I would vote to bring it back. This is a pull request to add that section back.	2025-01-07 13:20:51 +02:00
Adam Jones	885e31368d	docs: Fix main -> whisper-cli in download scripts (#2707)	2025-01-06 15:17:57 +02:00
Georgi Gerganov	8a9ad7844d	release : v1.7.4 v1.7.4	2025-01-06 15:13:48 +02:00
Georgi Gerganov	eb874b3a3c	ci : cont	2025-01-06 10:46:10 +02:00
Georgi Gerganov	eb78e3a3f1	ci : fix ubuntu runner names	2025-01-06 09:29:10 +02:00
Yusuf Redžić	ece3ff88f6	cli : fix segfault on missing argument (#2700)	2025-01-04 10:47:41 +02:00
Georgi Gerganov	9366544991	ci : fix arm builds	2025-01-04 10:45:01 +02:00
Georgi Gerganov	95583942ed	sync : ggml ... ggml-ci	2025-01-04 10:45:01 +02:00
Georgi Gerganov	2e93cb6a2f	ggml : do not install metal source when embed library (ggml/1054)	2025-01-04 10:45:01 +02:00
Georgi Gerganov	de5cd60d1c	metal : avoid uint (llama/11019)	2025-01-04 10:45:01 +02:00
Srihari-mcw	3fcba3e58b	ggml : fixes for AVXVNNI instruction set with MSVC and Clang (llama/11027) ... * Fixes for clang AVX VNNI * enable AVX VNNI and alder lake build for MSVC * Apply suggestions from code review --------- Co-authored-by: slaren <slarengh@gmail.com>	2025-01-04 10:45:01 +02:00
Jeff Bolz	cea5f1c52f	vulkan: optimize mul_mat for small values of N (llama/10991) ... Make the mul_mat_vec shaders support N>1 (as a spec constant, NUM_COLS) where the batch_strides are overloaded to hold the row strides. Put the loads from the B matrix in the innermost loop because it should cache better. Share some code for reducing the result values to memory in mul_mat_vec_base.	2025-01-04 10:45:01 +02:00
Jeff Bolz	2112462db4	vulkan: im2col and matmul optimizations for stable diffusion (llama/10942) ... * tests: Add im2col perf tests * vulkan: optimize im2col, more elements per thread * vulkan: increase small tile size for NV_coopmat2 * vulkan: change im2col to 512 elements per workgroup	2025-01-04 10:45:01 +02:00
Jeff Bolz	fc84ecd445	vulkan: Use push constant offset to handle misaligned descriptors (llama/10987)	2025-01-04 10:45:01 +02:00
Eve	8de1e99907	vulkan: multi-row k quants (llama/10846) ... * multi row k quant shaders! * better row selection * more row choices * readjust row selection * rm_kq=2 by default	2025-01-04 10:45:01 +02:00
Peter	499af9294a	examples, ggml : fix GCC compiler warnings (llama/10983) ... Warning types fixed (observed under MSYS2 GCC 14.2.0): * format '%ld' expects argument of type 'long int', but argument has type 'size_t' * llama.cpp/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp:81:46: warning: missing initializer for member '_STARTUPINFOA::lpDesktop' [-Wmissing-field-initializers] (emitted for all struct field except first)	2025-01-04 10:45:01 +02:00
Djip007	bcf937c216	ggml : more perfo with llamafile tinyblas on x86_64 (llama/10714) ... * more perfo with llamafile tinyblas on x86_64. - add bf16 suport - change dispache strategie (thanks: https://github.com/ikawrakow/ik_llama.cpp/pull/71 ) - reduce memory bandwidth simple tinyblas dispache and more cache freindly * tinyblas dynamic dispaching * sgemm: add M blocs. * - git 2.47 use short id of len 9. - show-progress is not part of GNU Wget2 * remove not stable test	2025-01-04 10:45:01 +02:00
Diego Devesa	b8d90953d7	ggml : use wstring for backend search paths (llama/10960) ... ggml-ci	2025-01-04 10:45:01 +02:00
Diego Devesa	60a422147b	ggml : fix arm enabled features check (llama/10961)	2025-01-04 10:45:01 +02:00