2328 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Georgi Gerganov	dbcc669e1a	sync : ggml	2025-02-04 13:03:09 +02:00
Christian Kastner	16245b35e4	cmake: Add ability to pass in GGML_BUILD_NUMBER (ggml/1096) ... This makes git as a dependency optional, and is useful in the case where ggml is built not from git, but from a tarball, or a distribution source package. This conditional also affects GGML_BUILD_COMMIT. Nothing seems to be using it, though, so there doesn't seem much value factor it out, or even require it.	2025-02-04 13:03:03 +02:00
Georgi Gerganov	898c0cb9d1	readme : add maintenance roadmap	2025-02-04 10:50:10 +02:00
Georgi Gerganov	eb9e5032c4	ci : add stalebot	2025-02-04 09:30:20 +02:00
billyct	cadfc50eab	node : add max_len params in node addon (#2760)	2025-02-03 22:49:06 +02:00
Georgi Gerganov	3f91832352	talk-llama : sync llama.cpp	2025-02-03 22:42:26 +02:00
mgrachten	cff8868b5f	coreml : always convert to "neuralnetwork" (#2770)	2025-02-03 22:36:32 +02:00
Georgi Gerganov	90e3c5fc40	ci : more git	2025-02-03 22:00:57 +02:00
Georgi Gerganov	e0f4cef867	ci : install git	2025-02-03 22:00:57 +02:00
Georgi Gerganov	234460987e	ci : use ubuntu-22.04 instead of ubuntu-latest	2025-02-03 22:00:57 +02:00
Georgi Gerganov	b8ab126343	cmake : sync cmake scripts	2025-02-03 22:00:57 +02:00
Georgi Gerganov	edc5d9267c	sync : ggml	2025-02-03 22:00:57 +02:00
Georgi Gerganov	344b98a44f	scripts : fix sync paths	2025-02-03 22:00:57 +02:00
Johannes Gäßler	dbeb7916b8	CUDA: fix Volta FlashAttention logic (llama/11615)	2025-02-03 22:00:57 +02:00
Johannes Gäßler	fad2806352	HIP: fix flash_attn_stream_k_fixup warning (llama/11604)	2025-02-03 22:00:57 +02:00
uvos	9906792ec3	CUDA/HIP: add support for selectable warp size to mmv (llama/11519) ... CUDA/HIP: add support for selectable warp size to mmv	2025-02-03 22:00:57 +02:00
uvos	c49ee07ff4	HIP: add GGML_CUDA_CC_IS_* for amd familys as increasing cc archtectures for amd gpus are not supersets of eatch other (llama/11601) ... This fixes a bug where RDNA1 gpus other than gfx1010 where not handled correctly	2025-02-03 22:00:57 +02:00
Johannes Gäßler	f8a831779e	CUDA: use mma PTX instructions for FlashAttention (llama/11583) ... * CUDA: use mma PTX instructions for FlashAttention * __shfl_sync workaround for movmatrix * add __shfl_sync to HIP Co-authored-by: Diego Devesa <slarengh@gmail.com>	2025-02-03 22:00:57 +02:00
Olivier Chafik	85451e3612	ci: use sccache on windows instead of ccache (llama/11545) ... * Use sccache on ci for windows * Detect sccache in cmake	2025-02-03 22:00:57 +02:00
uvos	43c744ce8b	HIP: require at least HIP 5.5	2025-02-03 22:00:57 +02:00
uvos	fc2e44490d	HIP: Prepare reduction operators for wave 64	2025-02-03 22:00:57 +02:00
uvos	f41fdad200	CUDA/HIP: add warp_size to cuda_device_info	2025-02-03 22:00:57 +02:00
Rémy Oudompheng	80fa576254	vulkan: implement initial support for IQ2 and IQ3 quantizations (llama/11360) ... * vulkan: initial support for IQ3_S * vulkan: initial support for IQ3_XXS * vulkan: initial support for IQ2_XXS * vulkan: initial support for IQ2_XS * vulkan: optimize Q3_K by removing branches * vulkan: implement dequantize variants for coopmat2 * vulkan: initial support for IQ2_S * vulkan: vertically realign code * port failing dequant callbacks from mul_mm * Fix array length mismatches * vulkan: avoid using workgroup size before it is referenced * tests: increase timeout for Vulkan llvmpipe backend --------- Co-authored-by: Jeff Bolz <jbolz@nvidia.com>	2025-02-03 22:00:57 +02:00
Jeff Bolz	75e7d0585e	vulkan: Catch pipeline creation failure and print an error message (llama/11436) ... * vulkan: Catch pipeline creation failure and print an error message Also, fix some warnings from my on-demand compile change. * vulkan: fix pipeline creation logging	2025-02-03 22:00:57 +02:00
uvos	682a6f5f87	HIP: Supress transformation warning in softmax.cu ... loops with bounds not known at compile time can not be unrolled. when ncols_template == 0, the bounds of the loop are not constexpr, thus llvm cant unroll the loops here.	2025-02-03 22:00:57 +02:00
Nikita Sarychev	115716d109	HIP: Only call rocblas_initialize on rocblas versions with the multiple instantation bug (llama/11080) ... This disables the workaround on rocblas fixed versions (>=4.0.0) to eliminate the runtime cost and unnecessary VRAM allocation of loading all tensile objects.	2025-02-03 22:00:57 +02:00
someone13574	b2cfef655b	cmake : don't fail on GGML_CPU=OFF (llama/11457)	2025-02-03 22:00:57 +02:00
Akarshan Biswas	22e3df0afa	SYCL : SOFTMAX F16 mask support and other fixes (llama/11261) ... Implemented ggml_sycl_op_soft_max() F16 src1(mask) support for which a pragma deprecation warning was added during #5021. To do this, had to decouple it from ggml_sycl_op_flatten which always considered src1 to be of fp32 type(many OP functions are dependent on it). * SYCL: SOFTMAX F16 mask support and other fixes * test-backend-ops: Add F16 mask test cases	2025-02-03 22:00:57 +02:00
Haus1	028511d349	AMD: parse the architecture as supplied by gcnArchName (llama/11244) ... The value provided by minor doesn't include stepping for AMD, parse the value returned by gcnArchName instead to retrieve an accurate ID.	2025-02-03 22:00:57 +02:00
Ihar Hrachyshka	70c4038842	metal: Handle null returned from MTLCreateSystemDefaultDevice() (llama/11441) ... This fixes segmentation fault error when running tests when no metal devices are available (for example, when not linked with Core Graphics framework or otherwise).	2025-02-03 22:00:57 +02:00
Georgi Gerganov	8639c003a9	metal : use residency sets (llama/11427) ... * metal : use residency sets ggml-ci * metal : restore commandBufferWithUnretainedReferences calls [no ci] * metal : release descriptors ggml-ci * metal : check env GGML_METAL_NO_RESIDENCY ggml-ci * metal : fix build + clean-up ggml-ci	2025-02-03 22:00:57 +02:00
bandoti	d5d831da65	cmake: add ggml find package (llama/11369) ... * Add initial ggml cmake package * Add build numbers to ggml find-package * Expand variables with GGML_ prefix * Guard against adding to cache variable twice * Add git to msys2 workflow * Handle ggml-cpu-* variants * Link ggml/ggml-base libraries to their targets * Replace main-cmake-pkg with simple-cmake-pkg * Interface features require c_std_90 * Fix typo * Removed unnecessary bracket from status message * Update examples/simple-cmake-pkg/README.md Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> * Update examples/simple-cmake-pkg/README.md Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2025-02-03 22:00:57 +02:00
Jeff Bolz	7230a6e1c8	vulkan: compile shaders on-demand (llama/11406) ... Reduce first-run startup time and memory consumption. Should fix #11339.	2025-02-03 22:00:57 +02:00
uvos	a160fa0f3a	Hip: disable VMM on hip as it seams that it dosent work in some configurations (llama/11420)	2025-02-03 22:00:57 +02:00
uvos	0282ad8fd1	hip : Add hipGraph and VMM support to ROCM (llama/11362) ... * Add hipGraph support * Enable VMM on rocm	2025-02-03 22:00:57 +02:00
Johannes Gäßler	9e467815d4	CUDA: fix FP16 cuBLAS GEMM (llama/11396)	2025-02-03 22:00:57 +02:00
uvos	727891d9bf	rocBLAS: Avoid fp32->fp16->fp32 conversion on cdna (llama/11356)	2025-02-03 22:00:57 +02:00
Johannes Gäßler	c262dc80e2	CPU/CUDA: fix (GQA) mul mat back, add CUDA support (llama/11380)	2025-02-03 22:00:57 +02:00
Bernhard M. Wiedemann	30767b4c4e	cmake : avoid -march=native when reproducible build is wanted (llama/11366) ... See https://reproducible-builds.org/ for why this is good and https://reproducible-builds.org/specs/source-date-epoch/ for the definition of this variable. Without this patch, compiling on different machines produced different binaries, which made verification of results difficult. Fixes: #11317 This patch was done while working on reproducible builds for openSUSE.	2025-02-03 22:00:57 +02:00
amd-dwang	16eeb31933	Vulkan-run-test: fix mmq_wg_denoms (llama/11343) ... There should be a copy-and-paste error here. *mmq_wg_denoms should be used together with *warptile_mmq, instead of wg_denoms.	2025-02-03 22:00:57 +02:00
Jeff Bolz	ba523d5e22	vulkan: sort shaders for more deterministic binary (llama/11315) ... Fixes #11306.	2025-02-03 22:00:57 +02:00
Jeff Bolz	3736706139	vulkan: fix diag_mask_inf (llama/11323) ... With robustbufferaccess disabled, this shader was showing OOB stores. There is a bounds check in the code, but the workgrouop dimensions were reversed vs CUDA and it was running the wrong number of threads. So fix the workgroup dimensions and disable robustness for this pipeline.	2025-02-03 22:00:57 +02:00
Radoslav Gerganov	58640aa456	rpc : better caching of the base buffer pointer (llama/11331) ... There is no need to use map, just store the base pointer in the buffer context.	2025-02-03 22:00:57 +02:00
Georgi Gerganov	5183a05e56	metal : fix out-of-bounds write (llama/11314) ... ggml-ci	2025-02-03 22:00:57 +02:00
Jeff Bolz	0dcada42d4	vulkan: fix coopmat2 validation failures (llama/11284) ... mul mat and flash attention shaders were loading f32 types directly into A/B matrices, which happens to work but is technically invalid usage. For FA, we can load it as an Accumulator matrix and convert and this is not in the inner loop and is cheap enough. For mul mat, it's more efficient to do this conversion in a separate pass and have the input(s) be f16. coopmat2 requires SPIR-V 1.6 (related using to LocalSizeId). LocalSizeId requires maintenance4 be enabled, and SPIR-V 1.6 requires Vulkan 1.3.	2025-02-03 22:00:57 +02:00
Nicolò Scipione	d507b4cebe	SYCL: Introducing memory host pool (llama/11251) ... * Implement host pool for matrix_info Creating a new memory pool on the host to store memory location for matrix_info needed to launch gemm_batch from oneMKL/oneMath. Removing complex support in gemm_batch since it is not used in llama.cpp * Remove unnecessary headers and cast * Reorder member variable to avoid warning on initialization * Formatting * Remove unused variable * Address PR review feedback - remove warning --------- Signed-off-by: nscipione <nicolo.scipione@codeplay.com>	2025-02-03 22:00:57 +02:00
Georgi Gerganov	90171055f3	cmake : add sanitizer flags for llama.cpp (llama/11279) ... * cmake : add sanitizer flags for llama.cpp ggml-ci * tests : fix compile warnings ggml-ci * cmake : move sanitizer flags to llama_add_compile_flags ggml-ci * cmake : move llama.cpp compile flags to top level lists ggml-ci * cmake : apply only sanitizer flags at top level ggml-ci * tests : fix gguf context use in same_tensor_data * gguf-test: tensor data comparison * dummy : trigger ggml-ci * unicode : silence gcc warnings ggml-ci * ci : use sanitizer builds only in Debug mode ggml-ci * cmake : add status messages [no ci] --------- Co-authored-by: Johannes Gäßler <johannesg@5d6.de>	2025-02-03 22:00:57 +02:00
Jeff Bolz	668306ff2b	vulkan: fix coopmat2 flash attention for non-contiguous inputs (llama/11281) ... Add code similar to mul_mm_cm2 to force alignment of strides, to avoid a performance regression. Add noncontiguous FA tests in test-backend-ops. Fixes #11268.	2025-02-03 22:00:57 +02:00
Radoslav Gerganov	fdc21fc87b	rpc : early register backend devices (llama/11262) ... Early register RPC devices and do not propagate RPC specifics in the llama model structures. ref: #10609	2025-02-03 22:00:57 +02:00
Jeff Bolz	7183a1eb72	vulkan: support copy from f32 to q4_0/q4_1/q5_0/q5_1/q8_0/iq4_nl (llama/11166) ... * vulkan: support copy from f32 to q4_0/q4_1/q5_0/q5_1/q8_0/iq4_nl Shaders are based on cpy.cu. * vulkan: support copy from q4_0/q4_1/q5_0/q5_1/q8_0/iq4_nl to f32 * ggml: copy q->f32 assumes some contiguity in the destination	2025-02-03 22:00:57 +02:00