whisper.cpp/ggml-cuda/template-instances/fattn-vec-f16-instance-hs128-q5_0-q4_1.cu

// This file has been autogenerated by generate_cu_files.py, do not edit manually.

#include "../fattn-vec-f16.cuh"

DECL_FATTN_VEC_F16_CASE(128, GGML_TYPE_Q5_0, GGML_TYPE_Q4_1);
CUDA: refactor mmq, dmmv, mmvq (llama/7716) * CUDA: refactor mmq, dmmv, mmvq * fix out-of-bounds write * struct for qk, qr, qi * fix cmake build * mmq_type_traits 2024-06-05 14:53:00 +00:00			`// This file has been autogenerated by generate_cu_files.py, do not edit manually.`
CUDA: quantized KV support for FA vec (llama/7527) * CUDA: quantized KV support for FA vec * try CI fix * fix commented-out kernel variants * add q8_0 q4_0 tests * fix nwarps > batch size * split fattn compile via extern templates * fix flake8 * fix metal tests * fix cmake * make generate_cu_files.py executable * add autogenerated .cu files * fix AMD * error if type_v != FP16 and not flash_attn * remove obsolete code 2024-06-01 06:44:14 +00:00
			`#include "../fattn-vec-f16.cuh"`

			`DECL_FATTN_VEC_F16_CASE(128, GGML_TYPE_Q5_0, GGML_TYPE_Q4_1);`