Commit Graph

88 Commits

Author SHA1 Message Date
Georgi Gerganov
41026c1e4b
cmake : temporary remove VLA check () 2024-01-22 14:51:42 +02:00
Georgi Gerganov
0b9af32a8b
release : v1.5.4 2024-01-05 17:11:27 +02:00
Georgi Gerganov
9962371f71
release : v1.5.3 2024-01-03 19:36:33 +02:00
Andreu Huguet
f39f9690ec
examples : fix WASM Stack Overflow ()
Fix for problem:

"""
RuntimeError: Aborted(Stack overflow! Stack cookie has been overwritten at 0x12be2b10, expected hex dwords 0x89BACDFE and 0x2135467, but received 0x00000000 0x00000000)
"""

That appears when executing the WASM example with the newer versions.
2024-01-02 16:50:04 +00:00
Georgi Gerganov
e77b27c331
sync : ggml (VMM, sync-ggml-am, dotprod ARM fixes, CUDA fixes) ()
* scripts : add sync-ggml-am.sh

* sync : ggml (VMM, ARM dot prod fix, etc.)

* build : fix CUDA build

* ggml : fix some mul mat cases + add tests for src1 F16

dbd02958fa
2023-12-29 11:30:47 +02:00
bobqianic
8986690c2a
cmake : set default CUDA architectures () 2023-12-21 15:44:04 +02:00
Georgi Gerganov
88112c8afb
release : v1.5.2 2023-12-14 17:56:39 +02:00
Gregor Jasny
73d5005880
cmake : install required ggml.h header () 2023-11-28 15:41:49 +02:00
Georgi Gerganov
9d6ebd877c
release : v1.5.1 2023-11-24 12:41:55 +02:00
Georgi Gerganov
d38af151a1
release : v1.5.0 2023-11-15 21:02:52 +02:00
Georgi Gerganov
6a5d195109
release : v1.4.3 2023-11-07 16:15:48 +02:00
Georgi Gerganov
f96e1c5b78
sync : ggml (backend v2, k-quants, CUDA opts, Metal opts, etc.) ()
* sync : ggml (backend v2, k-quants, CUDA opts, Metal opts, etc.)

* metal : allow env metal variable to override resource path ()

* Allow env variable to override resource path

* Update ggml-metal.m

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* sync : restore common / main from `master`

* sync : restore whisper from `master`

* talk-llama : update to latest llama.cpp

* ruby : fix build

* ggml : fix 32-bit ARM build

* ggml : fix MIN / MAX macro collisions + update ios bindings

* ggml : fix ifdefs and MIN / MAX again

* exampels : fix Obj-C and Swift examples

* ggml : fix 32-bit ARM compatibility

* ggml : one more attempt to fix 32-bit ARM compat

* whisper : fix support for larger graphs

---------

Co-authored-by: Chris Raethke <codesoda@users.noreply.github.com>
2023-11-03 21:35:05 +02:00
Marcin Mielniczuk
1b775cdd68
cmake : Abort the build if a requested feature could not be configured () 2023-10-07 20:01:18 +01:00
Marcin Mielniczuk
80bf931668
cmake : Prefer pkg-config while looking for BLAS () 2023-10-07 15:02:07 +01:00
Georgi Gerganov
93935980f8
whisper : Metal and ggml-alloc support ()
* metal : init

* whisper : factor out graph builds

* whisper : allocate encoder and decoder using ggml-alloc

* whisper : ggml-alloc is now supported

* whisper : CoreML support ggml-alloc

* build : fix ggml-alloc

* ios : update submodule

* extra : update sync-ggml.sh script to also sync ggml-alloc

* ci : see if this is causing the crash

* whisper : refactor ggml-alloc init

* whisper.android : try to fix build

* whisper : initial Metal version

* ci : try to debug vmem issue

* metal : decoder works on GPU!

* metal : add multi-decoder support

* ggml : fix ggml_nbytes (probably temp solution)

* metal : run "cross" step on the GPU

* whisper : remove ggml_repeat in the encoder

* whisper : offload the Encoder to Metal

* ggml : use simpler ggml_bytes() implementation

* ggml-alloc : try to make CI happy by reducing vram to 128GB

* whisper : add whisper_allocr to wrap ggml_allocr

* whisper : factor out alloc init in a function

* cmake : update to support Metal build

* whisper : add <functional> header

* objc : fix build (no Metal yet)

* ios : add Metal support

* swiftui : fix build

* metal : speed-up KQ multiplication

* metal : sync latest llama.cpp kernels

* readme : add Metal info

* ios : update submodule

* coreml : add code to toggle Core ML config (CPU, ANE, GPU)

* bench : fix timings by running a pre-heat

* bench : start benching the decoder

* whisper : add ggml_mul_mat_pad

* bench : fix uninitialized vars

* whisper : add comment for disabling mul-mat padding

* whisper : add description of ggml_mul_mat_pad

* whisper : clean-up ggml_mul_mat_pad

* metal : remove the "concurrent" flag

* bench : variable n_past

* ios : update SPM package
2023-09-15 12:18:18 +03:00
Georgi Gerganov
3e9edc6845
ci : upgrade gradle to 2.4.2 ()
* ci : upgrade gradle to 2.4.2

* cmake : add comment ()
2023-09-08 17:58:14 +03:00
Georgi Gerganov
f00c9bba33
cmake : noramlize case () 2023-09-08 14:50:03 +03:00
Przemysław Pawełczyk
b55b505690
build : do not use _GNU_SOURCE gratuitously ()
* Do not use _GNU_SOURCE gratuitously.

What is needed to build whisper.cpp and examples is availability of
stuff defined in The Open Group Base Specifications Issue 6
(https://pubs.opengroup.org/onlinepubs/009695399/) known also as
Single Unix Specification v3 (SUSv3) or POSIX.1-2001 + XSI extensions,
plus some stuff from BSD that is not specified in POSIX.1.

Well, that was true until NUMA support was added recently in ggml,
so enable GNU libc extensions for Linux builds to cover that.

There is no need to penalize musl libc which simply follows standards.

Not having feature test macros in source code gives greater flexibility
to those wanting to reuse it in 3rd party app, as they can build it with
minimal FTM (_XOPEN_SOURCE=600) or other FTM depending on their needs.

It builds without issues in Alpine (musl libc), Ubuntu (glibc), MSYS2.

* examples : include SDL headers before other headers

Avoid macOS build error when _DARWIN_C_SOURCE is not defined, brought by
SDL2 relying on Darwin extension memset_pattern4/8/16 (from string.h).

* make : enable BSD extensions for DragonFlyBSD to expose RLIMIT_MEMLOCK

* make : use BSD-specific FTMs to enable alloca on BSDs

* make : fix OpenBSD build by exposing newer POSIX definitions

* cmake : follow recent FTM improvements from Makefile
2023-09-07 12:36:14 +03:00
ardfork
cb5fb0a12d
whisper : initial hipBLAS support () 2023-08-27 20:03:58 +03:00
Marcin Mielniczuk
66f2078878
build : fix OpenBLAS detection under Arch Linux () 2023-08-25 19:26:34 +03:00
Marcin Mielniczuk
7decc85eb7
cmake : fix PowerPC build failures introduced in () 2023-08-25 15:19:48 +03:00
Marcin Mielniczuk
21e8c67a4f
Fix AVX etc. under GCC/CMake () 2023-08-19 21:39:03 +03:00
bobqianic
a792c4079c
cmake : fix MSVC compile error C3688 ()
Instead of simply using 'add_compile_options(/utf-8)' to address the MSVC compile error C3688, a better approach would be to handle it in a way that prevents passing '/utf-8' to NVCC.
2023-07-26 18:57:25 +03:00
Georgi Gerganov
7b374c9ac9
Revert "cmake : fix MSVC compile error C3688 on non-unicode Windows ()"
This reverts commit fe5c1a7341.
2023-07-26 10:25:09 +03:00
bobqianic
a195bf899a
cmake : enable OpenBLAS on Windows ()
Fixed the issue of not being able to find OpenBLAS on the Windows platform. Even though the name of the previously released binary file was whisper-blas-bin-x64.zip, BLAS was actually not enabled. After enabling, the inference speed can increase by 3-4 times.
2023-07-25 19:15:08 +03:00
Gang Chen
fe5c1a7341
cmake : fix MSVC compile error C3688 on non-unicode Windows ()
Co-authored-by: Gang Chen <cg@upiot.net>
2023-07-25 19:08:37 +03:00
alonfaraj
3998465721
ci : more platforms coverage ()
* add multi platform

* add image name

* fix

* fix /bin/sh path

* add missing \

* add all platforms for check

* remove platforms

* remove s390x

* - add arm v6
- format run cmd

* remove arm v6

* - bump checkout to v3
- use setup emsdk action
- add arch to all ubuntu jobs

* mymindstorm/setup-emsdk to v12

* add missing QEMU step

* add fail-fast: false for debug

* add freebsd

* remark all jobs except freebsd for test

* add sudo

* enable all tests again

* format

* check __AVX__ support before include immintrin.h

* try auto detect flag by cmake

* fix check for immintrin.h

* fix include check for immintrin.h

* Remove all platforms for sanitizer build except amd64

We have no clue why they failed.

---------

Co-authored-by: Alon Faraj <alon.faraj@mapcore.com>
2023-07-16 23:00:34 +03:00
Ryan Metcalfe
62b81276e0
whisper : add OpenVINO support ()
* openvino: use OpenVINO encoder inference

* openvino: add python script for OpenVINO model generation

* whisper: Fix 'unused' warnings when OpenVINO isn't enabled in build

* Apply suggestions from code review

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* whisper: Fix compilation error

* whisper: revert whisper_get_openvino_path_encoder & whisper_get_openvino_path_cache to non-const func signatures

* cmake: Add openvino-encoder as separate object target

* whisper : minor style fixes

* minor : indentation fixes

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2023-07-04 15:56:11 +03:00
Georgi Gerganov
6c25fae1c4
opencl : sync latest ggml-opencl 2023-06-25 15:38:30 +03:00
Alexey Kharlamov
041be06d58
cmake : build with any BLAS compatible library ()
* Build with any BLAS library

* ci: Removed explicit CUDA nvcc path
2023-05-20 21:23:45 +03:00
Georgi Gerganov
a5defbc1b9
release : v1.4.2 2023-05-14 19:06:45 +03:00
Jonathan Soo
bf2449dfae
cmake : fix define used for COREML_ALLOW_FALLBACK () 2023-05-08 21:08:09 +03:00
ZaBlazzingZephyrus
1a548c048e
cmake : fix options disabling AVX and AVX2 flags () 2023-05-08 20:45:53 +03:00
Georgi Gerganov
14bee39b29
cmake : add options to disable CPU flags () 2023-05-04 19:31:04 +03:00
RelatedTitle
d458fcbc15
ci : add cuBLAS build workflow and fix error causing lines in CMakeLists ()
* Add windows build with cuBLAS

* Remove error causing lines for cuBLAS on Windows
2023-05-03 23:47:37 +03:00
Vulcan
05bef0f0e9
build : CLBlast support as in llama.cpp ()
* ggml : CLBlast support as in llama.cpp

Building with CLBlast speeds up whisper.cpp ~2x on low end / older AMD APUs (CPU with integrated GPU) such as the A9.

Usage:
WHISPER_CLBLAST=1 make

* CMake/Makefile : CLBlast support as in llama.cpp

Building with CLBlast speeds up whisper.cpp ~2x on low end / older AMD APUs (CPU with integrated GPU) such as the A9.

Usage:
```
Makefile:
cd whisper.cpp
WHISPER_CLBLAST=1 make

CMake:
cd whisper.cpp ; mkdir build ; cd build
cmake -DWHISPER_CLBLAST=ON  ..
make
```
2023-05-02 22:50:32 +03:00
Georgi Gerganov
9c61f5f585
release : v1.4.1 2023-04-30 22:57:42 +03:00
Georgi Gerganov
fa8dbdc888
release : v1.4.0 2023-04-30 19:23:37 +03:00
Georgi Gerganov
794b162a46
whisper : add integer quantization support ()
* whisper : add integer quantization support

* examples : add common-ggml + prepare to add "quantize" tool

* whisper : quantization tool ready

* whisper : fix F32 support

* whisper : try to fix shared lib linkage

* wasm : update quantized models to Q5

* bench.wasm : remove "medium" button

* bench.wasm : fix custom model button

* ggml : add Q5_0 and Q5_1 WASM SIMD

* wasm : add quantized models to all WASM examples

* wasm : bump DB version number to 2

* talk-llama : update example to latest llama.cpp

* node : increase test timeout to 10s

* readme : add information for model quantization

* wasm : add links to other examples
2023-04-30 18:51:57 +03:00
Georgi Gerganov
5fd1bdd7fc
whisper : add GPU support via cuBLAS ()
* make : add WHISPER_CUBLAS

* make : fix CUBLAS build

* whisper : disable Flash Attention + adjust memory buffers

* whisper : remove old commented code

* readme : add cuBLAS instructions

* cmake : add WHISPER_CUBLAS option

* gitignore : ignore build-cublas
2023-04-30 12:14:33 +03:00
Georgi Gerganov
3efb81dec6
build : add WHISPER_COREML_ALLOW_FALLBACK to make / CMake () 2023-04-29 10:55:24 +03:00
Georgi Gerganov
c23588cc4b
release : v1.3.0 2023-04-15 17:30:44 +03:00
Georgi Gerganov
5e47e223bd
whisper : add Core ML support ()
* coreml : use Core ML encoder inference

* coreml : simlpify whisper_encode + log messages

* whisper : resolve rebase conflicts

* coreml : add scripts for CoreML model generation

* bench-all : recognize COREML flag
2023-04-15 13:21:27 +03:00
LittleLoli
61128870b8
cmake : add msvc compiler args /utf-8 fix error C3688 ()
* force msvc compiler use utf-8 encode

* only enable on msvc
2023-04-14 19:36:38 +03:00
Andrew Huynh
fc49c44426
cmake : add a flag to disable F16C () 2023-03-22 22:30:40 +02:00
Georgi Gerganov
ad1389003d
release : v1.2.1 2023-02-28 22:29:12 +02:00
Avik Sengupta
113fcec513
cmake : install whisper.h header ()
Including the header file in the install bundle helps projects that ship binaries.
2023-02-11 09:13:32 +02:00
Qianhe Chen
ab1916fc59
ci : add node addon test and optimize compilation configuration ()
* addon: implement node addon call whisper through cpp

* addon: modify the license to MIT

* addon: remove iostream

* addon: rename dir

* addon: fix typo

* addon: configure cmake to build when cmake-js is used

* ci: add addon.node test ci

* addon: remove build WHISPER_BUILD_TESTS

* addon: update build command

* addon: add test

* addon: add test file

* addon: adapt to compile on Windows

* addon: fix typo

* addon: reuse jfk.wav

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* addon: reuse jfk.wav

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2023-02-05 15:02:08 +02:00
Georgi Gerganov
b2083c5d02
release : v1.2.0 2023-02-04 09:49:49 +02:00
Georgi Gerganov
2c3f50a021
release : v1.1.1 2023-01-23 20:23:44 +02:00