2279 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Dan Johansson	21d890d534	whisper : add support for backends with multiple ggml_backend_buffer_type (#2863) ... * whisper : add support for ggml_backend_buffer_type Signed-off-by: Dan Johansson <dan.johansson@arm.com> * fix compile error when building on Ubuntu Signed-off-by: Dan Johansson <dan.johansson@arm.com> * remove copyright header from include file Signed-off-by: Dan Johansson <dan.johansson@arm.com> --------- Signed-off-by: Dan Johansson <dan.johansson@arm.com>	2025-03-26 16:54:02 +02:00
Daniel Bevenius	0b43a02be8	bindings.java : enable copyLibs task [no ci] (#2949) ... * bindings.java : enable copyLibs task [no ci] This commit adds a dependency on the copyLibs task to the sourcesJar and jar tasks. This ensures that the libwhisper.so file is copied to the correct location before the jar is built. It also sets the executable bit on the gradlew file. * bindings.java : add copyLibs dep for processResources [no ci] This will otherwise cause builds to fail after doing an initial build. * bindings.java : pass structs by value to native code This commit refactors the code to pass the structs by value to the native code. This is done by creating a ByValue class for each struct and using it in the Java code. The motivation for this change is that without this application crashes due to what I believe was memory mis-alignement. When the structs were passed to the native code they would be att different memory locations. Passing by value overcomes this issue and considering that the structs hold parementers (context and full params) it might be alright do to this. These changes allow all the tests to pass. * bindings.java : fix javadoc warnings [no ci] * bindings.java : fix libwhisper.dylib path in build.gradle [no ci] This commit fixes the copyLibwhisperDynlib task in the build.gradle file to copy the correct libwhisper.dylib file from build/src.	2025-03-26 15:01:28 +01:00
Daniel Bevenius	2699e1485a	bindings.javascript : update test instructions [no ci] (#2951) ... This commit updates the instructions for running the test in the JavaScript bindings README file. The motivation for this is for Node.js versions after v16.4.0 the `--experimental-wasm-threads` and `--experimental-wasm-simd` flags are no longer required and they generate the following errors: ```console $ node --experimental-wasm-threads --experimental-wasm-simd ../tests/test-whisper.js node: bad option: --experimental-wasm-threads node: bad option: --experimental-wasm-simd ```	2025-03-26 14:49:12 +01:00
Page-MS	594a121f3e	readme : add note about SDL2 (#2946) ... Precise the README section about real time audio processing, stating that sdl2 is needed.	2025-03-26 09:30:59 +02:00
Daniel Bevenius	996581c5e2	whisper.android : add GGML_USE_CPU compile definition (#2945) ... This commit add GGML_USE_CPU to built target library to enable CPU backend. The motivation for this that without the compile definition the CPU backend is not enabled and the app will crash when trying to use it.	2025-03-25 18:01:18 +01:00
Daniel Bevenius	226d344f56	whisper.android.java : update build with ggml source changes (#2942) ... * whisper.android.java : update build with ggml source changes This commit updates the whisper.android.java build to include the new ggml source files and directories. The gradle build configuration is also updated to include the aliyun maven repository.	2025-03-25 16:01:59 +01:00
Akarshan Biswas	bb9f68129f	ci: fix SYCL build (#2943)	2025-03-25 11:20:37 +02:00
Daniel Bevenius	30cf30ca82	examples : reduce initial memory to 512MB (#2939) ... * examples : reduce initial memory to 512MB This commit reduces the initial memory size to 512MB. This is done to to avoid WebAssembly memory allocation issues on some platforms. It also adds a flag to allow the memory to grow dynamically (up to the maximum). The motivation for this change is that currently the initial memory is set to 2GB which might be to large for some platforms. This will lead to an error being thrown from the JavaScript code generated by Emscripten when trying to allocate memory. More details can be found in the referenced issue below. * examples : set MAXIMUM_MEMORY instead of TOTAL_MEMORY This commit sets MAXIMUM_MEMORY instead of TOTAL_MEMORY in the whisper.wasm example. The motivation for this is that TOTAL_MEMORY and INITIAL_MEMORY are actually the same thing. Instead we want to set MAXIMUM_MEMORY to 2GB. Refs: https://github.com/ggerganov/whisper.cpp/issues/2920 Refs: https://emscripten.org/docs/tools_reference/settings_reference.html#initial-memory	2025-03-24 14:42:12 +01:00
Daniel Bevenius	ee6286c35d	examples : fix nthread parsing in whisper.wasm (#2938) ... This commit fixes the nthread parsing in the whisper.wasm example when using the `Threads` slider to change the number of threads to be used. Currently this results in the following error: ```console main.js:5597 Uncaught TypeError: Cannot convert "5" to int at checkAssertions (main.js:5597:21) at Object.toWireType (main.js:5611:15) at Object.full_default (eval at new_ (main.js:5292:27), <anonymous>:10:26) at whisper.wasm/:649:42 ```	2025-03-24 14:40:00 +01:00
Daniel Bevenius	c7941d5ccc	examples : fix request path for local worker files (#2937) ... This commit adds a fix to the server.py file to handle requests for web worker files when running the local python server to test the wasm examples. The motivation for this is that currently the server is serving files from the build-em/bin directory which is where the .worker.js files exist. But when examples access these resources they do so with the application context path, for example /whisper.wasm/libmain.worker.js but this will not be found as it currently works.	2025-03-24 14:33:45 +01:00
Daniel Bevenius	b82ac32a6c	ggml : add logging for native build options/vars (#2935) ... This commit adds debug level logging for the native build options and variables to ggml/CMakeLists.txt. The motivation for this is that it can be useful to see the effective result of `GGML_NATIVE`, `GGML_NATIVE_DEFAULT`, and `INS_ENB` for a cmake build. I've found myself adding similar logging a few times now, so I thought it might be a good idea to add this. Example output, specifying `-DCMAKE_MESSAGE_LOG_LEVEL=DEBUG` when running cmake produces the following output: ```console -- GGML_NATIVE : OFF -- GGML_NATIVE_DEFAULT : OFF -- INS_ENB : OFF ```	2025-03-24 09:53:38 +01:00
Peter	edf1ee1ef8	whisper : enhance model download scripts functionality and resolve compiler warning (#2925) ... * whisper : improve whisper-cli executable path detection in model download shell scripts If whisper-cli is found on the path, do not suggest invoking from build directory. This improves flexibility and usability for distribution and packaging scenarios. * whisper : enhance Windows model download batch script to have comparable functionality and behaviour as shell scripts * Download models to the current directory if the script is executed from the \bin\ directory (for future distribution scenarios where the script is in the \bin\ subdirectory of a Windows build) * Add model_path command line argument * If whisper-cli is found on the path, do not suggest invoking from build directory * whisper : resolve compiler warning by removing duplicate definition of NOMINMAX in whisper-cli code	2025-03-24 10:39:50 +02:00
Daniel Bevenius	cf5ddb8c21	whisper : initialize decoder's rng with unique seed (#2932) ... This change initializes each decoder's random number generator with a unique seed. The motivation for this is that currently all decoders are initialized with the same seed value, 0. The result of this is that for the same state (logits, probs, and logprobs) they will produce the same output.	2025-03-24 09:36:07 +01:00
Daniel Bevenius	7fe4979f25	ci : remove CMAKE_CUDA_ARCHITECTURES in windows-cublas (#2923) ... This commit removes the -DCMAKE_CUDA_ARCHITECTURES=all flag from the windows-cublas job in the build.yml file. The motivation for this is that building for all architectures is unnecessary and takes a long time. Without this flag the architectures will instead be set by ggml-cuda. Refs: https://github.com/ggerganov/whisper.cpp/pull/2915#issuecomment-2743160743	2025-03-22 15:40:28 +01:00
Peter	9bc0dc7235	whisper : update default model download directory behavior to use current working directory when script is in /bin/ directory (#2924) ... This change ensures that when the script is packaged and distributed, models are downloaded to the current directory instead of the script's location, preventing conflicts with system directories. This improves flexibility and usability for distribution and packaging scenarios.	2025-03-22 16:27:57 +02:00
Daniel Bevenius	3fc6ad97a3	whisper.swiftui : Add Core ML support to README [no ci] (#2921) ... This commit updates the README to include instructions on how to use a Core ML model with the example.	2025-03-21 11:38:32 +01:00
Daniel Bevenius	663cafc1e8	readme : update Python version to 3.11 for Core ML support [no -ci] (#2919) ... This commit updates the recommended version of Python to 3.11 for Core ML conversion support. It also adds the `-e` flag to the `generate-coreml-model.sh` script to ensure that the script exits on the first error. The motivation for this that when following the installation instructions using Python 3.10 I get the following error: ```console (venv) $ ./models/generate-coreml-model.sh base.en A module that was compiled using NumPy 1.x cannot be run in NumPy 2.1.3 as it may crash. To support both 1.x and 2.x versions of NumPy, modules must be compiled with NumPy 2.0. Some module may need to rebuild instead e.g. with 'pybind11>=2.12'. If you are a user of the module, the easiest solution will be to downgrade to 'numpy<2' or try to upgrade the affected module. We expect that some modules will need time to support NumPy 2. Traceback (most recent call last): File "/whisper-work/models/convert-whisper-to-coreml.py", line 2, in <module> import torch File "/whisper-work/venv/lib/python3.10/site-packages/torch/__init__.py", line 870, in <module> from . import _masked File "/whisper-work/venv/lib/python3.10/site-packages/torch/_masked/__init__.py", line 420, in <module> def sum(input: Tensor, File "/whisper-work/venv/lib/python3.10/site-packages/torch/_masked/__init__.py", line 223, in _apply_docstring_templates example_input = torch.tensor([[-3, -2, -1], [0, 1, 2]]) /whisper-work/venv/lib/python3.10/site-packages/torch/_masked/__init__.py:223: UserWarning: Failed to initialize NumPy: _ARRAY_API not found (Triggered internally at /Users/distiller/project/pytorch/torch/csrc/utils/tensor_numpy.cpp:68.) example_input = torch.tensor([[-3, -2, -1], [0, 1, 2]]) Minimum required torch version for importing coremltools.optimize.torch is 2.1.0. Got torch version 1.11.0. Traceback (most recent call last): File "/whisper-work/models/convert-whisper-to-coreml.py", line 4, in <module> import coremltools as ct File "/whisper-work/venv/lib/python3.10/site-packages/coremltools/__init__.py", line 120, in <module> from . import converters, models, optimize, proto File "/whisper-work/venv/lib/python3.10/site-packages/coremltools/converters/__init__.py", line 7, in <module> from . import libsvm, sklearn, xgboost File "/Users/danbev/work/ai/whisper-work/venv/lib/python3.10/site-packages/coremltools/converters/xgboost/__init__.py", line 6, in <module> from ._tree import convert File "/Users/danbev/work/ai/whisper-work/venv/lib/python3.10/site-packages/coremltools/converters/xgboost/_tree.py", line 9, in <module> from ._tree_ensemble import convert_tree_ensemble as _convert_tree_ensemble File "/Users/danbev/work/ai/whisper-work/venv/lib/python3.10/site-packages/coremltools/converters/xgboost/_tree_ensemble.py", line 11, in <module> from ...models.tree_ensemble import TreeEnsembleClassifier File "/Users/danbev/work/ai/whisper-work/venv/lib/python3.10/site-packages/coremltools/models/__init__.py", line 6, in <module> from . import ( File "/Users/danbev/work/ai/whisper-work/venv/lib/python3.10/site-packages/coremltools/models/ml_program/__init__.py", line 6, in <module> from . import compression_utils File "/Users/danbev/work/ai/whisper-work/venv/lib/python3.10/site-packages/coremltools/models/ml_program/compression_utils.py", line 8, in <module> from coremltools.converters.mil.mil import Operation as _Operation File "/Users/danbev/work/ai/whisper-work/venv/lib/python3.10/site-packages/coremltools/converters/mil/__init__.py", line 7, in <module> from .frontend.tensorflow.tf_op_registry import register_tf_op File "/Users/danbev/work/ai/whisper-work/venv/lib/python3.10/site-packages/coremltools/converters/mil/frontend/__init__.py", line 6, in <module> from . import tensorflow, tensorflow2, torch File "/Users/danbev/work/ai/whisper-work/venv/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/__init__.py", line 11, in <module> from . import ops, quantization_ops File "/Users/danbev/work/ai/whisper-work/venv/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 36, in <module> from .internal_graph import InternalTorchIRGraph, InternalTorchIRNode File "/Users/danbev/work/ai/whisper-work/venv/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/internal_graph.py", line 15, in <module> from .exir_utils import extract_io_from_exir_program File "/Users/danbev/work/ai/whisper-work/venv/lib/python3.10/site-packages/coremltools/converters/mil/frontend/torch/exir_utils.py", line 99, in <module> ) -> Dict[str, torch.fx.Node]: AttributeError: module 'torch' has no attribute 'fx' ``` Using Python3.11 the conversion script runs without any errors.	2025-03-21 10:31:55 +01:00
Daniel Bevenius	be9de81171	whisper : add check for CPU backend initialization (#2918) ... This commit adds a check for the CPU backend initialization in the whisper library. If the initialization fails, an exception is thrown. The motivation for this change is to make the library more robust and handle the case when the CPU backend initialization fails. Resolves: https://github.com/ggerganov/whisper.cpp/issues/2917	2025-03-21 09:53:26 +01:00
Daniel Bevenius	21fb513ef1	examples : update whisper.objc README.md (#2916) ... This commit updates the hisper.objc README.md to reflect the changes of using the xcframework and the new build process. Since whisper.cpp is no longer compiled by the example project, instead the library from the xframework will be used, the build instructions have been removed.	2025-03-21 09:52:53 +01:00
Daniel Bevenius	4e56747944	ci : increase windows-cublas evict-old-files to 5d (#2915) ... This commit updates the evict-old-files parameter for the windows-cublas build job to 5 days. The motivation for this change is to avoid the full rebuild which takes around 1.5 hours for the windows-cublas build job. Considering that there are periods of low traffic on whisper.cpp (like weekends etc.) it might be better to have a longer eviction policy to avoid the full rebuild.	2025-03-21 08:19:24 +01:00
Daniel Bevenius	ca75449a92	xcframework : add support for CoreML to ios/macOS (#2912) ... * xcframework : add support for CoreML to ios/macOS This commit add support for compiling whisper with CoreML support for iOS and macOS. The motivation for this change is it will allow users to use a Core ML model or fall back to a ggml model if Core ML is not available. With the updated xcframework, I was able to run the whisper.objc example and successfully load a Core ML model: ```console whisper_init_state: loading Core ML model from '/Users/danbev/Library/Developer/CoreSimulator/Devices/25E8C27D-0253-4281-AF17-C3F2A4D1D8F4/data/Containers/Bundle/Application/B81F6FF0-BF1A-40DF-AC2A-3908EC4BCC9A/whisper.objc.app/ggml-base.en-encoder.mlmodelc' whisper_init_state: first run on a device may take a while ... whisper_init_state: Core ML model loaded ``` * squash! xcframework : add support for CoreML to ios/macOS Fix grammar in output message.	2025-03-20 18:39:08 +01:00
Daniel Bevenius	80dad86b2c	examples : add WHISPER_SDL2 check to deprecation executables (#2911) ... This commit adds a check for `WHISPER_SDL2` to the deprecation warning examples. This is to prevent the examples from being built when WHISPER_SDL2 is not enabled. The motivation for this is that currently these deprecation executables are generate and when run they refer the user to examples with other names, for example `whisper-command` but unless they have built with `WHISPER_SDL2` those executable will not be present: ```console $ ls build/bin/ bench command main quantize stream whisper-bench whisper-cli whisper-server $ ./build/bin/command WARNING: The binary 'command' is deprecated. Please use 'whisper-command' instead. See https://github.com/ggerganov/whisper.cpp/tree/master/examples/deprecation-warning/README.md for more information. ```	2025-03-20 18:36:02 +01:00
Daniel Bevenius	485ece6725	ci : use ninja and fix caching for windows-cublas (#2910) ... This commit updates the windows-cublas job to use Ninja as the build system instead of msbuild/msvc. The motivation for this is that msbuild/mscv does not seem to handle ccache/sccache well, for example it ignores the `CMAKE_C_COMPILER_LAUNCHER` etc. variables. But using Ninja as the build caching works and the build is initially the same speed as it is currently (without caching) subsequently builds are much faster. Refs: https://github.com/ggerganov/whisper.cpp/issues/2781	2025-03-20 17:01:48 +01:00
Daniel Bevenius	e7d9d8687a	examples : update wasm examples to include server.py [no ci] (#2908) ... This commit updates the README files for the wasm examples to include instructions on how to run the examples using the provided server.py which was included in Commit 6e8242f7fe166b7798bbf49b4c65aba8afe1e131 ("examples : command.wasm updates (#2904)"). The motivation for this is consistency with the command.wasm example.	2025-03-20 09:07:43 +01:00
Daniel Bevenius	6e8242f7fe	examples : command.wasm updates (#2904) ... This commit updates the command.wasm example by adding a server.py script to make it easy to start a local http server to try out the example, updates the build instructions, and also addresses some of the compiler warnings that were being generated. * emscripten : fix TOTAL_STACK for wasm This commit moves the TOTAL_STACK setting from the compile flags to the linker flags. This is because the TOTAL_STACK setting is a linker setting. The motivation for this change is that currently the following warnings are generated when building: ```console em++: warning: linker setting ignored during compilation: 'TOTAL_STACK' [-Wunused-command-line-argument] em++: warning: linker setting ignored during compilation: 'TOTAL_STACK' [-Wunused-command-line-argument] em++: warning: linker setting ignored during compilation: 'TOTAL_STACK' [-Wunused-command-line-argument] em++: warning: linker setting ignored during compilation: 'TOTAL_STACK' [-Wunused-command-line-argument] em++: warning: linker setting ignored during compilation: 'TOTAL_STACK' [-Wunused-command-line-argument] em++: warning: linker setting ignored during compilation: 'TOTAL_STACK' [-Wunused-command-line-argument] ``` * examples : suppress C++17 deprecation warning for std::codecvt_utf8 This commit suppresses the C++17 deprecation warning for std::codecvt_utf8 similar to what is done in examples/talk-llama/unicode.cpp. The motivation for this change is to suppress these warnings: ```console /Users/danbev/work/ai/whisper-work/examples/common.cpp:251:31: warning: 'codecvt_utf8<wchar_t>' is deprecated [-Wdeprecated-declarations] 251 | std::wstring_convert<std::codecvt_utf8<wchar_t>> converter; | ^ /Users/danbev/work/wasm/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/codecvt:193:28: note: 'codecvt_utf8<wchar_t>' has been explicitly marked deprecated here 193 | class _LIBCPP_TEMPLATE_VIS _LIBCPP_DEPRECATED_IN_CXX17 codecvt_utf8 : public __codecvt_utf8<_Elem> { | ^ /Users/danbev/work/wasm/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/__config:723:41: note: expanded from macro '_LIBCPP_DEPRECATED_IN_CXX17' 723 | # define _LIBCPP_DEPRECATED_IN_CXX17 _LIBCPP_DEPRECATED | ^ /Users/danbev/work/wasm/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/__config:688:49: note: expanded from macro '_LIBCPP_DEPRECATED' 688 | # define _LIBCPP_DEPRECATED __attribute__((__deprecated__)) | ^ /Users/danbev/work/ai/whisper-work/examples/common.cpp:251:10: warning: 'wstring_convert<std::codecvt_utf8<wchar_t>>' is deprecated [-Wdeprecated-declarations] 251 | std::wstring_convert<std::codecvt_utf8<wchar_t>> converter; | ^ /Users/danbev/work/wasm/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/locale:3145:28: note: 'wstring_convert<std::codecvt_utf8<wchar_t>>' has been explicitly marked deprecated here 3145 | class _LIBCPP_TEMPLATE_VIS _LIBCPP_DEPRECATED_IN_CXX17 wstring_convert { | ^ /Users/danbev/work/wasm/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/__config:723:41: note: expanded from macro '_LIBCPP_DEPRECATED_IN_CXX17' 723 | # define _LIBCPP_DEPRECATED_IN_CXX17 _LIBCPP_DEPRECATED | ^ /Users/danbev/work/wasm/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/__config:688:49: note: expanded from macro '_LIBCPP_DEPRECATED' 688 | # define _LIBCPP_DEPRECATED __attribute__((__deprecated__)) | ^ /Users/danbev/work/ai/whisper-work/examples/common.cpp:257:31: warning: 'codecvt_utf8<wchar_t>' is deprecated [-Wdeprecated-declarations] 257 | std::wstring_convert<std::codecvt_utf8<wchar_t>> converter; | ^ /Users/danbev/work/wasm/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/codecvt:193:28: note: 'codecvt_utf8<wchar_t>' has been explicitly marked deprecated here 193 | class _LIBCPP_TEMPLATE_VIS _LIBCPP_DEPRECATED_IN_CXX17 codecvt_utf8 : public __codecvt_utf8<_Elem> { | ^ /Users/danbev/work/wasm/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/__config:723:41: note: expanded from macro '_LIBCPP_DEPRECATED_IN_CXX17' 723 | # define _LIBCPP_DEPRECATED_IN_CXX17 _LIBCPP_DEPRECATED | ^ /Users/danbev/work/wasm/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/__config:688:49: note: expanded from macro '_LIBCPP_DEPRECATED' 688 | # define _LIBCPP_DEPRECATED __attribute__((__deprecated__)) | ^ /Users/danbev/work/ai/whisper-work/examples/common.cpp:257:10: warning: 'wstring_convert<std::codecvt_utf8<wchar_t>>' is deprecated [-Wdeprecated-declarations] 257 | std::wstring_convert<std::codecvt_utf8<wchar_t>> converter; | ^ /Users/danbev/work/wasm/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/locale:3145:28: note: 'wstring_convert<std::codecvt_utf8<wchar_t>>' has been explicitly marked deprecated here 3145 | class _LIBCPP_TEMPLATE_VIS _LIBCPP_DEPRECATED_IN_CXX17 wstring_convert { | ^ /Users/danbev/work/wasm/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/__config:723:41: note: expanded from macro '_LIBCPP_DEPRECATED_IN_CXX17' 723 | # define _LIBCPP_DEPRECATED_IN_CXX17 _LIBCPP_DEPRECATED | ^ /Users/danbev/work/wasm/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/__config:688:49: note: expanded from macro '_LIBCPP_DEPRECATED' 688 | # define _LIBCPP_DEPRECATED __attribute__((__deprecated__)) | ^ 4 warnings generated. ``` * ggml : suppress double-promotion warning in GGML_F16x4_REDUCE This commit adds a cast to `ggml_float` in the `GGML_F16x4_REDUCE` macro to suppress a double-promotion warning. Currently the following warning is generated when compiling the command.wasm example: ```console /whisper-work/ggml/src/ggml-cpu/ggml-cpu.c:1592:5: warning: implicit conversion increases floating-point precision: 'float' to 'ggml_float' (aka 'double') [-Wdouble-promotion] 1592 | GGML_F16_VEC_REDUCE(sumf, sum); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /Users/danbev/work/ai/whisper-work/ggml/src/ggml-cpu/ggml-cpu.c:932:37: note: expanded from macro 'GGML_F16_VEC_REDUCE' 932 | #define GGML_F16_VEC_REDUCE GGML_F16x4_REDUCE | ^ /Users/danbev/work/ai/whisper-work/ggml/src/ggml-cpu/ggml-cpu.c:920:44: note: expanded from macro 'GGML_F16x4_REDUCE' 918 | res = wasm_f32x4_extract_lane(x[0], 0) + \ | ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 919 | wasm_f32x4_extract_lane(x[0], 1) + \ | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 920 | wasm_f32x4_extract_lane(x[0], 2) + \ | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~ 921 | wasm_f32x4_extract_lane(x[0], 3); \ | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /whisper-work/ggml/src/ggml-cpu/ggml-cpu.c:1640:9: warning: implicit conversion increases floating-point precision: 'float' to 'ggml_float' (aka 'double') [-Wdouble-promotion] 1640 | GGML_F16_VEC_REDUCE(sumf[k], sum[k]); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /Users/danbev/work/ai/whisper-work/ggml/src/ggml-cpu/ggml-cpu.c:932:37: note: expanded from macro 'GGML_F16_VEC_REDUCE' 932 | #define GGML_F16_VEC_REDUCE GGML_F16x4_REDUCE | ^ /Users/danbev/work/ai/whisper-work/ggml/src/ggml-cpu/ggml-cpu.c:920:44: note: expanded from macro 'GGML_F16x4_REDUCE' 918 | res = wasm_f32x4_extract_lane(x[0], 0) + \ | ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 919 | wasm_f32x4_extract_lane(x[0], 1) + \ | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 920 | wasm_f32x4_extract_lane(x[0], 2) + \ | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~ 921 | wasm_f32x4_extract_lane(x[0], 3); \ | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2 warnings generated. ``` wasm_f32x4_extract_lane returns a 32-bit float and this is what the addition is performed on. But there is an implicit conversion from 32-bit float to 64-bit double when the result is assigned to `res`, which is of type `ggml_float`. My understanding here is that this is intentional and adding a cast to `ggml_float` should suppress the warning. * emscripten : add -Wno-deprecated to for emscripten This commit adds -Wno-deprecated to the CMAKE_CXX_FLAGS for emscripten builds. The motivation for this is that currently there a number of warnings generated like the following: ```console warning: JS library symbol '$print' is deprecated. Please open a bug if you have a continuing need for this symbol [-Wdeprecated] warning: JS library symbol '$printErr' is deprecated. Please open a bug if you have a continuing need for this symbol [-Wdeprecated] em++: warning: warnings in JS library compilation [-Wjs-compiler] em++: warning: linker setting ignored during compilation: 'ENVIRONMENT' [-Wunused-command-line-argument] warning: JS library symbol '$print' is deprecated. Please open a bug if you have a continuing need for this symbol [-Wdeprecated] warning: JS library symbol '$printErr' is deprecated. Please open a bug if you have a continuing need for this symbol [-Wdeprecated] em++: warning: warnings in JS library compilation [-Wjs-compiler] warning: JS library symbol '$print' is deprecated. Please open a bug if you have a continuing need for this symbol [-Wdeprecated] warning: JS library symbol '$printErr' is deprecated. Please open a bug if you have a continuing need for this symbol [-Wdeprecated] em++: warning: warnings in JS library compilation [-Wjs-compiler] em++: warning: linker setting ignored during compilation: 'ENVIRONMENT' [-Wunused-command-line-argument] em++: warning: linker setting ignored during compilation: 'ENVIRONMENT' [-Wunused-command-line-argument] ``` The downside of this is that we might miss other deprecation warnings in the future so I'm not sure if this is acceptable. But it make the wasm examples cleaner without the warnings. * examples : fix tautological-compare warning in stb_vorbis.c [no ci] This commit applies a fix to address a tautological-compare warning in stb_vorbis.c. The motivation for this is that currently the following warning is generated when compiling the commmand-wasm example: ```console /Users/danbev/work/ai/whisper-work/examples/stb_vorbis.c:1404:75: warning: pointer comparison always evaluates to false [-Wtautological-compare] 1404 | if (f->stream_start + loc >= f->stream_end || f->stream_start + loc < f->stream_start) { | ^ 1 warning generated. ``` This fix was taken from an open pull request on the stb repository that addreses this issue: https://github.com/nothings/stb/pull/1746 * squash! examples : update command.wasm instructions [no ci] This commit adds a Python script to serve the the wasm examples build in the `build-em` directory. Initially I thought that it would be enough to start a simple python server but I did not notice that there was an error in the browser console when I did that: ```console command.js:1 Uncaught (in promise) DataCloneError: Failed to execute 'postMessage' on 'Worker': SharedArrayBuffer transfer requires self.crossOriginIsolated. at command.js:1:1206224 at new Promise (<anonymous>) at loadWasmModuleToWorker (command.js:1:1204981) at Array.map (<anonymous>) at Object.loadWasmModuleToAllWorkers (command.js:1:1206428) at command.js:1:1204318 at callRuntimeCallbacks (command.js:1:1202062) at preRun (command.js:1:6136) at run (command.js:1:1294094) at removeRunDependency (command.js:1:7046) ``` We need a few CORS headers to be set and in order hopefully make this easy for users a Python script is added to the examples directory. This should be able to server all the wasm examples provided they have been built. command.wasm's README.md is updated to reflect this change. * examples : remove unused functions This commit removed the unused functions convert_to_utf8 and convert_to_wstring from examples/common.cpp. * Revert "examples : fix tautological-compare warning in stb_vorbis.c [no ci]" This reverts commit 8e3c47d96141c7675c985562ebdc705e839e338a. We should not make this change here and instead when the upstream PR is merged we can sync with it. Refs: https://github.com/ggerganov/whisper.cpp/issues/2784	2025-03-20 07:02:18 +01:00
Daniel Bevenius	e27fd6f0c0	ci : refactor cuda toolkit installation steps (#2902) ... The commit updates the CUDA tookkit installation steps to use variables for the CUDA version and the components versions. The motivation for this change is that the currently the versions for the components are used in multiple places and it is hard to update and maintain.	2025-03-19 09:41:14 +01:00
Amanda Der Bedrosian	96db0c5a9c	go : add Encoder Begin Callback (#2900) ... Adding in EncoderBeginCallback to the Context's Process callback. This optional callback function returns false if computation should be aborted. Co-authored-by: Amanda Der Bedrosian <aderbedr@gmail.com>	2025-03-19 09:05:04 +02:00
Daniel Bevenius	d2aaffd5d9	ci : add ccache action to windows-cublas job (#2893) ... * ci : add ccache action to windows-cublas job This commit adds the ccache action to the windows-cublas job. This will allow us to cache the build artifacts and hopefully speed up the build process. Refs: https://github.com/ggerganov/whisper.cpp/issues/2781	2025-03-19 04:53:08 +01:00
Daniel Bevenius	215990abde	whisper : fix compiler warnings in whisper.cpp (#2895) ... This commit fixes compiler warnings in whisper.cpp by changing the type of the loop index variable from int64_t to size_t. Currently the following warnings are generated by the compiler: ```console /whisper.cpp/src/whisper.cpp:209:27: warning: comparison of integers of different signs: 'int64_t' (aka 'long long') and 'size_t' (aka 'unsigned long') [-Wsign-compare] 209 | for (int64_t i = 0; i < nels; ++i) { | ~ ^ ~~~~ /whisper.cpp/src/whisper.cpp:219:27: warning: comparison of integers of different signs: 'int64_t' (aka 'long long') and 'size_t' (aka 'unsigned long') [-Wsign-compare] 219 | for (int64_t i = 0; i < nels; ++i) { | ~ ^ ~~~~ ```	2025-03-18 13:38:41 +01:00
Daniel Bevenius	7e23d8c64a	ci : add missing env.branch_name to build.yml (#2896) ... This commit adds the missing env.branch_name to the build.yml file. The motivation for this is that the currently the build is failing during the release job because the branch_name is not set in the an invalid tag is being used.	2025-03-18 13:38:21 +01:00
Daniel Bevenius	740bf7f6a1	whisper : enable compiler warnings for src (#2891) ... * whisper : enable compiler warnings for src This commit enables compiler warnings for the src directory. Currently when the WHISPER_ALL_WARNINGS flag is set to ON is only enables warnings in ggml, by setting GGML_ALL_WARNINGS to ON. This commit adds the same compiler flags for whisper's src directory. The motivation for this is to catch potential bugs and issues early on in the development process. * squash! whisper : enable compiler warnings for src Remove GF_C_FLAGS and GF_CXX_FLAGS from add_compile_options.	2025-03-18 05:19:18 +01:00
Daniel Bevenius	c8e12f59dd	ci : add release job and include xcframework (#2889) ... * ci : add release job and include xcframework This commit adds a release job that uploads the xcframework as an artifact and creates a release with the xcframework as an asset. This job can be triggered manually and enables a pre-release tag name to be specified to that these releases can be distinguished from the regular releases more easily. Resolves: https://github.com/ggerganov/whisper.cpp/issues/2886	2025-03-18 05:18:20 +01:00
Daniel Bevenius	83b14c357c	examples : use xcframework in whisper.objc example (#2882) ... * examples : use xcframework in whisper.objc example This commit updates the whisper.objc example to use the xcframework. The motivation for this to be consistent with the swift example and to also act as a reference for how to use the xcframework in an objc project. Resolves: https://github.com/ggerganov/whisper.cpp/issues/2881 * examples : setup audio session viewDidload This commit adds the setup of the audio session in the viewDidload method of the ViewController.m file. This is necessary to allow the app to record audio. The motivation for this is that without this it was not possible to caputue audio from the microphone. It was possible to click on the Capture button but nothing happened after that, and the button was not marked red indicating that the button could be clicked again to stop capturing. With this change it is possible to capture audio from the microphone and get it transcribed.	2025-03-17 13:01:24 +01:00
Peter	60b481d881	whisper : add option to use system-installed GGML (#2887)	2025-03-17 09:54:48 +02:00
Anders Bjarby	4854789751	convert : update convert-h5-to-ggml.py (#2840) ... improved handling of missing max_length	2025-03-17 09:41:05 +02:00
Daniel Bevenius	e0f3c9d4dd	examples : add GGML_USE_CPU=ON flag to whisper.objc (#2880) ... This commit adds the GGML_USE_CPU=ON flag to the whisper.objc project in order to enable the CPU backend for the whisper.objc project. The motivation for this change is that currently the following error is generated when running the example: ```console ggml_backend_buffer_type_t ggml_backend_get_default_buffer_type(ggml_backend_t backend) { return ggml_backend_dev_buffer_type(backend->device); <- Thread 1: EXC_BAD_ACCESS (code=1, address=0x70) } ``` If we inspect the `backend` variable we can see that it is a `nullptr`. ```console (lldb) p backend (ggml_backend_t) nullptr ``` When running in a simulator and that automatically means that there will be no gpu as there is a check for this in the code. But the CPU backend should still be present. The objective-c code will compile the whisper sources including the ggml sources. And if `-DGGMLL_USE_CPU` is not defined then there will be no CPU backend, and in this particular case of backend at all. Resolves: https://github.com/ggerganov/whisper.cpp/issues/2870	2025-03-14 15:40:20 +01:00
Benjamin Ryan	1f4886b40d	ggml-ci: update input env variables to GG_BUILD_ (#2879)	2025-03-14 10:53:29 +02:00
Benjamin Ryan	f11de0e73c	ggml-ci: add run.sh (#2877)	2025-03-14 09:29:55 +02:00
Daniel Bevenius	d5cc27ee4d	examples : add dl to the list of libraries linked (#2875) ... * examples : add dl to the list of libraries linked This commit adds the dynamic linker library to the list of libraries linked by the examples. The motivation for this change is that when building the examples on ubuntu 20.04, which uses GCC 9.4.0, the dynamic linker requires explicit linking or the following error is generated: ```console [ 64%] Linking CXX executable ../../bin/whisper-cli cd /app/whisper.cpp/build/examples/cli && /usr/bin/cmake -E cmake_link_script CMakeFiles/whisper-cli.dir/link.txt --verbose=1 /usr/bin/c++ -O3 -DNDEBUG CMakeFiles/whisper-cli.dir/cli.cpp.o -o ../../bin/whisper-cli -Wl,-rpath,/app/whisper.cpp/build/src:/app/whisper.cpp/build/ggml/src: ../libcommon.a ../../src/libwhisper.so.1.7.4 -pthread ../../ggml/src/libggml.so ../../ggml/src/libggml-cpu.so ../../ggml/src/libggml-base.so /usr/bin/ld: ../libcommon.a(common-whisper.cpp.o): undefined reference to symbol 'dlclose@@GLIBC_2.2.5' /usr/bin/ld: /lib/x86_64-linux-gnu/libdl.so.2: error adding symbols: DSO missing from command line collect2: error: ld returned 1 exit status make[2]: *** [examples/cli/CMakeFiles/whisper-cli.dir/build.make:89: bin/whisper-cli] Error 1 make[2]: Leaving directory '/app/whisper.cpp/build' make[1]: *** [CMakeFiles/Makefile2:433: examples/cli/CMakeFiles/whisper-cli.dir/all] Error 2 make[1]: Leaving directory '/app/whisper.cpp/build' make: *** [Makefile:130: all] Error 2 ``` Resolves: https://github.com/ggerganov/whisper.cpp/issues/2854	2025-03-14 04:42:20 +01:00
Martin Destagnol	5bb1d58c6a	whisper: add xcframework build script (#2873) ... * whisper: add xcframework build script * added apple validation scripts * fixed Readme * validation script fix	2025-03-13 13:56:39 +01:00
Georgi Gerganov	7d14005717	objc : fix build, tmp remove GPU support, use C++17	2025-03-08 15:13:01 +02:00
Georgi Gerganov	4ffb8e3e4d	cmake : fix ggml-config (ggml/0)	2025-03-08 15:13:01 +02:00
Georgi Gerganov	1d8d8ae55e	sync : ggml	2025-03-08 15:13:01 +02:00
Rémy O	eebf6bc0bd	ggml-cpu: faster AVX2 variant for IQ1_M (llama/12216)	2025-03-08 15:13:01 +02:00
BB-fat	dc8f423b40	metal : simplify kernel arguments using a struct (ggml/3229) (llama/12194) ... * metal : refactor im2col parameters into a struct * metal: Change im2col offset types from int32_t to uint64_t to support larger memory offsets * metal : refactor sum_rows parameters into a struct * metal : refactor soft_max parameters into a struct * metal : refactor diag_mask_inf parameters into a struct * metal : refactor ssm_conv parameters into a struct * metal : refactor ssm_scan parameters into a struct * metal : refactor get_rows parameters into a struct * metal : refactor group_norm parameters into a struct * metal : refactor conv_transpose_1d parameters into a struct * metal : refactor upscale parameters into a struct * metal : refactor pad parameters into a struct * metal : refactor pad_reflect_1d parameters into a struct * metal : refactor arange parameters into a struct * metal : refactor timestep_embedding parameters into a struct * metal : refactor argsort parameters into a struct * metal : refactor leaky_relu parameters into a struct * metal : refactor pool_2d parameters into a struct * metal : fix trailing whitespace --------- Co-authored-by: alexju <alexju@tencent.com>	2025-03-08 15:13:01 +02:00
Daniel Bevenius	548e7052f1	metal : fix default.metallib build (llama/12224) ... This commit updates the custom command to build the default.metallib file to use the correct path to ../ggml-common.h by using the variable METALLIB_COMMON. The motivation for this change is that currently when building and specifying GGML_METAL_EMBED_LIBRARY=OFF the following error is generated: ```console [ 11%] Linking CXX shared library ../../bin/libggml.dylib [ 11%] Built target ggml make[2]: *** No rule to make target `ggml/src/ggml-metal/ggml-common.h', needed by `bin/default.metallib'. Stop. make[1]: *** [ggml/src/ggml-metal/CMakeFiles/ggml-metal-lib.dir/all] Error 2 ``` With the above change the build could progress but there was a follow on error about not being able to find the ggml-common.h file in ggml-metal.metal where is was included as a relative path: ```console [ 11%] Compiling Metal kernels /Users/danbev/work/llama.cpp/build/bin/ggml-metal.metal:6:10: error: '../ggml-common.h' file not found, did you mean 'ggml-common.h'? ^~~~~~~~~~~~~~~~~~ "ggml-common.h" 1 error generated. ``` Removing the relative path then allowed the build to complete successfully.	2025-03-08 15:13:01 +02:00
lhez	a34cb73dc2	opencl: Noncontiguous norm, rms_norm, disable fp16 for some ops (llama/12217) ... * opencl: support noncontiguous `norm` * opencl: support noncontiguous `rms_norm` * opencl: disable fp16 for `ADD`, `MUL`, `SCALE`, `RELU`, `GELU`, `SILU`, `CLAMP`	2025-03-08 15:13:01 +02:00
xiaofei	82f9496657	cmake : fix undefined reference errors for std::filesystem in ggml (#12092) (llama/12094) ... Signed-off-by: Ray Lee <hburaylee@gmail.com> Co-authored-by: Ray Lee <hburaylee@gmail.com>	2025-03-08 15:13:01 +02:00
Johannes Gäßler	e3c85e75bd	CUDA: fix FA logic for PTX 7.0 and CC >= 7.5 (llama/12222)	2025-03-08 15:13:01 +02:00
uvos	b9eab73fa2	HIP/CUDA: set the paramerter value in maintain_cuda_graph instead of replaceing it. (llama/12209) ... This avoids conflict with internal cuda/hip runtimes memory managment behavior.	2025-03-08 15:13:01 +02:00