mirror of
https://github.com/ggerganov/whisper.cpp.git
synced 2024-12-30 01:08:52 +00:00
53 lines
1.9 KiB
Markdown
53 lines
1.9 KiB
Markdown
# bench
|
|
|
|
A very basic tool for benchmarking the inference performance on your device. The tool simply runs the Encoder part of the transformer on some random audio data and records the execution time. This way we can have an objective comparison of the performance of the model for various setups.
|
|
|
|
Benchmark results are tracked in the following Github issue: https://github.com/ggerganov/whisper.cpp/issues/89
|
|
|
|
```bash
|
|
# build the bench tool
|
|
$ make bench
|
|
|
|
# run it on the small.en model using 4 threads
|
|
$ ./bench -m ./models/ggml-small.en.bin -t 4
|
|
|
|
whisper_model_load: loading model from './models/ggml-small.en.bin'
|
|
whisper_model_load: n_vocab = 51864
|
|
whisper_model_load: n_audio_ctx = 1500
|
|
whisper_model_load: n_audio_state = 768
|
|
whisper_model_load: n_audio_head = 12
|
|
whisper_model_load: n_audio_layer = 12
|
|
whisper_model_load: n_text_ctx = 448
|
|
whisper_model_load: n_text_state = 768
|
|
whisper_model_load: n_text_head = 12
|
|
whisper_model_load: n_text_layer = 12
|
|
whisper_model_load: n_mels = 80
|
|
whisper_model_load: f16 = 1
|
|
whisper_model_load: type = 3
|
|
whisper_model_load: mem_required = 1048.00 MB
|
|
whisper_model_load: adding 1607 extra tokens
|
|
whisper_model_load: ggml ctx size = 533.05 MB
|
|
whisper_model_load: memory size = 68.48 MB
|
|
whisper_model_load: model size = 464.44 MB
|
|
|
|
whisper_print_timings: load time = 240.82 ms
|
|
whisper_print_timings: mel time = 0.00 ms
|
|
whisper_print_timings: sample time = 0.00 ms
|
|
whisper_print_timings: encode time = 1062.21 ms / 88.52 ms per layer
|
|
whisper_print_timings: decode time = 0.00 ms / 0.00 ms per layer
|
|
whisper_print_timings: total time = 1303.04 ms
|
|
|
|
system_info: n_threads = 4 | AVX2 = 0 | AVX512 = 0 | NEON = 1 | FP16_VA = 1 | WASM_SIMD = 0 | BLAS = 1 |
|
|
|
|
If you wish, you can submit these results here:
|
|
|
|
https://github.com/ggerganov/whisper.cpp/issues/89
|
|
|
|
Please include the following information:
|
|
|
|
- CPU model
|
|
- Operating system
|
|
- Compiler
|
|
|
|
```
|