ExternalVendorCode/whisper.cpp
1
0
Fork 0
mirror of https://github.com/ggerganov/whisper.cpp.git synced 2025-01-27 06:39:26 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

whisper : add comment about the KV cache size

Browse Source
This commit is contained in:
Georgi Gerganov 2023-11-14 23:00:57 +02:00
parent ae1bd69041
commit 6c8a003acf
No known key found for this signature in database
GPG Key ID: 449E073F9DC10735
1 changed files with 3 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
whisper.cpp
View File

@ -3040,8 +3040,9 @@ struct whisper_state * whisper_init_state(whisper_context * ctx) {
state->backend = whisper_backend_init(ctx->params);
// TODO: determine how large the cache should be
const int factor = 2;
// at this point, we don't know yet how many decoders will be used, so we overallocate 3x ctx
// in theory, there can be a case where this is not enough, but in practice it should always be enough
const int factor = 3;
if (!kv_cache_init(ctx->model.hparams, state->kv_self, ctx->backend, ctx->itype, factor*ctx->model.hparams.n_text_ctx)) {
WHISPER_LOG_ERROR("%s: kv_cache_init() failed for self-attention cache\n", __func__);