72 Commits

Author	SHA1	Message	Date
dependabot[bot]	fff35d5528	chore(deps): Bump sentence-transformers from 3.3.1 to 3.4.0 in /backend/python/transformers (#4702) ... chore(deps): Bump sentence-transformers in /backend/python/transformers Bumps [sentence-transformers](https://github.com/UKPLab/sentence-transformers) from 3.3.1 to 3.4.0. - [Release notes](https://github.com/UKPLab/sentence-transformers/releases) - [Commits](https://github.com/UKPLab/sentence-transformers/compare/v3.3.1...v3.4.0) --- updated-dependencies: - dependency-name: sentence-transformers dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2025-01-27 21:09:50 +00:00
Ettore Di Giacinto	4d44ebc2f2	chore(deps): bump grpcio to 1.70.0 (#4682) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-01-24 10:18:22 +01:00
Ettore Di Giacinto	89429a439b	feat(transformers): add support to Mamba (#4669) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-01-23 09:30:47 +01:00
Ettore Di Giacinto	f496d0113b	chore(deps): pin numba ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-01-19 09:07:56 +01:00
Ettore Di Giacinto	1e9bf19c8d	feat(transformers): merge sentencetransformers backend (#4624) ... * merge sentencetransformers Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add alias to silently redirect sentencetransformers to transformers Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add alias also for transformers-musicgen Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Drop from makefile Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Move tests from sentencetransformers Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Remove sentencetransformers Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Remove tests from CI (part of transformers) Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Do not always try to load the tokenizer Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Adapt tests Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Fix typo Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Tiny adjustments Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-01-18 18:30:30 +01:00
dependabot[bot]	cbdbe59f16	chore(deps): Bump scipy from 1.14.0 to 1.15.1 in /backend/python/transformers (#4621) ... chore(deps): Bump scipy in /backend/python/transformers Bumps [scipy](https://github.com/scipy/scipy) from 1.14.0 to 1.15.1. - [Release notes](https://github.com/scipy/scipy/releases) - [Commits](https://github.com/scipy/scipy/compare/v1.14.0...v1.15.1) --- updated-dependencies: - dependency-name: scipy dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2025-01-17 22:14:11 +00:00
Ettore Di Giacinto	ee7904f170	feat(transformers): add support to OuteTTS (#4622) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-01-17 19:33:25 +01:00
Ettore Di Giacinto	8027fdf1c7	feat(transformers): merge musicgen functionalities to a single backend (#4620) ... * feat(transformers): merge musicgen functionalities to a single backend So we optimize space Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * specify type in tests Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Some adaptations for the MusicgenForConditionalGeneration type Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-01-17 17:01:16 +01:00
Ettore Di Giacinto	0d7550ad54	chore(deps): bump grpcio to 1.69.0 (#4543) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2025-01-05 15:01:49 +01:00
Ettore Di Giacinto	b325807c60	fix(intel): pin torch and intel-extensions (#4435) ... * fix(intel): pin torch version Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(intel): pin intel packages version Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-12-19 15:39:32 +01:00
Ettore Di Giacinto	7ca0e2d925	fix(python): remove pin to setuptools, pin python version (#4395) ... fix(setuptools): remove pin Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-12-16 10:55:02 +01:00
Ettore Di Giacinto	e51792784a	chore(deps): bump grpcio to 1.68.1 (#4301) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-12-02 19:13:26 +01:00
Ettore Di Giacinto	57e793482a	chore(deps): bump grpcio to 1.68.0 (#4166) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-11-16 10:33:47 +01:00
Ettore Di Giacinto	bc25890a65	Revert "chore(deps): Bump setuptools from 69.5.1 to 75.4.0 in /backend/python/transformers" (#4123) ... Revert "chore(deps): Bump setuptools from 69.5.1 to 75.4.0 in /backend/python…" This reverts commit 066fcce57bff41042eb6476684a06e13aa209c31.	2024-11-11 22:53:38 +01:00
dependabot[bot]	066fcce57b	chore(deps): Bump setuptools from 69.5.1 to 75.4.0 in /backend/python/transformers (#4117) ... chore(deps): Bump setuptools in /backend/python/transformers Bumps [setuptools](https://github.com/pypa/setuptools) from 69.5.1 to 75.4.0. - [Release notes](https://github.com/pypa/setuptools/releases) - [Changelog](https://github.com/pypa/setuptools/blob/main/NEWS.rst) - [Commits](https://github.com/pypa/setuptools/compare/v69.5.1...v75.4.0) --- updated-dependencies: - dependency-name: setuptools dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-11-11 21:42:24 +00:00
Ettore Di Giacinto	b897d47e0f	chore(deps): bump grpcio to 1.67.1 (#4009) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-10-29 15:04:21 +01:00
Ettore Di Giacinto	3e77a17b26	fix(dependencies): pin pytorch version (#3872) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-10-18 09:11:59 +02:00
Ettore Di Giacinto	1b44a5a3b7	chore(deps): bump grpcio to 1.67.0 (#3851) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-10-16 18:39:28 +02:00
Josh Bennett	85a3cc8d8f	feat(transformers): Use downloaded model for Transformers backend if it already exists. (#3777) ... * signing commit Signed-off-by: Josh Bennett <562773+joshbtn@users.noreply.github.com> * Update transformers backend to check for existing model directory Signed-off-by: Josh Bennett <562773+joshbtn@users.noreply.github.com> --------- Signed-off-by: Josh Bennett <562773+joshbtn@users.noreply.github.com>	2024-10-10 08:42:59 +00:00
Ettore Di Giacinto	078942fc9f	chore(deps): bump grpcio to 1.66.2 (#3690) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-09-30 09:09:51 +02:00
Ettore Di Giacinto	075e5015c0	Revert "chore(deps): Bump setuptools from 69.5.1 to 75.1.0 in /backend/python/transformers" (#3586) ... Revert "chore(deps): Bump setuptools from 69.5.1 to 75.1.0 in /backend/python…" This reverts commit e95cb8eaacdac6426c085197ec5acf790206c042.	2024-09-17 09:06:07 +02:00
dependabot[bot]	e95cb8eaac	chore(deps): Bump setuptools from 69.5.1 to 75.1.0 in /backend/python/transformers (#3579) ... chore(deps): Bump setuptools in /backend/python/transformers Bumps [setuptools](https://github.com/pypa/setuptools) from 69.5.1 to 75.1.0. - [Release notes](https://github.com/pypa/setuptools/releases) - [Changelog](https://github.com/pypa/setuptools/blob/main/NEWS.rst) - [Commits](https://github.com/pypa/setuptools/compare/v69.5.1...v75.1.0) --- updated-dependencies: - dependency-name: setuptools dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-09-17 03:33:52 +00:00
Ettore Di Giacinto	11d960b2a6	chore(cli): be consistent between workers and expose ExtraLLamaCPPArgs to both (#3428) ... * chore(cli): be consistent between workers and expose ExtraLLamaCPPArgs to both Fixes: https://github.com/mudler/LocalAI/issues/3427 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * bump grpcio Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-08-30 00:10:17 +02:00
Ettore Di Giacinto	a9c521eb41	fix(deps): bump grpcio (#3362) ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-08-23 10:29:04 +02:00
dependabot[bot]	0c31d1a4c8	chore(deps): Bump grpcio from 1.65.4 to 1.65.5 in /backend/python/transformers (#3296) ... chore(deps): Bump grpcio in /backend/python/transformers Bumps [grpcio](https://github.com/grpc/grpc) from 1.65.4 to 1.65.5. - [Release notes](https://github.com/grpc/grpc/releases) - [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md) - [Commits](https://github.com/grpc/grpc/compare/v1.65.4...v1.65.5) --- updated-dependencies: - dependency-name: grpcio dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-08-20 03:37:40 +00:00
Ettore Di Giacinto	61b5602111	fix(python): move accelerate and GPU-specific libs to build-type (#3194) ... Some of the dependencies in `requirements.txt`, even if generic, pulls down the line CUDA libraries. This changes moves mostly all GPU-specific libs to the build-type, and tries a safer approach. In `requirements.txt` now are listed only "first-level" dependencies, for instance, grpc, but libs-dependencies are moved down to the respective build-type `requirements.txt` to avoid any mixin. This should fix #2737 and #1592. Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-08-07 17:02:32 +02:00
dependabot[bot]	ada35e428e	chore(deps): Bump grpcio from 1.65.1 to 1.65.4 in /backend/python/transformers (#3161) ... chore(deps): Bump grpcio in /backend/python/transformers Bumps [grpcio](https://github.com/grpc/grpc) from 1.65.1 to 1.65.4. - [Release notes](https://github.com/grpc/grpc/releases) - [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md) - [Commits](https://github.com/grpc/grpc/compare/v1.65.1...v1.65.4) --- updated-dependencies: - dependency-name: grpcio dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-08-06 04:46:39 +00:00
cryptk	ed322bf59f	fix: ensure correct version of torch is always installed based on BUILD_TYPE(#2890) ... * fix: ensure correct version of torch is always installed based on BUILD_TYPE Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * Move causal-conv1d installation to build_types Signed-off-by: mudler <mudler@localai.io> * Move mamba-ssd install to build-type requirements.txt Signed-off-by: mudler <mudler@localai.io> --------- Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> Signed-off-by: mudler <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com> Co-authored-by: mudler <mudler@localai.io>	2024-08-05 16:38:33 +00:00
Ettore Di Giacinto	a7dbeb36ca	Revert "chore(deps): Bump setuptools from 69.5.1 to 72.1.0 in /backend/python/transformers" (#3078) ... Revert "chore(deps): Bump setuptools from 69.5.1 to 72.1.0 in /backend/python…" This reverts commit 3feb8690250b6d0c958df41c0532eff1918f0c1f.	2024-07-30 09:21:09 +02:00
dependabot[bot]	3feb869025	chore(deps): Bump setuptools from 69.5.1 to 72.1.0 in /backend/python/transformers (#3071) ... chore(deps): Bump setuptools in /backend/python/transformers Bumps [setuptools](https://github.com/pypa/setuptools) from 69.5.1 to 72.1.0. - [Release notes](https://github.com/pypa/setuptools/releases) - [Changelog](https://github.com/pypa/setuptools/blob/main/NEWS.rst) - [Commits](https://github.com/pypa/setuptools/compare/v69.5.1...v72.1.0) --- updated-dependencies: - dependency-name: setuptools dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-07-30 04:02:15 +00:00
dependabot[bot]	99324eeef0	chore(deps): Bump grpcio from 1.65.0 to 1.65.1 in /backend/python/transformers (#2970) ... chore(deps): Bump grpcio in /backend/python/transformers Bumps [grpcio](https://github.com/grpc/grpc) from 1.65.0 to 1.65.1. - [Release notes](https://github.com/grpc/grpc/releases) - [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md) - [Commits](https://github.com/grpc/grpc/compare/v1.65.0...v1.65.1) --- updated-dependencies: - dependency-name: grpcio dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-07-23 02:39:44 +00:00
fakezeta	7d61de63ae	fix: pin setuptools 69.5.1 (#2949) ... pin setuptools 69.5.1	2024-07-22 15:40:34 +02:00
cryptk	a3eb6e04c1	fix: update grpcio version to match version used in builds (#2888) ... Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>	2024-07-16 01:39:10 +00:00
dependabot[bot]	bda2222fee	chore(deps): Bump grpcio from 1.64.0 to 1.64.1 in /backend/python/transformers (#2837) ... chore(deps): Bump grpcio in /backend/python/transformers Bumps [grpcio](https://github.com/grpc/grpc) from 1.64.0 to 1.64.1. - [Release notes](https://github.com/grpc/grpc/releases) - [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md) - [Commits](https://github.com/grpc/grpc/compare/v1.64.0...v1.64.1) --- updated-dependencies: - dependency-name: grpcio dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-07-13 15:03:35 +00:00
dependabot[bot]	705f54257e	chore(deps): Bump setuptools from 69.5.1 to 70.3.0 in /backend/python/transformers (#2817) ... chore(deps): Bump setuptools in /backend/python/transformers Bumps [setuptools](https://github.com/pypa/setuptools) from 69.5.1 to 70.3.0. - [Release notes](https://github.com/pypa/setuptools/releases) - [Changelog](https://github.com/pypa/setuptools/blob/main/NEWS.rst) - [Commits](https://github.com/pypa/setuptools/compare/v69.5.1...v70.3.0) --- updated-dependencies: - dependency-name: setuptools dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-07-13 05:47:24 +00:00
fakezeta	6ef78ef7f6	bugfix: CUDA acceleration not working (#2475) ... * bugfix: CUDA acceleration not working CUDA not working after #2286. Refactored the code to be more polish * Update requirements.txt Missing imports Signed-off-by: fakezeta <fakezeta@gmail.com> * Update requirements.txt Signed-off-by: fakezeta <fakezeta@gmail.com> --------- Signed-off-by: fakezeta <fakezeta@gmail.com>	2024-06-03 22:41:42 +02:00
fakezeta	4a239a4bff	feat(transformers): various enhancements to the transformers backend (#2468) ... update transformers *Handle Temperature = 0 as greedy search *Handle custom works as stop words *Implement KV cache *Phi 3 no more requires trust_remote_code: true	2024-06-03 08:52:55 +02:00
cryptk	ba984c7097	fix: pin version of setuptools for intel builds to work around #2406 (#2414) ... Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>	2024-05-26 18:27:07 +00:00
cryptk	16433d2e8e	fix: install pytorch from proper index for hipblas builds (#2413) ... Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>	2024-05-26 18:05:52 +00:00
Ettore Di Giacinto	1a3dedece0	dependencies(grpcio): bump to fix CI issues (#2362) ... feat(grpcio): bump to fix CI issues Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2024-05-21 14:33:47 +02:00
fakezeta	5b79bd04a7	add setuptools for openvino (#2301)	2024-05-12 19:31:43 +00:00
cryptk	88942e4761	fix: add missing openvino/optimum/etc libraries for Intel, fixes #2289 (#2292) ... Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>	2024-05-12 09:01:45 +02:00
cryptk	e2de8a88f7	feat: create bash library to handle install/run/test of python backends (#2286) ... * feat: create bash library to handle install/run/test of python backends Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * chore: minor cleanup Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * fix: remove incorrect LIMIT_TARGETS from parler-tts Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * fix: update runUnitests to handle running tests from a custom test file Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * chore: document runUnittests Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> --------- Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>	2024-05-11 18:32:46 +02:00
cryptk	28a421cb1d	feat: migrate python backends from conda to uv (#2215) ... * feat: migrate diffusers backend from conda to uv - replace conda with UV for diffusers install (prototype for all extras backends) - add ability to build docker with one/some/all extras backends instead of all or nothing Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat: migrate autogtpq bark coqui from conda to uv Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat: convert exllama over to uv Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat: migrate exllama2 to uv Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat: migrate mamba to uv Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat: migrate parler to uv Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat: migrate petals to uv Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * fix: fix tests Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat: migrate rerankers to uv Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat: migrate sentencetransformers to uv Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * fix: install uv for tests-linux Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * fix: make sure file exists before installing on intel images Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat: migrate transformers backend to uv Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat: migrate transformers-musicgen to uv Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat: migrate vall-e-x to uv Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat: migrate vllm to uv Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * fix: add uv install to the rest of test-extra.yml Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * fix: adjust file perms on all install/run/test scripts Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * fix: add missing acclerate dependencies Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * fix: add some more missing dependencies to python backends Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * fix: parler tests venv py dir fix Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * fix: correct filename for transformers-musicgen tests Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * fix: adjust the pwd for valle tests Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat: cleanup and optimization work for uv migration Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * fix: add setuptools to requirements-install for mamba Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat: more size optimization work Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * feat: make installs and tests more consistent, cleanup some deps Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * fix: cleanup Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * fix: mamba backend is cublas only Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> * fix: uncomment lines in makefile Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com> --------- Signed-off-by: Chris Jowett <421501+cryptk@users.noreply.github.com>	2024-05-10 15:08:08 +02:00
fakezeta	fea9522982	fix: OpenVINO winograd always disabled (#2252) ... Winograd convolutions were always disabled giving error when inference device was CPU. This commit implement logic to disable Winograd convolutions only if CPU or NPU are declared.	2024-05-07 08:38:58 +02:00
fakezeta	4690b534e0	feat: user defined inference device for CUDA and OpenVINO (#2212) ... user defined inference device configuration via main_gpu parameter	2024-05-02 09:54:29 +02:00
fakezeta	e38610e521	feat: OpenVINO acceleration for embeddings in transformer backend (#2190) ... OpenVINO acceleration for embeddings New argument type: OVModelForFeatureExtraction	2024-04-30 10:13:04 +02:00
fakezeta	c9451cb604	Bump oneapi-basekit, optimum and openvino (#2139) ... * Bump oneapi-basekit, optimum and openvino * Changed PERFORMANCE HINT to CUMULATIVE_THROUGHPUT Minor latency change for first token but about 10-15% speedup on token generation.	2024-04-26 16:20:43 +02:00
fakezeta	f718a391c0	fix missing TrustRemoteCode in OpenVINO model load (#2114)	2024-04-24 00:45:37 +00:00
fakezeta	8e36fe9b6f	Transformers Backend: max_tokens adherence to OpenAI API (#2108) ... max token adherence to OpenAI API improve adherence to OpenAI API when max tokens is omitted or equal to 0 in the request	2024-04-23 18:42:17 +02:00