feat(vllm): Allow to set quantization (#1094)

Browse Source

This particularly useful to set AWQ

**Description**

Follow up of #1015 

**Notes for Reviewers**


**[Signed
commits](../CONTRIBUTING.md#signing-off-on-commits-developer-certificate-of-origin)**
- [ ] Yes, I signed my commits.
 

<!--
Thank you for contributing to LocalAI! 

Contributing Conventions:

1. Include descriptive PR titles with [<component-name>] prepended.
2. Build and test your changes before submitting a PR. 
3. Sign your commits

By following the community's contribution conventions upfront, the
review process will
be accelerated and your PR merged more quickly.
-->

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

...

This commit is contained in:

Ettore Di Giacinto

2023-09-22 15:52:38 +02:00

committed by GitHub

parent 048b81373d

commit a28ab18987

13 changed files with 357 additions and 332 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

56

extra/grpc/diffusers/backend_pb2.py

File diff suppressed because one or more lines are too long