ExternalVendorCode/ChainForge
1
0
Fork 0
mirror of https://github.com/ianarawjo/ChainForge.git synced 2025-03-14 16:26:45 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
ChainForge/chainforge/providers/dalaipy/README.md
ianarawjo 0134dbf59b
Custom providers (#122) 
* Removed Python backend files that are no longer used (everything in `promptengine`)

* Added `providers` subdomain, with `CustomProviderProtocol`, `provider` decorator, and global singleton `ProviderRegistry`

* Added a tab for custom providers, and a dropzone, in ChainForge global settings UI

* List custom providers in the Global Settings screen once added. 

* Added ability to remove custom providers by clicking X. 

* Make custom funcs sync but call them async'ly.

* Add Cohere custom provider example in examples/

*Cache the custom provider scripts and load them upon page load

* Rebuild react and update package version

* Bug fix when custom provider is deleted and settings screen is opened on the deleted custom provider
2023-08-27 15:11:42 -04:00

2.1 KiB
Raw Permalink Blame History

dalaipy

A Python Wrapper for Dalai!

Dalai is a simple, and easy way to run LLaMa and Alpaca locally.

Installation

pip install dalaipy==2.0.2

https://pypi.org/project/dalaipy/2.0.2/

Instructions

  1. Go to Dalai, and set up your model of choice on your system (either Mac, Windows, or Linux). The readme provides clear explanations!
  2. Once you can run npx dalai serve, run the server and test out your model of choice.
  3. Install dalaipy per the instructions above, and make your first request:
from dalaipy.src import Dalai

model = Dalai()
# your_model can be one of the following, "alpaca.7B", "alpaca.13B", "llama.7B", "llama.13B", "llama.30B", or "llama.65B", and is dictated by which model you installed
request_dict = model.generate_request("What is the capital of the United States?", your_model)
print(model.request(request_dict))

Credits

@cocktailpeanut - the owner of Dalai

@quadrismegistus - made a notebook with the original idea of using python-socketio to communicate with the web server

Docs

Dalai Class

  • generate_request() - model.generate_request(prompt, model)

    • prompt: (required) the prompt string
    • model: (required) the model that should be used, in the form of a string
      • alpaca.7B
      • alpaca.13B
      • llama.7B
      • llama.13B
      • llama.30B
      • llama.65B
    • id: the request ID (defaut is '0')
    • n_predict: the number of tokens to return (default is 128)
    • repeat_last_n: default is 64
    • repeat_penalty: default is 1.3
    • seed: the seed (default is -1)
    • temp: the temperature of the request (default is 0.5)
    • threads: the number of threads to use (default is 4)
    • top_k: default is 40
    • top_p: default is 0.9

  • request() - model.request(prompt)

    • prompt: (required) the prompt string
    • prettify: whether or not to clean and output just the string, or the whole request dictionary (default is True)