ExternalVendorCode/lollms
1
0
Fork 0
mirror of https://github.com/ParisNeo/lollms.git synced 2024-12-20 05:08:00 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
2a00bede8c
lollms/notebooks/lollms_service.ipynb
Saifeddine ALOUI 2a00bede8c enhanced core
2023-10-31 00:15:26 +01:00

33 KiB
Raw Blame History

In [4]:
!pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
!pip install lollms --upgrade

Looking in indexes: https://download.pytorch.org/whl/cu118
Requirement already satisfied: torch in /usr/local/lib/python3.10/dist-packages (2.1.0+cu118)
Requirement already satisfied: torchvision in /usr/local/lib/python3.10/dist-packages (0.16.0+cu118)
Requirement already satisfied: torchaudio in /usr/local/lib/python3.10/dist-packages (2.1.0+cu118)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch) (3.12.4)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch) (4.5.0)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch) (1.12)
Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch) (3.2)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch) (3.1.2)
Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch) (2023.6.0)
Requirement already satisfied: triton==2.1.0 in /usr/local/lib/python3.10/dist-packages (from torch) (2.1.0)
Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from torchvision) (1.23.5)
Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from torchvision) (2.31.0)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.10/dist-packages (from torchvision) (9.4.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch) (2.1.3)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->torchvision) (3.3.1)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->torchvision) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->torchvision) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->torchvision) (2023.7.22)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch) (1.3.0)
Requirement already satisfied: lollms in /usr/local/lib/python3.10/dist-packages (5.9.4)
Collecting lollms
  Downloading lollms-5.9.5-py3-none-any.whl (164 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 164.7/164.7 kB 4.2 MB/s eta 0:00:00
Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from lollms) (4.66.1)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from lollms) (6.0.1)
Requirement already satisfied: Pillow in /usr/local/lib/python3.10/dist-packages (from lollms) (9.4.0)
Requirement already satisfied: flask in /usr/local/lib/python3.10/dist-packages (from lollms) (2.2.5)
Requirement already satisfied: flask-socketio in /usr/local/lib/python3.10/dist-packages (from lollms) (5.3.6)
Requirement already satisfied: flask-cors in /usr/local/lib/python3.10/dist-packages (from lollms) (4.0.0)
Requirement already satisfied: simple-websocket in /usr/local/lib/python3.10/dist-packages (from lollms) (1.0.0)
Requirement already satisfied: eventlet in /usr/local/lib/python3.10/dist-packages (from lollms) (0.33.3)
Requirement already satisfied: wget in /usr/local/lib/python3.10/dist-packages (from lollms) (3.2)
Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from lollms) (67.7.2)
Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from lollms) (2.31.0)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from lollms) (3.7.1)
Requirement already satisfied: seaborn in /usr/local/lib/python3.10/dist-packages (from lollms) (0.12.2)
Requirement already satisfied: mplcursors in /usr/local/lib/python3.10/dist-packages (from lollms) (0.5.2)
Requirement already satisfied: scikit-learn in /usr/local/lib/python3.10/dist-packages (from lollms) (1.2.2)
Requirement already satisfied: ascii-colors in /usr/local/lib/python3.10/dist-packages (from lollms) (0.1.3)
Requirement already satisfied: safe-store in /usr/local/lib/python3.10/dist-packages (from lollms) (0.2.4)
Requirement already satisfied: dnspython>=1.15.0 in /usr/local/lib/python3.10/dist-packages (from eventlet->lollms) (2.4.2)
Requirement already satisfied: greenlet>=0.3 in /usr/local/lib/python3.10/dist-packages (from eventlet->lollms) (3.0.0)
Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.10/dist-packages (from eventlet->lollms) (1.16.0)
Requirement already satisfied: Werkzeug>=2.2.2 in /usr/local/lib/python3.10/dist-packages (from flask->lollms) (3.0.1)
Requirement already satisfied: Jinja2>=3.0 in /usr/local/lib/python3.10/dist-packages (from flask->lollms) (3.1.2)
Requirement already satisfied: itsdangerous>=2.0 in /usr/local/lib/python3.10/dist-packages (from flask->lollms) (2.1.2)
Requirement already satisfied: click>=8.0 in /usr/local/lib/python3.10/dist-packages (from flask->lollms) (8.1.7)
Requirement already satisfied: python-socketio>=5.0.2 in /usr/local/lib/python3.10/dist-packages (from flask-socketio->lollms) (5.10.0)
Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->lollms) (1.1.1)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib->lollms) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->lollms) (4.43.1)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->lollms) (1.4.5)
Requirement already satisfied: numpy>=1.20 in /usr/local/lib/python3.10/dist-packages (from matplotlib->lollms) (1.23.5)
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->lollms) (23.2)
Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->lollms) (3.1.1)
Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/dist-packages (from matplotlib->lollms) (2.8.2)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->lollms) (3.3.1)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->lollms) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->lollms) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->lollms) (2023.7.22)
Requirement already satisfied: python-docx in /usr/local/lib/python3.10/dist-packages (from safe-store->lollms) (1.0.1)
Requirement already satisfied: python-pptx in /usr/local/lib/python3.10/dist-packages (from safe-store->lollms) (0.6.22)
Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from safe-store->lollms) (1.5.3)
Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.10/dist-packages (from safe-store->lollms) (4.11.2)
Requirement already satisfied: PyPDF2 in /usr/local/lib/python3.10/dist-packages (from safe-store->lollms) (3.0.1)
Requirement already satisfied: scipy>=1.3.2 in /usr/local/lib/python3.10/dist-packages (from scikit-learn->lollms) (1.11.3)
Requirement already satisfied: joblib>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from scikit-learn->lollms) (1.3.2)
Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn->lollms) (3.2.0)
Requirement already satisfied: wsproto in /usr/local/lib/python3.10/dist-packages (from simple-websocket->lollms) (1.2.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from Jinja2>=3.0->flask->lollms) (2.1.3)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->safe-store->lollms) (2023.3.post1)
Requirement already satisfied: bidict>=0.21.0 in /usr/local/lib/python3.10/dist-packages (from python-socketio>=5.0.2->flask-socketio->lollms) (0.22.1)
Requirement already satisfied: python-engineio>=4.8.0 in /usr/local/lib/python3.10/dist-packages (from python-socketio>=5.0.2->flask-socketio->lollms) (4.8.0)
Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.10/dist-packages (from beautifulsoup4->safe-store->lollms) (2.5)
Requirement already satisfied: lxml>=3.1.0 in /usr/local/lib/python3.10/dist-packages (from python-docx->safe-store->lollms) (4.9.3)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from python-docx->safe-store->lollms) (4.5.0)
Requirement already satisfied: XlsxWriter>=0.5.7 in /usr/local/lib/python3.10/dist-packages (from python-pptx->safe-store->lollms) (3.1.9)
Requirement already satisfied: h11<1,>=0.9.0 in /usr/local/lib/python3.10/dist-packages (from wsproto->simple-websocket->lollms) (0.14.0)
Installing collected packages: lollms
  Attempting uninstall: lollms
    Found existing installation: lollms 5.9.4
    Uninstalling lollms-5.9.4:
      Successfully uninstalled lollms-5.9.4
Successfully installed lollms-5.9.5
In [2]:
!mkdir zoos
!git clone https://github.com/ParisNeo/lollms_bindings_zoo.git zoos/bindings_zoo
!git clone https://github.com/ParisNeo/lollms_extensions_zoo.git zoos/extensions_zoo
!git clone https://github.com/ParisNeo/models_zoo.git zoos/models_zoo
!git clone https://github.com/ParisNeo/lollms_personalities_zoo.git zoos/personalities_zoo

Cloning into 'lollms_bindings_zoo'...
remote: Enumerating objects: 2153, done.
remote: Counting objects: 100% (1095/1095), done.
remote: Compressing objects: 100% (337/337), done.
remote: Total 2153 (delta 832), reused 995 (delta 754), pack-reused 1058
Receiving objects: 100% (2153/2153), 6.34 MiB | 21.72 MiB/s, done.
Resolving deltas: 100% (1577/1577), done.
Cloning into 'lollms_extensions_zoo'...
remote: Enumerating objects: 82, done.
remote: Counting objects: 100% (82/82), done.
remote: Compressing objects: 100% (48/48), done.
remote: Total 82 (delta 31), reused 51 (delta 12), pack-reused 0
Receiving objects: 100% (82/82), 928.75 KiB | 6.07 MiB/s, done.
Resolving deltas: 100% (31/31), done.
Cloning into 'models_zoo'...
remote: Enumerating objects: 55, done.
remote: Counting objects: 100% (55/55), done.
remote: Compressing objects: 100% (39/39), done.
remote: Total 55 (delta 27), reused 39 (delta 15), pack-reused 0
Receiving objects: 100% (55/55), 225.38 KiB | 2.21 MiB/s, done.
Resolving deltas: 100% (27/27), done.
Cloning into 'lollms_personalities_zoo'...
remote: Enumerating objects: 5986, done.
remote: Counting objects: 100% (1657/1657), done.
remote: Compressing objects: 100% (598/598), done.
remote: Total 5986 (delta 1046), reused 1443 (delta 959), pack-reused 4329
Receiving objects: 100% (5986/5986), 65.03 MiB | 44.66 MiB/s, done.
Resolving deltas: 100% (2745/2745), done.
In [ ]:
!lollms-settings --silent --set_personal_folder_path ./personal_data --install_binding exllama2 --install_model TheBloke/Airoboros-L2-13B-3.1.1-GPTQ

Trying to use Configuration at :lollms_server_global_paths_cfg.yaml
lollms_server_global_paths_cfg.yaml not found! Searching in your home folder.
Selected: ./personal_data
Execution path : /content
Local zoos folder found
----------------------Paths information-----------------------
personal_path:/content/personal_data
personal_configuration_path:/content/personal_data/configs
personal_databases_path:/content/personal_data/databases
personal_models_path:/content/personal_data/models
personal_user_infos_path:/content/personal_data/user_infos
personal_trainers_path:/content/personal_data/trainers
personal_trainers_path:/content/personal_data/trainers/gptqlora
personal_data_path:/content/personal_data/data
custom_personalities_path:/content/personal_data/custom_personalities
-------------------------------------------------------------
----------------------Zoo information-----------------------
bindings_zoo_path:/content/zoos/bindings_zoo
personalities_zoo_path:/content/zoos/personalities_zoo
extensions_zoo_path:/content/zoos/extensions_zoo
models_zoo_path:/content/zoos/models_zoo
-------------------------------------------------------------
No bindings found in your personal space.
Cloning the personalities zoo
Cloning into '/content/zoos/bindings_zoo'...
remote: Enumerating objects: 2153, done.
remote: Counting objects: 100% (1095/1095), done.
remote: Compressing objects: 100% (337/337), done.
remote: Total 2153 (delta 832), reused 995 (delta 754), pack-reused 1058
Receiving objects: 100% (2153/2153), 6.34 MiB | 21.51 MiB/s, done.
Resolving deltas: 100% (1577/1577), done.
No personalities found in your personal space.
Cloning the personalities zoo
Cloning into '/content/zoos/personalities_zoo'...
remote: Enumerating objects: 5986, done.
remote: Counting objects: 100% (1657/1657), done.
remote: Compressing objects: 100% (598/598), done.
remote: Total 5986 (delta 1046), reused 1443 (delta 959), pack-reused 4329
Receiving objects: 100% (5986/5986), 65.03 MiB | 46.31 MiB/s, done.
Resolving deltas: 100% (2745/2745), done.
No extensions found in your personal space.
Cloning the extensions zoo
Cloning into '/content/zoos/extensions_zoo'...
remote: Enumerating objects: 82, done.
remote: Counting objects: 100% (82/82), done.
remote: Compressing objects: 100% (48/48), done.
remote: Total 82 (delta 31), reused 51 (delta 12), pack-reused 0
Receiving objects: 100% (82/82), 928.75 KiB | 6.07 MiB/s, done.
Resolving deltas: 100% (31/31), done.
No models found in your personal space.
Cloning the models zoo
Cloning into '/content/zoos/models_zoo'...
remote: Enumerating objects: 55, done.
remote: Counting objects: 100% (55/55), done.
remote: Compressing objects: 100% (39/39), done.
remote: Total 55 (delta 27), reused 39 (delta 15), pack-reused 0
Receiving objects: 100% (55/55), 225.38 KiB | 2.75 MiB/s, done.
Resolving deltas: 100% (27/27), done.
Selected personal path: ./personal_data
Trying to use Configuration at :/root/lollms_server_global_paths_cfg.yaml
/root/lollms_server_global_paths_cfg.yaml found!
Execution path : /content
Local zoos folder found
----------------------Paths information-----------------------
personal_path:/content/personal_data
personal_configuration_path:/content/personal_data/configs
personal_databases_path:/content/personal_data/databases
personal_models_path:/content/personal_data/models
personal_user_infos_path:/content/personal_data/user_infos
personal_trainers_path:/content/personal_data/trainers
personal_trainers_path:/content/personal_data/trainers/gptqlora
personal_data_path:/content/personal_data/data
custom_personalities_path:/content/personal_data/custom_personalities
-------------------------------------------------------------
----------------------Zoo information-----------------------
bindings_zoo_path:/content/zoos/bindings_zoo
personalities_zoo_path:/content/zoos/personalities_zoo
extensions_zoo_path:/content/zoos/extensions_zoo
models_zoo_path:/content/zoos/models_zoo
-------------------------------------------------------------
Bindings zoo found in your personal space.
Pulling last personalities zoo
Already up to date.
Personalities zoo found in your personal space.
Pulling last personalities zoo
Already up to date.
Extensions zoo found in your personal space.
Pulling last Extensions zoo
Already up to date.
Models zoo found in your personal space.
Pulling last Models zoo
Already up to date.
No binding selected
Personality  lollms mounted successfully but no model is selected

      ___       ___           ___       ___       ___           ___      
     /\__\     /\  \         /\__\     /\__\     /\__\         /\  \     
    /:/  /    /::\  \       /:/  /    /:/  /    /::|  |       /::\  \    
   /:/  /    /:/\:\  \     /:/  /    /:/  /    /:|:|  |      /:/\ \  \   
  /:/  /    /:/  \:\  \   /:/  /    /:/  /    /:/|:|__|__   _\:\~\ \  \  
 /:/__/    /:/__/ \:\__\ /:/__/    /:/__/    /:/ |::::\__\ /\ \:\ \ \__\ 
 \:\  \    \:\  \ /:/  / \:\  \    \:\  \    \/__/~~/:/  / \:\ \:\ \/__/ 
  \:\  \    \:\  /:/  /   \:\  \    \:\  \         /:/  /   \:\ \:\__\   
   \:\  \    \:\/:/  /     \:\  \    \:\  \       /:/  /     \:\/:/  /   
    \:\__\    \::/  /       \:\__\    \:\__\     /:/  /       \::/  /    
     \/__/     \/__/         \/__/     \/__/     \/__/         \/__/     

Version: 5.9.5
By : ParisNeo


Current personality : generic/lollms
Version : 1.0.0
Author : ParisNeo
Description : This personality is a helpful and Kind AI ready to help you solve your problems 



Current binding : None
Current model : None

freeing memory
The variable 'ExLlamaGenerator' does not exist in the local or global namespace.
The variable 'ExLlama' does not exist in the local or global namespace.
The variable 'ExLlamaCache' does not exist in the local or global namespace.
The variable 'ExLlamaConfig' does not exist in the local or global namespace.
The variable 'ExLlamaTokenizer' does not exist in the local or global namespace.
The variable 'torch_version' does not exist in the local or global namespace.
*-*-*-*-*-*-*-*
Cuda VRAM usage
*-*-*-*-*-*-*-*
{'nb_gpus': 1, 'gpu_0_total_vram': 16106127360, 'gpu_0_used_vram': 0, 'gpu_0_model': 'Tesla T4'}
Cleared cache
*-*-*-*-*-*-*-*
Cuda VRAM usage
*-*-*-*-*-*-*-*
{'nb_gpus': 1, 'gpu_0_total_vram': 16106127360, 'gpu_0_used_vram': 0, 'gpu_0_model': 'Tesla T4'}
freed memory
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
Installing exllama2
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
This is the first time you are using this binding.
Checking pytorch
CUDA is supported.
Collecting exllamav2==0.0.6+cu118
  Downloading https://github.com/turboderp/exllamav2/releases/download/v0.0.6/exllamav2-0.0.6+cu118-cp310-cp310-linux_x86_64.whl (12.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.2/12.2 MB 28.8 MB/s eta 0:00:00
Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from exllamav2==0.0.6+cu118) (1.5.3)
Collecting ninja (from exllamav2==0.0.6+cu118)
  Downloading ninja-1.11.1.1-py2.py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl (307 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 307.2/307.2 kB 4.1 MB/s eta 0:00:00
Collecting fastparquet (from exllamav2==0.0.6+cu118)
  Downloading fastparquet-2023.10.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 10.8 MB/s eta 0:00:00
Requirement already satisfied: torch>=2.0.1 in /usr/local/lib/python3.10/dist-packages (from exllamav2==0.0.6+cu118) (2.1.0+cu118)
Collecting safetensors>=0.3.2 (from exllamav2==0.0.6+cu118)
  Downloading safetensors-0.4.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 16.3 MB/s eta 0:00:00
Collecting sentencepiece>=0.1.97 (from exllamav2==0.0.6+cu118)
  Downloading sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 21.8 MB/s eta 0:00:00
Requirement already satisfied: pygments in /usr/local/lib/python3.10/dist-packages (from exllamav2==0.0.6+cu118) (2.16.1)
Collecting websockets (from exllamav2==0.0.6+cu118)
  Downloading websockets-12.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (130 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 130.2/130.2 kB 19.3 MB/s eta 0:00:00
Requirement already satisfied: regex in /usr/local/lib/python3.10/dist-packages (from exllamav2==0.0.6+cu118) (2023.6.3)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=2.0.1->exllamav2==0.0.6+cu118) (3.12.4)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch>=2.0.1->exllamav2==0.0.6+cu118) (4.5.0)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=2.0.1->exllamav2==0.0.6+cu118) (1.12)
Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=2.0.1->exllamav2==0.0.6+cu118) (3.2)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=2.0.1->exllamav2==0.0.6+cu118) (3.1.2)
Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch>=2.0.1->exllamav2==0.0.6+cu118) (2023.6.0)
Requirement already satisfied: triton==2.1.0 in /usr/local/lib/python3.10/dist-packages (from torch>=2.0.1->exllamav2==0.0.6+cu118) (2.1.0)
Requirement already satisfied: numpy>=1.20.3 in /usr/local/lib/python3.10/dist-packages (from fastparquet->exllamav2==0.0.6+cu118) (1.23.5)
Collecting cramjam>=2.3 (from fastparquet->exllamav2==0.0.6+cu118)
  Downloading cramjam-2.7.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 26.2 MB/s eta 0:00:00
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from fastparquet->exllamav2==0.0.6+cu118) (23.2)
Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->exllamav2==0.0.6+cu118) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->exllamav2==0.0.6+cu118) (2023.3.post1)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.1->pandas->exllamav2==0.0.6+cu118) (1.16.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=2.0.1->exllamav2==0.0.6+cu118) (2.1.3)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=2.0.1->exllamav2==0.0.6+cu118) (1.3.0)
Installing collected packages: sentencepiece, ninja, websockets, safetensors, cramjam, fastparquet, exllamav2
Successfully installed cramjam-2.7.0 exllamav2-0.0.6+cu118 fastparquet-2023.10.1 ninja-1.11.1.1 safetensors-0.4.0 sentencepiece-0.1.99 websockets-12.0
Installed successfully
Download Progress:   0%|                                 | 0/100 [00:00<?, ?%/s]
Downloading:   0% 0/100 [00:00<?, ?step/s]
Downloading https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GPTQ/resolve/main/README.md

Downloading:   0% 0.0/100 [00:00<?, ?step/s]
Downloading https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GPTQ/resolve/main/config.json

Downloading:   0% 0.0/100 [00:00<00:00, 996.17step/s]
Downloading https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GPTQ/resolve/main/generation_config.json

Downloading https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GPTQ/resolve/main/model.safetensors

Download Progress: 21569113792512% [00:19, 2119479570613.05%/s]
Downloading:  15% 14.606740773295085/100 [00:19<00:00, 517.09step/s]
Download Progress: 22522237976576% [00:20, 2348664474469.81%/s]
Download Progress: 76488317976576% [00:35, 5160444535118.35%/s]