From 2b2007ae9e745037d36de5a0fd9ad041ec0274f4 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Fri, 1 Dec 2023 19:11:45 +0100 Subject: [PATCH] docs: add fine-tuning example (#1374) Signed-off-by: Ettore Di Giacinto --- README.md | 30 +- docs/content/_index.en.md | 11 +- docs/content/advanced/fine-tuning.md | 134 ++ examples/README.md | 8 + examples/e2e-fine-tuning/README.md | 83 ++ examples/e2e-fine-tuning/axolotl.yaml | 63 + examples/e2e-fine-tuning/notebook.ipynb | 1655 +++++++++++++++++++++++ 7 files changed, 1969 insertions(+), 15 deletions(-) create mode 100644 docs/content/advanced/fine-tuning.md create mode 100644 examples/e2e-fine-tuning/README.md create mode 100644 examples/e2e-fine-tuning/axolotl.yaml create mode 100644 examples/e2e-fine-tuning/notebook.ipynb diff --git a/README.md b/README.md index c974c186..12bbd71d 100644 --- a/README.md +++ b/README.md @@ -81,10 +81,15 @@ Note that this started just as a [fun weekend project](https://localai.io/#backs ## 🔥🔥 Hot topics / Roadmap -- [Roadmap](https://github.com/mudler/LocalAI/issues?q=is%3Aissue+is%3Aopen+label%3Aroadmap) +[Roadmap](https://github.com/mudler/LocalAI/issues?q=is%3Aissue+is%3Aopen+label%3Aroadmap) -Hot topics: -- https://github.com/mudler/LocalAI/issues/1126 +🆕 New! [LLM finetuning guide](https://localai.io/advanced/fine-tuning/) + +Hot topics (looking for contributors): +- Backends v2: https://github.com/mudler/LocalAI/issues/1126 +- Improving UX v2: https://github.com/mudler/LocalAI/issues/1373 + +If you want to help and contribute, issues up for grabs: https://github.com/mudler/LocalAI/issues?q=is%3Aissue+is%3Aopen+label%3A%22up+for+grabs%22 ## 🚀 [Features](https://localai.io/features/) @@ -98,20 +103,13 @@ Hot topics: - 🖼️ [Download Models directly from Huggingface ](https://localai.io/models/) - 🆕 [Vision API](https://localai.io/features/gpt-vision/) -## :book: 🎥 [Media, Blogs, Social](https://localai.io/basics/news/#media-blogs-social) - -- [Create a slackbot for teams and OSS projects that answer to documentation](https://mudler.pm/posts/smart-slackbot-for-teams/) -- [LocalAI meets k8sgpt](https://www.youtube.com/watch?v=PKrDNuJ_dfE) -- [Question Answering on Documents locally with LangChain, LocalAI, Chroma, and GPT4All](https://mudler.pm/posts/localai-question-answering/) -- [Tutorial to use k8sgpt with LocalAI](https://medium.com/@tyler_97636/k8sgpt-localai-unlock-kubernetes-superpowers-for-free-584790de9b65) - ## 💻 Usage Check out the [Getting started](https://localai.io/basics/getting_started/index.html) section in our documentation. -### Community +### 🔗 Community and integrations -WebUI +WebUIs: - https://github.com/Jirubizu/localai-admin - https://github.com/go-skynet/LocalAI-frontend @@ -123,11 +121,19 @@ Other: ### 🔗 Resources +- 🆕 New! [LLM finetuning guide](https://localai.io/advanced/fine-tuning/) - [How to build locally](https://localai.io/basics/build/index.html) - [How to install in Kubernetes](https://localai.io/basics/getting_started/index.html#run-localai-in-kubernetes) - [Projects integrating LocalAI](https://localai.io/integrations/) - [How tos section](https://localai.io/howtos/) (curated by our community) +## :book: 🎥 [Media, Blogs, Social](https://localai.io/basics/news/#media-blogs-social) + +- [Create a slackbot for teams and OSS projects that answer to documentation](https://mudler.pm/posts/smart-slackbot-for-teams/) +- [LocalAI meets k8sgpt](https://www.youtube.com/watch?v=PKrDNuJ_dfE) +- [Question Answering on Documents locally with LangChain, LocalAI, Chroma, and GPT4All](https://mudler.pm/posts/localai-question-answering/) +- [Tutorial to use k8sgpt with LocalAI](https://medium.com/@tyler_97636/k8sgpt-localai-unlock-kubernetes-superpowers-for-free-584790de9b65) + ## Citation If you utilize this repository, data in a downstream project, please consider citing it with: diff --git a/docs/content/_index.en.md b/docs/content/_index.en.md index 121e4782..de8d7496 100644 --- a/docs/content/_index.en.md +++ b/docs/content/_index.en.md @@ -89,10 +89,15 @@ Note that this started just as a [fun weekend project](https://localai.io/#backs ## 🔥🔥 Hot topics / Roadmap -- [Roadmap](https://github.com/mudler/LocalAI/issues?q=is%3Aissue+is%3Aopen+label%3Aroadmap) +[Roadmap](https://github.com/mudler/LocalAI/issues?q=is%3Aissue+is%3Aopen+label%3Aroadmap) -Hot topics: -- https://github.com/mudler/LocalAI/issues/1126 +🆕 New! [LLM finetuning guide](https://localai.io/advanced/fine-tuning/) + +Hot topics (looking for contributors): +- Backends v2: https://github.com/mudler/LocalAI/issues/1126 +- Improving UX v2: https://github.com/mudler/LocalAI/issues/1373 + +If you want to help and contribute, issues up for grabs: https://github.com/mudler/LocalAI/issues?q=is%3Aissue+is%3Aopen+label%3A%22up+for+grabs%22 ## How does it work? diff --git a/docs/content/advanced/fine-tuning.md b/docs/content/advanced/fine-tuning.md new file mode 100644 index 00000000..7cf14161 --- /dev/null +++ b/docs/content/advanced/fine-tuning.md @@ -0,0 +1,134 @@ + ++++ +disableToc = false +title = "Fine-tuning LLMs for text generation" +weight = 3 ++++ + +{{% notice note %}} +Section under construction +{{% /notice %}} + +This section covers how to fine-tune a language model for text generation and consume it in LocalAI. + +## Requirements + +For this example you will need at least a 12GB VRAM of GPU and a Linux box. + +## Fine-tuning + +Fine-tuning a language model is a process that requires a lot of computational power and time. + +Currently LocalAI doesn't support the fine-tuning endpoint as LocalAI but there are are [plans](https://github.com/mudler/LocalAI/issues/596) to support that. For the time being a guide is proposed here to give a simple starting point on how to fine-tune a model and use it with LocalAI (but also with llama.cpp). + +There is an e2e example of fine-tuning a LLM model to use with [LocalAI](https://github/mudler/LocalAI) written by [@mudler](https://github.com/mudler) available [here](https://github.com/mudler/LocalAI/tree/master/examples/e2e-fine-tuning/). + +The steps involved are: + +- Preparing a dataset +- Prepare the environment and install dependencies +- Fine-tune the model +- Merge the Lora base with the model +- Convert the model to gguf +- Use the model with LocalAI + +## Dataset preparation + +We are going to need a dataset or a set of datasets. + +Axolotl supports a variety of formats, in the notebook and in this example we are aiming for a very simple dataset and build that manually, so we are going to use the `completion` format which requires the full text to be used for fine-tuning. + +A dataset for an instructor model (like Alpaca) can look like the following: + +```json +[ + { + "text": "As an AI language model you are trained to reply to an instruction. Try to be as much polite as possible\n\n## Instruction\n\nWrite a poem about a tree.\n\n## Response\n\nTrees are beautiful, ...", + }, + { + "text": "As an AI language model you are trained to reply to an instruction. Try to be as much polite as possible\n\n## Instruction\n\nWrite a poem about a tree.\n\n## Response\n\nTrees are beautiful, ...", + } +] +``` + +Every block in the text is the whole text that is used to fine-tune. For example, for an instructor model it follows the following format (more or less): + +``` + + +## Instruction + + + +## Response + + +``` + +The instruction format works such as when we are going to inference with the model, we are going to feed it only the first part up to the `## Instruction` block, and the model is going to complete the text with the `## Response` block. + +Prepare a dataset, and upload it to your Google Drive in case you are using the Google colab. Otherwise place it next the `axolotl.yaml` file as `dataset.json`. + +### Install dependencies + +```bash +# Install axolotl and dependencies +git clone https://github.com/OpenAccess-AI-Collective/axolotl && pushd axolotl && git checkout 797f3dd1de8fd8c0eafbd1c9fdb172abd9ff840a && popd #0.3.0 +pip install packaging +pushd axolotl && pip install -e '.[flash-attn,deepspeed]' && popd + +# https://github.com/oobabooga/text-generation-webui/issues/4238 +pip install https://github.com/Dao-AILab/flash-attention/releases/download/v2.3.0/flash_attn-2.3.0+cu117torch2.0cxx11abiFALSE-cp310-cp310-linux_x86_64.whl +``` + +Configure accelerate: + +```bash +accelerate config default +``` + +## Fine-tuning + +We will need to configure axolotl. In this example is provided a file to use `axolotl.yaml` that uses openllama-3b for fine-tuning. Copy the `axolotl.yaml` file and edit it to your needs. The dataset needs to be next to it as `dataset.json`. You can find the axolotl.yaml file [here](https://github.com/mudler/LocalAI/tree/master/examples/e2e-fine-tuning/). + +If you have a big dataset, you can pre-tokenize it to speedup the fine-tuning process: + +```bash +# Optional pre-tokenize (run only if big dataset) +python -m axolotl.cli.preprocess axolotl.yaml +``` + +Now we are ready to start the fine-tuning process: +```bash +# Fine-tune +accelerate launch -m axolotl.cli.train axolotl.yaml +``` + +After we have finished the fine-tuning, we merge the Lora base with the model: +```bash +# Merge lora +python3 -m axolotl.cli.merge_lora axolotl.yaml --lora_model_dir="./qlora-out" --load_in_8bit=False --load_in_4bit=False +``` + +And we convert it to the gguf format that LocalAI can consume: + +```bash + +# Convert to gguf +git clone https://github.com/ggerganov/llama.cpp.git +pushd llama.cpp && make LLAMA_CUBLAS=1 && popd + +# We need to convert the pytorch model into ggml for quantization +# It crates 'ggml-model-f16.bin' in the 'merged' directory. +pushd llama.cpp && python convert.py --outtype f16 \ + ../qlora-out/merged/pytorch_model-00001-of-00002.bin && popd + +# Start off by making a basic q4_0 4-bit quantization. +# It's important to have 'ggml' in the name of the quant for some +# software to recognize it's file format. +pushd llama.cpp && ./quantize ../qlora-out/merged/ggml-model-f16.gguf \ + ../custom-model-q4_0.bin q4_0 + +``` + +Now you should have ended up with a `custom-model-q4_0.bin` file that you can copy in the LocalAI models directory and use it with LocalAI. diff --git a/examples/README.md b/examples/README.md index 7f6dccb9..debfe1a5 100644 --- a/examples/README.md +++ b/examples/README.md @@ -41,6 +41,14 @@ This example show how to use LocalAI inside Kubernetes with [k8sgpt](https://k8s ![Screenshot from 2023-06-19 23-58-47](https://github.com/go-skynet/go-ggml-transformers.cpp/assets/2420543/cab87409-ee68-44ae-8d53-41627fb49509) +### Fine-tuning a model and convert it to gguf to use it with LocalAI + +_by [@mudler](https://github.com/mudler)_ + +This example is an e2e example on how to fine-tune a model with [axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) and convert it to gguf to use it with LocalAI. + +[Check it out here](https://github.com/mudler/LocalAI/tree/master/examples/e2e-fine-tuning/) + ### Flowise _by [@mudler](https://github.com/mudler)_ diff --git a/examples/e2e-fine-tuning/README.md b/examples/e2e-fine-tuning/README.md new file mode 100644 index 00000000..a9258c01 --- /dev/null +++ b/examples/e2e-fine-tuning/README.md @@ -0,0 +1,83 @@ +This is an example of fine-tuning a LLM model to use with [LocalAI](https://github/mudler/LocalAI) written by [@mudler](https://github.com/mudler). + +Specifically, this example shows how to use [axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) to fine-tune a LLM model to consume with LocalAI as a `gguf` model. + +A notebook is provided that currently works on _very small_ datasets on Google colab on the free instance. It is far from producing good models, but it gives a sense of how to use the code to use with a better dataset and configurations, and how to use the model produced with LocalAI. + +## Requirements + +For this example you will need at least a 12GB VRAM of GPU and a Linux box. +The notebook is tested on Google Colab with a Tesla T4 GPU. + +## Clone this directory + +Clone the repository and enter the example directory: + +```bash +git clone http://github.com/mudler/LocalAI +cd LocalAI/examples/e2e-fine-tuning +``` + +## Install dependencies + +```bash +# Install axolotl and dependencies +git clone https://github.com/OpenAccess-AI-Collective/axolotl && pushd axolotl && git checkout 797f3dd1de8fd8c0eafbd1c9fdb172abd9ff840a && popd #0.3.0 +pip install packaging +pushd axolotl && pip install -e '.[flash-attn,deepspeed]' && popd + +# https://github.com/oobabooga/text-generation-webui/issues/4238 +pip install https://github.com/Dao-AILab/flash-attention/releases/download/v2.3.0/flash_attn-2.3.0+cu117torch2.0cxx11abiFALSE-cp310-cp310-linux_x86_64.whl +``` + +Configure accelerate: + +```bash +accelerate config default +``` + +## Fine-tuning + +We will need to configure axolotl. In this example is provided a file to use `axolotl.yaml` that uses openllama-3b for fine-tuning. Copy the `axolotl.yaml` file and edit it to your needs. The dataset needs to be next to it as `dataset.json`. The format used is `completion` which is a list of JSON objects with a `text` field with the full text to train the LLM with. + +If you have a big dataset, you can pre-tokenize it to speedup the fine-tuning process: + +```bash +# Optional pre-tokenize (run only if big dataset) +python -m axolotl.cli.preprocess axolotl.yaml +``` + +Now we are ready to start the fine-tuning process: +```bash +# Fine-tune +accelerate launch -m axolotl.cli.train axolotl.yaml +``` + +After we have finished the fine-tuning, we merge the Lora base with the model: +```bash +# Merge lora +python3 -m axolotl.cli.merge_lora axolotl.yaml --lora_model_dir="./qlora-out" --load_in_8bit=False --load_in_4bit=False +``` + +And we convert it to the gguf format that LocalAI can consume: + +```bash + +# Convert to gguf +git clone https://github.com/ggerganov/llama.cpp.git +pushd llama.cpp && make LLAMA_CUBLAS=1 && popd + +# We need to convert the pytorch model into ggml for quantization +# It crates 'ggml-model-f16.bin' in the 'merged' directory. +pushd llama.cpp && python convert.py --outtype f16 \ + ../qlora-out/merged/pytorch_model-00001-of-00002.bin && popd + +# Start off by making a basic q4_0 4-bit quantization. +# It's important to have 'ggml' in the name of the quant for some +# software to recognize it's file format. +pushd llama.cpp && ./quantize ../qlora-out/merged/ggml-model-f16.gguf \ + ../custom-model-q4_0.bin q4_0 + +``` + +Now you should have ended up with a `custom-model-q4_0.bin` file that you can copy in the LocalAI models directory and use it with LocalAI. diff --git a/examples/e2e-fine-tuning/axolotl.yaml b/examples/e2e-fine-tuning/axolotl.yaml new file mode 100644 index 00000000..ea956dd4 --- /dev/null +++ b/examples/e2e-fine-tuning/axolotl.yaml @@ -0,0 +1,63 @@ + +base_model: openlm-research/open_llama_3b_v2 +model_type: LlamaForCausalLM +tokenizer_type: LlamaTokenizer +load_in_8bit: false +load_in_4bit: true +strict: false +push_dataset_to_hub: false +datasets: +- path: dataset.json + ds_type: json + type: completion +dataset_prepared_path: +val_set_size: 0.05 +adapter: qlora +lora_model_dir: +sequence_len: 1024 +sample_packing: true +lora_r: 8 +lora_alpha: 32 +lora_dropout: 0.05 +lora_target_modules: +lora_target_linear: true +lora_fan_in_fan_out: +wandb_project: +wandb_entity: +wandb_watch: +wandb_run_id: +wandb_log_model: +output_dir: ./qlora-out +gradient_accumulation_steps: 1 +micro_batch_size: 2 +num_epochs: 4 +optimizer: paged_adamw_32bit +torchdistx_path: +lr_scheduler: cosine +learning_rate: 0.0002 +train_on_inputs: false +group_by_length: false +bf16: false +fp16: true +tf32: false +gradient_checkpointing: true +early_stopping_patience: +resume_from_checkpoint: +local_rank: +logging_steps: 1 +xformers_attention: +flash_attention: false +gptq_groupsize: +gptq_model_v1: +warmup_steps: 20 +eval_steps: 0.05 +save_steps: +debug: +deepspeed: +weight_decay: 0.1 +fsdp: +fsdp_config: +special_tokens: +bos_token: "" +eos_token: "" +unk_token: "" diff --git a/examples/e2e-fine-tuning/notebook.ipynb b/examples/e2e-fine-tuning/notebook.ipynb new file mode 100644 index 00000000..9efb57d2 --- /dev/null +++ b/examples/e2e-fine-tuning/notebook.ipynb @@ -0,0 +1,1655 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Finetuning a model and using it with LocalAI\n", + "\n", + "This is an example of fine-tuning a LLM model to use with [LocalAI](https://github/mudler/LocalAI) written by [@mudler](https://github.com/mudler).\n", + "\n", + "Specifically, this example shows how to use [axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) to fine-tune a LLM model to consume with LocalAI as a `gguf` model." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "puRhZeHvuHgB" + }, + "source": [ + "# Important!\n", + "\n", + "Before starting, make sure you have selected GPU runtime : Runtime -> Change runtime type -> GPU (T4)!" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "xcUAOoASZUV1" + }, + "source": [ + "Change the model to link to your dataset. Upload the dataset as `output.jsonl` in the root tree and edit the model file (model.yml) with:\n", + "\n", + "```\n", + "# local\n", + "datasets:\n", + " - path: /content/output.jsonl\n", + " ds_type: json\n", + " type: completion\n", + "\n", + "```\n", + "\n", + "A full example:\n", + "\n", + "```yaml\n", + "\n", + "base_model: openlm-research/open_llama_3b_v2\n", + "model_type: LlamaForCausalLM\n", + "tokenizer_type: LlamaTokenizer\n", + "load_in_8bit: false\n", + "load_in_4bit: true\n", + "strict: false\n", + "push_dataset_to_hub: false\n", + "datasets:\n", + " - path: /content/output.jsonl\n", + " ds_type: json\n", + " type: completion\n", + "dataset_prepared_path:\n", + "val_set_size: 0.05\n", + "adapter: qlora\n", + "lora_model_dir:\n", + "sequence_len: 1024\n", + "sample_packing: true\n", + "lora_r: 8\n", + "lora_alpha: 32\n", + "lora_dropout: 0.05\n", + "lora_target_modules:\n", + "lora_target_linear: true\n", + "lora_fan_in_fan_out:\n", + "wandb_project:\n", + "wandb_entity:\n", + "wandb_watch:\n", + "wandb_run_id:\n", + "wandb_log_model:\n", + "output_dir: ./qlora-out\n", + "gradient_accumulation_steps: 1\n", + "micro_batch_size: 2\n", + "num_epochs: 4\n", + "optimizer: paged_adamw_32bit\n", + "torchdistx_path:\n", + "lr_scheduler: cosine\n", + "learning_rate: 0.0002\n", + "train_on_inputs: false\n", + "group_by_length: false\n", + "bf16: false\n", + "fp16: true\n", + "tf32: false\n", + "gradient_checkpointing: true\n", + "early_stopping_patience:\n", + "resume_from_checkpoint:\n", + "local_rank:\n", + "logging_steps: 1\n", + "xformers_attention:\n", + "flash_attention: false\n", + "gptq_groupsize:\n", + "gptq_model_v1:\n", + "warmup_steps: 20\n", + "eval_steps: 0.05\n", + "save_steps:\n", + "debug:\n", + "deepspeed:\n", + "weight_decay: 0.1\n", + "fsdp:\n", + "fsdp_config:\n", + "special_tokens:\n", + " bos_token: \"\"\n", + " eos_token: \"\"\n", + " unk_token: \"\"\n", + "\n", + "```" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "CBVQikr2WiFP", + "outputId": "236f9f0e-b2b5-4ba9-9127-27f804c511db" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Cloning into 'axolotl'...\n", + "remote: Enumerating objects: 7525, done.\u001b[K\n", + "remote: Counting objects: 100% (1726/1726), done.\u001b[K\n", + "remote: Compressing objects: 100% (385/385), done.\u001b[K\n", + "remote: Total 7525 (delta 1525), reused 1409 (delta 1319), pack-reused 5799\u001b[K\n", + "Receiving objects: 100% (7525/7525), 2.64 MiB | 10.52 MiB/s, done.\n", + "Resolving deltas: 100% (4854/4854), done.\n", + "Note: switching to '797f3dd1de8fd8c0eafbd1c9fdb172abd9ff840a'.\n", + "\n", + "You are in 'detached HEAD' state. You can look around, make experimental\n", + "changes and commit them, and you can discard any commits you make in this\n", + "state without impacting any branches by switching back to a branch.\n", + "\n", + "If you want to create a new branch to retain commits you create, you may\n", + "do so (now or later) by using -c with the switch command. Example:\n", + "\n", + " git switch -c \n", + "\n", + "Or undo this operation with:\n", + "\n", + " git switch -\n", + "\n", + "Turn off this advice by setting config variable advice.detachedHead to false\n", + "\n", + "HEAD is now at 797f3dd don't train if eval split is too small (#873)\n", + "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (23.2)\n", + "Obtaining file:///content/axolotl\n", + " Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", + "Collecting auto-gptq==0.5.1 (from axolotl==0.3.0)\n", + " Downloading auto_gptq-0.5.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.8 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.8/4.8 MB\u001b[0m \u001b[31m14.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from axolotl==0.3.0) (23.2)\n", + "Collecting peft==0.6.0 (from axolotl==0.3.0)\n", + " Downloading peft-0.6.0-py3-none-any.whl (134 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m134.9/134.9 kB\u001b[0m \u001b[31m20.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting transformers==4.35.1 (from axolotl==0.3.0)\n", + " Downloading transformers-4.35.1-py3-none-any.whl (7.9 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.9/7.9 MB\u001b[0m \u001b[31m44.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting bitsandbytes>=0.41.1 (from axolotl==0.3.0)\n", + " Downloading bitsandbytes-0.41.2.post2-py3-none-any.whl (92.6 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m92.6/92.6 MB\u001b[0m \u001b[31m9.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting accelerate==0.24.1 (from axolotl==0.3.0)\n", + " Downloading accelerate-0.24.1-py3-none-any.whl (261 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m261.4/261.4 kB\u001b[0m \u001b[31m31.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting addict (from axolotl==0.3.0)\n", + " Downloading addict-2.4.0-py3-none-any.whl (3.8 kB)\n", + "Collecting fire (from axolotl==0.3.0)\n", + " Downloading fire-0.5.0.tar.gz (88 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m88.3/88.3 kB\u001b[0m \u001b[31m13.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", + "Requirement already satisfied: PyYAML>=6.0 in /usr/local/lib/python3.10/dist-packages (from axolotl==0.3.0) (6.0.1)\n", + "Collecting datasets>=2.14.0 (from axolotl==0.3.0)\n", + " Downloading datasets-2.15.0-py3-none-any.whl (521 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m521.2/521.2 kB\u001b[0m \u001b[31m51.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting sentencepiece (from axolotl==0.3.0)\n", + " Downloading sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m76.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting wandb (from axolotl==0.3.0)\n", + " Downloading wandb-0.16.0-py3-none-any.whl (2.1 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.1/2.1 MB\u001b[0m \u001b[31m90.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting einops (from axolotl==0.3.0)\n", + " Downloading einops-0.7.0-py3-none-any.whl (44 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m44.6/44.6 kB\u001b[0m \u001b[31m6.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting xformers==0.0.22 (from axolotl==0.3.0)\n", + " Downloading xformers-0.0.22-cp310-cp310-manylinux2014_x86_64.whl (211.6 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m211.6/211.6 MB\u001b[0m \u001b[31m6.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting optimum==1.13.2 (from axolotl==0.3.0)\n", + " Downloading optimum-1.13.2.tar.gz (300 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m301.0/301.0 kB\u001b[0m \u001b[31m37.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n", + " Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n", + " Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n", + "Collecting hf_transfer (from axolotl==0.3.0)\n", + " Downloading hf_transfer-0.1.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.9 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.9/3.9 MB\u001b[0m \u001b[31m48.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting colorama (from axolotl==0.3.0)\n", + " Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB)\n", + "Requirement already satisfied: numba in /usr/local/lib/python3.10/dist-packages (from axolotl==0.3.0) (0.58.1)\n", + "Collecting numpy>=1.24.4 (from axolotl==0.3.0)\n", + " Downloading numpy-1.26.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m18.2/18.2 MB\u001b[0m \u001b[31m57.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting bert-score==0.3.13 (from axolotl==0.3.0)\n", + " Downloading bert_score-0.3.13-py3-none-any.whl (61 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m61.1/61.1 kB\u001b[0m \u001b[31m8.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting evaluate==0.4.0 (from axolotl==0.3.0)\n", + " Downloading evaluate-0.4.0-py3-none-any.whl (81 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m81.4/81.4 kB\u001b[0m \u001b[31m7.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting rouge-score==0.1.2 (from axolotl==0.3.0)\n", + " Downloading rouge_score-0.1.2.tar.gz (17 kB)\n", + " Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", + "Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from axolotl==0.3.0) (1.11.3)\n", + "Requirement already satisfied: scikit-learn==1.2.2 in /usr/local/lib/python3.10/dist-packages (from axolotl==0.3.0) (1.2.2)\n", + "Collecting pynvml (from axolotl==0.3.0)\n", + " Downloading pynvml-11.5.0-py3-none-any.whl (53 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m53.1/53.1 kB\u001b[0m \u001b[31m6.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting art (from axolotl==0.3.0)\n", + " Downloading art-6.1-py3-none-any.whl (599 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m599.8/599.8 kB\u001b[0m \u001b[31m38.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting fschat==0.2.29 (from axolotl==0.3.0)\n", + " Downloading fschat-0.2.29-py3-none-any.whl (200 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m200.7/200.7 kB\u001b[0m \u001b[31m21.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio (from axolotl==0.3.0)\n", + " Downloading gradio-4.4.1-py3-none-any.whl (15.4 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m15.4/15.4 MB\u001b[0m \u001b[31m70.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: tensorboard in /usr/local/lib/python3.10/dist-packages (from axolotl==0.3.0) (2.14.1)\n", + "Collecting s3fs (from axolotl==0.3.0)\n", + " Downloading s3fs-2023.10.0-py3-none-any.whl (28 kB)\n", + "Requirement already satisfied: gcsfs in /usr/local/lib/python3.10/dist-packages (from axolotl==0.3.0) (2023.6.0)\n", + "Collecting flash-attn>=2.3.0 (from axolotl==0.3.0)\n", + " Downloading flash_attn-2.3.3.tar.gz (2.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.3/2.3 MB\u001b[0m \u001b[31m81.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", + "Collecting deepspeed (from axolotl==0.3.0)\n", + " Downloading deepspeed-0.12.3.tar.gz (1.2 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.2/1.2 MB\u001b[0m \u001b[31m84.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", + "Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate==0.24.1->axolotl==0.3.0) (5.9.5)\n", + "Requirement already satisfied: torch>=1.10.0 in /usr/local/lib/python3.10/dist-packages (from accelerate==0.24.1->axolotl==0.3.0) (2.1.0+cu118)\n", + "Requirement already satisfied: huggingface-hub in /usr/local/lib/python3.10/dist-packages (from accelerate==0.24.1->axolotl==0.3.0) (0.19.3)\n", + "Collecting rouge (from auto-gptq==0.5.1->axolotl==0.3.0)\n", + " Downloading rouge-1.0.1-py3-none-any.whl (13 kB)\n", + "Collecting gekko (from auto-gptq==0.5.1->axolotl==0.3.0)\n", + " Downloading gekko-1.0.6-py3-none-any.whl (12.2 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m12.2/12.2 MB\u001b[0m \u001b[31m91.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: safetensors in /usr/local/lib/python3.10/dist-packages (from auto-gptq==0.5.1->axolotl==0.3.0) (0.4.0)\n", + "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from auto-gptq==0.5.1->axolotl==0.3.0) (4.66.1)\n", + "Requirement already satisfied: pandas>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from bert-score==0.3.13->axolotl==0.3.0) (1.5.3)\n", + "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from bert-score==0.3.13->axolotl==0.3.0) (2.31.0)\n", + "Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from bert-score==0.3.13->axolotl==0.3.0) (3.7.1)\n", + "Collecting dill (from evaluate==0.4.0->axolotl==0.3.0)\n", + " Downloading dill-0.3.7-py3-none-any.whl (115 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m115.3/115.3 kB\u001b[0m \u001b[31m16.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: xxhash in /usr/local/lib/python3.10/dist-packages (from evaluate==0.4.0->axolotl==0.3.0) (3.4.1)\n", + "Collecting multiprocess (from evaluate==0.4.0->axolotl==0.3.0)\n", + " Downloading multiprocess-0.70.15-py310-none-any.whl (134 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m134.8/134.8 kB\u001b[0m \u001b[31m18.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: fsspec[http]>=2021.05.0 in /usr/local/lib/python3.10/dist-packages (from evaluate==0.4.0->axolotl==0.3.0) (2023.6.0)\n", + "Collecting responses<0.19 (from evaluate==0.4.0->axolotl==0.3.0)\n", + " Downloading responses-0.18.0-py3-none-any.whl (38 kB)\n", + "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from fschat==0.2.29->axolotl==0.3.0) (3.8.6)\n", + "Collecting fastapi (from fschat==0.2.29->axolotl==0.3.0)\n", + " Downloading fastapi-0.104.1-py3-none-any.whl (92 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m92.9/92.9 kB\u001b[0m \u001b[31m14.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting httpx (from fschat==0.2.29->axolotl==0.3.0)\n", + " Downloading httpx-0.25.1-py3-none-any.whl (75 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.0/75.0 kB\u001b[0m \u001b[31m10.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting markdown2[all] (from fschat==0.2.29->axolotl==0.3.0)\n", + " Downloading markdown2-2.4.10-py2.py3-none-any.whl (39 kB)\n", + "Collecting nh3 (from fschat==0.2.29->axolotl==0.3.0)\n", + " Downloading nh3-0.2.14-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.7/1.7 MB\u001b[0m \u001b[31m82.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: prompt-toolkit>=3.0.0 in /usr/local/lib/python3.10/dist-packages (from fschat==0.2.29->axolotl==0.3.0) (3.0.41)\n", + "Requirement already satisfied: pydantic<2,>=1 in /usr/local/lib/python3.10/dist-packages (from fschat==0.2.29->axolotl==0.3.0) (1.10.13)\n", + "Requirement already satisfied: rich>=10.0.0 in /usr/local/lib/python3.10/dist-packages (from fschat==0.2.29->axolotl==0.3.0) (13.7.0)\n", + "Collecting shortuuid (from fschat==0.2.29->axolotl==0.3.0)\n", + " Downloading shortuuid-1.0.11-py3-none-any.whl (10 kB)\n", + "Collecting tiktoken (from fschat==0.2.29->axolotl==0.3.0)\n", + " Downloading tiktoken-0.5.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.0 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.0/2.0 MB\u001b[0m \u001b[31m76.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting uvicorn (from fschat==0.2.29->axolotl==0.3.0)\n", + " Downloading uvicorn-0.24.0.post1-py3-none-any.whl (59 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m59.7/59.7 kB\u001b[0m \u001b[31m8.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting coloredlogs (from optimum==1.13.2->axolotl==0.3.0)\n", + " Downloading coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m46.0/46.0 kB\u001b[0m \u001b[31m6.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from optimum==1.13.2->axolotl==0.3.0) (1.12)\n", + "Requirement already satisfied: transformers[sentencepiece]>=4.26.0 in /usr/local/lib/python3.10/dist-packages (from optimum==1.13.2->axolotl==0.3.0) (4.35.2)\n", + "Requirement already satisfied: absl-py in /usr/local/lib/python3.10/dist-packages (from rouge-score==0.1.2->axolotl==0.3.0) (1.4.0)\n", + "Requirement already satisfied: nltk in /usr/local/lib/python3.10/dist-packages (from rouge-score==0.1.2->axolotl==0.3.0) (3.8.1)\n", + "Requirement already satisfied: six>=1.14.0 in /usr/local/lib/python3.10/dist-packages (from rouge-score==0.1.2->axolotl==0.3.0) (1.16.0)\n", + "Requirement already satisfied: joblib>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from scikit-learn==1.2.2->axolotl==0.3.0) (1.3.2)\n", + "Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn==1.2.2->axolotl==0.3.0) (3.2.0)\n", + "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers==4.35.1->axolotl==0.3.0) (3.13.1)\n", + "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.35.1->axolotl==0.3.0) (2023.6.3)\n", + "Collecting tokenizers<0.15,>=0.14 (from transformers==4.35.1->axolotl==0.3.0)\n", + " Downloading tokenizers-0.14.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.8 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.8/3.8 MB\u001b[0m \u001b[31m116.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting torch>=1.10.0 (from accelerate==0.24.1->axolotl==0.3.0)\n", + " Downloading torch-2.0.1-cp310-cp310-manylinux1_x86_64.whl (619.9 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m619.9/619.9 MB\u001b[0m \u001b[31m2.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate==0.24.1->axolotl==0.3.0) (4.5.0)\n", + "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate==0.24.1->axolotl==0.3.0) (3.2.1)\n", + "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate==0.24.1->axolotl==0.3.0) (3.1.2)\n", + "Collecting nvidia-cuda-nvrtc-cu11==11.7.99 (from torch>=1.10.0->accelerate==0.24.1->axolotl==0.3.0)\n", + " Downloading nvidia_cuda_nvrtc_cu11-11.7.99-2-py3-none-manylinux1_x86_64.whl (21.0 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m21.0/21.0 MB\u001b[0m \u001b[31m63.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting nvidia-cuda-runtime-cu11==11.7.99 (from torch>=1.10.0->accelerate==0.24.1->axolotl==0.3.0)\n", + " Downloading nvidia_cuda_runtime_cu11-11.7.99-py3-none-manylinux1_x86_64.whl (849 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m849.3/849.3 kB\u001b[0m \u001b[31m58.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting nvidia-cuda-cupti-cu11==11.7.101 (from torch>=1.10.0->accelerate==0.24.1->axolotl==0.3.0)\n", + " Downloading nvidia_cuda_cupti_cu11-11.7.101-py3-none-manylinux1_x86_64.whl (11.8 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m11.8/11.8 MB\u001b[0m \u001b[31m79.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting nvidia-cudnn-cu11==8.5.0.96 (from torch>=1.10.0->accelerate==0.24.1->axolotl==0.3.0)\n", + " Downloading nvidia_cudnn_cu11-8.5.0.96-2-py3-none-manylinux1_x86_64.whl (557.1 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m557.1/557.1 MB\u001b[0m \u001b[31m3.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting nvidia-cublas-cu11==11.10.3.66 (from torch>=1.10.0->accelerate==0.24.1->axolotl==0.3.0)\n", + " Downloading nvidia_cublas_cu11-11.10.3.66-py3-none-manylinux1_x86_64.whl (317.1 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m317.1/317.1 MB\u001b[0m \u001b[31m4.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting nvidia-cufft-cu11==10.9.0.58 (from torch>=1.10.0->accelerate==0.24.1->axolotl==0.3.0)\n", + " Downloading nvidia_cufft_cu11-10.9.0.58-py3-none-manylinux1_x86_64.whl (168.4 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m168.4/168.4 MB\u001b[0m \u001b[31m7.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting nvidia-curand-cu11==10.2.10.91 (from torch>=1.10.0->accelerate==0.24.1->axolotl==0.3.0)\n", + " Downloading nvidia_curand_cu11-10.2.10.91-py3-none-manylinux1_x86_64.whl (54.6 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m54.6/54.6 MB\u001b[0m \u001b[31m12.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting nvidia-cusolver-cu11==11.4.0.1 (from torch>=1.10.0->accelerate==0.24.1->axolotl==0.3.0)\n", + " Downloading nvidia_cusolver_cu11-11.4.0.1-2-py3-none-manylinux1_x86_64.whl (102.6 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m102.6/102.6 MB\u001b[0m \u001b[31m8.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting nvidia-cusparse-cu11==11.7.4.91 (from torch>=1.10.0->accelerate==0.24.1->axolotl==0.3.0)\n", + " Downloading nvidia_cusparse_cu11-11.7.4.91-py3-none-manylinux1_x86_64.whl (173.2 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m173.2/173.2 MB\u001b[0m \u001b[31m6.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting nvidia-nccl-cu11==2.14.3 (from torch>=1.10.0->accelerate==0.24.1->axolotl==0.3.0)\n", + " Downloading nvidia_nccl_cu11-2.14.3-py3-none-manylinux1_x86_64.whl (177.1 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m177.1/177.1 MB\u001b[0m \u001b[31m6.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting nvidia-nvtx-cu11==11.7.91 (from torch>=1.10.0->accelerate==0.24.1->axolotl==0.3.0)\n", + " Downloading nvidia_nvtx_cu11-11.7.91-py3-none-manylinux1_x86_64.whl (98 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m98.6/98.6 kB\u001b[0m \u001b[31m13.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting triton==2.0.0 (from torch>=1.10.0->accelerate==0.24.1->axolotl==0.3.0)\n", + " Downloading triton-2.0.0-1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (63.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m63.3/63.3 MB\u001b[0m \u001b[31m9.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from nvidia-cublas-cu11==11.10.3.66->torch>=1.10.0->accelerate==0.24.1->axolotl==0.3.0) (67.7.2)\n", + "Requirement already satisfied: wheel in /usr/local/lib/python3.10/dist-packages (from nvidia-cublas-cu11==11.10.3.66->torch>=1.10.0->accelerate==0.24.1->axolotl==0.3.0) (0.41.3)\n", + "Requirement already satisfied: cmake in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.10.0->accelerate==0.24.1->axolotl==0.3.0) (3.27.7)\n", + "Collecting lit (from triton==2.0.0->torch>=1.10.0->accelerate==0.24.1->axolotl==0.3.0)\n", + " Downloading lit-17.0.5.tar.gz (153 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m153.0/153.0 kB\u001b[0m \u001b[31m20.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n", + " Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n", + " Installing backend dependencies ... \u001b[?25l\u001b[?25hdone\n", + " Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n", + "Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets>=2.14.0->axolotl==0.3.0) (9.0.0)\n", + "Collecting pyarrow-hotfix (from datasets>=2.14.0->axolotl==0.3.0)\n", + " Downloading pyarrow_hotfix-0.5-py3-none-any.whl (7.8 kB)\n", + "Collecting ninja (from flash-attn>=2.3.0->axolotl==0.3.0)\n", + " Downloading ninja-1.11.1.1-py2.py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl (307 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m307.2/307.2 kB\u001b[0m \u001b[31m37.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting hjson (from deepspeed->axolotl==0.3.0)\n", + " Downloading hjson-3.1.0-py3-none-any.whl (54 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m54.0/54.0 kB\u001b[0m \u001b[31m9.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: py-cpuinfo in /usr/local/lib/python3.10/dist-packages (from deepspeed->axolotl==0.3.0) (9.0.0)\n", + "Requirement already satisfied: termcolor in /usr/local/lib/python3.10/dist-packages (from fire->axolotl==0.3.0) (2.3.0)\n", + "Requirement already satisfied: decorator>4.1.2 in /usr/local/lib/python3.10/dist-packages (from gcsfs->axolotl==0.3.0) (4.4.2)\n", + "Requirement already satisfied: google-auth>=1.2 in /usr/local/lib/python3.10/dist-packages (from gcsfs->axolotl==0.3.0) (2.17.3)\n", + "Requirement already satisfied: google-auth-oauthlib in /usr/local/lib/python3.10/dist-packages (from gcsfs->axolotl==0.3.0) (1.0.0)\n", + "Requirement already satisfied: google-cloud-storage in /usr/local/lib/python3.10/dist-packages (from gcsfs->axolotl==0.3.0) (2.8.0)\n", + "Collecting aiofiles<24.0,>=22.0 (from gradio->axolotl==0.3.0)\n", + " Downloading aiofiles-23.2.1-py3-none-any.whl (15 kB)\n", + "Requirement already satisfied: altair<6.0,>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from gradio->axolotl==0.3.0) (4.2.2)\n", + "Collecting ffmpy (from gradio->axolotl==0.3.0)\n", + " Downloading ffmpy-0.3.1.tar.gz (5.5 kB)\n", + " Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", + "Collecting gradio-client==0.7.0 (from gradio->axolotl==0.3.0)\n", + " Downloading gradio_client-0.7.0-py3-none-any.whl (302 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m302.7/302.7 kB\u001b[0m \u001b[31m37.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: importlib-resources<7.0,>=1.3 in /usr/local/lib/python3.10/dist-packages (from gradio->axolotl==0.3.0) (6.1.1)\n", + "Requirement already satisfied: markupsafe~=2.0 in /usr/local/lib/python3.10/dist-packages (from gradio->axolotl==0.3.0) (2.1.3)\n", + "Collecting orjson~=3.0 (from gradio->axolotl==0.3.0)\n", + " Downloading orjson-3.9.10-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m138.7/138.7 kB\u001b[0m \u001b[31m20.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: pillow<11.0,>=8.0 in /usr/local/lib/python3.10/dist-packages (from gradio->axolotl==0.3.0) (9.4.0)\n", + "INFO: pip is looking at multiple versions of gradio to determine which version is compatible with other requirements. This could take a while.\n", + "Collecting gradio (from axolotl==0.3.0)\n", + " Downloading gradio-4.4.0-py3-none-any.whl (15.4 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m15.4/15.4 MB\u001b[0m \u001b[31m84.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-4.3.0-py3-none-any.whl (15.4 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m15.4/15.4 MB\u001b[0m \u001b[31m76.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-4.2.0-py3-none-any.whl (15.4 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m15.4/15.4 MB\u001b[0m \u001b[31m81.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-4.1.2-py3-none-any.whl (15.4 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m15.4/15.4 MB\u001b[0m \u001b[31m83.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-4.1.1-py3-none-any.whl (15.4 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m15.4/15.4 MB\u001b[0m \u001b[31m78.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-4.1.0-py3-none-any.whl (15.4 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m15.4/15.4 MB\u001b[0m \u001b[31m17.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-4.0.2-py3-none-any.whl (25.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m25.3/25.3 MB\u001b[0m \u001b[31m49.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hINFO: pip is looking at multiple versions of gradio to determine which version is compatible with other requirements. This could take a while.\n", + " Downloading gradio-4.0.1-py3-none-any.whl (25.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m25.3/25.3 MB\u001b[0m \u001b[31m56.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-4.0.0-py3-none-any.whl (25.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m25.3/25.3 MB\u001b[0m \u001b[31m15.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.50.2-py3-none-any.whl (20.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.3/20.3 MB\u001b[0m \u001b[31m66.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio-client==0.6.1 (from gradio->axolotl==0.3.0)\n", + " Downloading gradio_client-0.6.1-py3-none-any.whl (299 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m299.2/299.2 kB\u001b[0m \u001b[31m33.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting pydub (from gradio->axolotl==0.3.0)\n", + " Downloading pydub-0.25.1-py2.py3-none-any.whl (32 kB)\n", + "Collecting python-multipart (from gradio->axolotl==0.3.0)\n", + " Downloading python_multipart-0.0.6-py3-none-any.whl (45 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m45.7/45.7 kB\u001b[0m \u001b[31m7.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting semantic-version~=2.0 (from gradio->axolotl==0.3.0)\n", + " Downloading semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)\n", + "Collecting websockets<12.0,>=10.0 (from gradio->axolotl==0.3.0)\n", + " Downloading websockets-11.0.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (129 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m129.9/129.9 kB\u001b[0m \u001b[31m18.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: llvmlite<0.42,>=0.41.0dev0 in /usr/local/lib/python3.10/dist-packages (from numba->axolotl==0.3.0) (0.41.1)\n", + "Collecting aiobotocore~=2.7.0 (from s3fs->axolotl==0.3.0)\n", + " Downloading aiobotocore-2.7.0-py3-none-any.whl (73 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m73.5/73.5 kB\u001b[0m \u001b[31m10.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hINFO: pip is looking at multiple versions of s3fs to determine which version is compatible with other requirements. This could take a while.\n", + "Collecting s3fs (from axolotl==0.3.0)\n", + " Downloading s3fs-2023.9.2-py3-none-any.whl (28 kB)\n", + "Collecting aiobotocore~=2.5.4 (from s3fs->axolotl==0.3.0)\n", + " Downloading aiobotocore-2.5.4-py3-none-any.whl (73 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m73.4/73.4 kB\u001b[0m \u001b[31m8.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting s3fs (from axolotl==0.3.0)\n", + " Downloading s3fs-2023.9.1-py3-none-any.whl (28 kB)\n", + " Downloading s3fs-2023.9.0-py3-none-any.whl (28 kB)\n", + " Downloading s3fs-2023.6.0-py3-none-any.whl (28 kB)\n", + "Requirement already satisfied: grpcio>=1.48.2 in /usr/local/lib/python3.10/dist-packages (from tensorboard->axolotl==0.3.0) (1.59.2)\n", + "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.10/dist-packages (from tensorboard->axolotl==0.3.0) (3.5.1)\n", + "Requirement already satisfied: protobuf>=3.19.6 in /usr/local/lib/python3.10/dist-packages (from tensorboard->axolotl==0.3.0) (3.20.3)\n", + "Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard->axolotl==0.3.0) (0.7.2)\n", + "Requirement already satisfied: werkzeug>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from tensorboard->axolotl==0.3.0) (3.0.1)\n", + "Requirement already satisfied: Click!=8.0.0,>=7.1 in /usr/local/lib/python3.10/dist-packages (from wandb->axolotl==0.3.0) (8.1.7)\n", + "Collecting GitPython!=3.1.29,>=1.0.0 (from wandb->axolotl==0.3.0)\n", + " Downloading GitPython-3.1.40-py3-none-any.whl (190 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m190.6/190.6 kB\u001b[0m \u001b[31m26.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting sentry-sdk>=1.0.0 (from wandb->axolotl==0.3.0)\n", + " Downloading sentry_sdk-1.35.0-py2.py3-none-any.whl (248 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m248.6/248.6 kB\u001b[0m \u001b[31m27.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting docker-pycreds>=0.4.0 (from wandb->axolotl==0.3.0)\n", + " Downloading docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)\n", + "Collecting setproctitle (from wandb->axolotl==0.3.0)\n", + " Downloading setproctitle-1.3.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (30 kB)\n", + "Requirement already satisfied: appdirs>=1.4.3 in /usr/local/lib/python3.10/dist-packages (from wandb->axolotl==0.3.0) (1.4.4)\n", + "Collecting botocore<1.31.18,>=1.31.17 (from aiobotocore~=2.5.4->s3fs->axolotl==0.3.0)\n", + " Downloading botocore-1.31.17-py3-none-any.whl (11.1 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m11.1/11.1 MB\u001b[0m \u001b[31m122.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: wrapt<2.0.0,>=1.10.10 in /usr/local/lib/python3.10/dist-packages (from aiobotocore~=2.5.4->s3fs->axolotl==0.3.0) (1.14.1)\n", + "Collecting aioitertools<1.0.0,>=0.5.1 (from aiobotocore~=2.5.4->s3fs->axolotl==0.3.0)\n", + " Downloading aioitertools-0.11.0-py3-none-any.whl (23 kB)\n", + "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->fschat==0.2.29->axolotl==0.3.0) (23.1.0)\n", + "Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->fschat==0.2.29->axolotl==0.3.0) (3.3.2)\n", + "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->fschat==0.2.29->axolotl==0.3.0) (6.0.4)\n", + "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.10/dist-packages (from aiohttp->fschat==0.2.29->axolotl==0.3.0) (4.0.3)\n", + "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->fschat==0.2.29->axolotl==0.3.0) (1.9.2)\n", + "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->fschat==0.2.29->axolotl==0.3.0) (1.4.0)\n", + "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->fschat==0.2.29->axolotl==0.3.0) (1.3.1)\n", + "Requirement already satisfied: entrypoints in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio->axolotl==0.3.0) (0.4)\n", + "Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio->axolotl==0.3.0) (4.19.2)\n", + "Requirement already satisfied: toolz in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio->axolotl==0.3.0) (0.12.0)\n", + "Collecting gitdb<5,>=4.0.1 (from GitPython!=3.1.29,>=1.0.0->wandb->axolotl==0.3.0)\n", + " Downloading gitdb-4.0.11-py3-none-any.whl (62 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m62.7/62.7 kB\u001b[0m \u001b[31m10.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from google-auth>=1.2->gcsfs->axolotl==0.3.0) (5.3.2)\n", + "Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from google-auth>=1.2->gcsfs->axolotl==0.3.0) (0.3.0)\n", + "Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.10/dist-packages (from google-auth>=1.2->gcsfs->axolotl==0.3.0) (4.9)\n", + "Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from google-auth-oauthlib->gcsfs->axolotl==0.3.0) (1.3.1)\n", + "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->bert-score==0.3.13->axolotl==0.3.0) (1.2.0)\n", + "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib->bert-score==0.3.13->axolotl==0.3.0) (0.12.1)\n", + "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->bert-score==0.3.13->axolotl==0.3.0) (4.44.3)\n", + "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->bert-score==0.3.13->axolotl==0.3.0) (1.4.5)\n", + "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->bert-score==0.3.13->axolotl==0.3.0) (3.1.1)\n", + "Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/dist-packages (from matplotlib->bert-score==0.3.13->axolotl==0.3.0) (2.8.2)\n", + "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas>=1.0.1->bert-score==0.3.13->axolotl==0.3.0) (2023.3.post1)\n", + "Requirement already satisfied: wcwidth in /usr/local/lib/python3.10/dist-packages (from prompt-toolkit>=3.0.0->fschat==0.2.29->axolotl==0.3.0) (0.2.10)\n", + "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->bert-score==0.3.13->axolotl==0.3.0) (3.4)\n", + "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->bert-score==0.3.13->axolotl==0.3.0) (2.0.7)\n", + "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->bert-score==0.3.13->axolotl==0.3.0) (2023.7.22)\n", + "Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from rich>=10.0.0->fschat==0.2.29->axolotl==0.3.0) (3.0.0)\n", + "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.10/dist-packages (from rich>=10.0.0->fschat==0.2.29->axolotl==0.3.0) (2.16.1)\n", + "INFO: pip is looking at multiple versions of tokenizers to determine which version is compatible with other requirements. This could take a while.\n", + "Collecting tokenizers<0.15,>=0.14 (from transformers==4.35.1->axolotl==0.3.0)\n", + " Downloading tokenizers-0.14.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.8 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.8/3.8 MB\u001b[0m \u001b[31m103.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hINFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.\n", + "Collecting gradio (from axolotl==0.3.0)\n", + " Downloading gradio-3.50.1-py3-none-any.whl (20.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.3/20.3 MB\u001b[0m \u001b[31m88.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.50.0-py3-none-any.whl (20.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.3/20.3 MB\u001b[0m \u001b[31m37.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.49.0-py3-none-any.whl (20.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.3/20.3 MB\u001b[0m \u001b[31m57.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hINFO: pip is looking at multiple versions of tokenizers to determine which version is compatible with other requirements. This could take a while.\n", + " Downloading gradio-3.48.0-py3-none-any.whl (20.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.3/20.3 MB\u001b[0m \u001b[31m63.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.47.1-py3-none-any.whl (20.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.3/20.3 MB\u001b[0m \u001b[31m18.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio-client==0.6.0 (from gradio->axolotl==0.3.0)\n", + " Downloading gradio_client-0.6.0-py3-none-any.whl (298 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m298.8/298.8 kB\u001b[0m \u001b[31m37.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio (from axolotl==0.3.0)\n", + " Downloading gradio-3.47.0-py3-none-any.whl (20.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.3/20.3 MB\u001b[0m \u001b[31m98.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hINFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.\n", + " Downloading gradio-3.46.1-py3-none-any.whl (20.2 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.2/20.2 MB\u001b[0m \u001b[31m95.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio-client==0.5.3 (from gradio->axolotl==0.3.0)\n", + " Downloading gradio_client-0.5.3-py3-none-any.whl (298 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m298.4/298.4 kB\u001b[0m \u001b[31m39.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio (from axolotl==0.3.0)\n", + " Downloading gradio-3.46.0-py3-none-any.whl (20.2 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.2/20.2 MB\u001b[0m \u001b[31m95.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.45.2-py3-none-any.whl (20.2 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.2/20.2 MB\u001b[0m \u001b[31m28.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.45.1-py3-none-any.whl (20.2 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.2/20.2 MB\u001b[0m \u001b[31m57.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio-client==0.5.2 (from gradio->axolotl==0.3.0)\n", + " Downloading gradio_client-0.5.2-py3-none-any.whl (298 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m298.3/298.3 kB\u001b[0m \u001b[31m37.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio (from axolotl==0.3.0)\n", + " Downloading gradio-3.45.0-py3-none-any.whl (20.2 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.2/20.2 MB\u001b[0m \u001b[31m96.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.44.4-py3-none-any.whl (20.2 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.2/20.2 MB\u001b[0m \u001b[31m22.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio-client==0.5.1 (from gradio->axolotl==0.3.0)\n", + " Downloading gradio_client-0.5.1-py3-none-any.whl (298 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m298.2/298.2 kB\u001b[0m \u001b[31m19.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio (from axolotl==0.3.0)\n", + " Downloading gradio-3.44.3-py3-none-any.whl (20.2 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.2/20.2 MB\u001b[0m \u001b[31m96.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio-client==0.5.0 (from gradio->axolotl==0.3.0)\n", + " Downloading gradio_client-0.5.0-py3-none-any.whl (298 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m298.2/298.2 kB\u001b[0m \u001b[31m37.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio (from axolotl==0.3.0)\n", + " Downloading gradio-3.44.2-py3-none-any.whl (20.2 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.2/20.2 MB\u001b[0m \u001b[31m91.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.44.1-py3-none-any.whl (20.2 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.2/20.2 MB\u001b[0m \u001b[31m104.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.44.0-py3-none-any.whl (20.2 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.2/20.2 MB\u001b[0m \u001b[31m18.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.43.2-py3-none-any.whl (20.1 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.1/20.1 MB\u001b[0m \u001b[31m67.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.43.1-py3-none-any.whl (20.1 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.1/20.1 MB\u001b[0m \u001b[31m64.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.43.0-py3-none-any.whl (20.1 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.1/20.1 MB\u001b[0m \u001b[31m75.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.42.0-py3-none-any.whl (20.1 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.1/20.1 MB\u001b[0m \u001b[31m16.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.41.2-py3-none-any.whl (20.1 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.1/20.1 MB\u001b[0m \u001b[31m75.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.41.1-py3-none-any.whl (20.1 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.1/20.1 MB\u001b[0m \u001b[31m40.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.41.0-py3-none-any.whl (20.1 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.1/20.1 MB\u001b[0m \u001b[31m25.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.40.1-py3-none-any.whl (20.0 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.0/20.0 MB\u001b[0m \u001b[31m53.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting mdit-py-plugins<=0.3.3 (from gradio->axolotl==0.3.0)\n", + " Downloading mdit_py_plugins-0.3.3-py3-none-any.whl (50 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.5/50.5 kB\u001b[0m \u001b[31m6.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio-client>=0.4.0 (from gradio->axolotl==0.3.0)\n", + " Downloading gradio_client-0.4.0-py3-none-any.whl (297 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m297.4/297.4 kB\u001b[0m \u001b[31m32.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio (from axolotl==0.3.0)\n", + " Downloading gradio-3.40.0-py3-none-any.whl (20.0 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.0/20.0 MB\u001b[0m \u001b[31m54.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.39.0-py3-none-any.whl (19.9 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m19.9/19.9 MB\u001b[0m \u001b[31m67.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio-client>=0.3.0 (from gradio->axolotl==0.3.0)\n", + " Downloading gradio_client-0.3.0-py3-none-any.whl (294 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m294.2/294.2 kB\u001b[0m \u001b[31m37.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio (from axolotl==0.3.0)\n", + " Downloading gradio-3.38.0-py3-none-any.whl (19.8 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m19.8/19.8 MB\u001b[0m \u001b[31m18.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio-client>=0.2.10 (from gradio->axolotl==0.3.0)\n", + " Downloading gradio_client-0.2.10-py3-none-any.whl (288 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m289.0/289.0 kB\u001b[0m \u001b[31m35.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio (from axolotl==0.3.0)\n", + " Downloading gradio-3.37.0-py3-none-any.whl (19.8 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m19.8/19.8 MB\u001b[0m \u001b[31m90.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.36.1-py3-none-any.whl (19.8 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m19.8/19.8 MB\u001b[0m \u001b[31m103.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting websockets>=10.0 (from gradio->axolotl==0.3.0)\n", + " Downloading websockets-12.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (130 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m130.2/130.2 kB\u001b[0m \u001b[31m18.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio-client>=0.2.7 (from gradio->axolotl==0.3.0)\n", + " Downloading gradio_client-0.2.9-py3-none-any.whl (288 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m288.8/288.8 kB\u001b[0m \u001b[31m32.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio_client-0.2.8-py3-none-any.whl (288 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m288.8/288.8 kB\u001b[0m \u001b[31m27.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio_client-0.2.7-py3-none-any.whl (288 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m288.4/288.4 kB\u001b[0m \u001b[31m36.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio (from axolotl==0.3.0)\n", + " Downloading gradio-3.36.0-py3-none-any.whl (19.8 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m19.8/19.8 MB\u001b[0m \u001b[31m74.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.35.2-py3-none-any.whl (19.7 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m19.7/19.7 MB\u001b[0m \u001b[31m22.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.35.1-py3-none-any.whl (19.7 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m19.7/19.7 MB\u001b[0m \u001b[31m52.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.35.0-py3-none-any.whl (19.7 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m19.7/19.7 MB\u001b[0m \u001b[31m93.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.34.0-py3-none-any.whl (20.0 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.0/20.0 MB\u001b[0m \u001b[31m92.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio-client>=0.2.6 (from gradio->axolotl==0.3.0)\n", + " Downloading gradio_client-0.2.6-py3-none-any.whl (288 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m288.3/288.3 kB\u001b[0m \u001b[31m30.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio (from axolotl==0.3.0)\n", + " Downloading gradio-3.33.1-py3-none-any.whl (20.0 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.0/20.0 MB\u001b[0m \u001b[31m18.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio-client>=0.2.4 (from gradio->axolotl==0.3.0)\n", + " Downloading gradio_client-0.2.5-py3-none-any.whl (288 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m288.1/288.1 kB\u001b[0m \u001b[31m34.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio_client-0.2.4-py3-none-any.whl (287 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m287.9/287.9 kB\u001b[0m \u001b[31m30.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio (from axolotl==0.3.0)\n", + " Downloading gradio-3.33.0-py3-none-any.whl (20.0 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.0/20.0 MB\u001b[0m \u001b[31m95.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.32.0-py3-none-any.whl (19.9 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m19.9/19.9 MB\u001b[0m \u001b[31m95.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.31.0-py3-none-any.whl (17.4 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m17.4/17.4 MB\u001b[0m \u001b[31m86.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.30.0-py3-none-any.whl (17.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m17.3/17.3 MB\u001b[0m \u001b[31m73.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.29.0-py3-none-any.whl (17.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m17.3/17.3 MB\u001b[0m \u001b[31m15.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio-client>=0.2.1 (from gradio->axolotl==0.3.0)\n", + " Downloading gradio_client-0.2.3-py3-none-any.whl (287 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m287.9/287.9 kB\u001b[0m \u001b[31m32.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio_client-0.2.2-py3-none-any.whl (287 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m287.9/287.9 kB\u001b[0m \u001b[31m33.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio_client-0.2.1-py3-none-any.whl (287 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m287.9/287.9 kB\u001b[0m \u001b[31m34.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio (from axolotl==0.3.0)\n", + " Downloading gradio-3.28.3-py3-none-any.whl (17.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m17.3/17.3 MB\u001b[0m \u001b[31m75.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio-client>=0.1.3 (from gradio->axolotl==0.3.0)\n", + " Downloading gradio_client-0.2.0-py3-none-any.whl (287 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m287.9/287.9 kB\u001b[0m \u001b[31m30.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio_client-0.1.4-py3-none-any.whl (286 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m286.7/286.7 kB\u001b[0m \u001b[31m32.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio_client-0.1.3-py3-none-any.whl (286 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m286.2/286.2 kB\u001b[0m \u001b[31m31.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio (from axolotl==0.3.0)\n", + " Downloading gradio-3.28.2-py3-none-any.whl (17.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m17.3/17.3 MB\u001b[0m \u001b[31m68.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.28.1-py3-none-any.whl (17.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m17.3/17.3 MB\u001b[0m \u001b[31m64.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.28.0-py3-none-any.whl (17.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m17.3/17.3 MB\u001b[0m \u001b[31m98.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.27.0-py3-none-any.whl (17.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m17.3/17.3 MB\u001b[0m \u001b[31m33.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.26.0-py3-none-any.whl (17.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m17.3/17.3 MB\u001b[0m \u001b[31m101.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio-client==0.1.2 (from gradio->axolotl==0.3.0)\n", + " Downloading gradio_client-0.1.2-py3-none-any.whl (286 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m286.1/286.1 kB\u001b[0m \u001b[31m36.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio (from axolotl==0.3.0)\n", + " Downloading gradio-3.25.0-py3-none-any.whl (17.5 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m17.5/17.5 MB\u001b[0m \u001b[31m101.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio-client>=0.0.8 (from gradio->axolotl==0.3.0)\n", + " Downloading gradio_client-0.1.0-py3-none-any.whl (24 kB)\n", + " Downloading gradio_client-0.0.10-py3-none-any.whl (23 kB)\n", + " Downloading gradio_client-0.0.9-py3-none-any.whl (23 kB)\n", + " Downloading gradio_client-0.0.8-py3-none-any.whl (20 kB)\n", + "Collecting gradio (from axolotl==0.3.0)\n", + " Downloading gradio-3.24.1-py3-none-any.whl (15.7 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m15.7/15.7 MB\u001b[0m \u001b[31m86.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting gradio-client>=0.0.5 (from gradio->axolotl==0.3.0)\n", + " Downloading gradio_client-0.0.7-py3-none-any.whl (14 kB)\n", + " Downloading gradio_client-0.0.6-py3-none-any.whl (14 kB)\n", + " Downloading gradio_client-0.0.5-py3-none-any.whl (12 kB)\n", + "Collecting gradio (from axolotl==0.3.0)\n", + " Downloading gradio-3.24.0-py3-none-any.whl (15.7 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m15.7/15.7 MB\u001b[0m \u001b[31m15.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.23.0-py3-none-any.whl (15.8 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m15.8/15.8 MB\u001b[0m \u001b[31m107.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.22.1-py3-none-any.whl (15.8 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m15.8/15.8 MB\u001b[0m \u001b[31m91.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.22.0-py3-none-any.whl (15.8 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m15.8/15.8 MB\u001b[0m \u001b[31m90.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.21.0-py3-none-any.whl (15.8 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m15.8/15.8 MB\u001b[0m \u001b[31m102.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Downloading gradio-3.20.1-py3-none-any.whl (14.3 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m14.3/14.3 MB\u001b[0m \u001b[31m106.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting pycryptodome (from gradio->axolotl==0.3.0)\n", + " Downloading pycryptodome-3.19.0-cp35-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.1 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.1/2.1 MB\u001b[0m \u001b[31m92.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting datasets>=2.14.0 (from axolotl==0.3.0)\n", + " Downloading datasets-2.14.7-py3-none-any.whl (520 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m520.4/520.4 kB\u001b[0m \u001b[31m53.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting huggingface-hub (from accelerate==0.24.1->axolotl==0.3.0)\n", + " Downloading huggingface_hub-0.17.3-py3-none-any.whl (295 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m295.0/295.0 kB\u001b[0m \u001b[31m32.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hINFO: pip is looking at multiple versions of s3fs to determine which version is compatible with other requirements. This could take a while.\n", + "INFO: pip is looking at multiple versions of transformers[sentencepiece] to determine which version is compatible with other requirements. This could take a while.\n", + "Collecting h11>=0.8 (from uvicorn->fschat==0.2.29->axolotl==0.3.0)\n", + " Downloading h11-0.14.0-py3-none-any.whl (58 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m8.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting humanfriendly>=9.1 (from coloredlogs->optimum==1.13.2->axolotl==0.3.0)\n", + " Downloading humanfriendly-10.0-py2.py3-none-any.whl (86 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m86.8/86.8 kB\u001b[0m \u001b[31m319.1 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: anyio<4.0.0,>=3.7.1 in /usr/local/lib/python3.10/dist-packages (from fastapi->fschat==0.2.29->axolotl==0.3.0) (3.7.1)\n", + "Collecting starlette<0.28.0,>=0.27.0 (from fastapi->fschat==0.2.29->axolotl==0.3.0)\n", + " Downloading starlette-0.27.0-py3-none-any.whl (66 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.0/67.0 kB\u001b[0m \u001b[31m8.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting typing-extensions (from torch>=1.10.0->accelerate==0.24.1->axolotl==0.3.0)\n", + " Downloading typing_extensions-4.8.0-py3-none-any.whl (31 kB)\n", + "Requirement already satisfied: google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5 in /usr/local/lib/python3.10/dist-packages (from google-cloud-storage->gcsfs->axolotl==0.3.0) (2.11.1)\n", + "Requirement already satisfied: google-cloud-core<3.0dev,>=2.3.0 in /usr/local/lib/python3.10/dist-packages (from google-cloud-storage->gcsfs->axolotl==0.3.0) (2.3.3)\n", + "Requirement already satisfied: google-resumable-media>=2.3.2 in /usr/local/lib/python3.10/dist-packages (from google-cloud-storage->gcsfs->axolotl==0.3.0) (2.6.0)\n", + "Collecting httpcore (from httpx->fschat==0.2.29->axolotl==0.3.0)\n", + " Downloading httpcore-1.0.2-py3-none-any.whl (76 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m76.9/76.9 kB\u001b[0m \u001b[31m9.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx->fschat==0.2.29->axolotl==0.3.0) (1.3.0)\n", + "Collecting wavedrom (from markdown2[all]->fschat==0.2.29->axolotl==0.3.0)\n", + " Downloading wavedrom-2.0.3.post3.tar.gz (137 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m137.7/137.7 kB\u001b[0m \u001b[31m19.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n", + "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->optimum==1.13.2->axolotl==0.3.0) (1.3.0)\n", + "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<4.0.0,>=3.7.1->fastapi->fschat==0.2.29->axolotl==0.3.0) (1.1.3)\n", + "Collecting jmespath<2.0.0,>=0.7.1 (from botocore<1.31.18,>=1.31.17->aiobotocore~=2.5.4->s3fs->axolotl==0.3.0)\n", + " Downloading jmespath-1.0.1-py3-none-any.whl (20 kB)\n", + "Collecting urllib3<3,>=1.21.1 (from requests->bert-score==0.3.13->axolotl==0.3.0)\n", + " Downloading urllib3-1.26.18-py2.py3-none-any.whl (143 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m143.8/143.8 kB\u001b[0m \u001b[31m19.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hCollecting smmap<6,>=3.0.1 (from gitdb<5,>=4.0.1->GitPython!=3.1.29,>=1.0.0->wandb->axolotl==0.3.0)\n", + " Downloading smmap-5.0.1-py3-none-any.whl (24 kB)\n", + "Requirement already satisfied: googleapis-common-protos<2.0.dev0,>=1.56.2 in /usr/local/lib/python3.10/dist-packages (from google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-storage->gcsfs->axolotl==0.3.0) (1.61.0)\n", + "Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in /usr/local/lib/python3.10/dist-packages (from google-resumable-media>=2.3.2->google-cloud-storage->gcsfs->axolotl==0.3.0) (1.5.0)\n", + "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio->axolotl==0.3.0) (2023.11.1)\n", + "Requirement already satisfied: referencing>=0.28.4 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio->axolotl==0.3.0) (0.31.0)\n", + "Requirement already satisfied: rpds-py>=0.7.1 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio->axolotl==0.3.0) (0.12.0)\n", + "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py>=2.2.0->rich>=10.0.0->fschat==0.2.29->axolotl==0.3.0) (0.1.2)\n", + "Requirement already satisfied: pyasn1<0.6.0,>=0.4.6 in /usr/local/lib/python3.10/dist-packages (from pyasn1-modules>=0.2.1->google-auth>=1.2->gcsfs->axolotl==0.3.0) (0.5.0)\n", + "Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.10/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib->gcsfs->axolotl==0.3.0) (3.2.2)\n", + "Collecting svgwrite (from wavedrom->markdown2[all]->fschat==0.2.29->axolotl==0.3.0)\n", + " Downloading svgwrite-1.4.3-py3-none-any.whl (67 kB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.1/67.1 kB\u001b[0m \u001b[31m10.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hBuilding wheels for collected packages: optimum, rouge-score, flash-attn, deepspeed, fire, ffmpy, wavedrom, lit\n", + " Building wheel for optimum (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n", + " Created wheel for optimum: filename=optimum-1.13.2-py3-none-any.whl size=395598 sha256=c50241754999443c85c875c140c77aa5bf3a40f56a5ffd196d1f3c5199609431\n", + " Stored in directory: /root/.cache/pip/wheels/6e/b7/2c/79405d98f0943373d8546daeae25a3d377f7659ca0cbe48699\n", + " Building wheel for rouge-score (setup.py) ... \u001b[?25l\u001b[?25hdone\n", + " Created wheel for rouge-score: filename=rouge_score-0.1.2-py3-none-any.whl size=24933 sha256=cc9084a7eefdb136724effd732f6cb7c0fd01f0d7c9e7852043f1cd1d23b45a9\n", + " Stored in directory: /root/.cache/pip/wheels/5f/dd/89/461065a73be61a532ff8599a28e9beef17985c9e9c31e541b4\n", + " Building wheel for flash-attn (setup.py) ... \u001b[?25l\u001b[?25hdone\n", + " Created wheel for flash-attn: filename=flash_attn-2.3.3-cp310-cp310-linux_x86_64.whl size=57075008 sha256=bcb63b64213ab61590b340b77de84e448a442e19c100480895194df39ad7673d\n", + " Stored in directory: /root/.cache/pip/wheels/e5/e6/fa/941802ec61d1afd320d27160ab1db98e6dba65381f84b76d4a\n", + " Building wheel for deepspeed (setup.py) ... \u001b[?25l\u001b[?25hdone\n", + " Created wheel for deepspeed: filename=deepspeed-0.12.3-py3-none-any.whl size=1279165 sha256=e678c7b56fa15c9218f88c9a8a48728bee271215c554abcb4049f609e47adb25\n", + " Stored in directory: /root/.cache/pip/wheels/ee/2b/c5/892ceee06964ce8aa2a98d4260848d0d9a3f1e743862e4b45a\n", + " Building wheel for fire (setup.py) ... \u001b[?25l\u001b[?25hdone\n", + " Created wheel for fire: filename=fire-0.5.0-py2.py3-none-any.whl size=116933 sha256=2e829d74a791417c18e5d80016d0125dc65481ae4577bfd4041e51aa64558256\n", + " Stored in directory: /root/.cache/pip/wheels/90/d4/f7/9404e5db0116bd4d43e5666eaa3e70ab53723e1e3ea40c9a95\n", + " Building wheel for ffmpy (setup.py) ... \u001b[?25l\u001b[?25hdone\n", + " Created wheel for ffmpy: filename=ffmpy-0.3.1-py3-none-any.whl size=5579 sha256=dd713b1fe6d28ec101c96dbf7033241901635fcddd42a07d9e97fafada70eca4\n", + " Stored in directory: /root/.cache/pip/wheels/01/a6/d1/1c0828c304a4283b2c1639a09ad86f83d7c487ef34c6b4a1bf\n", + " Building wheel for wavedrom (setup.py) ... \u001b[?25l\u001b[?25hdone\n", + " Created wheel for wavedrom: filename=wavedrom-2.0.3.post3-py2.py3-none-any.whl size=30053 sha256=65b629500b343fc851f1c23dd2065fa414153974ae25603bba9f99e559ecbf8c\n", + " Stored in directory: /root/.cache/pip/wheels/9c/52/8c/38b454b42f712f325e26f633287484c7dc1ad469e1580c5954\n", + " Building wheel for lit (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n", + " Created wheel for lit: filename=lit-17.0.5-py3-none-any.whl size=93256 sha256=209fa0a842c16d9479d3626694714ebd0b9f4afaaacf487e3fb92d19ecfa9fcf\n", + " Stored in directory: /root/.cache/pip/wheels/1c/87/8e/5a42c0d4be23362b68bbff33b17f3c35a3df44f1cd2f5a24b4\n", + "Successfully built optimum rouge-score flash-attn deepspeed fire ffmpy wavedrom lit\n", + "Installing collected packages: sentencepiece, pydub, ninja, nh3, lit, hjson, ffmpy, bitsandbytes, addict, websockets, urllib3, typing-extensions, svgwrite, smmap, shortuuid, setproctitle, semantic-version, rouge, python-multipart, pynvml, pyarrow-hotfix, orjson, nvidia-nvtx-cu11, nvidia-nccl-cu11, nvidia-cusparse-cu11, nvidia-curand-cu11, nvidia-cufft-cu11, nvidia-cuda-runtime-cu11, nvidia-cuda-nvrtc-cu11, nvidia-cuda-cupti-cu11, nvidia-cublas-cu11, numpy, markdown2, jmespath, humanfriendly, hf_transfer, h11, fire, einops, docker-pycreds, dill, colorama, art, aioitertools, aiofiles, wavedrom, uvicorn, starlette, sentry-sdk, rouge-score, nvidia-cusolver-cu11, nvidia-cudnn-cu11, multiprocess, httpcore, gitdb, gekko, coloredlogs, botocore, tiktoken, responses, huggingface-hub, httpx, GitPython, fastapi, aiobotocore, wandb, tokenizers, s3fs, gradio-client, fschat, datasets, transformers, gradio, evaluate, triton, torch, accelerate, peft, xformers, optimum, bert-score, auto-gptq, flash-attn, deepspeed, axolotl\n", + " Attempting uninstall: urllib3\n", + " Found existing installation: urllib3 2.0.7\n", + " Uninstalling urllib3-2.0.7:\n", + " Successfully uninstalled urllib3-2.0.7\n", + " Attempting uninstall: typing-extensions\n", + " Found existing installation: typing_extensions 4.5.0\n", + " Uninstalling typing_extensions-4.5.0:\n", + " Successfully uninstalled typing_extensions-4.5.0\n", + " Attempting uninstall: numpy\n", + " Found existing installation: numpy 1.23.5\n", + " Uninstalling numpy-1.23.5:\n", + " Successfully uninstalled numpy-1.23.5\n", + " Attempting uninstall: huggingface-hub\n", + " Found existing installation: huggingface-hub 0.19.3\n", + " Uninstalling huggingface-hub-0.19.3:\n", + " Successfully uninstalled huggingface-hub-0.19.3\n", + " Attempting uninstall: tokenizers\n", + " Found existing installation: tokenizers 0.15.0\n", + " Uninstalling tokenizers-0.15.0:\n", + " Successfully uninstalled tokenizers-0.15.0\n", + " Attempting uninstall: transformers\n", + " Found existing installation: transformers 4.35.2\n", + " Uninstalling transformers-4.35.2:\n", + " Successfully uninstalled transformers-4.35.2\n", + " Attempting uninstall: triton\n", + " Found existing installation: triton 2.1.0\n", + " Uninstalling triton-2.1.0:\n", + " Successfully uninstalled triton-2.1.0\n", + " Attempting uninstall: torch\n", + " Found existing installation: torch 2.1.0+cu118\n", + " Uninstalling torch-2.1.0+cu118:\n", + " Successfully uninstalled torch-2.1.0+cu118\n", + " Running setup.py develop for axolotl\n", + "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n", + "lida 0.0.10 requires kaleido, which is not installed.\n", + "llmx 0.0.15a0 requires cohere, which is not installed.\n", + "llmx 0.0.15a0 requires openai, which is not installed.\n", + "cupy-cuda11x 11.0.0 requires numpy<1.26,>=1.20, but you have numpy 1.26.2 which is incompatible.\n", + "tensorflow-probability 0.22.0 requires typing-extensions<4.6.0, but you have typing-extensions 4.8.0 which is incompatible.\n", + "torchaudio 2.1.0+cu118 requires torch==2.1.0, but you have torch 2.0.1 which is incompatible.\n", + "torchdata 0.7.0 requires torch==2.1.0, but you have torch 2.0.1 which is incompatible.\n", + "torchtext 0.16.0 requires torch==2.1.0, but you have torch 2.0.1 which is incompatible.\n", + "torchvision 0.16.0+cu118 requires torch==2.1.0, but you have torch 2.0.1 which is incompatible.\u001b[0m\u001b[31m\n", + "\u001b[0mSuccessfully installed GitPython-3.1.40 accelerate-0.24.1 addict-2.4.0 aiobotocore-2.5.4 aiofiles-23.2.1 aioitertools-0.11.0 art-6.1 auto-gptq-0.5.1 axolotl-0.3.0 bert-score-0.3.13 bitsandbytes-0.41.2.post2 botocore-1.31.17 colorama-0.4.6 coloredlogs-15.0.1 datasets-2.14.7 deepspeed-0.12.3 dill-0.3.7 docker-pycreds-0.4.0 einops-0.7.0 evaluate-0.4.0 fastapi-0.104.1 ffmpy-0.3.1 fire-0.5.0 flash-attn-2.3.3 fschat-0.2.29 gekko-1.0.6 gitdb-4.0.11 gradio-3.50.2 gradio-client-0.6.1 h11-0.14.0 hf_transfer-0.1.4 hjson-3.1.0 httpcore-1.0.2 httpx-0.25.1 huggingface-hub-0.17.3 humanfriendly-10.0 jmespath-1.0.1 lit-17.0.5 markdown2-2.4.10 multiprocess-0.70.15 nh3-0.2.14 ninja-1.11.1.1 numpy-1.26.2 nvidia-cublas-cu11-11.10.3.66 nvidia-cuda-cupti-cu11-11.7.101 nvidia-cuda-nvrtc-cu11-11.7.99 nvidia-cuda-runtime-cu11-11.7.99 nvidia-cudnn-cu11-8.5.0.96 nvidia-cufft-cu11-10.9.0.58 nvidia-curand-cu11-10.2.10.91 nvidia-cusolver-cu11-11.4.0.1 nvidia-cusparse-cu11-11.7.4.91 nvidia-nccl-cu11-2.14.3 nvidia-nvtx-cu11-11.7.91 optimum-1.13.2 orjson-3.9.10 peft-0.6.0 pyarrow-hotfix-0.5 pydub-0.25.1 pynvml-11.5.0 python-multipart-0.0.6 responses-0.18.0 rouge-1.0.1 rouge-score-0.1.2 s3fs-2023.6.0 semantic-version-2.10.0 sentencepiece-0.1.99 sentry-sdk-1.35.0 setproctitle-1.3.3 shortuuid-1.0.11 smmap-5.0.1 starlette-0.27.0 svgwrite-1.4.3 tiktoken-0.5.1 tokenizers-0.14.1 torch-2.0.1 transformers-4.35.1 triton-2.0.0 typing-extensions-4.8.0 urllib3-1.26.18 uvicorn-0.24.0.post1 wandb-0.16.0 wavedrom-2.0.3.post3 websockets-11.0.3 xformers-0.0.22\n" + ] + } + ], + "source": [ + "# Install axolotl\n", + "!git clone https://github.com/OpenAccess-AI-Collective/axolotl && cd axolotl && git checkout 797f3dd1de8fd8c0eafbd1c9fdb172abd9ff840a #0.3.0\n", + "!cd axolotl\n", + "!pip install packaging\n", + "!cd axolotl && pip install -e '.[flash-attn,deepspeed]'" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "ET82VllsW6gU", + "outputId": "27e4d16a-da64-46ed-b927-ce12b3f9af6d" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "accelerate configuration saved at /root/.cache/huggingface/accelerate/default_config.yaml\n" + ] + } + ], + "source": [ + "!accelerate config default" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "n_xquNdQYsMX", + "outputId": "68de83a9-2e5a-49e6-ff06-5013eb085370" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Requirement already satisfied: accelerate in /usr/local/lib/python3.10/dist-packages (0.24.1)\n", + "Requirement already satisfied: bitsandbytes in /usr/local/lib/python3.10/dist-packages (0.41.2.post2)\n", + "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from accelerate) (1.26.2)\n", + "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from accelerate) (23.2)\n", + "Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate) (5.9.5)\n", + "Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from accelerate) (6.0.1)\n", + "Requirement already satisfied: torch>=1.10.0 in /usr/local/lib/python3.10/dist-packages (from accelerate) (2.0.1)\n", + "Requirement already satisfied: huggingface-hub in /usr/local/lib/python3.10/dist-packages (from accelerate) (0.17.3)\n", + "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (3.13.1)\n", + "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (4.8.0)\n", + "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (1.12)\n", + "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (3.2.1)\n", + "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (3.1.2)\n", + "Requirement already satisfied: nvidia-cuda-nvrtc-cu11==11.7.99 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (11.7.99)\n", + "Requirement already satisfied: nvidia-cuda-runtime-cu11==11.7.99 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (11.7.99)\n", + "Requirement already satisfied: nvidia-cuda-cupti-cu11==11.7.101 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (11.7.101)\n", + "Requirement already satisfied: nvidia-cudnn-cu11==8.5.0.96 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (8.5.0.96)\n", + "Requirement already satisfied: nvidia-cublas-cu11==11.10.3.66 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (11.10.3.66)\n", + "Requirement already satisfied: nvidia-cufft-cu11==10.9.0.58 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (10.9.0.58)\n", + "Requirement already satisfied: nvidia-curand-cu11==10.2.10.91 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (10.2.10.91)\n", + "Requirement already satisfied: nvidia-cusolver-cu11==11.4.0.1 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (11.4.0.1)\n", + "Requirement already satisfied: nvidia-cusparse-cu11==11.7.4.91 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (11.7.4.91)\n", + "Requirement already satisfied: nvidia-nccl-cu11==2.14.3 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (2.14.3)\n", + "Requirement already satisfied: nvidia-nvtx-cu11==11.7.91 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (11.7.91)\n", + "Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (2.0.0)\n", + "Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from nvidia-cublas-cu11==11.10.3.66->torch>=1.10.0->accelerate) (67.7.2)\n", + "Requirement already satisfied: wheel in /usr/local/lib/python3.10/dist-packages (from nvidia-cublas-cu11==11.10.3.66->torch>=1.10.0->accelerate) (0.41.3)\n", + "Requirement already satisfied: cmake in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.10.0->accelerate) (3.27.7)\n", + "Requirement already satisfied: lit in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.10.0->accelerate) (17.0.5)\n", + "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->accelerate) (2023.6.0)\n", + "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->accelerate) (2.31.0)\n", + "Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->accelerate) (4.66.1)\n", + "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=1.10.0->accelerate) (2.1.3)\n", + "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (3.3.2)\n", + "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (3.4)\n", + "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (1.26.18)\n", + "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (2023.7.22)\n", + "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.10.0->accelerate) (1.3.0)\n", + "/content\n" + ] + } + ], + "source": [ + "!pip install accelerate bitsandbytes\n", + "!pwd" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "IZv_VnRrtTSz", + "outputId": "36439ab2-c4de-46b5-dd36-b90d09db8358" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "True" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import torch\n", + "torch.cuda.is_available()" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "9SwNYGmisJU6", + "outputId": "963ffc20-dd38-48e4-f72d-8a4d78e36461" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Requirement already satisfied: transformers in /usr/local/lib/python3.10/dist-packages (4.35.1)\n", + "Collecting transformers\n", + " Downloading transformers-4.35.2-py3-none-any.whl (7.9 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.9/7.9 MB\u001b[0m \u001b[31m24.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers) (3.13.1)\n", + "Requirement already satisfied: huggingface-hub<1.0,>=0.16.4 in /usr/local/lib/python3.10/dist-packages (from transformers) (0.17.3)\n", + "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from transformers) (1.26.2)\n", + "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers) (23.2)\n", + "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers) (6.0.1)\n", + "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers) (2023.6.3)\n", + "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers) (2.31.0)\n", + "Requirement already satisfied: tokenizers<0.19,>=0.14 in /usr/local/lib/python3.10/dist-packages (from transformers) (0.14.1)\n", + "Requirement already satisfied: safetensors>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from transformers) (0.4.0)\n", + "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers) (4.66.1)\n", + "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.16.4->transformers) (2023.6.0)\n", + "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.16.4->transformers) (4.8.0)\n", + "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (3.3.2)\n", + "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (3.4)\n", + "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (1.26.18)\n", + "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (2023.7.22)\n", + "Installing collected packages: transformers\n", + " Attempting uninstall: transformers\n", + " Found existing installation: transformers 4.35.1\n", + " Uninstalling transformers-4.35.1:\n", + " Successfully uninstalled transformers-4.35.1\n", + "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n", + "axolotl 0.3.0 requires transformers==4.35.1, but you have transformers 4.35.2 which is incompatible.\u001b[0m\u001b[31m\n", + "\u001b[0mSuccessfully installed transformers-4.35.2\n" + ] + } + ], + "source": [ + "!pip install --upgrade transformers" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "jCsnQhmave0Z", + "outputId": "6c9f39ef-0e4a-41cb-ee22-8c71438e254e" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Collecting flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE\n", + " Downloading https://github.com/Dao-AILab/flash-attention/releases/download/v2.3.0/flash_attn-2.3.0+cu117torch2.0cxx11abiFALSE-cp310-cp310-linux_x86_64.whl (30.0 MB)\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m30.0/30.0 MB\u001b[0m \u001b[31m46.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25hRequirement already satisfied: torch in /usr/local/lib/python3.10/dist-packages (from flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (2.0.1)\n", + "Requirement already satisfied: einops in /usr/local/lib/python3.10/dist-packages (from flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (0.7.0)\n", + "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (23.2)\n", + "Requirement already satisfied: ninja in /usr/local/lib/python3.10/dist-packages (from flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (1.11.1.1)\n", + "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (3.13.1)\n", + "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (4.8.0)\n", + "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (1.12)\n", + "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (3.2.1)\n", + "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (3.1.2)\n", + "Requirement already satisfied: nvidia-cuda-nvrtc-cu11==11.7.99 in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (11.7.99)\n", + "Requirement already satisfied: nvidia-cuda-runtime-cu11==11.7.99 in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (11.7.99)\n", + "Requirement already satisfied: nvidia-cuda-cupti-cu11==11.7.101 in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (11.7.101)\n", + "Requirement already satisfied: nvidia-cudnn-cu11==8.5.0.96 in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (8.5.0.96)\n", + "Requirement already satisfied: nvidia-cublas-cu11==11.10.3.66 in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (11.10.3.66)\n", + "Requirement already satisfied: nvidia-cufft-cu11==10.9.0.58 in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (10.9.0.58)\n", + "Requirement already satisfied: nvidia-curand-cu11==10.2.10.91 in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (10.2.10.91)\n", + "Requirement already satisfied: nvidia-cusolver-cu11==11.4.0.1 in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (11.4.0.1)\n", + "Requirement already satisfied: nvidia-cusparse-cu11==11.7.4.91 in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (11.7.4.91)\n", + "Requirement already satisfied: nvidia-nccl-cu11==2.14.3 in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (2.14.3)\n", + "Requirement already satisfied: nvidia-nvtx-cu11==11.7.91 in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (11.7.91)\n", + "Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.10/dist-packages (from torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (2.0.0)\n", + "Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from nvidia-cublas-cu11==11.10.3.66->torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (67.7.2)\n", + "Requirement already satisfied: wheel in /usr/local/lib/python3.10/dist-packages (from nvidia-cublas-cu11==11.10.3.66->torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (0.41.3)\n", + "Requirement already satisfied: cmake in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (3.27.7)\n", + "Requirement already satisfied: lit in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (17.0.5)\n", + "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (2.1.3)\n", + "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch->flash-attn==2.3.0+cu117torch2.0cxx11abiFALSE) (1.3.0)\n", + "Installing collected packages: flash-attn\n", + " Attempting uninstall: flash-attn\n", + " Found existing installation: flash-attn 2.3.3\n", + " Uninstalling flash-attn-2.3.3:\n", + " Successfully uninstalled flash-attn-2.3.3\n", + "Successfully installed flash-attn-2.3.0\n" + ] + } + ], + "source": [ + "# https://github.com/oobabooga/text-generation-webui/issues/4238\n", + "!pip install https://github.com/Dao-AILab/flash-attention/releases/download/v2.3.0/flash_attn-2.3.0+cu117torch2.0cxx11abiFALSE-cp310-cp310-linux_x86_64.whl" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "3xU248nEtTxg" + }, + "source": [ + "Start the training process (fine-tuning)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "19jClLyTbumJ", + "outputId": "9b74bad8-956d-4434-953a-d5a5be229043" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "2023-11-18 10:15:30.581758: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n", + "2023-11-18 10:15:30.581829: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n", + "2023-11-18 10:15:30.581870: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n", + "2023-11-18 10:15:32.302565: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT\n", + "/usr/local/lib/python3.10/dist-packages/transformers/deepspeed.py:23: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations\n", + " warnings.warn(\n", + " dP dP dP \n", + " 88 88 88 \n", + " .d8888b. dP. .dP .d8888b. 88 .d8888b. d8888P 88 \n", + " 88' `88 `8bd8' 88' `88 88 88' `88 88 88 \n", + " 88. .88 .d88b. 88. .88 88 88. .88 88 88 \n", + " `88888P8 dP' `dP `88888P' dP `88888P' dP dP \n", + " \n", + " \n", + "\n", + "\u001b[33m[2023-11-18 10:15:36,028] [WARNING] [axolotl.validate_config:169] [PID:4655] [RANK:0] eval_batch_size != micro_batch_size. This can lead to VRAM instability.\u001b[39m\n", + "[2023-11-18 10:15:36,239] [INFO] [axolotl.normalize_config:128] [PID:4655] [RANK:0] GPU memory usage baseline: 0.000GB (+0.255GB misc)\u001b[39m\n", + "\u001b[33m[2023-11-18 10:15:36,239] [WARNING] [axolotl.scripts.check_accelerate_default_config:343] [PID:4655] [RANK:0] accelerate config file found at /root/.cache/huggingface/accelerate/default_config.yaml. This can lead to unexpected errors\u001b[39m\n", + "\u001b[33m[2023-11-18 10:15:36,239] [WARNING] [axolotl.scripts.check_user_token:355] [PID:4655] [RANK:0] Error verifying HuggingFace token. Remember to log in using `huggingface-cli login` and get your access token from https://huggingface.co/settings/tokens if you want to use gated models or datasets.\u001b[39m\n", + "[2023-11-18 10:15:36,594] [DEBUG] [axolotl.load_tokenizer:100] [PID:4655] [RANK:0] EOS: 2 / \u001b[39m\n", + "[2023-11-18 10:15:36,595] [DEBUG] [axolotl.load_tokenizer:101] [PID:4655] [RANK:0] BOS: 1 / \u001b[39m\n", + "[2023-11-18 10:15:36,595] [DEBUG] [axolotl.load_tokenizer:102] [PID:4655] [RANK:0] PAD: 0 / \u001b[39m\n", + "[2023-11-18 10:15:36,595] [DEBUG] [axolotl.load_tokenizer:103] [PID:4655] [RANK:0] UNK: 0 / \u001b[39m\n", + "[2023-11-18 10:15:36,595] [INFO] [axolotl.load_tokenized_prepared_datasets:147] [PID:4655] [RANK:0] Unable to find prepared dataset in last_run_prepared/5dca4483042d16053f3cd9eeaf5ac8af\u001b[39m\n", + "[2023-11-18 10:15:36,595] [INFO] [axolotl.load_tokenized_prepared_datasets:148] [PID:4655] [RANK:0] Loading raw datasets...\u001b[39m\n", + "[2023-11-18 10:15:36,595] [INFO] [axolotl.load_tokenized_prepared_datasets:153] [PID:4655] [RANK:0] No seed provided, using default seed of 42\u001b[39m\n", + "Map (num_proc=2): 100% 846/846 [00:00<00:00, 991.41 examples/s] \n", + "[2023-11-18 10:15:37,890] [INFO] [axolotl.load_tokenized_prepared_datasets:355] [PID:4655] [RANK:0] merging datasets\u001b[39m\n", + "[2023-11-18 10:15:37,892] [INFO] [axolotl.load_tokenized_prepared_datasets:362] [PID:4655] [RANK:0] Saving merged prepared dataset to disk... last_run_prepared/5dca4483042d16053f3cd9eeaf5ac8af\u001b[39m\n", + "Saving the dataset (1/1 shards): 100% 846/846 [00:00<00:00, 118881.71 examples/s]\n", + "Filter (num_proc=2): 100% 803/803 [00:00<00:00, 3171.26 examples/s]\n", + "Filter (num_proc=2): 100% 43/43 [00:00<00:00, 301.68 examples/s]\n", + "Map (num_proc=2): 100% 803/803 [00:00<00:00, 2783.35 examples/s]\n", + "[2023-11-18 10:15:38,745] [DEBUG] [axolotl.log:60] [PID:4655] [RANK:0] total_num_tokens: 77893\u001b[39m\n", + "[2023-11-18 10:15:38,753] [DEBUG] [axolotl.log:60] [PID:4655] [RANK:0] `total_supervised_tokens: 77893`\u001b[39m\n", + "[2023-11-18 10:15:44,265] [INFO] [axolotl.utils.samplers.multipack._len_est:178] [PID:4655] [RANK:0] packing_efficiency_estimate: 1.0 total_num_tokens per device: 77893\u001b[39m\n", + "[2023-11-18 10:15:44,265] [DEBUG] [axolotl.log:60] [PID:4655] [RANK:0] data_loader_len: 8\u001b[39m\n", + "[2023-11-18 10:15:44,265] [INFO] [axolotl.log:60] [PID:4655] [RANK:0] sample_packing_eff_est across ranks: [0.95084228515625]\u001b[39m\n", + "[2023-11-18 10:15:44,265] [DEBUG] [axolotl.log:60] [PID:4655] [RANK:0] sample_packing_eff_est: 0.96\u001b[39m\n", + "[2023-11-18 10:15:44,266] [DEBUG] [axolotl.log:60] [PID:4655] [RANK:0] total_num_steps: 32\u001b[39m\n", + "[2023-11-18 10:15:44,266] [DEBUG] [axolotl.train.log:60] [PID:4655] [RANK:0] loading tokenizer... NousResearch/Llama-2-7b-hf\u001b[39m\n", + "[2023-11-18 10:15:44,629] [DEBUG] [axolotl.load_tokenizer:100] [PID:4655] [RANK:0] EOS: 2 / \u001b[39m\n", + "[2023-11-18 10:15:44,629] [DEBUG] [axolotl.load_tokenizer:101] [PID:4655] [RANK:0] BOS: 1 / \u001b[39m\n", + "[2023-11-18 10:15:44,629] [DEBUG] [axolotl.load_tokenizer:102] [PID:4655] [RANK:0] PAD: 0 / \u001b[39m\n", + "[2023-11-18 10:15:44,629] [DEBUG] [axolotl.load_tokenizer:103] [PID:4655] [RANK:0] UNK: 0 / \u001b[39m\n", + "[2023-11-18 10:15:44,630] [DEBUG] [axolotl.train.log:60] [PID:4655] [RANK:0] loading model and peft_config...\u001b[39m\n", + "[2023-11-18 10:15:44,713] [INFO] [axolotl.load_model:201] [PID:4655] [RANK:0] patching _expand_mask\u001b[39m\n", + "Downloading (…)fetensors.index.json: 100% 26.8k/26.8k [00:00<00:00, 34.9MB/s]\n", + "Downloading shards: 0% 0/2 [00:00