ExternalVendorCode/lollms-webui
1
0
Fork 0
mirror of https://github.com/ParisNeo/lollms-webui.git synced 2024-12-22 05:37:48 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge branch 'main' into code-block-style

Browse Source
This commit is contained in:
AndzejsP 2023-05-15 12:36:12 +03:00
commit ab0b1f57ed
43 changed files with 1768 additions and 765 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
.gitignore vendored
View File

@ -160,12 +160,6 @@ databases/*
extensions/ extensions/
!extensions/.keep !extensions/.keep
# backends
backends/
!backends/gpt4all
!backends/llama_cpp
!backends/gptj
!backends/__init__.py
web/.env.build web/.env.build
web/.env.dev web/.env.dev
web/.env.development web/.env.development

2
Dockerfile
View File

@ -7,7 +7,7 @@ RUN python3 -m venv venv && . venv/bin/activate
RUN python3 -m pip install --no-cache-dir -r requirements.txt --upgrade pip RUN python3 -m pip install --no-cache-dir -r requirements.txt --upgrade pip
COPY ./app.py /srv/app.py COPY ./app.py /srv/app.py
COPY ./pyGpt4All /srv/pyGpt4All COPY ./gpt4all_api /srv/gpt4all_api
COPY ./backends /srv/backends COPY ./backends /srv/backends
COPY ./static /srv/static COPY ./static /srv/static
COPY ./templates /srv/templates COPY ./templates /srv/templates

29
README.md
View File

@ -41,7 +41,7 @@ It's worth noting that the model has recently been launched, and it's expected t
# Installation and running # Installation and running
Make sure that your CPU supports `AVX2` instruction set. Without it, this application won't run out of the box. To check your CPU features, please visit the website of your CPU manufacturer for more information and look for `Instruction set extension: AVX2`. Make sure that your CPU supports `AVX2` instruction set. Without it, this application won't run out of the box (for the pyllamacpp backend). To check your CPU features, please visit the website of your CPU manufacturer for more information and look for `Instruction set extension: AVX2`.
> **Note** > **Note**
> >
>Default model `gpt4all-lora-quantized-ggml.bin` is roughly 4GB in size. >Default model `gpt4all-lora-quantized-ggml.bin` is roughly 4GB in size.
@ -60,6 +60,7 @@ Make sure that your CPU supports `AVX2` instruction set. Without it, this applic
> **Note** > **Note**
> During installtion, it may ask you to download a model. Feel free to accept or to download your own models depending on the backends you are using. > During installtion, it may ask you to download a model. Feel free to accept or to download your own models depending on the backends you are using.
Once installed, you can run the app by using `webui.bat` or `webui.sh`. The script will check for any new updates Once installed, you can run the app by using `webui.bat` or `webui.sh`. The script will check for any new updates
[If you want to use a more advanced install procedure, please click here](docs/usage/AdvancedInstallInstructions.md) [If you want to use a more advanced install procedure, please click here](docs/usage/AdvancedInstallInstructions.md)
@ -96,9 +97,11 @@ Now you're ready to work!
# Supported backends # Supported backends
Two backends are now supported: Two backends are now supported:
1 - [The llama_cpp backend](https://github.com/nomic-ai/pygpt4all)
2 - [The GPT-j backend](https://github.com/marella/gpt4all-j) 1- [The llama_cpp backend by Abdeladim](https://github.com/abdeladim-s/pyllamacpp)
3 - Hugging face's Transformers (under construction) 2- [The GPT-j backend by Abdeladim](https://github.com/abdeladim-s/pygptj)
3- [The GPT-j backend by marella](https://github.com/marella/gpt4all-j)
4- Hugging face's Transformers (under construction)
# Supported models # Supported models
You can also refuse to download the model during the install procedure and download it manually. You can also refuse to download the model during the install procedure and download it manually.
@ -144,14 +147,18 @@ Just download the model into the `models/<backend name>` folder and start using
You can find hundreds of personalities in my personal [Personalities repository](https://github.com/ParisNeo/PyAIPersonality). This new personalities format can be used for any third party applications, it builds a simple structure and format to define personalities. This format is evolutive and new fields and assets will be added in the future like personality voice or 3d animated character with prebaked motions that should allow AI to be more alive. The format is baked to support old versions while adding new capabilities for new versions making it ideal as a personality defintition format. You can find hundreds of personalities in my personal [Personalities repository](https://github.com/ParisNeo/PyAIPersonality). This new personalities format can be used for any third party applications, it builds a simple structure and format to define personalities. This format is evolutive and new fields and assets will be added in the future like personality voice or 3d animated character with prebaked motions that should allow AI to be more alive. The format is baked to support old versions while adding new capabilities for new versions making it ideal as a personality defintition format.
## Personality install ### How to Install Personalities from the Zoo
if you are on windows you can install new personalities directly using the `add_personality.bat` code:
```bash 1. Navigate to the root directory of your repository.
add_personality.bat 2. Run either `installations/add_personality.bat` or `installations/add_personality.sh`, depending on your operating system.
``` 3. Select the desired language, category, and personality from the provided options.
4. The selected personality will be added to the list of available options.
5. Choose the current personality:
- Option 1: Use the UI by going to "Settings" and selecting "Personalities".
- Option 2: Update the configuration file `configs/default_local.yaml` with the appropriate language, category, and personality name.
Note: Ensure that you have the necessary permissions and dependencies installed before performing the above steps.
```bash
bash add_personality.sh
``` ```
Please don't forget to take time and give a Star if you like the project. This helps the visibility of the project. Please don't forget to take time and give a Star if you like the project. This helps the visibility of the project.

284
app.py
View File

@ -20,11 +20,10 @@ import argparse
import json import json
import re import re
import traceback import traceback
import threading
import sys import sys
from tqdm import tqdm from tqdm import tqdm
from pyaipersonality import AIPersonality from pyaipersonality import AIPersonality
from pyGpt4All.db import DiscussionsDB, Discussion from gpt4all_api.db import DiscussionsDB, Discussion
from flask import ( from flask import (
Flask, Flask,
Response, Response,
@ -41,9 +40,12 @@ from geventwebsocket.handler import WebSocketHandler
from gevent.pywsgi import WSGIServer from gevent.pywsgi import WSGIServer
import requests import requests
from concurrent.futures import ThreadPoolExecutor, as_completed from concurrent.futures import ThreadPoolExecutor, as_completed
import logging
log = logging.getLogger('werkzeug')
log.setLevel(logging.ERROR)
app = Flask("GPT4All-WebUI", static_url_path="/static", static_folder="static") app = Flask("GPT4All-WebUI", static_url_path="/static", static_folder="static")
socketio = SocketIO(app, cors_allowed_origins="*", async_mode='gevent', ping_timeout=30, ping_interval=15) socketio = SocketIO(app, cors_allowed_origins="*", async_mode='gevent', ping_timeout=60, ping_interval=15)
app.config['SECRET_KEY'] = 'secret!' app.config['SECRET_KEY'] = 'secret!'
# Set the logging level to WARNING or higher # Set the logging level to WARNING or higher
@ -54,19 +56,19 @@ log = logging.getLogger('werkzeug')
log.setLevel(logging.ERROR) log.setLevel(logging.ERROR)
import time import time
from pyGpt4All.config import load_config, save_config from gpt4all_api.config import load_config, save_config
from pyGpt4All.api import GPT4AllAPI from gpt4all_api.api import GPT4AllAPI
import shutil import shutil
import markdown import markdown
class Gpt4AllWebUI(GPT4AllAPI): class Gpt4AllWebUI(GPT4AllAPI):
def __init__(self, _app, _socketio, config:dict, personality:dict, config_file_path) -> None: def __init__(self, _app, _socketio, config:dict, config_file_path) -> None:
super().__init__(config, personality, config_file_path) super().__init__(config, _socketio, config_file_path)
self.app = _app self.app = _app
self.cancel_gen = False self.cancel_gen = False
self.socketio = _socketio
if "use_new_ui" in self.config: if "use_new_ui" in self.config:
if self.config["use_new_ui"]: if self.config["use_new_ui"]:
@ -109,7 +111,7 @@ class Gpt4AllWebUI(GPT4AllAPI):
self.add_endpoint("/", "", self.index, methods=["GET"]) self.add_endpoint("/", "", self.index, methods=["GET"])
self.add_endpoint("/<path:filename>", "serve_static", self.serve_static, methods=["GET"]) self.add_endpoint("/<path:filename>", "serve_static", self.serve_static, methods=["GET"])
self.add_endpoint("/personalities/<path:filename>", "serve_personalities", self.serve_personalities, methods=["GET"]) self.add_endpoint("/personalities/<path:filename>", "serve_personalities", self.serve_personalities, methods=["GET"])
self.add_endpoint("/export_discussion", "export_discussion", self.export_discussion, methods=["GET"]) self.add_endpoint("/export_discussion", "export_discussion", self.export_discussion, methods=["GET"])
self.add_endpoint("/export", "export", self.export, methods=["GET"]) self.add_endpoint("/export", "export", self.export, methods=["GET"])
@ -199,121 +201,6 @@ class Gpt4AllWebUI(GPT4AllAPI):
"/get_current_personality", "get_current_personality", self.get_current_personality, methods=["GET"] "/get_current_personality", "get_current_personality", self.get_current_personality, methods=["GET"]
) )
# =========================================================================================
# Socket IO stuff
# =========================================================================================
@socketio.on('connect')
def connect():
print('Client connected')
@socketio.on('disconnect')
def disconnect():
print('Client disconnected')
@socketio.on('install_model')
def install_model(data):
model_path = data["path"]
progress = 0
installation_dir = Path(f'./models/{self.config["backend"]}/')
filename = Path(model_path).name
installation_path = installation_dir / filename
print("Model install requested")
print(f"Model path : {model_path}")
if installation_path.exists():
print("Error: Model already exists")
data.installing = False
socketio.emit('install_progress',{'status': 'failed', 'error': 'model already exists'})
socketio.emit('install_progress',{'status': 'progress', 'progress': progress})
response = requests.get(model_path, stream=True)
file_size = int(response.headers.get('Content-Length'))
downloaded_size = 0
CHUNK_SIZE = 8192
def download_chunk(url, start_byte, end_byte, fileobj):
headers = {'Range': f'bytes={start_byte}-{end_byte}'}
response = requests.get(url, headers=headers, stream=True)
downloaded_bytes = 0
for chunk in response.iter_content(chunk_size=CHUNK_SIZE):
if chunk:
fileobj.seek(start_byte)
fileobj.write(chunk)
downloaded_bytes += len(chunk)
start_byte += len(chunk)
return downloaded_bytes
def download_file(url, file_path, num_threads=4):
response = requests.head(url)
file_size = int(response.headers.get('Content-Length'))
chunk_size = file_size // num_threads
progress = 0
with open(file_path, 'wb') as fileobj:
with tqdm(total=file_size, unit='B', unit_scale=True, unit_divisor=1024) as pbar:
with ThreadPoolExecutor(max_workers=num_threads) as executor:
futures = []
for i in range(num_threads):
start_byte = i * chunk_size
end_byte = start_byte + chunk_size - 1 if i < num_threads - 1 else file_size - 1
futures.append(executor.submit(download_chunk, url, start_byte, end_byte, fileobj))
for future in tqdm(as_completed(futures), total=num_threads):
downloaded_bytes = future.result()
progress += downloaded_bytes
pbar.update(downloaded_bytes)
socketio.emit('install_progress', {'status': 'progress', 'progress': progress})
# Usage example
download_file(model_path, installation_path, num_threads=4)
socketio.emit('install_progress',{'status': 'succeeded', 'error': ''})
@socketio.on('uninstall_model')
def uninstall_model(data):
model_path = data['path']
installation_dir = Path(f'./models/{self.config["backend"]}/')
filename = Path(model_path).name
installation_path = installation_dir / filename
if not installation_path.exists():
socketio.emit('install_progress',{'status': 'failed', 'error': ''})
installation_path.unlink()
socketio.emit('install_progress',{'status': 'succeeded', 'error': ''})
@socketio.on('generate_msg')
def generate_msg(data):
if self.current_discussion is None:
if self.db.does_last_discussion_have_messages():
self.current_discussion = self.db.create_discussion()
else:
self.current_discussion = self.db.load_last_discussion()
message = data["prompt"]
message_id = self.current_discussion.add_message(
"user", message, parent=self.message_id
)
self.current_user_message_id = message_id
tpe = threading.Thread(target=self.start_message_generation, args=(message, message_id))
tpe.start()
@socketio.on('generate_msg_from')
def handle_connection(data):
message_id = int(data['id'])
message = data["prompt"]
self.current_user_message_id = message_id
tpe = threading.Thread(target=self.start_message_generation, args=(message, message_id))
tpe.start()
def save_settings(self): def save_settings(self):
@ -405,9 +292,9 @@ class Gpt4AllWebUI(GPT4AllAPI):
elif setting_name== "model": elif setting_name== "model":
self.config["model"]=data['setting_value'] self.config["model"]=data['setting_value']
print("New model selected") print("update_settings : New model selected")
# Build chatbot # Build chatbot
self.chatbot_bindings = self.create_chatbot() self.process.set_config(self.config)
elif setting_name== "backend": elif setting_name== "backend":
print("New backend selected") print("New backend selected")
@ -415,13 +302,13 @@ class Gpt4AllWebUI(GPT4AllAPI):
print("New backend selected") print("New backend selected")
self.config["backend"]=data['setting_value'] self.config["backend"]=data['setting_value']
backend_ =self.load_backend(self.BACKENDS_LIST[self.config["backend"]]) backend_ =self.process.rebuild_backend(self.config)
models = backend_.list_models(self.config) models = backend_.list_models(self.config)
if len(models)>0: if len(models)>0:
self.backend = backend_ self.backend = backend_
self.config['model'] = models[0] self.config['model'] = models[0]
# Build chatbot # Build chatbot
self.chatbot_bindings = self.create_chatbot() self.process.set_config(self.config)
if self.config["debug"]: if self.config["debug"]:
print(f"Configuration {data['setting_name']} set to {data['setting_value']}") print(f"Configuration {data['setting_name']} set to {data['setting_value']}")
return jsonify({'setting_name': data['setting_name'], "status":True}) return jsonify({'setting_name': data['setting_name'], "status":True})
@ -441,6 +328,9 @@ class Gpt4AllWebUI(GPT4AllAPI):
if self.config["debug"]: if self.config["debug"]:
print(f"Configuration {data['setting_name']} set to {data['setting_value']}") print(f"Configuration {data['setting_name']} set to {data['setting_value']}")
print("Configuration updated")
self.process.set_config(self.config)
# Tell that the setting was changed # Tell that the setting was changed
return jsonify({'setting_name': data['setting_name'], "status":True}) return jsonify({'setting_name': data['setting_name'], "status":True})
@ -549,69 +439,15 @@ class Gpt4AllWebUI(GPT4AllAPI):
return jsonify({"discussion_text":self.get_discussion_to()}) return jsonify({"discussion_text":self.get_discussion_to()})
def start_message_generation(self, message, message_id):
bot_says = ""
# send the message to the bot
print(f"Received message : {message}")
if self.current_discussion:
# First we need to send the new message ID to the client
self.current_ai_message_id = self.current_discussion.add_message(
self.personality.name, "", parent = self.current_user_message_id
) # first the content is empty, but we'll fill it at the end
socketio.emit('infos',
{
"type": "input_message_infos",
"bot": self.personality.name,
"user": self.personality.user_name,
"message":message,#markdown.markdown(message),
"user_message_id": self.current_user_message_id,
"ai_message_id": self.current_ai_message_id,
}
)
# prepare query and reception
self.discussion_messages = self.prepare_query(message_id)
self.prepare_reception()
self.generating = True
# app.config['executor'] = ThreadPoolExecutor(max_workers=1)
# app.config['executor'].submit(self.generate_message)
print("## Generating message ##")
self.generate_message()
print()
print("## Done ##")
print()
# Send final message
self.socketio.emit('final', {
'data': self.bot_says,
'ai_message_id':self.current_ai_message_id,
'parent':self.current_user_message_id, 'discussion_id':self.current_discussion.discussion_id
}
)
self.current_discussion.update_message(self.current_ai_message_id, self.bot_says)
self.full_message_list.append(self.bot_says)
self.cancel_gen = False
return bot_says
else:
#No discussion available
print("No discussion selected!!!")
print("## Done ##")
print()
self.cancel_gen = False
return ""
def get_generation_status(self): def get_generation_status(self):
return jsonify({"status":self.generating}) return jsonify({"status":self.process.is_generating.value==1})
def stop_gen(self): def stop_gen(self):
self.cancel_gen = True self.cancel_gen = True
print("Stop generation received") print("Stop generation received")
return jsonify({"status": "ok"}) return jsonify({"status": "ok"})
def rename(self): def rename(self):
data = request.get_json() data = request.get_json()
@ -680,9 +516,6 @@ class Gpt4AllWebUI(GPT4AllAPI):
def new_discussion(self): def new_discussion(self):
title = request.args.get("title") title = request.args.get("title")
timestamp = self.create_new_discussion(title) timestamp = self.create_new_discussion(title)
# app.config['executor'] = ThreadPoolExecutor(max_workers=1)
# app.config['executor'].submit(self.create_chatbot)
# target=self.create_chatbot()
# Return a success response # Return a success response
return json.dumps({"id": self.current_discussion.discussion_id, "time": timestamp, "welcome_message":self.personality.welcome_message, "sender":self.personality.name}) return json.dumps({"id": self.current_discussion.discussion_id, "time": timestamp, "welcome_message":self.personality.welcome_message, "sender":self.personality.name})
@ -694,13 +527,13 @@ class Gpt4AllWebUI(GPT4AllAPI):
print("New backend selected") print("New backend selected")
self.config['backend'] = backend self.config['backend'] = backend
backend_ =self.load_backend(self.BACKENDS_LIST[self.config["backend"]]) backend_ =self.process.load_backend(Path("backends")/config["backend"])
models = backend_.list_models(self.config) models = backend_.list_models(self.config)
if len(models)>0: if len(models)>0:
self.backend = backend_ self.backend = backend_
self.config['model'] = models[0] self.config['model'] = models[0]
# Build chatbot # Build chatbot
self.chatbot_bindings = self.create_chatbot() self.process.set_config(self.config)
return jsonify({"status": "ok"}) return jsonify({"status": "ok"})
else: else:
return jsonify({"status": "no_models_found"}) return jsonify({"status": "no_models_found"})
@ -711,10 +544,10 @@ class Gpt4AllWebUI(GPT4AllAPI):
data = request.get_json() data = request.get_json()
model = str(data["model"]) model = str(data["model"])
if self.config['model']!= model: if self.config['model']!= model:
print("New model selected") print("set_model: New model selected")
self.config['model'] = model self.config['model'] = model
# Build chatbot # Build chatbot
self.chatbot_bindings = self.create_chatbot() self.process.set_config(self.config)
return jsonify({"status": "ok"}) return jsonify({"status": "ok"})
return jsonify({"status": "error"}) return jsonify({"status": "error"})
@ -728,11 +561,11 @@ class Gpt4AllWebUI(GPT4AllAPI):
personality = str(data["personality"]) personality = str(data["personality"])
if self.config['backend']!=backend or self.config['model'] != model: if self.config['backend']!=backend or self.config['model'] != model:
print("New model selected") print("update_model_params: New model selected")
self.config['backend'] = backend self.config['backend'] = backend
self.config['model'] = model self.config['model'] = model
self.create_chatbot() self.process.set_config(self.config)
self.config['personality_language'] = personality_language self.config['personality_language'] = personality_language
self.config['personality_category'] = personality_category self.config['personality_category'] = personality_category
@ -740,7 +573,6 @@ class Gpt4AllWebUI(GPT4AllAPI):
personality_fn = f"personalities/{self.config['personality_language']}/{self.config['personality_category']}/{self.config['personality']}" personality_fn = f"personalities/{self.config['personality_language']}/{self.config['personality_category']}/{self.config['personality']}"
print(f"Loading personality : {personality_fn}") print(f"Loading personality : {personality_fn}")
self.personality = AIPersonality(personality_fn)
self.config['n_predict'] = int(data["nPredict"]) self.config['n_predict'] = int(data["nPredict"])
self.config['seed'] = int(data["seed"]) self.config['seed'] = int(data["seed"])
@ -755,6 +587,10 @@ class Gpt4AllWebUI(GPT4AllAPI):
self.config['repeat_last_n'] = int(data["repeatLastN"]) self.config['repeat_last_n'] = int(data["repeatLastN"])
save_config(self.config, self.config_file_path) save_config(self.config, self.config_file_path)
self.process.set_config(self.config)
# Fixed missing argument
self.backend = self.process.rebuild_backend(self.config)
print("==============================================") print("==============================================")
print("Parameters changed to:") print("Parameters changed to:")
@ -778,24 +614,36 @@ class Gpt4AllWebUI(GPT4AllAPI):
def get_available_models(self): def get_available_models(self):
response = requests.get(f'https://gpt4all.io/models/models.json') """Get the available models
model_list = response.json()
Returns:
_type_: _description_
"""
model_list = self.backend.get_available_models()
models = [] models = []
for model in model_list: for model in model_list:
filename = model['filename'] try:
filesize = model['filesize'] filename = model['filename']
path = f'https://gpt4all.io/models/{filename}' server = model['server']
local_path = Path(f'./models/{self.config["backend"]}/{filename}') filesize = model['filesize']
is_installed = local_path.exists() if server.endswith("/"):
models.append({ path = f'{server}{filename}'
'title': model['filename'], else:
'icon': '/icons/default.png', # Replace with the path to the model icon path = f'{server}/{filename}'
'description': model['description'], local_path = Path(f'./models/{self.config["backend"]}/{filename}')
'isInstalled': is_installed, is_installed = local_path.exists()
'path': path, models.append({
'filesize': filesize, 'title': model['filename'],
}) 'icon': '/icons/default.png', # Replace with the path to the model icon
'description': model['description'],
'isInstalled': is_installed,
'path': path,
'filesize': filesize,
})
except:
print(f"Problem with model : {model}")
return jsonify(models) return jsonify(models)
@ -915,17 +763,9 @@ if __name__ == "__main__":
if arg_value is not None: if arg_value is not None:
config[arg_name] = arg_value config[arg_name] = arg_value
try:
personality_path = f"personalities/{config['personality_language']}/{config['personality_category']}/{config['personality']}"
personality = AIPersonality(personality_path)
except Exception as ex:
print("Personality file not found. Please verify that the personality you have selected exists or select another personality. Some updates may lead to change in personality name or category, so check the personality selection in settings to be sure.")
if config["debug"]:
print(ex)
personality = AIPersonality()
# executor = ThreadPoolExecutor(max_workers=1) # executor = ThreadPoolExecutor(max_workers=1)
# app.config['executor'] = executor # app.config['executor'] = executor
bot = Gpt4AllWebUI(app, socketio, config, personality, config_file_path) bot = Gpt4AllWebUI(app, socketio, config, config_file_path)
# chong Define custom WebSocketHandler with error handling # chong Define custom WebSocketHandler with error handling
class CustomWebSocketHandler(WebSocketHandler): class CustomWebSocketHandler(WebSocketHandler):
@ -948,7 +788,3 @@ if __name__ == "__main__":
http_server = WSGIServer((config["host"], config["port"]), app, handler_class=WebSocketHandler) http_server = WSGIServer((config["host"], config["port"]), app, handler_class=WebSocketHandler)
http_server.serve_forever() http_server.serve_forever()
#if config["debug"]:
# app.run(debug=True, host=config["host"], port=config["port"])
#else:
# app.run(host=config["host"], port=config["port"])

76
backends/gpt_4all/__init__.py Normal file
View File

@ -0,0 +1,76 @@
######
# Project : GPT4ALL-UI
# File : backend.py
# Author : ParisNeo with the help of the community
# Supported by Nomic-AI
# Licence : Apache 2.0
# Description :
# This is an interface class for GPT4All-ui backends.
######
from pathlib import Path
from typing import Callable
from gpt4all import GPT4All
from gpt4all_api.backend import GPTBackend
import yaml
__author__ = "parisneo"
__github__ = "https://github.com/nomic-ai/gpt4all-ui"
__copyright__ = "Copyright 2023, "
__license__ = "Apache 2.0"
backend_name = "GPT4ALL"
class GPT4ALL(GPTBackend):
file_extension='*.bin'
def __init__(self, config:dict) -> None:
"""Builds a GPT4ALL backend
Args:
config (dict): The configuration file
"""
super().__init__(config, False)
self.model = GPT4All.get_model_from_name(self.config['model'])
self.model.load_model(
model_path=f"./models/gpt_4all/{self.config['model']}",
)
def generate(self,
prompt:str,
n_predict: int = 128,
new_text_callback: Callable[[str], None] = bool,
verbose: bool = False,
**gpt_params ):
"""Generates text out of a prompt
Args:
prompt (str): The prompt to use for generation
n_predict (int, optional): Number of tokens to prodict. Defaults to 128.
new_text_callback (Callable[[str], None], optional): A callback function that is called everytime a new text element is generated. Defaults to None.
verbose (bool, optional): If true, the code will spit many informations about the generation process. Defaults to False.
"""
try:
for tok in self.model.generate(prompt,
n_predict=n_predict,
temp=self.config['temperature'],
top_k=self.config['top_k'],
top_p=self.config['top_p'],
repeat_penalty=self.config['repeat_penalty'],
repeat_last_n = self.config['repeat_last_n'],
# n_threads=self.config['n_threads'],
):
if not new_text_callback(tok):
return
except Exception as ex:
print(ex)
@staticmethod
def get_available_models():
# Create the file path relative to the child class's directory
backend_path = Path(__file__).parent
file_path = backend_path/"models.yaml"
with open(file_path, 'r') as file:
yaml_data = yaml.safe_load(file)
return yaml_data

27
backends/gpt_4all/json2yaml.py Normal file
View File

@ -0,0 +1,27 @@
import json
import yaml
from pathlib import Path
import argparse
def json_to_yaml(json_file):
# Read JSON file
with open(json_file, 'r') as file:
json_data = json.load(file)
# Create YAML file path
yaml_file = Path(json_file).with_suffix('.yaml')
# Convert JSON to YAML
with open(yaml_file, 'w') as file:
yaml.dump(json_data, file)
print(f"Conversion complete. YAML file saved as: {yaml_file}")
if __name__ == "__main__":
# Parse command-line arguments
parser = argparse.ArgumentParser(description='Convert JSON file to YAML.')
parser.add_argument('json_file', help='Path to the JSON file')
args = parser.parse_args()
# Convert JSON to YAML
json_to_yaml(args.json_file)

13
zoos/gpt4all_models_zoo.json → backends/gpt_4all/models.json
View File

@ -2,6 +2,7 @@
{ {
"md5sum": "81a09a0ddf89690372fc296ff7f625af", "md5sum": "81a09a0ddf89690372fc296ff7f625af",
"filename": "ggml-gpt4all-j-v1.3-groovy.bin", "filename": "ggml-gpt4all-j-v1.3-groovy.bin",
"server":"https://gpt4all.io/models/",
"filesize": "3785248281", "filesize": "3785248281",
"isDefault": "true", "isDefault": "true",
"bestGPTJ": "true", "bestGPTJ": "true",
@ -10,6 +11,7 @@
{ {
"md5sum": "91f886b68fbce697e9a3cd501951e455", "md5sum": "91f886b68fbce697e9a3cd501951e455",
"filename": "ggml-gpt4all-l13b-snoozy.bin", "filename": "ggml-gpt4all-l13b-snoozy.bin",
"server":"https://gpt4all.io/models/",
"filesize": "8136770688", "filesize": "8136770688",
"bestLlama": "true", "bestLlama": "true",
"description": "Current best non-commercially licensable model based on Llama 13b and trained by Nomic AI on the latest curated GPT4All dataset." "description": "Current best non-commercially licensable model based on Llama 13b and trained by Nomic AI on the latest curated GPT4All dataset."
@ -17,6 +19,7 @@
{ {
"md5sum": "756249d3d6abe23bde3b1ae272628640", "md5sum": "756249d3d6abe23bde3b1ae272628640",
"filename": "ggml-mpt-7b-chat.bin", "filename": "ggml-mpt-7b-chat.bin",
"server":"https://gpt4all.io/models/",
"filesize": "4854401050", "filesize": "4854401050",
"isDefault": "true", "isDefault": "true",
"bestMPT": "true", "bestMPT": "true",
@ -26,48 +29,56 @@
{ {
"md5sum": "879344aaa9d62fdccbda0be7a09e7976", "md5sum": "879344aaa9d62fdccbda0be7a09e7976",
"filename": "ggml-gpt4all-j-v1.2-jazzy.bin", "filename": "ggml-gpt4all-j-v1.2-jazzy.bin",
"server":"https://gpt4all.io/models/",
"filesize": "3785248281", "filesize": "3785248281",
"description": "A commercially licensable model based on GPT-J and trained by Nomic AI on the v2 GPT4All dataset." "description": "A commercially licensable model based on GPT-J and trained by Nomic AI on the v2 GPT4All dataset."
}, },
{ {
"md5sum": "61d48a82cb188cceb14ebb8082bfec37", "md5sum": "61d48a82cb188cceb14ebb8082bfec37",
"filename": "ggml-gpt4all-j-v1.1-breezy.bin", "filename": "ggml-gpt4all-j-v1.1-breezy.bin",
"server":"https://gpt4all.io/models/",
"filesize": "3785248281", "filesize": "3785248281",
"description": "A commercially licensable model based on GPT-J and trained by Nomic AI on the v1 GPT4All dataset." "description": "A commercially licensable model based on GPT-J and trained by Nomic AI on the v1 GPT4All dataset."
}, },
{ {
"md5sum": "5b5a3f9b858d33b29b52b89692415595", "md5sum": "5b5a3f9b858d33b29b52b89692415595",
"filename": "ggml-gpt4all-j.bin", "filename": "ggml-gpt4all-j.bin",
"server":"https://gpt4all.io/models/",
"filesize": "3785248281", "filesize": "3785248281",
"description": "A commercially licensable model based on GPT-J and trained by Nomic AI on the v0 GPT4All dataset." "description": "A commercially licensable model based on GPT-J and trained by Nomic AI on the v0 GPT4All dataset."
}, },
{ {
"md5sum": "29119f8fa11712704c6b22ac5ab792ea", "md5sum": "29119f8fa11712704c6b22ac5ab792ea",
"filename": "ggml-vicuna-7b-1.1-q4_2.bin", "filename": "ggml-vicuna-7b-1.1-q4_2.bin",
"server":"https://gpt4all.io/models/",
"filesize": "4212859520", "filesize": "4212859520",
"description": "A non-commercially licensable model based on Llama 7b and trained by teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego." "description": "A non-commercially licensable model based on Llama 7b and trained by teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego."
}, },
{ {
"md5sum": "95999b7b0699e2070af63bf5d34101a8", "md5sum": "95999b7b0699e2070af63bf5d34101a8",
"filename": "ggml-vicuna-13b-1.1-q4_2.bin", "filename": "ggml-vicuna-13b-1.1-q4_2.bin",
"server":"https://gpt4all.io/models/",
"filesize": "8136770688", "filesize": "8136770688",
"description": "A non-commercially licensable model based on Llama 13b and trained by teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego." "description": "A non-commercially licensable model based on Llama 13b and trained by teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego."
}, },
{ {
"md5sum": "99e6d129745a3f1fb1121abed747b05a", "md5sum": "99e6d129745a3f1fb1121abed747b05a",
"filename": "ggml-wizardLM-7B.q4_2.bin", "filename": "ggml-wizardLM-7B.q4_2.bin",
"server":"https://gpt4all.io/models/",
"filesize": "4212864640", "filesize": "4212864640",
"description": "A non-commercially licensable model based on Llama 7b and trained by Microsoft and Peking University." "description": "A non-commercially licensable model based on Llama 7b and trained by Microsoft and Peking University."
}, },
{ {
"md5sum": "6cb4ee297537c9133bddab9692879de0", "md5sum": "6cb4ee297537c9133bddab9692879de0",
"filename": "ggml-stable-vicuna-13B.q4_2.bin", "filename": "ggml-stable-vicuna-13B.q4_2.bin",
"server":"https://gpt4all.io/models/",
"filesize": "8136777088", "filesize": "8136777088",
"description": "A non-commercially licensable model based on Llama 13b and RLHF trained by Stable AI." "description": "A non-commercially licensable model based on Llama 13b and RLHF trained by Stable AI."
}, },
{ {
"md5sum": "120c32a51d020066288df045ef5d52b9", "md5sum": "120c32a51d020066288df045ef5d52b9",
"filename": "ggml-mpt-7b-base.bin", "filename": "ggml-mpt-7b-base.bin",
"server":"https://gpt4all.io/models/",
"filesize": "4854401028", "filesize": "4854401028",
"requires": "2.4.1", "requires": "2.4.1",
"description": "A commercially licensable model base pre-trained by Mosaic ML." "description": "A commercially licensable model base pre-trained by Mosaic ML."
@ -75,12 +86,14 @@
{ {
"md5sum": "d5eafd5b0bd0d615cfd5fd763f642dfe", "md5sum": "d5eafd5b0bd0d615cfd5fd763f642dfe",
"filename": "ggml-nous-gpt4-vicuna-13b.bin", "filename": "ggml-nous-gpt4-vicuna-13b.bin",
"server":"https://gpt4all.io/models/",
"filesize": "8136777088", "filesize": "8136777088",
"description": "A non-commercially licensable model based on Vicuna 13b, fine-tuned on ~180,000 instructions, trained by Nous Research." "description": "A non-commercially licensable model based on Vicuna 13b, fine-tuned on ~180,000 instructions, trained by Nous Research."
}, },
{ {
"md5sum": "1cfa4958f489f0a0d1ffdf6b37322809", "md5sum": "1cfa4958f489f0a0d1ffdf6b37322809",
"filename": "ggml-mpt-7b-instruct.bin", "filename": "ggml-mpt-7b-instruct.bin",
"server":"https://gpt4all.io/models/",
"filesize": "4854401028", "filesize": "4854401028",
"requires": "2.4.1", "requires": "2.4.1",
"description": "A commericially licensable instruct model based on MPT and trained by Mosaic ML." "description": "A commericially licensable instruct model based on MPT and trained by Mosaic ML."

85
backends/gpt_4all/models.yaml Normal file
View File

@ -0,0 +1,85 @@
- bestGPTJ: 'true'
description: Current best commercially licensable model based on GPT-J and trained
by Nomic AI on the latest curated GPT4All dataset.
filename: ggml-gpt4all-j-v1.3-groovy.bin
filesize: '3785248281'
isDefault: 'true'
md5sum: 81a09a0ddf89690372fc296ff7f625af
server: https://gpt4all.io/models/
- bestLlama: 'true'
description: Current best non-commercially licensable model based on Llama 13b and
trained by Nomic AI on the latest curated GPT4All dataset.
filename: ggml-gpt4all-l13b-snoozy.bin
filesize: '8136770688'
md5sum: 91f886b68fbce697e9a3cd501951e455
server: https://gpt4all.io/models/
- bestMPT: 'true'
description: Current best non-commercially licensable chat model based on MPT and
trained by Mosaic ML.
filename: ggml-mpt-7b-chat.bin
filesize: '4854401050'
isDefault: 'true'
md5sum: 756249d3d6abe23bde3b1ae272628640
requires: 2.4.1
server: https://gpt4all.io/models/
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v2 GPT4All dataset.
filename: ggml-gpt4all-j-v1.2-jazzy.bin
filesize: '3785248281'
md5sum: 879344aaa9d62fdccbda0be7a09e7976
server: https://gpt4all.io/models/
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v1 GPT4All dataset.
filename: ggml-gpt4all-j-v1.1-breezy.bin
filesize: '3785248281'
md5sum: 61d48a82cb188cceb14ebb8082bfec37
server: https://gpt4all.io/models/
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v0 GPT4All dataset.
filename: ggml-gpt4all-j.bin
filesize: '3785248281'
md5sum: 5b5a3f9b858d33b29b52b89692415595
server: https://gpt4all.io/models/
- description: A non-commercially licensable model based on Llama 7b and trained by
teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego.
filename: ggml-vicuna-7b-1.1-q4_2.bin
filesize: '4212859520'
md5sum: 29119f8fa11712704c6b22ac5ab792ea
server: https://gpt4all.io/models/
- description: A non-commercially licensable model based on Llama 13b and trained
by teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego.
filename: ggml-vicuna-13b-1.1-q4_2.bin
filesize: '8136770688'
md5sum: 95999b7b0699e2070af63bf5d34101a8
server: https://gpt4all.io/models/
- description: A non-commercially licensable model based on Llama 7b and trained by
Microsoft and Peking University.
filename: ggml-wizardLM-7B.q4_2.bin
filesize: '4212864640'
md5sum: 99e6d129745a3f1fb1121abed747b05a
server: https://gpt4all.io/models/
- description: A non-commercially licensable model based on Llama 13b and RLHF trained
by Stable AI.
filename: ggml-stable-vicuna-13B.q4_2.bin
filesize: '8136777088'
md5sum: 6cb4ee297537c9133bddab9692879de0
server: https://gpt4all.io/models/
- description: A commercially licensable model base pre-trained by Mosaic ML.
filename: ggml-mpt-7b-base.bin
filesize: '4854401028'
md5sum: 120c32a51d020066288df045ef5d52b9
requires: 2.4.1
server: https://gpt4all.io/models/
- description: A non-commercially licensable model based on Vicuna 13b, fine-tuned
on ~180,000 instructions, trained by Nous Research.
filename: ggml-nous-gpt4-vicuna-13b.bin
filesize: '8136777088'
md5sum: d5eafd5b0bd0d615cfd5fd763f642dfe
server: https://gpt4all.io/models/
- description: A commericially licensable instruct model based on MPT and trained
by Mosaic ML.
filename: ggml-mpt-7b-instruct.bin
filesize: '4854401028'
md5sum: 1cfa4958f489f0a0d1ffdf6b37322809
requires: 2.4.1
server: https://gpt4all.io/models/

24
backends/gpt4all/__init__.py → backends/gpt_j_a/__init__.py
View File

@ -2,6 +2,7 @@
# Project : GPT4ALL-UI # Project : GPT4ALL-UI
# File : backend.py # File : backend.py
# Author : ParisNeo with the help of the community # Author : ParisNeo with the help of the community
# Underlying backend : Abdeladim's pygptj backend
# Supported by Nomic-AI # Supported by Nomic-AI
# Licence : Apache 2.0 # Licence : Apache 2.0
# Description : # Description :
@ -9,20 +10,20 @@
###### ######
from pathlib import Path from pathlib import Path
from typing import Callable from typing import Callable
from pygpt4all import GPT4All as Model from pygptj.model import Model
from pyGpt4All.backend import GPTBackend from gpt4all_api.backend import GPTBackend
__author__ = "parisneo" __author__ = "parisneo"
__github__ = "https://github.com/nomic-ai/gpt4all-ui" __github__ = "https://github.com/nomic-ai/gpt4all-ui"
__copyright__ = "Copyright 2023, " __copyright__ = "Copyright 2023, "
__license__ = "Apache 2.0" __license__ = "Apache 2.0"
backend_name = "GPT4ALL" backend_name = "GptJ"
class GPT4ALL(GPTBackend): class GptJ(GPTBackend):
file_extension='*.bin' file_extension='*.bin'
def __init__(self, config:dict) -> None: def __init__(self, config:dict) -> None:
"""Builds a GPT4ALL backend """Builds a LLAMACPP backend
Args: Args:
config (dict): The configuration file config (dict): The configuration file
@ -30,15 +31,10 @@ class GPT4ALL(GPTBackend):
super().__init__(config, False) super().__init__(config, False)
self.model = Model( self.model = Model(
model_path=f"./models/gpt4all/{self.config['model']}", model_path=f"./models/gpt_j/{self.config['model']}",
prompt_context="", prompt_prefix="", prompt_suffix="", prompt_context="", prompt_prefix="", prompt_suffix=""
n_ctx=self.config['ctx_size'],
seed=self.config['seed'],
) )
def stop_generation(self):
self.model._grab_text_callback()
def generate(self, def generate(self,
prompt:str, prompt:str,
n_predict: int = 128, n_predict: int = 128,
@ -60,8 +56,8 @@ class GPT4ALL(GPTBackend):
temp=self.config['temperature'], temp=self.config['temperature'],
top_k=self.config['top_k'], top_k=self.config['top_k'],
top_p=self.config['top_p'], top_p=self.config['top_p'],
repeat_penalty=self.config['repeat_penalty'], #repeat_penalty=self.config['repeat_penalty'],
repeat_last_n = self.config['repeat_last_n'], #repeat_last_n = self.config['repeat_last_n'],
n_threads=self.config['n_threads'], n_threads=self.config['n_threads'],
): ):
if not new_text_callback(tok): if not new_text_callback(tok):

80
backends/gpt_j_m/__init__.py Normal file
View File

@ -0,0 +1,80 @@
######
# Project : GPT4ALL-UI
# File : backend.py
# Author : ParisNeo with the help of the community
# Underlying backend : Abdeladim's pygptj backend
# Supported by Nomic-AI
# Licence : Apache 2.0
# Description :
# This is an interface class for GPT4All-ui backends.
######
from pathlib import Path
from typing import Callable
from gpt4allj import Model
from gpt4all_api.backend import GPTBackend
import yaml
__author__ = "parisneo"
__github__ = "https://github.com/nomic-ai/gpt4all-ui"
__copyright__ = "Copyright 2023, "
__license__ = "Apache 2.0"
backend_name = "GPTJ"
class GPTJ(GPTBackend):
file_extension='*.bin'
def __init__(self, config:dict) -> None:
"""Builds a LLAMACPP backend
Args:
config (dict): The configuration file
"""
super().__init__(config, False)
self.model = Model(
model=f"./models/llama_cpp/{self.config['model']}", avx2 = self.config["use_avx2"]
)
def generate(self,
prompt:str,
n_predict: int = 128,
new_text_callback: Callable[[str], None] = bool,
verbose: bool = False,
**gpt_params ):
"""Generates text out of a prompt
Args:
prompt (str): The prompt to use for generation
n_predict (int, optional): Number of tokens to prodict. Defaults to 128.
new_text_callback (Callable[[str], None], optional): A callback function that is called everytime a new text element is generated. Defaults to None.
verbose (bool, optional): If true, the code will spit many informations about the generation process. Defaults to False.
"""
try:
self.model.reset()
for tok in self.model.generate(
prompt,
seed=self.config['seed'],
n_threads=self.config['n_threads'],
n_predict=n_predict,
top_k=self.config['top_k'],
top_p=self.config['top_p'],
temp=self.config['temperature'],
repeat_penalty=self.config['repeat_penalty'],
repeat_last_n=self.config['repeat_last_n'],
n_batch=8,
reset=True,
):
if not new_text_callback(tok):
return
except Exception as ex:
print(ex)
@staticmethod
def get_available_models():
# Create the file path relative to the child class's directory
backend_path = Path(__file__).parent
file_path = backend_path/"models.yaml"
with open(file_path, 'r') as file:
yaml_data = yaml.safe_load(file)
return yaml_data

72
backends/gpt_j_m/models.yaml Normal file
View File

@ -0,0 +1,72 @@
- bestGPTJ: 'true'
description: Current best commercially licensable model based on GPT-J and trained
by Nomic AI on the latest curated GPT4All dataset.
filename: ggml-gpt4all-j-v1.3-groovy.bin
filesize: '3785248281'
isDefault: 'true'
md5sum: 81a09a0ddf89690372fc296ff7f625af
- bestLlama: 'true'
description: Current best non-commercially licensable model based on Llama 13b and
trained by Nomic AI on the latest curated GPT4All dataset.
filename: ggml-gpt4all-l13b-snoozy.bin
filesize: '8136770688'
md5sum: 91f886b68fbce697e9a3cd501951e455
- bestMPT: 'true'
description: Current best non-commercially licensable chat model based on MPT and
trained by Mosaic ML.
filename: ggml-mpt-7b-chat.bin
filesize: '4854401050'
isDefault: 'true'
md5sum: 756249d3d6abe23bde3b1ae272628640
requires: 2.4.1
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v2 GPT4All dataset.
filename: ggml-gpt4all-j-v1.2-jazzy.bin
filesize: '3785248281'
md5sum: 879344aaa9d62fdccbda0be7a09e7976
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v1 GPT4All dataset.
filename: ggml-gpt4all-j-v1.1-breezy.bin
filesize: '3785248281'
md5sum: 61d48a82cb188cceb14ebb8082bfec37
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v0 GPT4All dataset.
filename: ggml-gpt4all-j.bin
filesize: '3785248281'
md5sum: 5b5a3f9b858d33b29b52b89692415595
- description: A non-commercially licensable model based on Llama 7b and trained by
teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego.
filename: ggml-vicuna-7b-1.1-q4_2.bin
filesize: '4212859520'
md5sum: 29119f8fa11712704c6b22ac5ab792ea
- description: A non-commercially licensable model based on Llama 13b and trained
by teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego.
filename: ggml-vicuna-13b-1.1-q4_2.bin
filesize: '8136770688'
md5sum: 95999b7b0699e2070af63bf5d34101a8
- description: A non-commercially licensable model based on Llama 7b and trained by
Microsoft and Peking University.
filename: ggml-wizardLM-7B.q4_2.bin
filesize: '4212864640'
md5sum: 99e6d129745a3f1fb1121abed747b05a
- description: A non-commercially licensable model based on Llama 13b and RLHF trained
by Stable AI.
filename: ggml-stable-vicuna-13B.q4_2.bin
filesize: '8136777088'
md5sum: 6cb4ee297537c9133bddab9692879de0
- description: A commercially licensable model base pre-trained by Mosaic ML.
filename: ggml-mpt-7b-base.bin
filesize: '4854401028'
md5sum: 120c32a51d020066288df045ef5d52b9
requires: 2.4.1
- description: A non-commercially licensable model based on Vicuna 13b, fine-tuned
on ~180,000 instructions, trained by Nous Research.
filename: ggml-nous-gpt4-vicuna-13b.bin
filesize: '8136777088'
md5sum: d5eafd5b0bd0d615cfd5fd763f642dfe
- description: A commericially licensable instruct model based on MPT and trained
by Mosaic ML.
filename: ggml-mpt-7b-instruct.bin
filesize: '4854401028'
md5sum: 1cfa4958f489f0a0d1ffdf6b37322809
requires: 2.4.1

95
backends/gptq/__init__.py Normal file
View File

@ -0,0 +1,95 @@
######
# Project : GPT4ALL-UI
# File : backend.py
# Author : ParisNeo with the help of the community
# Supported by Nomic-AI
# Licence : Apache 2.0
# Description :
# This is an interface class for GPT4All-ui backends.
######
from pathlib import Path
from typing import Callable
from transformers import AutoTokenizer, TextGenerationPipeline
from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
from gpt4all_api.backend import GPTBackend
import torch
import yaml
__author__ = "parisneo"
__github__ = "https://github.com/ParisNeo/GPTQ_backend"
__copyright__ = "Copyright 2023, "
__license__ = "Apache 2.0"
backend_name = "GPTQ"
class GPTQ(GPTBackend):
file_extension='*'
def __init__(self, config:dict) -> None:
"""Builds a GPTQ backend
Args:
config (dict): The configuration file
"""
super().__init__(config, False)
self.model_dir = f'{config["model"]}'
# load quantized model, currently only support cpu or single gpu
self.model = AutoGPTQForCausalLM.from_pretrained(self.model_dir, BaseQuantizeConfig())
self.tokenizer = AutoTokenizer.from_pretrained(self.model_dir, use_fast=True )
def generate(self,
prompt:str,
n_predict: int = 128,
new_text_callback: Callable[[str], None] = bool,
verbose: bool = False,
**gpt_params ):
"""Generates text out of a prompt
Args:
prompt (str): The prompt to use for generation
n_predict (int, optional): Number of tokens to prodict. Defaults to 128.
new_text_callback (Callable[[str], None], optional): A callback function that is called everytime a new text element is generated. Defaults to None.
verbose (bool, optional): If true, the code will spit many informations about the generation process. Defaults to False.
"""
try:
tok = self.tokenizer.decode(self.model.generate(**self.tokenizer(prompt, return_tensors="pt").to("cuda:0"))[0])
new_text_callback(tok)
"""
self.model.reset()
for tok in self.model.generate(prompt,
n_predict=n_predict,
temp=self.config['temp'],
top_k=self.config['top_k'],
top_p=self.config['top_p'],
repeat_penalty=self.config['repeat_penalty'],
repeat_last_n = self.config['repeat_last_n'],
n_threads=self.config['n_threads'],
):
if not new_text_callback(tok):
return
"""
except Exception as ex:
print(ex)
@staticmethod
def list_models(config:dict):
"""Lists the models for this backend
"""
return [
"EleutherAI/gpt-j-6b",
"opt-125m-4bit"
"TheBloke/medalpaca-13B-GPTQ-4bit",
"TheBloke/stable-vicuna-13B-GPTQ",
]
@staticmethod
def get_available_models():
# Create the file path relative to the child class's directory
backend_path = Path(__file__).parent
file_path = backend_path/"models.yaml"
with open(file_path, 'r') as file:
yaml_data = yaml.safe_load(file)
return yaml_data

72
backends/gptq/models.yaml Normal file
View File

@ -0,0 +1,72 @@
- bestGPTJ: 'true'
description: Current best commercially licensable model based on GPT-J and trained
by Nomic AI on the latest curated GPT4All dataset.
filename: ggml-gpt4all-j-v1.3-groovy.bin
filesize: '3785248281'
isDefault: 'true'
md5sum: 81a09a0ddf89690372fc296ff7f625af
- bestLlama: 'true'
description: Current best non-commercially licensable model based on Llama 13b and
trained by Nomic AI on the latest curated GPT4All dataset.
filename: ggml-gpt4all-l13b-snoozy.bin
filesize: '8136770688'
md5sum: 91f886b68fbce697e9a3cd501951e455
- bestMPT: 'true'
description: Current best non-commercially licensable chat model based on MPT and
trained by Mosaic ML.
filename: ggml-mpt-7b-chat.bin
filesize: '4854401050'
isDefault: 'true'
md5sum: 756249d3d6abe23bde3b1ae272628640
requires: 2.4.1
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v2 GPT4All dataset.
filename: ggml-gpt4all-j-v1.2-jazzy.bin
filesize: '3785248281'
md5sum: 879344aaa9d62fdccbda0be7a09e7976
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v1 GPT4All dataset.
filename: ggml-gpt4all-j-v1.1-breezy.bin
filesize: '3785248281'
md5sum: 61d48a82cb188cceb14ebb8082bfec37
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v0 GPT4All dataset.
filename: ggml-gpt4all-j.bin
filesize: '3785248281'
md5sum: 5b5a3f9b858d33b29b52b89692415595
- description: A non-commercially licensable model based on Llama 7b and trained by
teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego.
filename: ggml-vicuna-7b-1.1-q4_2.bin
filesize: '4212859520'
md5sum: 29119f8fa11712704c6b22ac5ab792ea
- description: A non-commercially licensable model based on Llama 13b and trained
by teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego.
filename: ggml-vicuna-13b-1.1-q4_2.bin
filesize: '8136770688'
md5sum: 95999b7b0699e2070af63bf5d34101a8
- description: A non-commercially licensable model based on Llama 7b and trained by
Microsoft and Peking University.
filename: ggml-wizardLM-7B.q4_2.bin
filesize: '4212864640'
md5sum: 99e6d129745a3f1fb1121abed747b05a
- description: A non-commercially licensable model based on Llama 13b and RLHF trained
by Stable AI.
filename: ggml-stable-vicuna-13B.q4_2.bin
filesize: '8136777088'
md5sum: 6cb4ee297537c9133bddab9692879de0
- description: A commercially licensable model base pre-trained by Mosaic ML.
filename: ggml-mpt-7b-base.bin
filesize: '4854401028'
md5sum: 120c32a51d020066288df045ef5d52b9
requires: 2.4.1
- description: A non-commercially licensable model based on Vicuna 13b, fine-tuned
on ~180,000 instructions, trained by Nous Research.
filename: ggml-nous-gpt4-vicuna-13b.bin
filesize: '8136777088'
md5sum: d5eafd5b0bd0d615cfd5fd763f642dfe
- description: A commericially licensable instruct model based on MPT and trained
by Mosaic ML.
filename: ggml-mpt-7b-instruct.bin
filesize: '4854401028'
md5sum: 1cfa4958f489f0a0d1ffdf6b37322809
requires: 2.4.1

83
backends/hugging_face/__init__.py Normal file
View File

@ -0,0 +1,83 @@
######
# Project : GPT4ALL-UI
# File : backend.py
# Author : ParisNeo with the help of the community
# Supported by Nomic-AI
# Licence : Apache 2.0
# Description :
# This is an interface class for GPT4All-ui backends.
######
from pathlib import Path
from typing import Callable
from accelerate import init_empty_weights
from accelerate import load_checkpoint_and_dispatch
from transformers import AutoTokenizer
from transformers import AutoConfig, AutoModelForCausalLM
from gpt4all_api.backend import GPTBackend
import torch
__author__ = "parisneo"
__github__ = "https://github.com/ParisNeo/GPTQ_backend"
__copyright__ = "Copyright 2023, "
__license__ = "Apache 2.0"
backend_name = "HuggingFace"
class HuggingFace(GPTBackend):
file_extension='*'
def __init__(self, config:dict) -> None:
"""Builds a HuggingFace backend
Args:
config (dict): The configuration file
"""
super().__init__(config, False)
# load quantized model, currently only support cpu or single gpu
config_path = AutoConfig.from_pretrained(config["model"])
self.tokenizer = AutoTokenizer.from_pretrained(config["model"])
self.model = AutoModelForCausalLM.from_pretrained(config["model"], load_in_8bit=True, device_map='auto')
def generate(self,
prompt:str,
n_predict: int = 128,
new_text_callback: Callable[[str], None] = bool,
verbose: bool = False,
**gpt_params ):
"""Generates text out of a prompt
Args:
prompt (str): The prompt to use for generation
n_predict (int, optional): Number of tokens to prodict. Defaults to 128.
new_text_callback (Callable[[str], None], optional): A callback function that is called everytime a new text element is generated. Defaults to None.
verbose (bool, optional): If true, the code will spit many informations about the generation process. Defaults to False.
"""
try:
tok = self.tokenizer.decode(self.model.generate(**self.tokenizer(prompt, return_tensors="pt").to("cuda:0"))[0])
new_text_callback(tok)
"""
self.model.reset()
for tok in self.model.generate(prompt,
n_predict=n_predict,
temp=self.config['temp'],
top_k=self.config['top_k'],
top_p=self.config['top_p'],
repeat_penalty=self.config['repeat_penalty'],
repeat_last_n = self.config['repeat_last_n'],
n_threads=self.config['n_threads'],
):
if not new_text_callback(tok):
return
"""
except Exception as ex:
print(ex)
@staticmethod
def list_models(config:dict):
"""Lists the models for this backend
"""
return [
"EleutherAI/gpt-j-6B"
]

72
backends/hugging_face/models.yaml Normal file
View File

@ -0,0 +1,72 @@
- bestGPTJ: 'true'
description: Current best commercially licensable model based on GPT-J and trained
by Nomic AI on the latest curated GPT4All dataset.
filename: ggml-gpt4all-j-v1.3-groovy.bin
filesize: '3785248281'
isDefault: 'true'
md5sum: 81a09a0ddf89690372fc296ff7f625af
- bestLlama: 'true'
description: Current best non-commercially licensable model based on Llama 13b and
trained by Nomic AI on the latest curated GPT4All dataset.
filename: ggml-gpt4all-l13b-snoozy.bin
filesize: '8136770688'
md5sum: 91f886b68fbce697e9a3cd501951e455
- bestMPT: 'true'
description: Current best non-commercially licensable chat model based on MPT and
trained by Mosaic ML.
filename: ggml-mpt-7b-chat.bin
filesize: '4854401050'
isDefault: 'true'
md5sum: 756249d3d6abe23bde3b1ae272628640
requires: 2.4.1
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v2 GPT4All dataset.
filename: ggml-gpt4all-j-v1.2-jazzy.bin
filesize: '3785248281'
md5sum: 879344aaa9d62fdccbda0be7a09e7976
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v1 GPT4All dataset.
filename: ggml-gpt4all-j-v1.1-breezy.bin
filesize: '3785248281'
md5sum: 61d48a82cb188cceb14ebb8082bfec37
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v0 GPT4All dataset.
filename: ggml-gpt4all-j.bin
filesize: '3785248281'
md5sum: 5b5a3f9b858d33b29b52b89692415595
- description: A non-commercially licensable model based on Llama 7b and trained by
teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego.
filename: ggml-vicuna-7b-1.1-q4_2.bin
filesize: '4212859520'
md5sum: 29119f8fa11712704c6b22ac5ab792ea
- description: A non-commercially licensable model based on Llama 13b and trained
by teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego.
filename: ggml-vicuna-13b-1.1-q4_2.bin
filesize: '8136770688'
md5sum: 95999b7b0699e2070af63bf5d34101a8
- description: A non-commercially licensable model based on Llama 7b and trained by
Microsoft and Peking University.
filename: ggml-wizardLM-7B.q4_2.bin
filesize: '4212864640'
md5sum: 99e6d129745a3f1fb1121abed747b05a
- description: A non-commercially licensable model based on Llama 13b and RLHF trained
by Stable AI.
filename: ggml-stable-vicuna-13B.q4_2.bin
filesize: '8136777088'
md5sum: 6cb4ee297537c9133bddab9692879de0
- description: A commercially licensable model base pre-trained by Mosaic ML.
filename: ggml-mpt-7b-base.bin
filesize: '4854401028'
md5sum: 120c32a51d020066288df045ef5d52b9
requires: 2.4.1
- description: A non-commercially licensable model based on Vicuna 13b, fine-tuned
on ~180,000 instructions, trained by Nous Research.
filename: ggml-nous-gpt4-vicuna-13b.bin
filesize: '8136777088'
md5sum: d5eafd5b0bd0d615cfd5fd763f642dfe
- description: A commericially licensable instruct model based on MPT and trained
by Mosaic ML.
filename: ggml-mpt-7b-instruct.bin
filesize: '4854401028'
md5sum: 1cfa4958f489f0a0d1ffdf6b37322809
requires: 2.4.1

19
backends/llama_cpp/__init__.py
View File

@ -10,7 +10,8 @@
from pathlib import Path from pathlib import Path
from typing import Callable from typing import Callable
from pyllamacpp.model import Model from pyllamacpp.model import Model
from pyGpt4All.backend import GPTBackend from gpt4all_api.backend import GPTBackend
import yaml
__author__ = "parisneo" __author__ = "parisneo"
__github__ = "https://github.com/nomic-ai/gpt4all-ui" __github__ = "https://github.com/nomic-ai/gpt4all-ui"
@ -36,9 +37,6 @@ class LLAMACPP(GPTBackend):
seed=self.config['seed'], seed=self.config['seed'],
) )
def stop_generation(self):
self.model._grab_text_callback()
def generate(self, def generate(self,
prompt:str, prompt:str,
n_predict: int = 128, n_predict: int = 128,
@ -67,4 +65,15 @@ class LLAMACPP(GPTBackend):
if not new_text_callback(tok): if not new_text_callback(tok):
return return
except Exception as ex: except Exception as ex:
print(ex) print(ex)
@staticmethod
def get_available_models():
# Create the file path relative to the child class's directory
backend_path = Path(__file__).parent
file_path = backend_path/"models.yaml"
with open(file_path, 'r') as file:
yaml_data = yaml.safe_load(file)
return yaml_data

47
backends/llama_cpp/models.yaml Normal file
View File

@ -0,0 +1,47 @@
- bestLlama: 'false'
description: The model who started it all
filename: gpt4all-lora-quantized-ggml.new.bin
md5sum: 91f886b68fbce697e9a3cd501951e455
server: https://huggingface.co/ParisNeo/GPT4All/resolve/main/
- bestLlama: 'false'
description: The model who started it all (uncensored version)
filename: gpt4all-lora-unfiltered-quantized.new.bin
md5sum: 91f886b68fbce697e9a3cd501951e455
server: https://huggingface.co/ParisNeo/GPT4All/resolve/main/
- bestLlama: 'true'
description: Current best non-commercially licensable model based on Llama 13b and
trained by Nomic AI on the latest curated GPT4All dataset.
filename: ggml-gpt4all-l13b-snoozy.bin
filesize: '8136770688'
md5sum: 91f886b68fbce697e9a3cd501951e455
server: https://gpt4all.io/models/
- description: A non-commercially licensable model based on Llama 7b and trained by
teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego.
filename: ggml-vicuna-7b-1.1-q4_2.bin
filesize: '4212859520'
md5sum: 29119f8fa11712704c6b22ac5ab792ea
server: https://gpt4all.io/models/
- description: A non-commercially licensable model based on Llama 13b and trained
by teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego.
filename: ggml-vicuna-13b-1.1-q4_2.bin
filesize: '8136770688'
md5sum: 95999b7b0699e2070af63bf5d34101a8
server: https://gpt4all.io/models/
- description: A non-commercially licensable model based on Llama 7b and trained by
Microsoft and Peking University.
filename: ggml-wizardLM-7B.q4_2.bin
filesize: '4212864640'
md5sum: 99e6d129745a3f1fb1121abed747b05a
server: https://gpt4all.io/models/
- description: A non-commercially licensable model based on Llama 13b and RLHF trained
by Stable AI.
filename: ggml-stable-vicuna-13B.q4_2.bin
filesize: '8136777088'
md5sum: 6cb4ee297537c9133bddab9692879de0
server: https://gpt4all.io/models/
- description: A non-commercially licensable model based on Vicuna 13b, fine-tuned
on ~180,000 instructions, trained by Nous Research.
filename: ggml-nous-gpt4-vicuna-13b.bin
filesize: '8136777088'
md5sum: d5eafd5b0bd0d615cfd5fd763f642dfe
server: https://gpt4all.io/models/

2
configs/default.yaml
View File

@ -7,7 +7,7 @@ n_threads: 8
host: localhost host: localhost
language: en-US language: en-US
# Supported backends are llamacpp and gpt-j # Supported backends are llamacpp and gpt-j
backend: gpt4all backend: llama_cpp
model: null model: null
n_predict: 1024 n_predict: 1024
nb_messages_to_remember: 5 nb_messages_to_remember: 5

0
pyGpt4All/__init__.py → gpt4all_api/__init__.py
View File

628
gpt4all_api/api.py Normal file
View File

@ -0,0 +1,628 @@
######
# Project : GPT4ALL-UI
# File : api.py
# Author : ParisNeo with the help of the community
# Supported by Nomic-AI
# Licence : Apache 2.0
# Description :
# A simple api to communicate with gpt4all-ui and its models.
######
import gc
import sys
from datetime import datetime
from gpt4all_api.db import DiscussionsDB
from pathlib import Path
import importlib
from pyaipersonality import AIPersonality
import multiprocessing as mp
import threading
import time
import requests
import urllib.request
from tqdm import tqdm
__author__ = "parisneo"
__github__ = "https://github.com/nomic-ai/gpt4all-ui"
__copyright__ = "Copyright 2023, "
__license__ = "Apache 2.0"
class ModelProcess:
def __init__(self, config=None):
self.config = config
self.generate_queue = mp.Queue()
self.generation_queue = mp.Queue()
self.cancel_queue = mp.Queue(maxsize=1)
self.clear_queue_queue = mp.Queue(maxsize=1)
self.set_config_queue = mp.Queue(maxsize=1)
self.started_queue = mp.Queue()
self.process = None
self.is_generating = mp.Value('i', 0)
self.model_ready = mp.Value('i', 0)
self.ready = False
def load_backend(self, backend_path):
# define the full absolute path to the module
absolute_path = backend_path.resolve()
# infer the module name from the file path
module_name = backend_path.stem
# use importlib to load the module from the file path
loader = importlib.machinery.SourceFileLoader(module_name, str(absolute_path/"__init__.py"))
backend_module = loader.load_module()
backend_class = getattr(backend_module, backend_module.backend_name)
return backend_class
def start(self):
if self.process is None:
self.process = mp.Process(target=self._run)
self.process.start()
def stop(self):
if self.process is not None:
self.generate_queue.put(None)
self.process.join()
self.process = None
def set_backend(self, backend_path):
self.backend = backend_path
def set_model(self, model_path):
self.model = model_path
def set_config(self, config):
self.set_config_queue.put(config)
def generate(self, prompt, id, n_predict):
self.generate_queue.put((prompt, id, n_predict))
def cancel_generation(self):
self.cancel_queue.put(('cancel',))
def clear_queue(self):
self.clear_queue_queue.put(('clear_queue',))
def rebuild_backend(self, config):
try:
backend = self.load_backend(Path("backends")/config["backend"])
print("Backend loaded successfully")
except Exception as ex:
print("Couldn't build backend")
print(ex)
backend = None
return backend
def _rebuild_model(self):
try:
print("Rebuilding model")
self.backend = self.load_backend(Path("backends")/self.config["backend"])
print("Backend loaded successfully")
try:
model_file = Path("models")/self.config["backend"]/self.config["model"]
print(f"Loading model : {model_file}")
self.model = self.backend(self.config)
self.model_ready.value = 1
print("Model created successfully")
except Exception as ex:
print("Couldn't build model")
print(ex)
self.model = None
except Exception as ex:
print("Couldn't build backend")
print(ex)
self.backend = None
self.model = None
def rebuild_personality(self):
try:
personality_path = f"personalities/{self.config['personality_language']}/{self.config['personality_category']}/{self.config['personality']}"
personality = AIPersonality(personality_path)
except Exception as ex:
print("Personality file not found. Please verify that the personality you have selected exists or select another personality. Some updates may lead to change in personality name or category, so check the personality selection in settings to be sure.")
if self.config["debug"]:
print(ex)
personality = AIPersonality()
return personality
def _rebuild_personality(self):
try:
personality_path = f"personalities/{self.config['personality_language']}/{self.config['personality_category']}/{self.config['personality']}"
self.personality = AIPersonality(personality_path)
except Exception as ex:
print("Personality file not found. Please verify that the personality you have selected exists or select another personality. Some updates may lead to change in personality name or category, so check the personality selection in settings to be sure.")
if self.config["debug"]:
print(ex)
self.personality = AIPersonality()
def _run(self):
self._rebuild_model()
self._rebuild_personality()
if self.model_ready.value == 1:
self._generate("I",0,1)
print()
print("Ready to receive data")
else:
print("No model loaded. Waiting for new configuration instructions")
self.ready = True
print(f"Listening on :http://{self.config['host']}:{self.config['port']}")
while True:
try:
self._check_set_config_queue()
self._check_cancel_queue()
self._check_clear_queue()
if not self.generate_queue.empty():
command = self.generate_queue.get()
if command is None:
break
if self.cancel_queue.empty() and self.clear_queue_queue.empty():
self.is_generating.value = 1
self.started_queue.put(1)
self._generate(*command)
while not self.generation_queue.empty():
time.sleep(1)
self.is_generating.value = 0
time.sleep(1)
except Exception as ex:
time.sleep(1)
print(ex)
def _generate(self, prompt, id, n_predict):
if self.model is not None:
self.id = id
if self.config["override_personality_model_parameters"]:
self.model.generate(
prompt,
new_text_callback=self._callback,
n_predict=n_predict,
temp=self.config['temperature'],
top_k=self.config['top_k'],
top_p=self.config['top_p'],
repeat_penalty=self.config['repeat_penalty'],
repeat_last_n = self.config['repeat_last_n'],
seed=self.config['seed'],
n_threads=self.config['n_threads']
)
else:
self.model.generate(
prompt,
new_text_callback=self._callback,
n_predict=n_predict,
temp=self.personality.model_temperature,
top_k=self.personality.model_top_k,
top_p=self.personality.model_top_p,
repeat_penalty=self.personality.model_repeat_penalty,
repeat_last_n = self.personality.model_repeat_last_n,
#seed=self.config['seed'],
n_threads=self.config['n_threads']
)
else:
print("No model is installed or selected. Please make sure to install a model and select it inside your configuration before attempting to communicate with the model.")
print("To do this: Install the model to your models/<backend name> folder.")
print("Then set your model information in your local configuration file that you can find in configs/local_default.yaml")
print("You can also use the ui to set your model in the settings page.")
def _callback(self, text):
if not self.ready:
print(".",end="")
sys.stdout.flush()
return True
else:
# Stream the generated text to the main process
self.generation_queue.put((text,self.id))
self._check_set_config_queue()
self._check_cancel_queue()
self._check_clear_queue()
# if stop generation is detected then stop
if self.is_generating.value==1:
return True
else:
return False
def _check_cancel_queue(self):
while not self.cancel_queue.empty():
command = self.cancel_queue.get()
if command is not None:
self._cancel_generation()
def _check_clear_queue(self):
while not self.clear_queue_queue.empty():
command = self.clear_queue_queue.get()
if command is not None:
self._clear_queue()
def _check_set_config_queue(self):
while not self.set_config_queue.empty():
config = self.set_config_queue.get()
if config is not None:
self._set_config(config)
def _cancel_generation(self):
self.is_generating.value = 0
def _clear_queue(self):
while not self.generate_queue.empty():
self.generate_queue.get()
def _set_config(self, config):
bk_cfg = self.config
self.config = config
print("Changing configuration")
# verify that the backend is the same
if self.config["backend"]!=bk_cfg["backend"] or self.config["model"]!=bk_cfg["model"]:
self._rebuild_model()
# verify that the personality is the same
if self.config["personality"]!=bk_cfg["personality"] or self.config["personality_category"]!=bk_cfg["personality_category"] or self.config["personality_language"]!=bk_cfg["personality_language"]:
self._rebuild_personality()
class GPT4AllAPI():
def __init__(self, config:dict, socketio, config_file_path:str) -> None:
self.socketio = socketio
#Create and launch the process
self.process = ModelProcess(config)
self.process.start()
self.config = config
self.backend = self.process.rebuild_backend(self.config)
self.personality = self.process.rebuild_personality()
if config["debug"]:
print(print(f"{self.personality}"))
self.config_file_path = config_file_path
self.cancel_gen = False
# Keeping track of current discussion and message
self.current_discussion = None
self._current_user_message_id = 0
self._current_ai_message_id = 0
self._message_id = 0
self.db_path = config["db_path"]
# Create database object
self.db = DiscussionsDB(self.db_path)
# If the database is empty, populate it with tables
self.db.populate()
# This is used to keep track of messages
self.full_message_list = []
# =========================================================================================
# Socket IO stuff
# =========================================================================================
@socketio.on('connect')
def connect():
print('Client connected')
@socketio.on('disconnect')
def disconnect():
print('Client disconnected')
@socketio.on('install_model')
def install_model(data):
def install_model_():
print("Install model triggered")
model_path = data["path"]
progress = 0
installation_dir = Path(f'./models/{self.config["backend"]}/')
filename = Path(model_path).name
installation_path = installation_dir / filename
print("Model install requested")
print(f"Model path : {model_path}")
if installation_path.exists():
print("Error: Model already exists")
socketio.emit('install_progress',{'status': 'failed', 'error': 'model already exists'})
socketio.emit('install_progress',{'status': 'progress', 'progress': progress})
def callback(progress):
socketio.emit('install_progress',{'status': 'progress', 'progress': progress})
self.download_file(model_path, installation_path, callback)
socketio.emit('install_progress',{'status': 'succeeded', 'error': ''})
tpe = threading.Thread(target=install_model_, args=())
tpe.start()
@socketio.on('uninstall_model')
def uninstall_model(data):
model_path = data['path']
installation_dir = Path(f'./models/{self.config["backend"]}/')
filename = Path(model_path).name
installation_path = installation_dir / filename
if not installation_path.exists():
socketio.emit('install_progress',{'status': 'failed', 'error': 'The model does not exist'})
installation_path.unlink()
socketio.emit('install_progress',{'status': 'succeeded', 'error': ''})
@socketio.on('generate_msg')
def generate_msg(data):
if self.process.model_ready.value==1:
if self.current_discussion is None:
if self.db.does_last_discussion_have_messages():
self.current_discussion = self.db.create_discussion()
else:
self.current_discussion = self.db.load_last_discussion()
message = data["prompt"]
message_id = self.current_discussion.add_message(
"user", message, parent=self.message_id
)
self.current_user_message_id = message_id
tpe = threading.Thread(target=self.start_message_generation, args=(message, message_id))
tpe.start()
else:
self.socketio.emit('infos',
{
"status":'model_not_ready',
"type": "input_message_infos",
"bot": self.personality.name,
"user": self.personality.user_name,
"message":"",
"user_message_id": self.current_user_message_id,
"ai_message_id": self.current_ai_message_id,
}
)
@socketio.on('generate_msg_from')
def handle_connection(data):
message_id = int(data['id'])
message = data["prompt"]
self.current_user_message_id = message_id
tpe = threading.Thread(target=self.start_message_generation, args=(message, message_id))
tpe.start()
# generation status
self.generating=False
#properties
@property
def message_id(self):
return self._message_id
@property
def current_user_message_id(self):
return self._current_user_message_id
@current_user_message_id.setter
def current_user_message_id(self, id):
self._current_user_message_id=id
self._message_id = id
@property
def current_ai_message_id(self):
return self._current_ai_message_id
@current_ai_message_id.setter
def current_ai_message_id(self, id):
self._current_ai_message_id=id
self._message_id = id
def download_file(self, url, installation_path, callback=None):
"""
Downloads a file from a URL, reports the download progress using a callback function, and displays a progress bar.
Args:
url (str): The URL of the file to download.
installation_path (str): The path where the file should be saved.
callback (function, optional): A callback function to be called during the download
with the progress percentage as an argument. Defaults to None.
"""
try:
response = requests.get(url, stream=True)
# Get the file size from the response headers
total_size = int(response.headers.get('content-length', 0))
with open(installation_path, 'wb') as file:
downloaded_size = 0
with tqdm(total=total_size, unit='B', unit_scale=True, ncols=80) as progress_bar:
for chunk in response.iter_content(chunk_size=8192):
if chunk:
file.write(chunk)
downloaded_size += len(chunk)
if callback is not None:
percentage = (downloaded_size / total_size) * 100
callback(percentage)
progress_bar.update(len(chunk))
if callback is not None:
callback(100.0)
print("File downloaded successfully")
except Exception as e:
print("Couldn't download file:", str(e))
def load_backend(self, backend_path):
# define the full absolute path to the module
absolute_path = backend_path.resolve()
# infer the module name from the file path
module_name = backend_path.stem
# use importlib to load the module from the file path
loader = importlib.machinery.SourceFileLoader(module_name, str(absolute_path/"__init__.py"))
backend_module = loader.load_module()
backend_class = getattr(backend_module, backend_module.backend_name)
return backend_class
def condition_chatbot(self):
if self.current_discussion is None:
self.current_discussion = self.db.load_last_discussion()
if self.personality.welcome_message!="":
message_id = self.current_discussion.add_message(
self.personality.name, self.personality.welcome_message,
DiscussionsDB.MSG_TYPE_NORMAL,
0,
-1
)
self.current_ai_message_id = message_id
return message_id
def prepare_reception(self):
self.bot_says = ""
self.full_text = ""
self.is_bot_text_started = False
def create_new_discussion(self, title):
self.current_discussion = self.db.create_discussion(title)
# Get the current timestamp
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
# Chatbot conditionning
self.condition_chatbot()
return timestamp
def prepare_query(self, message_id=-1):
messages = self.current_discussion.get_messages()
self.full_message_list = []
for message in messages:
if message["id"]<= message_id or message_id==-1:
if message["type"]==self.db.MSG_TYPE_NORMAL:
if message["sender"]==self.personality.name:
self.full_message_list.append(self.personality.ai_message_prefix+message["content"])
else:
self.full_message_list.append(self.personality.user_message_prefix + message["content"])
link_text = self.personality.link_text
if len(self.full_message_list) > self.config["nb_messages_to_remember"]:
discussion_messages = self.personality.personality_conditioning+ link_text.join(self.full_message_list[-self.config["nb_messages_to_remember"]:])
else:
discussion_messages = self.personality.personality_conditioning+ link_text.join(self.full_message_list)
discussion_messages += link_text + self.personality.ai_message_prefix
return discussion_messages # Removes the last return
def get_discussion_to(self, message_id=-1):
messages = self.current_discussion.get_messages()
self.full_message_list = []
for message in messages:
if message["id"]<= message_id or message_id==-1:
if message["type"]!=self.db.MSG_TYPE_CONDITIONNING:
if message["sender"]==self.personality.name:
self.full_message_list.append(self.personality.ai_message_prefix+message["content"])
else:
self.full_message_list.append(self.personality.user_message_prefix + message["content"])
link_text = self.personality.link_text
if len(self.full_message_list) > self.config["nb_messages_to_remember"]:
discussion_messages = self.personality.personality_conditioning+ link_text.join(self.full_message_list[-self.config["nb_messages_to_remember"]:])
else:
discussion_messages = self.personality.personality_conditioning+ link_text.join(self.full_message_list)
return discussion_messages # Removes the last return
def remove_text_from_string(self, string, text_to_find):
"""
Removes everything from the first occurrence of the specified text in the string (case-insensitive).
Parameters:
string (str): The original string.
text_to_find (str): The text to find in the string.
Returns:
str: The updated string.
"""
index = string.lower().find(text_to_find.lower())
if index != -1:
string = string[:index]
return string
def process_chunk(self, chunk):
print(chunk[0],end="")
sys.stdout.flush()
self.bot_says += chunk[0]
if not self.personality.detect_antiprompt(self.bot_says):
self.socketio.emit('message', {
'data': self.bot_says,
'user_message_id':self.current_user_message_id,
'ai_message_id':self.current_ai_message_id,
'discussion_id':self.current_discussion.discussion_id
}
)
if self.cancel_gen:
print("Generation canceled")
self.process.cancel_generation()
self.cancel_gen = False
else:
self.bot_says = self.remove_text_from_string(self.bot_says, self.personality.user_message_prefix.strip())
self.process.cancel_generation()
print("The model is halucinating")
def start_message_generation(self, message, message_id):
bot_says = ""
# send the message to the bot
print(f"Received message : {message}")
if self.current_discussion:
# First we need to send the new message ID to the client
self.current_ai_message_id = self.current_discussion.add_message(
self.personality.name, "", parent = self.current_user_message_id
) # first the content is empty, but we'll fill it at the end
self.socketio.emit('infos',
{
"status":'generation_started',
"type": "input_message_infos",
"bot": self.personality.name,
"user": self.personality.user_name,
"message":message,#markdown.markdown(message),
"user_message_id": self.current_user_message_id,
"ai_message_id": self.current_ai_message_id,
}
)
# prepare query and reception
self.discussion_messages = self.prepare_query(message_id)
self.prepare_reception()
self.generating = True
print("## Generating message ##")
self.process.generate(self.discussion_messages, message_id, n_predict = self.config['n_predict'])
self.process.started_queue.get()
while(self.process.is_generating.value): # Simulating other commands being issued
while not self.process.generation_queue.empty():
self.process_chunk(self.process.generation_queue.get())
print()
print("## Done ##")
print()
# Send final message
self.socketio.emit('final', {
'data': self.bot_says,
'ai_message_id':self.current_ai_message_id,
'parent':self.current_user_message_id, 'discussion_id':self.current_discussion.discussion_id
}
)
self.current_discussion.update_message(self.current_ai_message_id, self.bot_says)
self.full_message_list.append(self.bot_says)
self.cancel_gen = False
return bot_says
else:
#No discussion available
print("No discussion selected!!!")
print("## Done ##")
print()
self.cancel_gen = False
return ""

14
pyGpt4All/backend.py → gpt4all_api/backend.py
View File

@ -9,6 +9,9 @@
###### ######
from pathlib import Path from pathlib import Path
from typing import Callable from typing import Callable
import inspect
import yaml
import sys
__author__ = "parisneo" __author__ = "parisneo"
__github__ = "https://github.com/nomic-ai/gpt4all-ui" __github__ = "https://github.com/nomic-ai/gpt4all-ui"
@ -18,6 +21,7 @@ __license__ = "Apache 2.0"
class GPTBackend: class GPTBackend:
file_extension='*.bin' file_extension='*.bin'
backend_path = Path(__file__).parent
def __init__(self, config:dict, inline:bool) -> None: def __init__(self, config:dict, inline:bool) -> None:
self.config = config self.config = config
self.inline = inline self.inline = inline
@ -45,3 +49,13 @@ class GPTBackend:
""" """
models_dir = Path('./models')/config["backend"] # replace with the actual path to the models folder models_dir = Path('./models')/config["backend"] # replace with the actual path to the models folder
return [f.name for f in models_dir.glob(GPTBackend.file_extension)] return [f.name for f in models_dir.glob(GPTBackend.file_extension)]
@staticmethod
def get_available_models():
# Create the file path relative to the child class's directory
backend_path = Path(__file__).parent
file_path = backend_path/"models.yaml"
with open(file_path, 'r') as file:
yaml_data = yaml.safe_load(file)
return yaml_data

0
pyGpt4All/config.py → gpt4all_api/config.py
View File

0
pyGpt4All/db.py → gpt4all_api/db.py
View File

0
pyGpt4All/extension.py → gpt4all_api/extension.py
View File

3
installations/add_personality.sh Normal file → Executable file
View File

@ -39,7 +39,8 @@ read -p "Enter the number of the desired personality: " SELECTED_PERSONALITY
PERSONALITY_FOLDER="$PERSONALITIES_FOLDER/${PERSONALITIES[$SELECTED_PERSONALITY]}" PERSONALITY_FOLDER="$PERSONALITIES_FOLDER/${PERSONALITIES[$SELECTED_PERSONALITY]}"
# Copy the selected personality folder to personalities/language/category folder # Copy the selected personality folder to personalities/language/category folder
OUTPUT_FOLDER="$(pwd)/personalities/${LANGUAGES[$SELECTED_LANGUAGE]}/${CATEGORIES[$SELECTED_CATEGORY]}/${PERSONALITIES[$SELECTED_PERSONALITY]}" CORRECTED_PATH="$(pwd)/.."
OUTPUT_FOLDER="$CORRECTED_PATH/personalities/${LANGUAGES[$SELECTED_LANGUAGE]}/${CATEGORIES[$SELECTED_CATEGORY]}/${PERSONALITIES[$SELECTED_PERSONALITY]}"
mkdir -p "$OUTPUT_FOLDER" mkdir -p "$OUTPUT_FOLDER"
cp -r "$PERSONALITY_FOLDER/." "$OUTPUT_FOLDER" cp -r "$PERSONALITY_FOLDER/." "$OUTPUT_FOLDER"

0
models/gpt_4all/.keep Normal file
View File

258
pyGpt4All/api.py
View File

@ -1,258 +0,0 @@
######
# Project : GPT4ALL-UI
# File : api.py
# Author : ParisNeo with the help of the community
# Supported by Nomic-AI
# Licence : Apache 2.0
# Description :
# A simple api to communicate with gpt4all-ui and its models.
######
import gc
import sys
from datetime import datetime
from pyGpt4All.db import DiscussionsDB
from pathlib import Path
import importlib
from pyaipersonality import AIPersonality
__author__ = "parisneo"
__github__ = "https://github.com/nomic-ai/gpt4all-ui"
__copyright__ = "Copyright 2023, "
__license__ = "Apache 2.0"
class GPT4AllAPI():
def __init__(self, config:dict, personality:AIPersonality, config_file_path:str) -> None:
self.config = config
self.personality = personality
if config["debug"]:
print(print(f"{personality}"))
self.config_file_path = config_file_path
self.cancel_gen = False
# Keeping track of current discussion and message
self.current_discussion = None
self._current_user_message_id = 0
self._current_ai_message_id = 0
self._message_id = 0
self.db_path = config["db_path"]
# Create database object
self.db = DiscussionsDB(self.db_path)
# If the database is empty, populate it with tables
self.db.populate()
# This is used to keep track of messages
self.full_message_list = []
# Select backend
self.BACKENDS_LIST = {f.stem:f for f in Path("backends").iterdir() if f.is_dir() and f.stem!="__pycache__"}
if self.config["backend"] is None:
self.backend = "gpt4all"
self.backend = self.load_backend(self.BACKENDS_LIST[self.config["backend"]])
else:
try:
self.backend = self.load_backend(self.BACKENDS_LIST[self.config["backend"]])
# Build chatbot
self.chatbot_bindings = self.create_chatbot()
print("Chatbot created successfully")
except Exception as ex:
self.config["backend"] = "gpt4all"
self.backend = self.load_backend(self.BACKENDS_LIST[self.config["backend"]])
self.config["model"] = None
print("No Models found, please select a backend and download a model for this tool to work")
# generation status
self.generating=False
#properties
@property
def message_id(self):
return self._message_id
@property
def current_user_message_id(self):
return self._current_user_message_id
@current_user_message_id.setter
def current_user_message_id(self, id):
self._current_user_message_id=id
self._message_id = id
@property
def current_ai_message_id(self):
return self._current_ai_message_id
@current_ai_message_id.setter
def current_ai_message_id(self, id):
self._current_ai_message_id=id
self._message_id = id
def load_backend(self, backend_path):
# define the full absolute path to the module
absolute_path = backend_path.resolve()
# infer the module name from the file path
module_name = backend_path.stem
# use importlib to load the module from the file path
loader = importlib.machinery.SourceFileLoader(module_name, str(absolute_path/"__init__.py"))
backend_module = loader.load_module()
backend_class = getattr(backend_module, backend_module.backend_name)
return backend_class
def create_chatbot(self):
return self.backend(self.config)
def condition_chatbot(self, conditionning_message):
if self.current_discussion is None:
self.current_discussion = self.db.load_last_discussion()
if self.personality.welcome_message!="":
message_id = self.current_discussion.add_message(
self.personality.name, self.personality.welcome_message,
DiscussionsDB.MSG_TYPE_NORMAL,
0,
-1
)
self.current_ai_message_id = message_id
return message_id
def prepare_reception(self):
self.bot_says = ""
self.full_text = ""
self.is_bot_text_started = False
def create_new_discussion(self, title):
self.current_discussion = self.db.create_discussion(title)
# Get the current timestamp
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
# Chatbot conditionning
self.condition_chatbot(self.personality.personality_conditioning)
return timestamp
def prepare_query(self, message_id=-1):
messages = self.current_discussion.get_messages()
self.full_message_list = []
for message in messages:
if message["id"]<= message_id or message_id==-1:
if message["type"]==self.db.MSG_TYPE_NORMAL:
if message["sender"]==self.personality.name:
self.full_message_list.append(self.personality.ai_message_prefix+message["content"])
else:
self.full_message_list.append(self.personality.user_message_prefix + message["content"])
link_text = self.personality.link_text
if len(self.full_message_list) > self.config["nb_messages_to_remember"]:
discussion_messages = self.personality.personality_conditioning+ link_text.join(self.full_message_list[-self.config["nb_messages_to_remember"]:])
else:
discussion_messages = self.personality.personality_conditioning+ link_text.join(self.full_message_list)
discussion_messages += link_text + self.personality.ai_message_prefix
return discussion_messages # Removes the last return
def get_discussion_to(self, message_id=-1):
messages = self.current_discussion.get_messages()
self.full_message_list = []
for message in messages:
if message["id"]<= message_id or message_id==-1:
if message["type"]!=self.db.MSG_TYPE_CONDITIONNING:
if message["sender"]==self.personality.name:
self.full_message_list.append(self.personality.ai_message_prefix+message["content"])
else:
self.full_message_list.append(self.personality.user_message_prefix + message["content"])
link_text = self.personality.link_text
if len(self.full_message_list) > self.config["nb_messages_to_remember"]:
discussion_messages = self.personality.personality_conditioning+ link_text.join(self.full_message_list[-self.config["nb_messages_to_remember"]:])
else:
discussion_messages = self.personality.personality_conditioning+ link_text.join(self.full_message_list)
return discussion_messages # Removes the last return
def remove_text_from_string(self, string, text_to_find):
"""
Removes everything from the first occurrence of the specified text in the string (case-insensitive).
Parameters:
string (str): The original string.
text_to_find (str): The text to find in the string.
Returns:
str: The updated string.
"""
index = string.lower().find(text_to_find.lower())
if index != -1:
string = string[:index]
return string
def new_text_callback(self, text: str):
if self.cancel_gen:
return False
print(text, end="")
sys.stdout.flush()
self.bot_says += text
if not self.personality.detect_antiprompt(self.bot_says):
self.socketio.emit('message', {
'data': self.bot_says,
'user_message_id':self.current_user_message_id,
'ai_message_id':self.current_ai_message_id,
'discussion_id':self.current_discussion.discussion_id
}
)
if self.cancel_gen:
print("Generation canceled")
self.cancel_gen = False
return False
else:
return True
else:
self.bot_says = self.remove_text_from_string(self.bot_says, self.personality.user_message_prefix.strip())
print("The model is halucinating")
return False
def generate_message(self):
self.generating=True
gc.collect()
total_n_predict = self.config['n_predict']
print(f"Generating {total_n_predict} outputs... ")
print(f"Input text :\n{self.discussion_messages}")
if self.config["override_personality_model_parameters"]:
self.chatbot_bindings.generate(
self.discussion_messages,
new_text_callback=self.new_text_callback,
n_predict=total_n_predict,
temp=self.config['temperature'],
top_k=self.config['top_k'],
top_p=self.config['top_p'],
repeat_penalty=self.config['repeat_penalty'],
repeat_last_n = self.config['repeat_last_n'],
seed=self.config['seed'],
n_threads=self.config['n_threads']
)
else:
self.chatbot_bindings.generate(
self.discussion_messages,
new_text_callback=self.new_text_callback,
n_predict=total_n_predict,
temp=self.personality.model_temperature,
top_k=self.personality.model_top_k,
top_p=self.personality.model_top_p,
repeat_penalty=self.personality.model_repeat_penalty,
repeat_last_n = self.personality.model_repeat_last_n,
#seed=self.config['seed'],
n_threads=self.config['n_threads']
)
self.generating=False

2
requirements.txt
View File

@ -7,7 +7,7 @@ markdown
pyllamacpp==2.1.1 pyllamacpp==2.1.1
gpt4all-j gpt4all-j
pygptj pygptj
pygpt4all gpt4all
--find-links https://download.pytorch.org/whl/cu117 --find-links https://download.pytorch.org/whl/cu117
torch==2.0.0 torch==2.0.0
torchvision torchvision

2
requirements_dev.txt
View File

@ -6,6 +6,6 @@ pyyaml
markdown markdown
pyllamacpp==2.0.0 pyllamacpp==2.0.0
gpt4all-j gpt4all-j
pygpt4all gpt4all
transformers transformers
pyaipersonality>=0.0.11 pyaipersonality>=0.0.11

28
static/js/websocket.js
View File

@ -14,7 +14,7 @@ var globals={
waitAnimation:undefined waitAnimation:undefined
} }
var socket = io.connect('http://' + document.domain + ':' + location.port); var socket = io.connect(location.protocol + '//' + document.domain + ':' + location.port);
socket.on('connect', function() { socket.on('connect', function() {
}); });
@ -22,14 +22,24 @@ socket.on('disconnect', function() {
console.log("Disconnected") console.log("Disconnected")
}); });
socket.on('infos', function(msg) { socket.on('infos', function(msg) {
if(globals.user_msg){ console.log(msg)
globals.user_msg.setSender(msg.user); if(msg["status"]=="generation_started"){
globals.user_msg.setMessage(msg.message); if(globals.user_msg){
globals.user_msg.setID(msg.id); globals.user_msg.setSender(msg.user);
} globals.user_msg.setMessage(msg.message);
globals.bot_msg.setSender(msg.bot); globals.user_msg.setID(msg.id);
globals.bot_msg.setID(msg.ai_message_id); }
globals.bot_msg.messageTextElement.innerHTML = `Generating answer. Please stand by...`; globals.bot_msg.setSender(msg.bot);
globals.bot_msg.setID(msg.ai_message_id);
globals.bot_msg.messageTextElement.innerHTML = `Generating answer. Please stand by...`;
}
else{
globals.sendbtn.style.display="block";
globals.waitAnimation.style.display="none";
globals.stopGeneration.style.display = "none";
globals.is_generating = false
alert("It seems that no model has been loaded. Please download and install a model first, then try again.");
}
}); });
socket.on('waiter', function(msg) { socket.on('waiter', function(msg) {

1
web/dist/assets/index-0f4879e0.css vendored Normal file
View File

File diff suppressed because one or more lines are too long

1
web/dist/assets/index-1c281352.css vendored
View File

File diff suppressed because one or more lines are too long

36
web/dist/assets/index-45c321b0.js vendored Normal file
View File

File diff suppressed because one or more lines are too long

36
web/dist/assets/index-a948d86d.js vendored
View File

File diff suppressed because one or more lines are too long

4
web/dist/index.html vendored
View File

@ -6,8 +6,8 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>GPT4All - WEBUI</title> <title>GPT4All - WEBUI</title>
<script type="module" crossorigin src="/assets/index-a948d86d.js"></script> <script type="module" crossorigin src="/assets/index-45c321b0.js"></script>
<link rel="stylesheet" href="/assets/index-1c281352.css"> <link rel="stylesheet" href="/assets/index-0f4879e0.css">
</head> </head>
<body> <body>
<div id="app"></div> <div id="app"></div>

23
web/src/components/ModelEntry.vue
View File

@ -5,14 +5,9 @@
</div> </div>
<div class="flex-1"> <div class="flex-1">
<h3 class="font-bold text-lg"> <h3 class="font-bold text-lg">
<input
type="radio"
:checked="selected"
:disabled="!isInstalled"
@change="handleSelection"
/>
{{ title }} {{ title }}
</h3> </h3>
<a :href="path" title="Download this manually (faster) and put it in the models/<your backend> folder then refresh">{{ title }}</a>
<p class="opacity-80">{{ description }}</p> <p class="opacity-80">{{ description }}</p>
</div> </div>
<div class="flex-shrink-0"> <div class="flex-shrink-0">
@ -24,13 +19,17 @@
> >
<template v-if="installing"> <template v-if="installing">
<div class="flex items-center space-x-2"> <div class="flex items-center space-x-2">
<div class="h-2 w-20 bg-gray-300 rounded"></div> <div class="h-2 w-20 bg-gray-300 rounded">
<span>Installing...</span> <div :style="{ width: progress + '%'}" class="h-full bg-red-500 rounded"></div>
</div>
<span>Installing...{{ Math.floor(progress) }}%</span>
</div> </div>
</template> </template>
<template v-else-if="uninstalling"> <template v-else-if="uninstalling">
<div class="flex items-center space-x-2"> <div class="flex items-center space-x-2">
<div class="h-2 w-20 bg-gray-300 rounded"></div> <div class="h-2 w-20 bg-gray-300 rounded">
<div :style="{ width: progress + '%' }" class="h-full bg-green-500"></div>
</div>
<span>Uninstalling...</span> <span>Uninstalling...</span>
</div> </div>
</template> </template>
@ -43,7 +42,7 @@
</template> </template>
<script> <script>
import { socket, state } from '@/services/websocket.js' import socket from '@/services/websocket.js'
export default { export default {
props: { props: {
title: String, title: String,
@ -58,6 +57,7 @@ export default {
}, },
data() { data() {
return { return {
progress: 0,
installing: false, installing: false,
uninstalling: false uninstalling: false
}; };
@ -76,8 +76,7 @@ export default {
}, },
handleSelection() { handleSelection() {
if (this.isInstalled && !this.selected) { if (this.isInstalled && !this.selected) {
this.selected=true; this.onSelected(this);
onSelected(this);
} }
} }
} }

65
web/src/components/PersonalityViewer.vue Normal file
View File

@ -0,0 +1,65 @@
<template>
<div class="p-4">
<div class="flex items-center mb-4">
<img :src="avatar" class="w-12 h-12 rounded-full mr-2" alt="Avatar">
<h2 class="text-lg font-semibold">{{ personalityName }}</h2>
</div>
<p><strong>Author:</strong> {{ personalityAuthor }}</p>
<p><strong>Description:</strong> {{ personalityDescription }}</p>
<p><strong>Language:</strong> {{ personalityLanguage }}</p>
<p><strong>Category:</strong> {{ personalityCategory }}</p>
<p v-if="disclaimer"><strong>Disclaimer:</strong> {{ disclaimer }}</p>
<p><strong>Conditioning Text:</strong> {{ conditioningText }}</p>
<p><strong>AI Prefix:</strong> {{ aiPrefix }}</p>
<p><strong>User Prefix:</strong> {{ userPrefix }}</p>
<div>
<strong>Antiprompts:</strong>
<ul>
<li v-for="antiprompt in antipromptsList" :key="antiprompt.id">
{{ antiprompt.text }}
</li>
</ul>
</div>
<button @click="editMode = true" class="mt-4 bg-blue-500 text-white px-4 py-2 rounded">
Edit
</button>
<button @click="commitChanges" v-if="editMode" class="mt-4 bg-green-500 text-white px-4 py-2 rounded">
Commit
</button>
</div>
</template>
<script>
export default {
data() {
return {
editMode: false,
avatar: 'path/to/avatar.jpg',
personalityName: 'Personality Name',
personalityAuthor: 'Author Name',
personalityDescription: 'Personality Description',
personalityLanguage: 'English',
personalityCategory: 'Category',
disclaimer: 'Disclaimer text',
conditioningText: 'Conditioning Text',
aiPrefix: 'AI Prefix',
userPrefix: 'User Prefix',
antipromptsList: [
{ id: 1, text: 'Antiprompt 1' },
{ id: 2, text: 'Antiprompt 2' },
{ id: 3, text: 'Antiprompt 3' }
]
};
},
methods: {
commitChanges() {
// Send the modified personality to the backend
// Implement your backend integration here
console.log('Personality changes committed');
this.editMode = false;
}
}
};
</script>

22
web/src/components/Toast.vue
View File

@ -5,6 +5,16 @@
role="alert"> role="alert">
<div class="flex flex-row"> <div class="flex flex-row">
<slot> <slot>
<div v-if="success"
class="inline-flex items-center justify-center flex-shrink-0 w-8 h-8 text-green-500 bg-green-100 rounded-lg dark:bg-green-800 dark:text-green-200">
<i data-feather="check"></i>
<span class="sr-only">Check icon</span>
</div>
<div v-if="!success" class="inline-flex items-center justify-center flex-shrink-0 w-8 h-8 text-red-500 bg-red-100 rounded-lg dark:bg-red-800 dark:text-red-200">
<i data-feather="x"></i>
<span class="sr-only">Cross icon</span>
</div>
<div class="ml-3 text-sm font-normal">{{ message }}</div>
</slot> </slot>
</div> </div>
@ -19,6 +29,7 @@
clip-rule="evenodd"></path> clip-rule="evenodd"></path>
</svg> </svg>
</button> </button>
</div> </div>
</div> </div>
</template> </template>
@ -34,12 +45,23 @@ export default {
data() { data() {
return { return {
show: false, show: false,
success: true,
message: ''
}; };
}, },
methods: { methods: {
close() { close() {
this.$emit('close') this.$emit('close')
this.show = false this.show = false
},
showToast(message, duration_s=3, success= true){
this.success = success;
this.message = message;
this.show = true;
setTimeout(() => {
this.$emit('close')
this.show = false
}, duration_s*1000);
} }
}, },
watch: { watch: {

19
web/src/services/websocket.js
View File

@ -3,12 +3,9 @@
// Description : // Description :
// All websocket stuff can be found here. // All websocket stuff can be found here.
// More info can be found here https://socket.io/how-to/use-with-vue // More info can be found here https://socket.io/how-to/use-with-vue
import io from 'socket.io-client'; import { createApp } from 'vue';
import { reactive } from "vue"; import io from 'socket.io-client';
const state = reactive({
connected: false,
});
const socket = new io(import.meta.env.VITE_GPT4ALL_API ); const socket = new io(import.meta.env.VITE_GPT4ALL_API );
@ -26,14 +23,18 @@ socket.onerror = (error) => {
}; };
socket.on("connect", () => { socket.on("connect", () => {
state.connected = true;
console.log('WebSocket connected (websocket)'); console.log('WebSocket connected (websocket)');
}); });
socket.on("disconnect", () => { socket.on("disconnect", () => {
state.connected = false;
console.log('WebSocket disonnected (websocket)'); console.log('WebSocket disonnected (websocket)');
}); });
export {socket, state}; const app = createApp(/* your root component */);
app.config.globalProperties.$socket = socket;
app.mount(/* your root element */);
export default socket;

80
web/src/views/DiscussionsView.vue
View File

@ -131,13 +131,7 @@
</div> </div>
</div> </div>
<Toast :showProp="isCopiedToClipboard" @close="isCopiedToClipboard = false"> <Toast ref="toast">
<div
class="inline-flex items-center justify-center flex-shrink-0 w-8 h-8 text-green-500 bg-green-100 rounded-lg dark:bg-green-800 dark:text-green-200">
<i data-feather="check"></i>
<span class="sr-only">Check icon</span>
</div>
<div class="ml-3 text-sm font-normal">Message content copied to clipboard!</div>
</Toast> </Toast>
</template> </template>
@ -148,6 +142,7 @@
</style> </style>
<script> <script>
export default { export default {
setup() { }, setup() { },
data() { data() {
@ -165,7 +160,7 @@ export default {
isSelectAll: false, isSelectAll: false,
showConfirmation: false, showConfirmation: false,
chime: new Audio("chime_aud.wav"), chime: new Audio("chime_aud.wav"),
isCopiedToClipboard: false, showToast: false,
isSearch: false, isSearch: false,
isDiscussionBottom: false, isDiscussionBottom: false,
} }
@ -426,40 +421,48 @@ export default {
// Update previous message with reponse user data // Update previous message with reponse user data
// //
// msgObj // msgObj
// // "status": "if the model is not ready this will inform the user that he can't promt the model"
// "type": "input_message_infos", // "type": "input_message_infos",
// "bot": self.personality.name, // "bot": self.personality.name,
// "user": self.personality.user_name, // "user": self.personality.user_name,
// "message":message,#markdown.markdown(message), // "message":message,#markdown.markdown(message),
// "user_message_id": self.current_user_message_id, // "user_message_id": self.current_user_message_id,
// "ai_message_id": self.current_ai_message_id, // "ai_message_id": self.current_ai_message_id,
console.log(msgObj);
this.updateLastUserMsg(msgObj) if(msgObj["status"]=="generation_started"){
// Create response message this.updateLastUserMsg(msgObj)
let responseMessage = { // Create response message
content: "✍ please stand by ...",//msgObj.message, let responseMessage = {
id: msgObj.ai_message_id, content: "✍ please stand by ...",//msgObj.message,
parent: msgObj.user_message_id, id: msgObj.ai_message_id,
rank: 0, parent: msgObj.user_message_id,
sender: msgObj.bot, rank: 0,
//type: msgObj.type sender: msgObj.bot,
} //type: msgObj.type
this.discussionArr.push(responseMessage)
nextTick(() => {
const msgList = document.getElementById('messages-list')
this.scrollBottom(msgList)
})
if (this.currentDiscussion.title === '' || this.currentDiscussion.title === null) {
if (msgObj.type == "input_message_infos") {
// This is a user input
this.changeTitleUsingUserMSG(this.currentDiscussion.id, msgObj.message)
} }
this.discussionArr.push(responseMessage)
nextTick(() => {
const msgList = document.getElementById('messages-list')
this.scrollBottom(msgList)
})
if (this.currentDiscussion.title === '' || this.currentDiscussion.title === null) {
if (msgObj.type == "input_message_infos") {
// This is a user input
this.changeTitleUsingUserMSG(this.currentDiscussion.id, msgObj.message)
}
}
console.log("infos", msgObj)
}
else{
this.$refs.toast.showToast("It seems that no model has been loaded. Please download and install a model first, then try again.",4, false)
this.isGenerating = false
this.setDiscussionLoading(this.currentDiscussion.id, this.isGenerating)
this.chime.play()
} }
console.log("infos", msgObj)
}, },
sendMsg(msg) { sendMsg(msg) {
// Sends message to backend // Sends message to backend
@ -745,15 +748,14 @@ export default {
this.chime.play() this.chime.play()
}, },
copyToClipBoard(content) { copyToClipBoard(content) {
this.$refs.toast.showToast("Copied to clipboard successfully")
this.isCopiedToClipboard = true
nextTick(() => { nextTick(() => {
feather.replace() feather.replace()
}) })
}, },
closeToast() { closeToast() {
this.isCopiedToClipboard = false this.showToast = false
}, },
@ -831,7 +833,7 @@ export default {
}, },
computed: { computed: {
socketConnected() { socketConnected() {
return state.connected return true
}, },
selectedDiscussions() { selectedDiscussions() {
nextTick(() => { nextTick(() => {
@ -856,7 +858,7 @@ import feather from 'feather-icons'
import axios from 'axios' import axios from 'axios'
import { nextTick } from 'vue' import { nextTick } from 'vue'
import { socket, state } from '@/services/websocket.js' import socket from '@/services/websocket.js'
import { onMounted } from 'vue' import { onMounted } from 'vue'
import { initFlowbite } from 'flowbite' import { initFlowbite } from 'flowbite'

135
web/src/views/SettingsView.vue
View File

@ -52,21 +52,32 @@
</select> </select>
</div> </div>
<div class="m-2">
<label for="model" class="block mb-2 text-sm font-medium text-gray-900 dark:text-white">
Model:
</label>
<select id="model" @change="update_model($event.target.value)"
class="bg-gray-50 border border-gray-300 text-gray-900 text-sm rounded-lg focus:ring-blue-500 focus:border-blue-500 block w-full p-2.5 dark:bg-gray-700 dark:border-gray-600 dark:placeholder-gray-400 dark:text-white dark:focus:ring-blue-500 dark:focus:border-blue-500">
<option v-for="item in modelsArr" :selected="item === configFile.model">{{ item }}</option>
</select>
</div>
</div> </div>
</div> </div>
<div <div
class="flex flex-col mb-2 p-3 rounded-lg bg-bg-light-tone dark:bg-bg-dark-tone hover:bg-bg-light-tone-panel hover:dark:bg-bg-dark-tone-panel duration-150 shadow-lg"> class="flex flex-col mb-2 p-3 rounded-lg bg-bg-light-tone dark:bg-bg-dark-tone hover:bg-bg-light-tone-panel hover:dark:bg-bg-dark-tone-panel duration-150 shadow-lg">
<div class="flex flex-row "> <div class="flex flex-row ">
<button @click.stop="bec_collapsed = !bec_collapsed" <button @click.stop="mzc_collapsed = !mzc_collapsed"
class="text-2xl hover:text-primary duration-75 p-2 -m-2 w-full text-left active:translate-y-1"> class="text-2xl hover:text-primary duration-75 p-2 -m-2 w-full text-left active:translate-y-1">
<!-- <i data-feather="chevron-right"></i> --> <!-- <i data-feather="chevron-right"></i> -->
<h3 class="text-lg font-semibold cursor-pointer select-none " <h3 class="text-lg font-semibold cursor-pointer select-none "
@click.stop="bec_collapsed = !bec_collapsed"> @click.stop="mzc_collapsed = !mzc_collapsed">
Models zoo</h3> Models zoo</h3>
</button> </button>
</div> </div>
<div :class="{ 'hidden': bec_collapsed }" class="flex flex-col mb-2 p-2"> <div :class="{ 'hidden': mzc_collapsed }" class="flex flex-col mb-2 p-2">
<div v-if="models.length > 0" class="my-2"> <div v-if="models.length > 0" class="my-2">
<label for="model" class="block ml-2 mb-2 text-sm font-medium text-gray-900 dark:text-white"> <label for="model" class="block ml-2 mb-2 text-sm font-medium text-gray-900 dark:text-white">
Install more models: Install more models:
@ -132,7 +143,29 @@
</div> </div>
</div> </div>
</div> </div>
<div
class="flex flex-col mb-2 p-3 rounded-lg bg-bg-light-tone dark:bg-bg-dark-tone hover:bg-bg-light-tone-panel hover:dark:bg-bg-dark-tone-panel duration-150 shadow-lg">
<div class="flex flex-row ">
<button @click.stop="pzc_collapsed = !pzc_collapsed"
class="text-2xl hover:text-primary duration-75 p-2 -m-2 w-full text-left active:translate-y-1">
<!-- <i data-feather="chevron-right"></i> -->
<h3 class="text-lg font-semibold cursor-pointer select-none "
@click.stop="pzc_collapsed = !pzc_collapsed">
Personalities zoo</h3>
</button>
</div>
<div :class="{ 'hidden': pzc_collapsed }" class="flex flex-col mb-2 p-2">
<div v-if="models.length > 0" class="my-2">
<label for="model" class="block ml-2 mb-2 text-sm font-medium text-gray-900 dark:text-white">
Install more models:
</label>
<div class="overflow-y-auto max-h-96 no-scrollbar p-2">
</div>
</div>
</div>
</div>
<!-- MODEL --> <!-- MODEL -->
<div <div
class="flex flex-col mb-2 p-3 rounded-lg bg-bg-light-tone dark:bg-bg-dark-tone hover:bg-bg-light-tone-panel hover:dark:bg-bg-dark-tone-panel duration-150 shadow-lg"> class="flex flex-col mb-2 p-3 rounded-lg bg-bg-light-tone dark:bg-bg-dark-tone hover:bg-bg-light-tone-panel hover:dark:bg-bg-dark-tone-panel duration-150 shadow-lg">
@ -171,7 +204,7 @@
</div> </div>
<input id="temperature" @change="update_setting('temperature', $event.target.value)" type="range" <input id="temperature" @change="update_setting('temperature', $event.target.value)" type="range"
v-model="configFile.temp" min="0" max="5" step="0.1" v-model="configFile.temperature" min="0" max="5" step="0.1"
class="flex-none h-2 mt-14 mb-2 w-full bg-gray-200 rounded-lg appearance-none cursor-pointer dark:bg-gray-700 focus:ring-blue-500 focus:border-blue-500 dark:border-gray-600 dark:placeholder-gray-400 dark:focus:ring-blue-500 dark:focus:border-blue-500"> class="flex-none h-2 mt-14 mb-2 w-full bg-gray-200 rounded-lg appearance-none cursor-pointer dark:bg-gray-700 focus:ring-blue-500 focus:border-blue-500 dark:border-gray-600 dark:placeholder-gray-400 dark:focus:ring-blue-500 dark:focus:border-blue-500">
</div> </div>
</div> </div>
@ -297,13 +330,15 @@ import { nextTick } from 'vue'
import MessageBox from "@/components/MessageBox.vue"; import MessageBox from "@/components/MessageBox.vue";
import YesNoDialog from "@/components/YesNoDialog.vue"; import YesNoDialog from "@/components/YesNoDialog.vue";
import ModelEntry from '@/components/ModelEntry.vue'; import ModelEntry from '@/components/ModelEntry.vue';
import { socket, state } from '@/services/websocket.js' import PersonalityViewer from '@/components/PersonalityViewer.vue';
import socket from '@/services/websocket.js'
axios.defaults.baseURL = import.meta.env.VITE_GPT4ALL_API_BASEURL axios.defaults.baseURL = import.meta.env.VITE_GPT4ALL_API_BASEURL
export default { export default {
components: { components: {
MessageBox, MessageBox,
YesNoDialog, YesNoDialog,
ModelEntry ModelEntry,
PersonalityViewer
}, },
setup() { setup() {
@ -315,16 +350,18 @@ export default {
data() { data() {
return { return {
// Websocket
socket: socket,
// Models zoo installer stuff // Models zoo installer stuff
models: [], models: [],
personalities:[],
// Accordeon stuff // Accordeon stuff
bec_collapsed: false, bec_collapsed: true,
pc_collapsed: false, mzc_collapsed: true, // models zoo
mc_collapsed: false, pzc_collapsed: true, // personalities zoo
pc_collapsed: true,
mc_collapsed: true,
// Settings stuff // Settings stuff
backendsArr: [], backendsArr: [],
modelsArr: [],
persLangArr: [], persLangArr: [],
persCatgArr: [], persCatgArr: [],
persArr: [], persArr: [],
@ -348,65 +385,64 @@ export default {
}, },
onSelected(model_object){ onSelected(model_object){
console.log("Selected model") console.log("Selected model")
update_setting('model', model_object.title) this.update_setting('model', model_object.title, (res)=>{console.log("Model selected"); })
}, },
// Model installation // Model installation
onInstall(model_object) { onInstall(model_object) {
let isInstalled = model_object.isInstalled let path = model_object.path;
let path = model_object.path
this.showProgress = true; this.showProgress = true;
this.progress = 0; this.progress = 0;
console.log("installing...") console.log("installing...");
// Use an arrow function for progressListener
const progressListener = (response) => { const progressListener = (response) => {
console.log("received something");
if (response.status === 'progress') { if (response.status === 'progress') {
this.progress = message.progress; console.log(`Progress = ${response.progress}`);
model_object.progress = response.progress
} else if (response.status === 'succeeded') { } else if (response.status === 'succeeded') {
// Installation completed socket.off('install_progress', progressListener);
model_object.installing = false; // Update the isInstalled property of the corresponding model
this.showProgress = false; const index = this.models.findIndex((model) => model.path === path);
// Update the isInstalled property of the corresponding model this.models[index].isInstalled = true;
const index = this.models.findIndex((model) => model.path === path); this.showProgress = false;
this.models[index].isInstalled = true;
this.socket.off('install_progress', progressListener);
} else if (response.status === 'failed') { } else if (response.status === 'failed') {
// Installation failed or encountered an error socket.off('install_progress', progressListener);
model_object.installing = false; // Installation failed or encountered an error
this.showProgress = false; model_object.installing = false;
this.socket.off('install_progress', progressListener); this.showProgress = false;
console.error('Installation failed:', message.error); console.error('Installation failed:', response.error);
} }
}; };
this.socket.on('install_progress', progressListener);
this.socket.emit('install_model', { path: path });
socket.on('install_progress', progressListener);
socket.emit('install_model', { path: path });
console.log("Started installation, please wait");
}, },
onUninstall(model_object) { onUninstall(model_object) {
console.log("uninstalling model...") console.log("uninstalling model...")
const progressListener = (response) => { const progressListener = (response) => {
if (response.status === 'progress') { if (response.status === 'progress') {
this.progress = message.progress; this.progress = response.progress;
} else if (response.status === 'succeeded') { } else if (response.status === 'succeeded') {
console.log(model_object)
// Installation completed // Installation completed
model_object.uninstalling = false; model_object.uninstalling = false;
socket.off('install_progress', progressListener);
this.showProgress = false; this.showProgress = false;
// Update the isInstalled property of the corresponding model const index = this.models.findIndex((model) => model.path === model_object.path);
model_object.isInstalled = false; this.models[index].isInstalled = false;
this.socket.off('install_progress', progressListener);
} else if (response.status === 'failed') { } else if (response.status === 'failed') {
// Installation failed or encountered an error // Installation failed or encountered an error
model_object.uninstalling = false; model_object.uninstalling = false;
this.showProgress = false; this.showProgress = false;
this.socket.off('install_progress', progressListener); socket.off('install_progress', progressListener);
console.error('Installation failed:', message.error); console.error('Installation failed:', message.error);
} }
}; };
this.socket.on('install_progress', progressListener); socket.on('install_progress', progressListener);
this.socket.emit('uninstall_model', { path: model_object.path }); socket.emit('uninstall_model', { path: model_object.path });
}, },
// messagebox ok stuff // messagebox ok stuff
onMessageBoxOk() { onMessageBoxOk() {
@ -426,7 +462,7 @@ export default {
this.api_get_req("get_config").then(response => { this.api_get_req("get_config").then(response => {
this.configFile = response this.configFile = response
console.log("selecting model") console.log("selecting model")
self.models.forEach(model => { this.models.forEach(model => {
console.log(`${model} -> ${response["model"]}`) console.log(`${model} -> ${response["model"]}`)
if(model.title==response["model"]){ if(model.title==response["model"]){
model.selected=true; model.selected=true;
@ -450,13 +486,21 @@ export default {
axios.post('/update_setting', obj).then((res) => { axios.post('/update_setting', obj).then((res) => {
if (res) { if (res) {
if (next !== undefined) { if (next !== undefined) {
next() next(res)
} }
return res.data; return res.data;
} }
}) })
.catch(error => { return { 'status': false } }); .catch(error => { return { 'status': false } });
}, },
update_backend(value) {
console.log("Upgrading backend")
this.update_setting('backend', value, (res)=>{console.log("Backend changed"); this.fetchModels(); })
},
update_model(value) {
console.log("Upgrading model")
this.update_setting('model', value, (res)=>{console.log("Model changed"); this.fetchModels(); })
},
save_configuration() { save_configuration() {
this.showConfirmation = false this.showConfirmation = false
axios.post('/save_settings', {}) axios.post('/save_settings', {})
@ -502,13 +546,7 @@ export default {
} }
}); });
}, },
update_backend(value) {
res = update_setting('backend', value)
if (res.status) {
console.log("Backend changed")
}
},
async api_get_req(endpoint) { async api_get_req(endpoint) {
try { try {
const res = await axios.get("/" + endpoint); const res = await axios.get("/" + endpoint);
@ -534,6 +572,7 @@ export default {
this.configFile = await this.api_get_req("get_config") this.configFile = await this.api_get_req("get_config")
this.backendsArr = await this.api_get_req("list_backends") this.backendsArr = await this.api_get_req("list_backends")
this.modelsArr = await this.api_get_req("list_models")
this.persLangArr = await this.api_get_req("list_personalities_languages") this.persLangArr = await this.api_get_req("list_personalities_languages")
this.persCatgArr = await this.api_get_req("list_personalities_categories") this.persCatgArr = await this.api_get_req("list_personalities_categories")
this.persArr = await this.api_get_req("list_personalities") this.persArr = await this.api_get_req("list_personalities")

40
webui.bat
View File

@ -262,46 +262,6 @@ if not exist \models (
md \models md \models
) )
dir ".\models\llama_cpp\*.bin" /b 2>&1
if errorlevel 1 (
echo.
choice /C YNB /M "The default model file (gpt4all-lora-quantized-ggml.bin) does not exist. Do you want to download it? Press B to download it with a browser (faster)."
if errorlevel 3 goto DOWNLOAD_WITH_BROWSER
if errorlevel 2 goto DOWNLOAD_SKIP
if errorlevel 1 goto MODEL_DOWNLOAD
) ELSE (
echo Model already installed
goto CONTINUE
)
:DOWNLOAD_WITH_BROWSER
start https://huggingface.co/ParisNeo/GPT4All/resolve/main/gpt4all-lora-quantized-ggml.bin
echo Link has been opened with the default web browser, make sure to save it into the models/llama_cpp folder before continuing. Press any key to continue...
pause
goto :CONTINUE
:MODEL_DOWNLOAD
echo.
echo Downloading latest model...
set clone_dir=%cd%
powershell -Command "Invoke-WebRequest -Uri 'https://huggingface.co/ParisNeo/GPT4All/resolve/main/gpt4all-lora-quantized-ggml.bin' -OutFile %clone_dir%'/models/llama_cpp/gpt4all-lora-quantized-ggml.bin'"
if errorlevel 1 (
echo Failed to download model. Please check your internet connection.
choice /C YN /M "Do you want to try downloading again?"
if errorlevel 2 goto DOWNLOAD_SKIP
if errorlevel 1 goto MODEL_DOWNLOAD
) else (
echo Model successfully downloaded.
)
goto :CONTINUE
:DOWNLOAD_SKIP
echo.
echo Skipping download of model file...
goto :CONTINUE
:CONTINUE
:END :END
if exist "./tmp" ( if exist "./tmp" (
echo Cleaning tmp folder echo Cleaning tmp folder

48
webui.sh
View File

@ -110,53 +110,9 @@ if ping -q -c 1 google.com >/dev/null 2>&1; then
else else
echo "is created" echo "is created"
fi fi
# Checking model
MODEL="./models/llama_cpp/gpt4all-lora-quantized-ggml.bin"
MODEL_URL="https://huggingface.co/ParisNeo/GPT4All/resolve/main/gpt4all-lora-quantized-ggml.bin"
if [ -f "$MODEL" ]; then
echo "File $MODEL already exists. Skipping download."
else
echo "File $MODEL does not exist."
echo "What would you like to do?"
select option in "Download" "Download using browser" "Skip"; do
case $option in
Download)
if [ -x "$(command -v wget)" ]; then
wget "$MODEL_URL" -P ./models/llama_cpp/
elif [ -x "$(command -v curl)" ]; then
curl -o "$MODEL" "$MODEL_URL"
else
echo "Error: neither wget nor curl is installed. Please install one of them and try again."
exit 1
fi
break
;;
"Download using browser")
if [ -x "$(command -v xdg-open)" ]; then
xdg-open "$MODEL_URL"
elif [ -x "$(command -v gnome-open)" ]; then
gnome-open "$MODEL_URL"
elif [ -x "$(command -v kde-open)" ]; then
kde-open "$MODEL_URL"
elif [ -x "$(command -v open)" ]; then
open "$MODEL_URL"
else
echo "Error: could not detect a default browser. Please open the link in your web browser manually and press any key to continue."
read -n 1 -s -r -p "Press any key to continue"$'\n'
fi
break
;;
Skip)
echo "Skipping downloading $MODEL"
break
;;
esac
done
fi
else
echo "Internet connection not available"
fi fi
# Activate the virtual environment # Activate the virtual environment
echo -n "Activating virtual environment..." echo -n "Activating virtual environment..."
source env/bin/activate source env/bin/activate