ExternalVendorCode/lollms-webui
1
0
Fork 0
mirror of https://github.com/ParisNeo/lollms-webui.git synced 2024-12-22 05:37:48 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge branch 'main' into code-block-style

Browse Source
This commit is contained in:
AndzejsP 2023-05-15 12:36:12 +03:00
commit ab0b1f57ed
43 changed files with 1768 additions and 765 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
.gitignore vendored
View File

@ -160,12 +160,6 @@ databases/*
extensions/
!extensions/.keep
# backends
backends/
!backends/gpt4all
!backends/llama_cpp
!backends/gptj
!backends/__init__.py
web/.env.build
web/.env.dev
web/.env.development

2
Dockerfile
View File

@ -7,7 +7,7 @@ RUN python3 -m venv venv && . venv/bin/activate
RUN python3 -m pip install --no-cache-dir -r requirements.txt --upgrade pip
COPY ./app.py /srv/app.py
COPY ./pyGpt4All /srv/pyGpt4All
COPY ./gpt4all_api /srv/gpt4all_api
COPY ./backends /srv/backends
COPY ./static /srv/static
COPY ./templates /srv/templates

29
README.md
View File

@ -41,7 +41,7 @@ It's worth noting that the model has recently been launched, and it's expected t
# Installation and running
Make sure that your CPU supports `AVX2` instruction set. Without it, this application won't run out of the box. To check your CPU features, please visit the website of your CPU manufacturer for more information and look for `Instruction set extension: AVX2`.
Make sure that your CPU supports `AVX2` instruction set. Without it, this application won't run out of the box (for the pyllamacpp backend). To check your CPU features, please visit the website of your CPU manufacturer for more information and look for `Instruction set extension: AVX2`.
> **Note**
>
>Default model `gpt4all-lora-quantized-ggml.bin` is roughly 4GB in size.
@ -60,6 +60,7 @@ Make sure that your CPU supports `AVX2` instruction set. Without it, this applic
> **Note**
> During installtion, it may ask you to download a model. Feel free to accept or to download your own models depending on the backends you are using.
Once installed, you can run the app by using `webui.bat` or `webui.sh`. The script will check for any new updates
[If you want to use a more advanced install procedure, please click here](docs/usage/AdvancedInstallInstructions.md)
@ -96,9 +97,11 @@ Now you're ready to work!
# Supported backends
Two backends are now supported:
1 - [The llama_cpp backend](https://github.com/nomic-ai/pygpt4all)
2 - [The GPT-j backend](https://github.com/marella/gpt4all-j)
3 - Hugging face's Transformers (under construction)
1- [The llama_cpp backend by Abdeladim](https://github.com/abdeladim-s/pyllamacpp)
2- [The GPT-j backend by Abdeladim](https://github.com/abdeladim-s/pygptj)
3- [The GPT-j backend by marella](https://github.com/marella/gpt4all-j)
4- Hugging face's Transformers (under construction)
# Supported models
You can also refuse to download the model during the install procedure and download it manually.
@ -144,14 +147,18 @@ Just download the model into the `models/<backend name>` folder and start using
You can find hundreds of personalities in my personal [Personalities repository](https://github.com/ParisNeo/PyAIPersonality). This new personalities format can be used for any third party applications, it builds a simple structure and format to define personalities. This format is evolutive and new fields and assets will be added in the future like personality voice or 3d animated character with prebaked motions that should allow AI to be more alive. The format is baked to support old versions while adding new capabilities for new versions making it ideal as a personality defintition format.
## Personality install
if you are on windows you can install new personalities directly using the `add_personality.bat` code:
```bash
add_personality.bat
```
### How to Install Personalities from the Zoo
1. Navigate to the root directory of your repository.
2. Run either `installations/add_personality.bat` or `installations/add_personality.sh`, depending on your operating system.
3. Select the desired language, category, and personality from the provided options.
4. The selected personality will be added to the list of available options.
5. Choose the current personality:
- Option 1: Use the UI by going to "Settings" and selecting "Personalities".
- Option 2: Update the configuration file `configs/default_local.yaml` with the appropriate language, category, and personality name.
Note: Ensure that you have the necessary permissions and dependencies installed before performing the above steps.
```bash
bash add_personality.sh
```
Please don't forget to take time and give a Star if you like the project. This helps the visibility of the project.

254
app.py
View File

@ -20,11 +20,10 @@ import argparse
import json
import re
import traceback
import threading
import sys
from tqdm import tqdm
from pyaipersonality import AIPersonality
from pyGpt4All.db import DiscussionsDB, Discussion
from gpt4all_api.db import DiscussionsDB, Discussion
from flask import (
Flask,
Response,
@ -41,9 +40,12 @@ from geventwebsocket.handler import WebSocketHandler
from gevent.pywsgi import WSGIServer
import requests
from concurrent.futures import ThreadPoolExecutor, as_completed
import logging
log = logging.getLogger('werkzeug')
log.setLevel(logging.ERROR)
app = Flask("GPT4All-WebUI", static_url_path="/static", static_folder="static")
socketio = SocketIO(app, cors_allowed_origins="*", async_mode='gevent', ping_timeout=30, ping_interval=15)
socketio = SocketIO(app, cors_allowed_origins="*", async_mode='gevent', ping_timeout=60, ping_interval=15)
app.config['SECRET_KEY'] = 'secret!'
# Set the logging level to WARNING or higher
@ -54,19 +56,19 @@ log = logging.getLogger('werkzeug')
log.setLevel(logging.ERROR)
import time
from pyGpt4All.config import load_config, save_config
from pyGpt4All.api import GPT4AllAPI
from gpt4all_api.config import load_config, save_config
from gpt4all_api.api import GPT4AllAPI
import shutil
import markdown
class Gpt4AllWebUI(GPT4AllAPI):
def __init__(self, _app, _socketio, config:dict, personality:dict, config_file_path) -> None:
super().__init__(config, personality, config_file_path)
def __init__(self, _app, _socketio, config:dict, config_file_path) -> None:
super().__init__(config, _socketio, config_file_path)
self.app = _app
self.cancel_gen = False
self.socketio = _socketio
if "use_new_ui" in self.config:
if self.config["use_new_ui"]:
@ -199,121 +201,6 @@ class Gpt4AllWebUI(GPT4AllAPI):
"/get_current_personality", "get_current_personality", self.get_current_personality, methods=["GET"]
)
# =========================================================================================
# Socket IO stuff
# =========================================================================================
@socketio.on('connect')
def connect():
print('Client connected')
@socketio.on('disconnect')
def disconnect():
print('Client disconnected')
@socketio.on('install_model')
def install_model(data):
model_path = data["path"]
progress = 0
installation_dir = Path(f'./models/{self.config["backend"]}/')
filename = Path(model_path).name
installation_path = installation_dir / filename
print("Model install requested")
print(f"Model path : {model_path}")
if installation_path.exists():
print("Error: Model already exists")
data.installing = False
socketio.emit('install_progress',{'status': 'failed', 'error': 'model already exists'})
socketio.emit('install_progress',{'status': 'progress', 'progress': progress})
response = requests.get(model_path, stream=True)
file_size = int(response.headers.get('Content-Length'))
downloaded_size = 0
CHUNK_SIZE = 8192
def download_chunk(url, start_byte, end_byte, fileobj):
headers = {'Range': f'bytes={start_byte}-{end_byte}'}
response = requests.get(url, headers=headers, stream=True)
downloaded_bytes = 0
for chunk in response.iter_content(chunk_size=CHUNK_SIZE):
if chunk:
fileobj.seek(start_byte)
fileobj.write(chunk)
downloaded_bytes += len(chunk)
start_byte += len(chunk)
return downloaded_bytes
def download_file(url, file_path, num_threads=4):
response = requests.head(url)
file_size = int(response.headers.get('Content-Length'))
chunk_size = file_size // num_threads
progress = 0
with open(file_path, 'wb') as fileobj:
with tqdm(total=file_size, unit='B', unit_scale=True, unit_divisor=1024) as pbar:
with ThreadPoolExecutor(max_workers=num_threads) as executor:
futures = []
for i in range(num_threads):
start_byte = i * chunk_size
end_byte = start_byte + chunk_size - 1 if i < num_threads - 1 else file_size - 1
futures.append(executor.submit(download_chunk, url, start_byte, end_byte, fileobj))
for future in tqdm(as_completed(futures), total=num_threads):
downloaded_bytes = future.result()
progress += downloaded_bytes
pbar.update(downloaded_bytes)
socketio.emit('install_progress', {'status': 'progress', 'progress': progress})
# Usage example
download_file(model_path, installation_path, num_threads=4)
socketio.emit('install_progress',{'status': 'succeeded', 'error': ''})
@socketio.on('uninstall_model')
def uninstall_model(data):
model_path = data['path']
installation_dir = Path(f'./models/{self.config["backend"]}/')
filename = Path(model_path).name
installation_path = installation_dir / filename
if not installation_path.exists():
socketio.emit('install_progress',{'status': 'failed', 'error': ''})
installation_path.unlink()
socketio.emit('install_progress',{'status': 'succeeded', 'error': ''})
@socketio.on('generate_msg')
def generate_msg(data):
if self.current_discussion is None:
if self.db.does_last_discussion_have_messages():
self.current_discussion = self.db.create_discussion()
else:
self.current_discussion = self.db.load_last_discussion()
message = data["prompt"]
message_id = self.current_discussion.add_message(
"user", message, parent=self.message_id
)
self.current_user_message_id = message_id
tpe = threading.Thread(target=self.start_message_generation, args=(message, message_id))
tpe.start()
@socketio.on('generate_msg_from')
def handle_connection(data):
message_id = int(data['id'])
message = data["prompt"]
self.current_user_message_id = message_id
tpe = threading.Thread(target=self.start_message_generation, args=(message, message_id))
tpe.start()
def save_settings(self):
@ -405,9 +292,9 @@ class Gpt4AllWebUI(GPT4AllAPI):
elif setting_name== "model":
self.config["model"]=data['setting_value']
print("New model selected")
print("update_settings : New model selected")
# Build chatbot
self.chatbot_bindings = self.create_chatbot()
self.process.set_config(self.config)
elif setting_name== "backend":
print("New backend selected")
@ -415,13 +302,13 @@ class Gpt4AllWebUI(GPT4AllAPI):
print("New backend selected")
self.config["backend"]=data['setting_value']
backend_ =self.load_backend(self.BACKENDS_LIST[self.config["backend"]])
backend_ =self.process.rebuild_backend(self.config)
models = backend_.list_models(self.config)
if len(models)>0:
self.backend = backend_
self.config['model'] = models[0]
# Build chatbot
self.chatbot_bindings = self.create_chatbot()
self.process.set_config(self.config)
if self.config["debug"]:
print(f"Configuration {data['setting_name']} set to {data['setting_value']}")
return jsonify({'setting_name': data['setting_name'], "status":True})
@ -441,6 +328,9 @@ class Gpt4AllWebUI(GPT4AllAPI):
if self.config["debug"]:
print(f"Configuration {data['setting_name']} set to {data['setting_value']}")
print("Configuration updated")
self.process.set_config(self.config)
# Tell that the setting was changed
return jsonify({'setting_name': data['setting_name'], "status":True})
@ -549,70 +439,16 @@ class Gpt4AllWebUI(GPT4AllAPI):
return jsonify({"discussion_text":self.get_discussion_to()})
def start_message_generation(self, message, message_id):
bot_says = ""
# send the message to the bot
print(f"Received message : {message}")
if self.current_discussion:
# First we need to send the new message ID to the client
self.current_ai_message_id = self.current_discussion.add_message(
self.personality.name, "", parent = self.current_user_message_id
) # first the content is empty, but we'll fill it at the end
socketio.emit('infos',
{
"type": "input_message_infos",
"bot": self.personality.name,
"user": self.personality.user_name,
"message":message,#markdown.markdown(message),
"user_message_id": self.current_user_message_id,
"ai_message_id": self.current_ai_message_id,
}
)
# prepare query and reception
self.discussion_messages = self.prepare_query(message_id)
self.prepare_reception()
self.generating = True
# app.config['executor'] = ThreadPoolExecutor(max_workers=1)
# app.config['executor'].submit(self.generate_message)
print("## Generating message ##")
self.generate_message()
print()
print("## Done ##")
print()
# Send final message
self.socketio.emit('final', {
'data': self.bot_says,
'ai_message_id':self.current_ai_message_id,
'parent':self.current_user_message_id, 'discussion_id':self.current_discussion.discussion_id
}
)
self.current_discussion.update_message(self.current_ai_message_id, self.bot_says)
self.full_message_list.append(self.bot_says)
self.cancel_gen = False
return bot_says
else:
#No discussion available
print("No discussion selected!!!")
print("## Done ##")
print()
self.cancel_gen = False
return ""
def get_generation_status(self):
return jsonify({"status":self.generating})
return jsonify({"status":self.process.is_generating.value==1})
def stop_gen(self):
self.cancel_gen = True
print("Stop generation received")
return jsonify({"status": "ok"})
def rename(self):
data = request.get_json()
title = data["title"]
@ -680,9 +516,6 @@ class Gpt4AllWebUI(GPT4AllAPI):
def new_discussion(self):
title = request.args.get("title")
timestamp = self.create_new_discussion(title)
# app.config['executor'] = ThreadPoolExecutor(max_workers=1)
# app.config['executor'].submit(self.create_chatbot)
# target=self.create_chatbot()
# Return a success response
return json.dumps({"id": self.current_discussion.discussion_id, "time": timestamp, "welcome_message":self.personality.welcome_message, "sender":self.personality.name})
@ -694,13 +527,13 @@ class Gpt4AllWebUI(GPT4AllAPI):
print("New backend selected")
self.config['backend'] = backend
backend_ =self.load_backend(self.BACKENDS_LIST[self.config["backend"]])
backend_ =self.process.load_backend(Path("backends")/config["backend"])
models = backend_.list_models(self.config)
if len(models)>0:
self.backend = backend_
self.config['model'] = models[0]
# Build chatbot
self.chatbot_bindings = self.create_chatbot()
self.process.set_config(self.config)
return jsonify({"status": "ok"})
else:
return jsonify({"status": "no_models_found"})
@ -711,10 +544,10 @@ class Gpt4AllWebUI(GPT4AllAPI):
data = request.get_json()
model = str(data["model"])
if self.config['model']!= model:
print("New model selected")
print("set_model: New model selected")
self.config['model'] = model
# Build chatbot
self.chatbot_bindings = self.create_chatbot()
self.process.set_config(self.config)
return jsonify({"status": "ok"})
return jsonify({"status": "error"})
@ -728,11 +561,11 @@ class Gpt4AllWebUI(GPT4AllAPI):
personality = str(data["personality"])
if self.config['backend']!=backend or self.config['model'] != model:
print("New model selected")
print("update_model_params: New model selected")
self.config['backend'] = backend
self.config['model'] = model
self.create_chatbot()
self.process.set_config(self.config)
self.config['personality_language'] = personality_language
self.config['personality_category'] = personality_category
@ -740,7 +573,6 @@ class Gpt4AllWebUI(GPT4AllAPI):
personality_fn = f"personalities/{self.config['personality_language']}/{self.config['personality_category']}/{self.config['personality']}"
print(f"Loading personality : {personality_fn}")
self.personality = AIPersonality(personality_fn)
self.config['n_predict'] = int(data["nPredict"])
self.config['seed'] = int(data["seed"])
@ -756,6 +588,10 @@ class Gpt4AllWebUI(GPT4AllAPI):
save_config(self.config, self.config_file_path)
self.process.set_config(self.config)
# Fixed missing argument
self.backend = self.process.rebuild_backend(self.config)
print("==============================================")
print("Parameters changed to:")
print(f"\tBackend:{self.config['backend']}")
@ -778,14 +614,24 @@ class Gpt4AllWebUI(GPT4AllAPI):
def get_available_models(self):
response = requests.get(f'https://gpt4all.io/models/models.json')
model_list = response.json()
"""Get the available models
Returns:
_type_: _description_
"""
model_list = self.backend.get_available_models()
models = []
for model in model_list:
try:
filename = model['filename']
server = model['server']
filesize = model['filesize']
path = f'https://gpt4all.io/models/{filename}'
if server.endswith("/"):
path = f'{server}{filename}'
else:
path = f'{server}/{filename}'
local_path = Path(f'./models/{self.config["backend"]}/{filename}')
is_installed = local_path.exists()
models.append({
@ -796,6 +642,8 @@ class Gpt4AllWebUI(GPT4AllAPI):
'path': path,
'filesize': filesize,
})
except:
print(f"Problem with model : {model}")
return jsonify(models)
@ -915,17 +763,9 @@ if __name__ == "__main__":
if arg_value is not None:
config[arg_name] = arg_value
try:
personality_path = f"personalities/{config['personality_language']}/{config['personality_category']}/{config['personality']}"
personality = AIPersonality(personality_path)
except Exception as ex:
print("Personality file not found. Please verify that the personality you have selected exists or select another personality. Some updates may lead to change in personality name or category, so check the personality selection in settings to be sure.")
if config["debug"]:
print(ex)
personality = AIPersonality()
# executor = ThreadPoolExecutor(max_workers=1)
# app.config['executor'] = executor
bot = Gpt4AllWebUI(app, socketio, config, personality, config_file_path)
bot = Gpt4AllWebUI(app, socketio, config, config_file_path)
# chong Define custom WebSocketHandler with error handling
class CustomWebSocketHandler(WebSocketHandler):
@ -948,7 +788,3 @@ if __name__ == "__main__":
http_server = WSGIServer((config["host"], config["port"]), app, handler_class=WebSocketHandler)
http_server.serve_forever()
#if config["debug"]:
# app.run(debug=True, host=config["host"], port=config["port"])
#else:
# app.run(host=config["host"], port=config["port"])

76
backends/gpt_4all/__init__.py Normal file
View File

@ -0,0 +1,76 @@
######
# Project : GPT4ALL-UI
# File : backend.py
# Author : ParisNeo with the help of the community
# Supported by Nomic-AI
# Licence : Apache 2.0
# Description :
# This is an interface class for GPT4All-ui backends.
######
from pathlib import Path
from typing import Callable
from gpt4all import GPT4All
from gpt4all_api.backend import GPTBackend
import yaml
__author__ = "parisneo"
__github__ = "https://github.com/nomic-ai/gpt4all-ui"
__copyright__ = "Copyright 2023, "
__license__ = "Apache 2.0"
backend_name = "GPT4ALL"
class GPT4ALL(GPTBackend):
file_extension='*.bin'
def __init__(self, config:dict) -> None:
"""Builds a GPT4ALL backend
Args:
config (dict): The configuration file
"""
super().__init__(config, False)
self.model = GPT4All.get_model_from_name(self.config['model'])
self.model.load_model(
model_path=f"./models/gpt_4all/{self.config['model']}",
)
def generate(self,
prompt:str,
n_predict: int = 128,
new_text_callback: Callable[[str], None] = bool,
verbose: bool = False,
**gpt_params ):
"""Generates text out of a prompt
Args:
prompt (str): The prompt to use for generation
n_predict (int, optional): Number of tokens to prodict. Defaults to 128.
new_text_callback (Callable[[str], None], optional): A callback function that is called everytime a new text element is generated. Defaults to None.
verbose (bool, optional): If true, the code will spit many informations about the generation process. Defaults to False.
"""
try:
for tok in self.model.generate(prompt,
n_predict=n_predict,
temp=self.config['temperature'],
top_k=self.config['top_k'],
top_p=self.config['top_p'],
repeat_penalty=self.config['repeat_penalty'],
repeat_last_n = self.config['repeat_last_n'],
# n_threads=self.config['n_threads'],
):
if not new_text_callback(tok):
return
except Exception as ex:
print(ex)
@staticmethod
def get_available_models():
# Create the file path relative to the child class's directory
backend_path = Path(__file__).parent
file_path = backend_path/"models.yaml"
with open(file_path, 'r') as file:
yaml_data = yaml.safe_load(file)
return yaml_data

27
backends/gpt_4all/json2yaml.py Normal file
View File

@ -0,0 +1,27 @@
import json
import yaml
from pathlib import Path
import argparse
def json_to_yaml(json_file):
# Read JSON file
with open(json_file, 'r') as file:
json_data = json.load(file)
# Create YAML file path
yaml_file = Path(json_file).with_suffix('.yaml')
# Convert JSON to YAML
with open(yaml_file, 'w') as file:
yaml.dump(json_data, file)
print(f"Conversion complete. YAML file saved as: {yaml_file}")
if __name__ == "__main__":
# Parse command-line arguments
parser = argparse.ArgumentParser(description='Convert JSON file to YAML.')
parser.add_argument('json_file', help='Path to the JSON file')
args = parser.parse_args()
# Convert JSON to YAML
json_to_yaml(args.json_file)

13
zoos/gpt4all_models_zoo.json → backends/gpt_4all/models.json
View File

@ -2,6 +2,7 @@
{
"md5sum": "81a09a0ddf89690372fc296ff7f625af",
"filename": "ggml-gpt4all-j-v1.3-groovy.bin",
"server":"https://gpt4all.io/models/",
"filesize": "3785248281",
"isDefault": "true",
"bestGPTJ": "true",
@ -10,6 +11,7 @@
{
"md5sum": "91f886b68fbce697e9a3cd501951e455",
"filename": "ggml-gpt4all-l13b-snoozy.bin",
"server":"https://gpt4all.io/models/",
"filesize": "8136770688",
"bestLlama": "true",
"description": "Current best non-commercially licensable model based on Llama 13b and trained by Nomic AI on the latest curated GPT4All dataset."
@ -17,6 +19,7 @@
{
"md5sum": "756249d3d6abe23bde3b1ae272628640",
"filename": "ggml-mpt-7b-chat.bin",
"server":"https://gpt4all.io/models/",
"filesize": "4854401050",
"isDefault": "true",
"bestMPT": "true",
@ -26,48 +29,56 @@
{
"md5sum": "879344aaa9d62fdccbda0be7a09e7976",
"filename": "ggml-gpt4all-j-v1.2-jazzy.bin",
"server":"https://gpt4all.io/models/",
"filesize": "3785248281",
"description": "A commercially licensable model based on GPT-J and trained by Nomic AI on the v2 GPT4All dataset."
},
{
"md5sum": "61d48a82cb188cceb14ebb8082bfec37",
"filename": "ggml-gpt4all-j-v1.1-breezy.bin",
"server":"https://gpt4all.io/models/",
"filesize": "3785248281",
"description": "A commercially licensable model based on GPT-J and trained by Nomic AI on the v1 GPT4All dataset."
},
{
"md5sum": "5b5a3f9b858d33b29b52b89692415595",
"filename": "ggml-gpt4all-j.bin",
"server":"https://gpt4all.io/models/",
"filesize": "3785248281",
"description": "A commercially licensable model based on GPT-J and trained by Nomic AI on the v0 GPT4All dataset."
},
{
"md5sum": "29119f8fa11712704c6b22ac5ab792ea",
"filename": "ggml-vicuna-7b-1.1-q4_2.bin",
"server":"https://gpt4all.io/models/",
"filesize": "4212859520",
"description": "A non-commercially licensable model based on Llama 7b and trained by teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego."
},
{
"md5sum": "95999b7b0699e2070af63bf5d34101a8",
"filename": "ggml-vicuna-13b-1.1-q4_2.bin",
"server":"https://gpt4all.io/models/",
"filesize": "8136770688",
"description": "A non-commercially licensable model based on Llama 13b and trained by teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego."
},
{
"md5sum": "99e6d129745a3f1fb1121abed747b05a",
"filename": "ggml-wizardLM-7B.q4_2.bin",
"server":"https://gpt4all.io/models/",
"filesize": "4212864640",
"description": "A non-commercially licensable model based on Llama 7b and trained by Microsoft and Peking University."
},
{
"md5sum": "6cb4ee297537c9133bddab9692879de0",
"filename": "ggml-stable-vicuna-13B.q4_2.bin",
"server":"https://gpt4all.io/models/",
"filesize": "8136777088",
"description": "A non-commercially licensable model based on Llama 13b and RLHF trained by Stable AI."
},
{
"md5sum": "120c32a51d020066288df045ef5d52b9",
"filename": "ggml-mpt-7b-base.bin",
"server":"https://gpt4all.io/models/",
"filesize": "4854401028",
"requires": "2.4.1",
"description": "A commercially licensable model base pre-trained by Mosaic ML."
@ -75,12 +86,14 @@
{
"md5sum": "d5eafd5b0bd0d615cfd5fd763f642dfe",
"filename": "ggml-nous-gpt4-vicuna-13b.bin",
"server":"https://gpt4all.io/models/",
"filesize": "8136777088",
"description": "A non-commercially licensable model based on Vicuna 13b, fine-tuned on ~180,000 instructions, trained by Nous Research."
},
{
"md5sum": "1cfa4958f489f0a0d1ffdf6b37322809",
"filename": "ggml-mpt-7b-instruct.bin",
"server":"https://gpt4all.io/models/",
"filesize": "4854401028",
"requires": "2.4.1",
"description": "A commericially licensable instruct model based on MPT and trained by Mosaic ML."

85
backends/gpt_4all/models.yaml Normal file
View File

@ -0,0 +1,85 @@
- bestGPTJ: 'true'
description: Current best commercially licensable model based on GPT-J and trained
by Nomic AI on the latest curated GPT4All dataset.
filename: ggml-gpt4all-j-v1.3-groovy.bin
filesize: '3785248281'
isDefault: 'true'
md5sum: 81a09a0ddf89690372fc296ff7f625af
server: https://gpt4all.io/models/
- bestLlama: 'true'
description: Current best non-commercially licensable model based on Llama 13b and
trained by Nomic AI on the latest curated GPT4All dataset.
filename: ggml-gpt4all-l13b-snoozy.bin
filesize: '8136770688'
md5sum: 91f886b68fbce697e9a3cd501951e455
server: https://gpt4all.io/models/
- bestMPT: 'true'
description: Current best non-commercially licensable chat model based on MPT and
trained by Mosaic ML.
filename: ggml-mpt-7b-chat.bin
filesize: '4854401050'
isDefault: 'true'
md5sum: 756249d3d6abe23bde3b1ae272628640
requires: 2.4.1
server: https://gpt4all.io/models/
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v2 GPT4All dataset.
filename: ggml-gpt4all-j-v1.2-jazzy.bin
filesize: '3785248281'
md5sum: 879344aaa9d62fdccbda0be7a09e7976
server: https://gpt4all.io/models/
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v1 GPT4All dataset.
filename: ggml-gpt4all-j-v1.1-breezy.bin
filesize: '3785248281'
md5sum: 61d48a82cb188cceb14ebb8082bfec37
server: https://gpt4all.io/models/
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v0 GPT4All dataset.
filename: ggml-gpt4all-j.bin
filesize: '3785248281'
md5sum: 5b5a3f9b858d33b29b52b89692415595
server: https://gpt4all.io/models/
- description: A non-commercially licensable model based on Llama 7b and trained by
teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego.
filename: ggml-vicuna-7b-1.1-q4_2.bin
filesize: '4212859520'
md5sum: 29119f8fa11712704c6b22ac5ab792ea
server: https://gpt4all.io/models/
- description: A non-commercially licensable model based on Llama 13b and trained
by teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego.
filename: ggml-vicuna-13b-1.1-q4_2.bin
filesize: '8136770688'
md5sum: 95999b7b0699e2070af63bf5d34101a8
server: https://gpt4all.io/models/
- description: A non-commercially licensable model based on Llama 7b and trained by
Microsoft and Peking University.
filename: ggml-wizardLM-7B.q4_2.bin
filesize: '4212864640'
md5sum: 99e6d129745a3f1fb1121abed747b05a
server: https://gpt4all.io/models/
- description: A non-commercially licensable model based on Llama 13b and RLHF trained
by Stable AI.
filename: ggml-stable-vicuna-13B.q4_2.bin
filesize: '8136777088'
md5sum: 6cb4ee297537c9133bddab9692879de0
server: https://gpt4all.io/models/
- description: A commercially licensable model base pre-trained by Mosaic ML.
filename: ggml-mpt-7b-base.bin
filesize: '4854401028'
md5sum: 120c32a51d020066288df045ef5d52b9
requires: 2.4.1
server: https://gpt4all.io/models/
- description: A non-commercially licensable model based on Vicuna 13b, fine-tuned
on ~180,000 instructions, trained by Nous Research.
filename: ggml-nous-gpt4-vicuna-13b.bin
filesize: '8136777088'
md5sum: d5eafd5b0bd0d615cfd5fd763f642dfe
server: https://gpt4all.io/models/
- description: A commericially licensable instruct model based on MPT and trained
by Mosaic ML.
filename: ggml-mpt-7b-instruct.bin
filesize: '4854401028'
md5sum: 1cfa4958f489f0a0d1ffdf6b37322809
requires: 2.4.1
server: https://gpt4all.io/models/

24
backends/gpt4all/__init__.py → backends/gpt_j_a/__init__.py
View File

@ -2,6 +2,7 @@
# Project : GPT4ALL-UI
# File : backend.py
# Author : ParisNeo with the help of the community
# Underlying backend : Abdeladim's pygptj backend
# Supported by Nomic-AI
# Licence : Apache 2.0
# Description :
@ -9,20 +10,20 @@
######
from pathlib import Path
from typing import Callable
from pygpt4all import GPT4All as Model
from pyGpt4All.backend import GPTBackend
from pygptj.model import Model
from gpt4all_api.backend import GPTBackend
__author__ = "parisneo"
__github__ = "https://github.com/nomic-ai/gpt4all-ui"
__copyright__ = "Copyright 2023, "
__license__ = "Apache 2.0"
backend_name = "GPT4ALL"
backend_name = "GptJ"
class GPT4ALL(GPTBackend):
class GptJ(GPTBackend):
file_extension='*.bin'
def __init__(self, config:dict) -> None:
"""Builds a GPT4ALL backend
"""Builds a LLAMACPP backend
Args:
config (dict): The configuration file
@ -30,15 +31,10 @@ class GPT4ALL(GPTBackend):
super().__init__(config, False)
self.model = Model(
model_path=f"./models/gpt4all/{self.config['model']}",
prompt_context="", prompt_prefix="", prompt_suffix="",
n_ctx=self.config['ctx_size'],
seed=self.config['seed'],
model_path=f"./models/gpt_j/{self.config['model']}",
prompt_context="", prompt_prefix="", prompt_suffix=""
)
def stop_generation(self):
self.model._grab_text_callback()
def generate(self,
prompt:str,
n_predict: int = 128,
@ -60,8 +56,8 @@ class GPT4ALL(GPTBackend):
temp=self.config['temperature'],
top_k=self.config['top_k'],
top_p=self.config['top_p'],
repeat_penalty=self.config['repeat_penalty'],
repeat_last_n = self.config['repeat_last_n'],
#repeat_penalty=self.config['repeat_penalty'],
#repeat_last_n = self.config['repeat_last_n'],
n_threads=self.config['n_threads'],
):
if not new_text_callback(tok):

80
backends/gpt_j_m/__init__.py Normal file
View File

@ -0,0 +1,80 @@
######
# Project : GPT4ALL-UI
# File : backend.py
# Author : ParisNeo with the help of the community
# Underlying backend : Abdeladim's pygptj backend
# Supported by Nomic-AI
# Licence : Apache 2.0
# Description :
# This is an interface class for GPT4All-ui backends.
######
from pathlib import Path
from typing import Callable
from gpt4allj import Model
from gpt4all_api.backend import GPTBackend
import yaml
__author__ = "parisneo"
__github__ = "https://github.com/nomic-ai/gpt4all-ui"
__copyright__ = "Copyright 2023, "
__license__ = "Apache 2.0"
backend_name = "GPTJ"
class GPTJ(GPTBackend):
file_extension='*.bin'
def __init__(self, config:dict) -> None:
"""Builds a LLAMACPP backend
Args:
config (dict): The configuration file
"""
super().__init__(config, False)
self.model = Model(
model=f"./models/llama_cpp/{self.config['model']}", avx2 = self.config["use_avx2"]
)
def generate(self,
prompt:str,
n_predict: int = 128,
new_text_callback: Callable[[str], None] = bool,
verbose: bool = False,
**gpt_params ):
"""Generates text out of a prompt
Args:
prompt (str): The prompt to use for generation
n_predict (int, optional): Number of tokens to prodict. Defaults to 128.
new_text_callback (Callable[[str], None], optional): A callback function that is called everytime a new text element is generated. Defaults to None.
verbose (bool, optional): If true, the code will spit many informations about the generation process. Defaults to False.
"""
try:
self.model.reset()
for tok in self.model.generate(
prompt,
seed=self.config['seed'],
n_threads=self.config['n_threads'],
n_predict=n_predict,
top_k=self.config['top_k'],
top_p=self.config['top_p'],
temp=self.config['temperature'],
repeat_penalty=self.config['repeat_penalty'],
repeat_last_n=self.config['repeat_last_n'],
n_batch=8,
reset=True,
):
if not new_text_callback(tok):
return
except Exception as ex:
print(ex)
@staticmethod
def get_available_models():
# Create the file path relative to the child class's directory
backend_path = Path(__file__).parent
file_path = backend_path/"models.yaml"
with open(file_path, 'r') as file:
yaml_data = yaml.safe_load(file)
return yaml_data

72
backends/gpt_j_m/models.yaml Normal file
View File

@ -0,0 +1,72 @@
- bestGPTJ: 'true'
description: Current best commercially licensable model based on GPT-J and trained
by Nomic AI on the latest curated GPT4All dataset.
filename: ggml-gpt4all-j-v1.3-groovy.bin
filesize: '3785248281'
isDefault: 'true'
md5sum: 81a09a0ddf89690372fc296ff7f625af
- bestLlama: 'true'
description: Current best non-commercially licensable model based on Llama 13b and
trained by Nomic AI on the latest curated GPT4All dataset.
filename: ggml-gpt4all-l13b-snoozy.bin
filesize: '8136770688'
md5sum: 91f886b68fbce697e9a3cd501951e455
- bestMPT: 'true'
description: Current best non-commercially licensable chat model based on MPT and
trained by Mosaic ML.
filename: ggml-mpt-7b-chat.bin
filesize: '4854401050'
isDefault: 'true'
md5sum: 756249d3d6abe23bde3b1ae272628640
requires: 2.4.1
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v2 GPT4All dataset.
filename: ggml-gpt4all-j-v1.2-jazzy.bin
filesize: '3785248281'
md5sum: 879344aaa9d62fdccbda0be7a09e7976
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v1 GPT4All dataset.
filename: ggml-gpt4all-j-v1.1-breezy.bin
filesize: '3785248281'
md5sum: 61d48a82cb188cceb14ebb8082bfec37
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v0 GPT4All dataset.
filename: ggml-gpt4all-j.bin
filesize: '3785248281'
md5sum: 5b5a3f9b858d33b29b52b89692415595
- description: A non-commercially licensable model based on Llama 7b and trained by
teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego.
filename: ggml-vicuna-7b-1.1-q4_2.bin
filesize: '4212859520'
md5sum: 29119f8fa11712704c6b22ac5ab792ea
- description: A non-commercially licensable model based on Llama 13b and trained
by teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego.
filename: ggml-vicuna-13b-1.1-q4_2.bin
filesize: '8136770688'
md5sum: 95999b7b0699e2070af63bf5d34101a8
- description: A non-commercially licensable model based on Llama 7b and trained by
Microsoft and Peking University.
filename: ggml-wizardLM-7B.q4_2.bin
filesize: '4212864640'
md5sum: 99e6d129745a3f1fb1121abed747b05a
- description: A non-commercially licensable model based on Llama 13b and RLHF trained
by Stable AI.
filename: ggml-stable-vicuna-13B.q4_2.bin
filesize: '8136777088'
md5sum: 6cb4ee297537c9133bddab9692879de0
- description: A commercially licensable model base pre-trained by Mosaic ML.
filename: ggml-mpt-7b-base.bin
filesize: '4854401028'
md5sum: 120c32a51d020066288df045ef5d52b9
requires: 2.4.1
- description: A non-commercially licensable model based on Vicuna 13b, fine-tuned
on ~180,000 instructions, trained by Nous Research.
filename: ggml-nous-gpt4-vicuna-13b.bin
filesize: '8136777088'
md5sum: d5eafd5b0bd0d615cfd5fd763f642dfe
- description: A commericially licensable instruct model based on MPT and trained
by Mosaic ML.
filename: ggml-mpt-7b-instruct.bin
filesize: '4854401028'
md5sum: 1cfa4958f489f0a0d1ffdf6b37322809
requires: 2.4.1

95
backends/gptq/__init__.py Normal file
View File

@ -0,0 +1,95 @@
######
# Project : GPT4ALL-UI
# File : backend.py
# Author : ParisNeo with the help of the community
# Supported by Nomic-AI
# Licence : Apache 2.0
# Description :
# This is an interface class for GPT4All-ui backends.
######
from pathlib import Path
from typing import Callable
from transformers import AutoTokenizer, TextGenerationPipeline
from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig
from gpt4all_api.backend import GPTBackend
import torch
import yaml
__author__ = "parisneo"
__github__ = "https://github.com/ParisNeo/GPTQ_backend"
__copyright__ = "Copyright 2023, "
__license__ = "Apache 2.0"
backend_name = "GPTQ"
class GPTQ(GPTBackend):
file_extension='*'
def __init__(self, config:dict) -> None:
"""Builds a GPTQ backend
Args:
config (dict): The configuration file
"""
super().__init__(config, False)
self.model_dir = f'{config["model"]}'
# load quantized model, currently only support cpu or single gpu
self.model = AutoGPTQForCausalLM.from_pretrained(self.model_dir, BaseQuantizeConfig())
self.tokenizer = AutoTokenizer.from_pretrained(self.model_dir, use_fast=True )
def generate(self,
prompt:str,
n_predict: int = 128,
new_text_callback: Callable[[str], None] = bool,
verbose: bool = False,
**gpt_params ):
"""Generates text out of a prompt
Args:
prompt (str): The prompt to use for generation
n_predict (int, optional): Number of tokens to prodict. Defaults to 128.
new_text_callback (Callable[[str], None], optional): A callback function that is called everytime a new text element is generated. Defaults to None.
verbose (bool, optional): If true, the code will spit many informations about the generation process. Defaults to False.
"""
try:
tok = self.tokenizer.decode(self.model.generate(**self.tokenizer(prompt, return_tensors="pt").to("cuda:0"))[0])
new_text_callback(tok)
"""
self.model.reset()
for tok in self.model.generate(prompt,
n_predict=n_predict,
temp=self.config['temp'],
top_k=self.config['top_k'],
top_p=self.config['top_p'],
repeat_penalty=self.config['repeat_penalty'],
repeat_last_n = self.config['repeat_last_n'],
n_threads=self.config['n_threads'],
):
if not new_text_callback(tok):
return
"""
except Exception as ex:
print(ex)
@staticmethod
def list_models(config:dict):
"""Lists the models for this backend
"""
return [
"EleutherAI/gpt-j-6b",
"opt-125m-4bit"
"TheBloke/medalpaca-13B-GPTQ-4bit",
"TheBloke/stable-vicuna-13B-GPTQ",
]
@staticmethod
def get_available_models():
# Create the file path relative to the child class's directory
backend_path = Path(__file__).parent
file_path = backend_path/"models.yaml"
with open(file_path, 'r') as file:
yaml_data = yaml.safe_load(file)
return yaml_data

72
backends/gptq/models.yaml Normal file
View File

@ -0,0 +1,72 @@
- bestGPTJ: 'true'
description: Current best commercially licensable model based on GPT-J and trained
by Nomic AI on the latest curated GPT4All dataset.
filename: ggml-gpt4all-j-v1.3-groovy.bin
filesize: '3785248281'
isDefault: 'true'
md5sum: 81a09a0ddf89690372fc296ff7f625af
- bestLlama: 'true'
description: Current best non-commercially licensable model based on Llama 13b and
trained by Nomic AI on the latest curated GPT4All dataset.
filename: ggml-gpt4all-l13b-snoozy.bin
filesize: '8136770688'
md5sum: 91f886b68fbce697e9a3cd501951e455
- bestMPT: 'true'
description: Current best non-commercially licensable chat model based on MPT and
trained by Mosaic ML.
filename: ggml-mpt-7b-chat.bin
filesize: '4854401050'
isDefault: 'true'
md5sum: 756249d3d6abe23bde3b1ae272628640
requires: 2.4.1
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v2 GPT4All dataset.
filename: ggml-gpt4all-j-v1.2-jazzy.bin
filesize: '3785248281'
md5sum: 879344aaa9d62fdccbda0be7a09e7976
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v1 GPT4All dataset.
filename: ggml-gpt4all-j-v1.1-breezy.bin
filesize: '3785248281'
md5sum: 61d48a82cb188cceb14ebb8082bfec37
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v0 GPT4All dataset.
filename: ggml-gpt4all-j.bin
filesize: '3785248281'
md5sum: 5b5a3f9b858d33b29b52b89692415595
- description: A non-commercially licensable model based on Llama 7b and trained by
teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego.
filename: ggml-vicuna-7b-1.1-q4_2.bin
filesize: '4212859520'
md5sum: 29119f8fa11712704c6b22ac5ab792ea
- description: A non-commercially licensable model based on Llama 13b and trained
by teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego.
filename: ggml-vicuna-13b-1.1-q4_2.bin
filesize: '8136770688'
md5sum: 95999b7b0699e2070af63bf5d34101a8
- description: A non-commercially licensable model based on Llama 7b and trained by
Microsoft and Peking University.
filename: ggml-wizardLM-7B.q4_2.bin
filesize: '4212864640'
md5sum: 99e6d129745a3f1fb1121abed747b05a
- description: A non-commercially licensable model based on Llama 13b and RLHF trained
by Stable AI.
filename: ggml-stable-vicuna-13B.q4_2.bin
filesize: '8136777088'
md5sum: 6cb4ee297537c9133bddab9692879de0
- description: A commercially licensable model base pre-trained by Mosaic ML.
filename: ggml-mpt-7b-base.bin
filesize: '4854401028'
md5sum: 120c32a51d020066288df045ef5d52b9
requires: 2.4.1
- description: A non-commercially licensable model based on Vicuna 13b, fine-tuned
on ~180,000 instructions, trained by Nous Research.
filename: ggml-nous-gpt4-vicuna-13b.bin
filesize: '8136777088'
md5sum: d5eafd5b0bd0d615cfd5fd763f642dfe
- description: A commericially licensable instruct model based on MPT and trained
by Mosaic ML.
filename: ggml-mpt-7b-instruct.bin
filesize: '4854401028'
md5sum: 1cfa4958f489f0a0d1ffdf6b37322809
requires: 2.4.1

83
backends/hugging_face/__init__.py Normal file
View File

@ -0,0 +1,83 @@
######
# Project : GPT4ALL-UI
# File : backend.py
# Author : ParisNeo with the help of the community
# Supported by Nomic-AI
# Licence : Apache 2.0
# Description :
# This is an interface class for GPT4All-ui backends.
######
from pathlib import Path
from typing import Callable
from accelerate import init_empty_weights
from accelerate import load_checkpoint_and_dispatch
from transformers import AutoTokenizer
from transformers import AutoConfig, AutoModelForCausalLM
from gpt4all_api.backend import GPTBackend
import torch
__author__ = "parisneo"
__github__ = "https://github.com/ParisNeo/GPTQ_backend"
__copyright__ = "Copyright 2023, "
__license__ = "Apache 2.0"
backend_name = "HuggingFace"
class HuggingFace(GPTBackend):
file_extension='*'
def __init__(self, config:dict) -> None:
"""Builds a HuggingFace backend
Args:
config (dict): The configuration file
"""
super().__init__(config, False)
# load quantized model, currently only support cpu or single gpu
config_path = AutoConfig.from_pretrained(config["model"])
self.tokenizer = AutoTokenizer.from_pretrained(config["model"])
self.model = AutoModelForCausalLM.from_pretrained(config["model"], load_in_8bit=True, device_map='auto')
def generate(self,
prompt:str,
n_predict: int = 128,
new_text_callback: Callable[[str], None] = bool,
verbose: bool = False,
**gpt_params ):
"""Generates text out of a prompt
Args:
prompt (str): The prompt to use for generation
n_predict (int, optional): Number of tokens to prodict. Defaults to 128.
new_text_callback (Callable[[str], None], optional): A callback function that is called everytime a new text element is generated. Defaults to None.
verbose (bool, optional): If true, the code will spit many informations about the generation process. Defaults to False.
"""
try:
tok = self.tokenizer.decode(self.model.generate(**self.tokenizer(prompt, return_tensors="pt").to("cuda:0"))[0])
new_text_callback(tok)
"""
self.model.reset()
for tok in self.model.generate(prompt,
n_predict=n_predict,
temp=self.config['temp'],
top_k=self.config['top_k'],
top_p=self.config['top_p'],
repeat_penalty=self.config['repeat_penalty'],
repeat_last_n = self.config['repeat_last_n'],
n_threads=self.config['n_threads'],
):
if not new_text_callback(tok):
return
"""
except Exception as ex:
print(ex)
@staticmethod
def list_models(config:dict):
"""Lists the models for this backend
"""
return [
"EleutherAI/gpt-j-6B"
]

72
backends/hugging_face/models.yaml Normal file
View File

@ -0,0 +1,72 @@
- bestGPTJ: 'true'
description: Current best commercially licensable model based on GPT-J and trained
by Nomic AI on the latest curated GPT4All dataset.
filename: ggml-gpt4all-j-v1.3-groovy.bin
filesize: '3785248281'
isDefault: 'true'
md5sum: 81a09a0ddf89690372fc296ff7f625af
- bestLlama: 'true'
description: Current best non-commercially licensable model based on Llama 13b and
trained by Nomic AI on the latest curated GPT4All dataset.
filename: ggml-gpt4all-l13b-snoozy.bin
filesize: '8136770688'
md5sum: 91f886b68fbce697e9a3cd501951e455
- bestMPT: 'true'
description: Current best non-commercially licensable chat model based on MPT and
trained by Mosaic ML.
filename: ggml-mpt-7b-chat.bin
filesize: '4854401050'
isDefault: 'true'
md5sum: 756249d3d6abe23bde3b1ae272628640
requires: 2.4.1
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v2 GPT4All dataset.
filename: ggml-gpt4all-j-v1.2-jazzy.bin
filesize: '3785248281'
md5sum: 879344aaa9d62fdccbda0be7a09e7976
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v1 GPT4All dataset.
filename: ggml-gpt4all-j-v1.1-breezy.bin
filesize: '3785248281'
md5sum: 61d48a82cb188cceb14ebb8082bfec37
- description: A commercially licensable model based on GPT-J and trained by Nomic
AI on the v0 GPT4All dataset.
filename: ggml-gpt4all-j.bin
filesize: '3785248281'
md5sum: 5b5a3f9b858d33b29b52b89692415595
- description: A non-commercially licensable model based on Llama 7b and trained by
teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego.
filename: ggml-vicuna-7b-1.1-q4_2.bin
filesize: '4212859520'
md5sum: 29119f8fa11712704c6b22ac5ab792ea
- description: A non-commercially licensable model based on Llama 13b and trained
by teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego.
filename: ggml-vicuna-13b-1.1-q4_2.bin
filesize: '8136770688'
md5sum: 95999b7b0699e2070af63bf5d34101a8
- description: A non-commercially licensable model based on Llama 7b and trained by
Microsoft and Peking University.
filename: ggml-wizardLM-7B.q4_2.bin
filesize: '4212864640'
md5sum: 99e6d129745a3f1fb1121abed747b05a
- description: A non-commercially licensable model based on Llama 13b and RLHF trained
by Stable AI.
filename: ggml-stable-vicuna-13B.q4_2.bin
filesize: '8136777088'
md5sum: 6cb4ee297537c9133bddab9692879de0
- description: A commercially licensable model base pre-trained by Mosaic ML.
filename: ggml-mpt-7b-base.bin
filesize: '4854401028'
md5sum: 120c32a51d020066288df045ef5d52b9
requires: 2.4.1
- description: A non-commercially licensable model based on Vicuna 13b, fine-tuned
on ~180,000 instructions, trained by Nous Research.
filename: ggml-nous-gpt4-vicuna-13b.bin
filesize: '8136777088'
md5sum: d5eafd5b0bd0d615cfd5fd763f642dfe
- description: A commericially licensable instruct model based on MPT and trained
by Mosaic ML.
filename: ggml-mpt-7b-instruct.bin
filesize: '4854401028'
md5sum: 1cfa4958f489f0a0d1ffdf6b37322809
requires: 2.4.1

17
backends/llama_cpp/__init__.py
View File

@ -10,7 +10,8 @@
from pathlib import Path
from typing import Callable
from pyllamacpp.model import Model
from pyGpt4All.backend import GPTBackend
from gpt4all_api.backend import GPTBackend
import yaml
__author__ = "parisneo"
__github__ = "https://github.com/nomic-ai/gpt4all-ui"
@ -36,9 +37,6 @@ class LLAMACPP(GPTBackend):
seed=self.config['seed'],
)
def stop_generation(self):
self.model._grab_text_callback()
def generate(self,
prompt:str,
n_predict: int = 128,
@ -68,3 +66,14 @@ class LLAMACPP(GPTBackend):
return
except Exception as ex:
print(ex)
@staticmethod
def get_available_models():
# Create the file path relative to the child class's directory
backend_path = Path(__file__).parent
file_path = backend_path/"models.yaml"
with open(file_path, 'r') as file:
yaml_data = yaml.safe_load(file)
return yaml_data

47
backends/llama_cpp/models.yaml Normal file
View File

@ -0,0 +1,47 @@
- bestLlama: 'false'
description: The model who started it all
filename: gpt4all-lora-quantized-ggml.new.bin
md5sum: 91f886b68fbce697e9a3cd501951e455
server: https://huggingface.co/ParisNeo/GPT4All/resolve/main/
- bestLlama: 'false'
description: The model who started it all (uncensored version)
filename: gpt4all-lora-unfiltered-quantized.new.bin
md5sum: 91f886b68fbce697e9a3cd501951e455
server: https://huggingface.co/ParisNeo/GPT4All/resolve/main/
- bestLlama: 'true'
description: Current best non-commercially licensable model based on Llama 13b and
trained by Nomic AI on the latest curated GPT4All dataset.
filename: ggml-gpt4all-l13b-snoozy.bin
filesize: '8136770688'
md5sum: 91f886b68fbce697e9a3cd501951e455
server: https://gpt4all.io/models/
- description: A non-commercially licensable model based on Llama 7b and trained by
teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego.
filename: ggml-vicuna-7b-1.1-q4_2.bin
filesize: '4212859520'
md5sum: 29119f8fa11712704c6b22ac5ab792ea
server: https://gpt4all.io/models/
- description: A non-commercially licensable model based on Llama 13b and trained
by teams from UC Berkeley, CMU, Stanford, MBZUAI, and UC San Diego.
filename: ggml-vicuna-13b-1.1-q4_2.bin
filesize: '8136770688'
md5sum: 95999b7b0699e2070af63bf5d34101a8
server: https://gpt4all.io/models/
- description: A non-commercially licensable model based on Llama 7b and trained by
Microsoft and Peking University.
filename: ggml-wizardLM-7B.q4_2.bin
filesize: '4212864640'
md5sum: 99e6d129745a3f1fb1121abed747b05a
server: https://gpt4all.io/models/
- description: A non-commercially licensable model based on Llama 13b and RLHF trained
by Stable AI.
filename: ggml-stable-vicuna-13B.q4_2.bin
filesize: '8136777088'
md5sum: 6cb4ee297537c9133bddab9692879de0
server: https://gpt4all.io/models/
- description: A non-commercially licensable model based on Vicuna 13b, fine-tuned
on ~180,000 instructions, trained by Nous Research.
filename: ggml-nous-gpt4-vicuna-13b.bin
filesize: '8136777088'
md5sum: d5eafd5b0bd0d615cfd5fd763f642dfe
server: https://gpt4all.io/models/

2
configs/default.yaml
View File

@ -7,7 +7,7 @@ n_threads: 8
host: localhost
language: en-US
# Supported backends are llamacpp and gpt-j
backend: gpt4all
backend: llama_cpp
model: null
n_predict: 1024
nb_messages_to_remember: 5

0
pyGpt4All/__init__.py → gpt4all_api/__init__.py
View File

628
gpt4all_api/api.py Normal file
View File

@ -0,0 +1,628 @@
######
# Project : GPT4ALL-UI
# File : api.py
# Author : ParisNeo with the help of the community
# Supported by Nomic-AI
# Licence : Apache 2.0
# Description :
# A simple api to communicate with gpt4all-ui and its models.
######
import gc
import sys
from datetime import datetime
from gpt4all_api.db import DiscussionsDB
from pathlib import Path
import importlib
from pyaipersonality import AIPersonality
import multiprocessing as mp
import threading
import time
import requests
import urllib.request
from tqdm import tqdm
__author__ = "parisneo"
__github__ = "https://github.com/nomic-ai/gpt4all-ui"
__copyright__ = "Copyright 2023, "
__license__ = "Apache 2.0"
class ModelProcess:
def __init__(self, config=None):
self.config = config
self.generate_queue = mp.Queue()
self.generation_queue = mp.Queue()
self.cancel_queue = mp.Queue(maxsize=1)
self.clear_queue_queue = mp.Queue(maxsize=1)
self.set_config_queue = mp.Queue(maxsize=1)
self.started_queue = mp.Queue()
self.process = None
self.is_generating = mp.Value('i', 0)
self.model_ready = mp.Value('i', 0)
self.ready = False
def load_backend(self, backend_path):
# define the full absolute path to the module
absolute_path = backend_path.resolve()
# infer the module name from the file path
module_name = backend_path.stem
# use importlib to load the module from the file path
loader = importlib.machinery.SourceFileLoader(module_name, str(absolute_path/"__init__.py"))
backend_module = loader.load_module()
backend_class = getattr(backend_module, backend_module.backend_name)
return backend_class
def start(self):
if self.process is None:
self.process = mp.Process(target=self._run)
self.process.start()
def stop(self):
if self.process is not None:
self.generate_queue.put(None)
self.process.join()
self.process = None
def set_backend(self, backend_path):
self.backend = backend_path
def set_model(self, model_path):
self.model = model_path
def set_config(self, config):
self.set_config_queue.put(config)
def generate(self, prompt, id, n_predict):
self.generate_queue.put((prompt, id, n_predict))
def cancel_generation(self):
self.cancel_queue.put(('cancel',))
def clear_queue(self):
self.clear_queue_queue.put(('clear_queue',))
def rebuild_backend(self, config):
try:
backend = self.load_backend(Path("backends")/config["backend"])
print("Backend loaded successfully")
except Exception as ex:
print("Couldn't build backend")
print(ex)
backend = None
return backend
def _rebuild_model(self):
try:
print("Rebuilding model")
self.backend = self.load_backend(Path("backends")/self.config["backend"])
print("Backend loaded successfully")
try:
model_file = Path("models")/self.config["backend"]/self.config["model"]
print(f"Loading model : {model_file}")
self.model = self.backend(self.config)
self.model_ready.value = 1
print("Model created successfully")
except Exception as ex:
print("Couldn't build model")
print(ex)
self.model = None
except Exception as ex:
print("Couldn't build backend")
print(ex)
self.backend = None
self.model = None
def rebuild_personality(self):
try:
personality_path = f"personalities/{self.config['personality_language']}/{self.config['personality_category']}/{self.config['personality']}"
personality = AIPersonality(personality_path)
except Exception as ex:
print("Personality file not found. Please verify that the personality you have selected exists or select another personality. Some updates may lead to change in personality name or category, so check the personality selection in settings to be sure.")
if self.config["debug"]:
print(ex)
personality = AIPersonality()
return personality
def _rebuild_personality(self):
try:
personality_path = f"personalities/{self.config['personality_language']}/{self.config['personality_category']}/{self.config['personality']}"
self.personality = AIPersonality(personality_path)
except Exception as ex:
print("Personality file not found. Please verify that the personality you have selected exists or select another personality. Some updates may lead to change in personality name or category, so check the personality selection in settings to be sure.")
if self.config["debug"]:
print(ex)
self.personality = AIPersonality()
def _run(self):
self._rebuild_model()
self._rebuild_personality()
if self.model_ready.value == 1:
self._generate("I",0,1)
print()
print("Ready to receive data")
else:
print("No model loaded. Waiting for new configuration instructions")
self.ready = True
print(f"Listening on :http://{self.config['host']}:{self.config['port']}")
while True:
try:
self._check_set_config_queue()
self._check_cancel_queue()
self._check_clear_queue()
if not self.generate_queue.empty():
command = self.generate_queue.get()
if command is None:
break
if self.cancel_queue.empty() and self.clear_queue_queue.empty():
self.is_generating.value = 1
self.started_queue.put(1)
self._generate(*command)
while not self.generation_queue.empty():
time.sleep(1)
self.is_generating.value = 0
time.sleep(1)
except Exception as ex:
time.sleep(1)
print(ex)
def _generate(self, prompt, id, n_predict):
if self.model is not None:
self.id = id
if self.config["override_personality_model_parameters"]:
self.model.generate(
prompt,
new_text_callback=self._callback,
n_predict=n_predict,
temp=self.config['temperature'],
top_k=self.config['top_k'],
top_p=self.config['top_p'],
repeat_penalty=self.config['repeat_penalty'],
repeat_last_n = self.config['repeat_last_n'],
seed=self.config['seed'],
n_threads=self.config['n_threads']
)
else:
self.model.generate(
prompt,
new_text_callback=self._callback,
n_predict=n_predict,
temp=self.personality.model_temperature,
top_k=self.personality.model_top_k,
top_p=self.personality.model_top_p,
repeat_penalty=self.personality.model_repeat_penalty,
repeat_last_n = self.personality.model_repeat_last_n,
#seed=self.config['seed'],
n_threads=self.config['n_threads']
)
else:
print("No model is installed or selected. Please make sure to install a model and select it inside your configuration before attempting to communicate with the model.")
print("To do this: Install the model to your models/<backend name> folder.")
print("Then set your model information in your local configuration file that you can find in configs/local_default.yaml")
print("You can also use the ui to set your model in the settings page.")
def _callback(self, text):
if not self.ready:
print(".",end="")
sys.stdout.flush()
return True
else:
# Stream the generated text to the main process
self.generation_queue.put((text,self.id))
self._check_set_config_queue()
self._check_cancel_queue()
self._check_clear_queue()
# if stop generation is detected then stop
if self.is_generating.value==1:
return True
else:
return False
def _check_cancel_queue(self):
while not self.cancel_queue.empty():
command = self.cancel_queue.get()
if command is not None:
self._cancel_generation()
def _check_clear_queue(self):
while not self.clear_queue_queue.empty():
command = self.clear_queue_queue.get()
if command is not None:
self._clear_queue()
def _check_set_config_queue(self):
while not self.set_config_queue.empty():
config = self.set_config_queue.get()
if config is not None:
self._set_config(config)
def _cancel_generation(self):
self.is_generating.value = 0
def _clear_queue(self):
while not self.generate_queue.empty():
self.generate_queue.get()
def _set_config(self, config):
bk_cfg = self.config
self.config = config
print("Changing configuration")
# verify that the backend is the same
if self.config["backend"]!=bk_cfg["backend"] or self.config["model"]!=bk_cfg["model"]:
self._rebuild_model()
# verify that the personality is the same
if self.config["personality"]!=bk_cfg["personality"] or self.config["personality_category"]!=bk_cfg["personality_category"] or self.config["personality_language"]!=bk_cfg["personality_language"]:
self._rebuild_personality()
class GPT4AllAPI():
def __init__(self, config:dict, socketio, config_file_path:str) -> None:
self.socketio = socketio
#Create and launch the process
self.process = ModelProcess(config)
self.process.start()
self.config = config
self.backend = self.process.rebuild_backend(self.config)
self.personality = self.process.rebuild_personality()
if config["debug"]:
print(print(f"{self.personality}"))
self.config_file_path = config_file_path
self.cancel_gen = False
# Keeping track of current discussion and message
self.current_discussion = None
self._current_user_message_id = 0
self._current_ai_message_id = 0
self._message_id = 0
self.db_path = config["db_path"]
# Create database object
self.db = DiscussionsDB(self.db_path)
# If the database is empty, populate it with tables
self.db.populate()
# This is used to keep track of messages
self.full_message_list = []
# =========================================================================================
# Socket IO stuff
# =========================================================================================
@socketio.on('connect')
def connect():
print('Client connected')
@socketio.on('disconnect')
def disconnect():
print('Client disconnected')
@socketio.on('install_model')
def install_model(data):
def install_model_():
print("Install model triggered")
model_path = data["path"]
progress = 0
installation_dir = Path(f'./models/{self.config["backend"]}/')
filename = Path(model_path).name
installation_path = installation_dir / filename
print("Model install requested")
print(f"Model path : {model_path}")
if installation_path.exists():
print("Error: Model already exists")
socketio.emit('install_progress',{'status': 'failed', 'error': 'model already exists'})
socketio.emit('install_progress',{'status': 'progress', 'progress': progress})
def callback(progress):
socketio.emit('install_progress',{'status': 'progress', 'progress': progress})
self.download_file(model_path, installation_path, callback)
socketio.emit('install_progress',{'status': 'succeeded', 'error': ''})
tpe = threading.Thread(target=install_model_, args=())
tpe.start()
@socketio.on('uninstall_model')
def uninstall_model(data):
model_path = data['path']
installation_dir = Path(f'./models/{self.config["backend"]}/')
filename = Path(model_path).name
installation_path = installation_dir / filename
if not installation_path.exists():
socketio.emit('install_progress',{'status': 'failed', 'error': 'The model does not exist'})
installation_path.unlink()
socketio.emit('install_progress',{'status': 'succeeded', 'error': ''})
@socketio.on('generate_msg')
def generate_msg(data):
if self.process.model_ready.value==1:
if self.current_discussion is None:
if self.db.does_last_discussion_have_messages():
self.current_discussion = self.db.create_discussion()
else:
self.current_discussion = self.db.load_last_discussion()
message = data["prompt"]
message_id = self.current_discussion.add_message(
"user", message, parent=self.message_id
)
self.current_user_message_id = message_id
tpe = threading.Thread(target=self.start_message_generation, args=(message, message_id))
tpe.start()
else:
self.socketio.emit('infos',
{
"status":'model_not_ready',
"type": "input_message_infos",
"bot": self.personality.name,
"user": self.personality.user_name,
"message":"",
"user_message_id": self.current_user_message_id,
"ai_message_id": self.current_ai_message_id,
}
)
@socketio.on('generate_msg_from')
def handle_connection(data):
message_id = int(data['id'])
message = data["prompt"]
self.current_user_message_id = message_id
tpe = threading.Thread(target=self.start_message_generation, args=(message, message_id))
tpe.start()
# generation status
self.generating=False
#properties
@property
def message_id(self):
return self._message_id
@property
def current_user_message_id(self):
return self._current_user_message_id
@current_user_message_id.setter
def current_user_message_id(self, id):
self._current_user_message_id=id
self._message_id = id
@property
def current_ai_message_id(self):
return self._current_ai_message_id
@current_ai_message_id.setter
def current_ai_message_id(self, id):
self._current_ai_message_id=id
self._message_id = id
def download_file(self, url, installation_path, callback=None):
"""
Downloads a file from a URL, reports the download progress using a callback function, and displays a progress bar.
Args:
url (str): The URL of the file to download.
installation_path (str): The path where the file should be saved.
callback (function, optional): A callback function to be called during the download
with the progress percentage as an argument. Defaults to None.
"""
try:
response = requests.get(url, stream=True)
# Get the file size from the response headers
total_size = int(response.headers.get('content-length', 0))
with open(installation_path, 'wb') as file:
downloaded_size = 0
with tqdm(total=total_size, unit='B', unit_scale=True, ncols=80) as progress_bar:
for chunk in response.iter_content(chunk_size=8192):
if chunk:
file.write(chunk)
downloaded_size += len(chunk)
if callback is not None:
percentage = (downloaded_size / total_size) * 100
callback(percentage)
progress_bar.update(len(chunk))
if callback is not None:
callback(100.0)
print("File downloaded successfully")
except Exception as e:
print("Couldn't download file:", str(e))
def load_backend(self, backend_path):
# define the full absolute path to the module
absolute_path = backend_path.resolve()
# infer the module name from the file path
module_name = backend_path.stem
# use importlib to load the module from the file path
loader = importlib.machinery.SourceFileLoader(module_name, str(absolute_path/"__init__.py"))
backend_module = loader.load_module()
backend_class = getattr(backend_module, backend_module.backend_name)
return backend_class
def condition_chatbot(self):
if self.current_discussion is None:
self.current_discussion = self.db.load_last_discussion()
if self.personality.welcome_message!="":
message_id = self.current_discussion.add_message(
self.personality.name, self.personality.welcome_message,
DiscussionsDB.MSG_TYPE_NORMAL,
0,
-1
)
self.current_ai_message_id = message_id
return message_id
def prepare_reception(self):
self.bot_says = ""
self.full_text = ""
self.is_bot_text_started = False
def create_new_discussion(self, title):
self.current_discussion = self.db.create_discussion(title)
# Get the current timestamp
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
# Chatbot conditionning
self.condition_chatbot()
return timestamp
def prepare_query(self, message_id=-1):
messages = self.current_discussion.get_messages()
self.full_message_list = []
for message in messages:
if message["id"]<= message_id or message_id==-1:
if message["type"]==self.db.MSG_TYPE_NORMAL:
if message["sender"]==self.personality.name:
self.full_message_list.append(self.personality.ai_message_prefix+message["content"])
else:
self.full_message_list.append(self.personality.user_message_prefix + message["content"])
link_text = self.personality.link_text
if len(self.full_message_list) > self.config["nb_messages_to_remember"]:
discussion_messages = self.personality.personality_conditioning+ link_text.join(self.full_message_list[-self.config["nb_messages_to_remember"]:])
else:
discussion_messages = self.personality.personality_conditioning+ link_text.join(self.full_message_list)
discussion_messages += link_text + self.personality.ai_message_prefix
return discussion_messages # Removes the last return
def get_discussion_to(self, message_id=-1):
messages = self.current_discussion.get_messages()
self.full_message_list = []
for message in messages:
if message["id"]<= message_id or message_id==-1:
if message["type"]!=self.db.MSG_TYPE_CONDITIONNING:
if message["sender"]==self.personality.name:
self.full_message_list.append(self.personality.ai_message_prefix+message["content"])
else:
self.full_message_list.append(self.personality.user_message_prefix + message["content"])
link_text = self.personality.link_text
if len(self.full_message_list) > self.config["nb_messages_to_remember"]:
discussion_messages = self.personality.personality_conditioning+ link_text.join(self.full_message_list[-self.config["nb_messages_to_remember"]:])
else:
discussion_messages = self.personality.personality_conditioning+ link_text.join(self.full_message_list)
return discussion_messages # Removes the last return
def remove_text_from_string(self, string, text_to_find):
"""
Removes everything from the first occurrence of the specified text in the string (case-insensitive).
Parameters:
string (str): The original string.
text_to_find (str): The text to find in the string.
Returns:
str: The updated string.
"""
index = string.lower().find(text_to_find.lower())
if index != -1:
string = string[:index]
return string
def process_chunk(self, chunk):
print(chunk[0],end="")
sys.stdout.flush()
self.bot_says += chunk[0]
if not self.personality.detect_antiprompt(self.bot_says):
self.socketio.emit('message', {
'data': self.bot_says,
'user_message_id':self.current_user_message_id,
'ai_message_id':self.current_ai_message_id,
'discussion_id':self.current_discussion.discussion_id
}
)
if self.cancel_gen:
print("Generation canceled")
self.process.cancel_generation()
self.cancel_gen = False
else:
self.bot_says = self.remove_text_from_string(self.bot_says, self.personality.user_message_prefix.strip())
self.process.cancel_generation()
print("The model is halucinating")
def start_message_generation(self, message, message_id):
bot_says = ""
# send the message to the bot
print(f"Received message : {message}")
if self.current_discussion:
# First we need to send the new message ID to the client
self.current_ai_message_id = self.current_discussion.add_message(
self.personality.name, "", parent = self.current_user_message_id
) # first the content is empty, but we'll fill it at the end
self.socketio.emit('infos',
{
"status":'generation_started',
"type": "input_message_infos",
"bot": self.personality.name,
"user": self.personality.user_name,
"message":message,#markdown.markdown(message),
"user_message_id": self.current_user_message_id,
"ai_message_id": self.current_ai_message_id,
}
)
# prepare query and reception
self.discussion_messages = self.prepare_query(message_id)
self.prepare_reception()
self.generating = True
print("## Generating message ##")
self.process.generate(self.discussion_messages, message_id, n_predict = self.config['n_predict'])
self.process.started_queue.get()
while(self.process.is_generating.value): # Simulating other commands being issued
while not self.process.generation_queue.empty():
self.process_chunk(self.process.generation_queue.get())
print()
print("## Done ##")
print()
# Send final message
self.socketio.emit('final', {
'data': self.bot_says,
'ai_message_id':self.current_ai_message_id,
'parent':self.current_user_message_id, 'discussion_id':self.current_discussion.discussion_id
}
)
self.current_discussion.update_message(self.current_ai_message_id, self.bot_says)
self.full_message_list.append(self.bot_says)
self.cancel_gen = False
return bot_says
else:
#No discussion available
print("No discussion selected!!!")
print("## Done ##")
print()
self.cancel_gen = False
return ""

14
pyGpt4All/backend.py → gpt4all_api/backend.py
View File

@ -9,6 +9,9 @@
######
from pathlib import Path
from typing import Callable
import inspect
import yaml
import sys
__author__ = "parisneo"
__github__ = "https://github.com/nomic-ai/gpt4all-ui"
@ -18,6 +21,7 @@ __license__ = "Apache 2.0"
class GPTBackend:
file_extension='*.bin'
backend_path = Path(__file__).parent
def __init__(self, config:dict, inline:bool) -> None:
self.config = config
self.inline = inline
@ -45,3 +49,13 @@ class GPTBackend:
"""
models_dir = Path('./models')/config["backend"] # replace with the actual path to the models folder
return [f.name for f in models_dir.glob(GPTBackend.file_extension)]
@staticmethod
def get_available_models():
# Create the file path relative to the child class's directory
backend_path = Path(__file__).parent
file_path = backend_path/"models.yaml"
with open(file_path, 'r') as file:
yaml_data = yaml.safe_load(file)
return yaml_data

0
pyGpt4All/config.py → gpt4all_api/config.py
View File

0
pyGpt4All/db.py → gpt4all_api/db.py
View File

0
pyGpt4All/extension.py → gpt4all_api/extension.py
View File

3
installations/add_personality.sh Normal file → Executable file
View File

@ -39,7 +39,8 @@ read -p "Enter the number of the desired personality: " SELECTED_PERSONALITY
PERSONALITY_FOLDER="$PERSONALITIES_FOLDER/${PERSONALITIES[$SELECTED_PERSONALITY]}"
# Copy the selected personality folder to personalities/language/category folder
OUTPUT_FOLDER="$(pwd)/personalities/${LANGUAGES[$SELECTED_LANGUAGE]}/${CATEGORIES[$SELECTED_CATEGORY]}/${PERSONALITIES[$SELECTED_PERSONALITY]}"
CORRECTED_PATH="$(pwd)/.."
OUTPUT_FOLDER="$CORRECTED_PATH/personalities/${LANGUAGES[$SELECTED_LANGUAGE]}/${CATEGORIES[$SELECTED_CATEGORY]}/${PERSONALITIES[$SELECTED_PERSONALITY]}"
mkdir -p "$OUTPUT_FOLDER"
cp -r "$PERSONALITY_FOLDER/." "$OUTPUT_FOLDER"

0
models/gpt_4all/.keep Normal file
View File

258
pyGpt4All/api.py
View File

@ -1,258 +0,0 @@
######
# Project : GPT4ALL-UI
# File : api.py
# Author : ParisNeo with the help of the community
# Supported by Nomic-AI
# Licence : Apache 2.0
# Description :
# A simple api to communicate with gpt4all-ui and its models.
######
import gc
import sys
from datetime import datetime
from pyGpt4All.db import DiscussionsDB
from pathlib import Path
import importlib
from pyaipersonality import AIPersonality
__author__ = "parisneo"
__github__ = "https://github.com/nomic-ai/gpt4all-ui"
__copyright__ = "Copyright 2023, "
__license__ = "Apache 2.0"
class GPT4AllAPI():
def __init__(self, config:dict, personality:AIPersonality, config_file_path:str) -> None:
self.config = config
self.personality = personality
if config["debug"]:
print(print(f"{personality}"))
self.config_file_path = config_file_path
self.cancel_gen = False
# Keeping track of current discussion and message
self.current_discussion = None
self._current_user_message_id = 0
self._current_ai_message_id = 0
self._message_id = 0
self.db_path = config["db_path"]
# Create database object
self.db = DiscussionsDB(self.db_path)
# If the database is empty, populate it with tables
self.db.populate()
# This is used to keep track of messages
self.full_message_list = []
# Select backend
self.BACKENDS_LIST = {f.stem:f for f in Path("backends").iterdir() if f.is_dir() and f.stem!="__pycache__"}
if self.config["backend"] is None:
self.backend = "gpt4all"
self.backend = self.load_backend(self.BACKENDS_LIST[self.config["backend"]])
else:
try:
self.backend = self.load_backend(self.BACKENDS_LIST[self.config["backend"]])
# Build chatbot
self.chatbot_bindings = self.create_chatbot()
print("Chatbot created successfully")
except Exception as ex:
self.config["backend"] = "gpt4all"
self.backend = self.load_backend(self.BACKENDS_LIST[self.config["backend"]])
self.config["model"] = None
print("No Models found, please select a backend and download a model for this tool to work")
# generation status
self.generating=False
#properties
@property
def message_id(self):
return self._message_id
@property
def current_user_message_id(self):
return self._current_user_message_id
@current_user_message_id.setter
def current_user_message_id(self, id):
self._current_user_message_id=id
self._message_id = id
@property
def current_ai_message_id(self):
return self._current_ai_message_id
@current_ai_message_id.setter
def current_ai_message_id(self, id):
self._current_ai_message_id=id
self._message_id = id
def load_backend(self, backend_path):
# define the full absolute path to the module
absolute_path = backend_path.resolve()
# infer the module name from the file path
module_name = backend_path.stem
# use importlib to load the module from the file path
loader = importlib.machinery.SourceFileLoader(module_name, str(absolute_path/"__init__.py"))
backend_module = loader.load_module()
backend_class = getattr(backend_module, backend_module.backend_name)
return backend_class
def create_chatbot(self):
return self.backend(self.config)
def condition_chatbot(self, conditionning_message):
if self.current_discussion is None:
self.current_discussion = self.db.load_last_discussion()
if self.personality.welcome_message!="":
message_id = self.current_discussion.add_message(
self.personality.name, self.personality.welcome_message,
DiscussionsDB.MSG_TYPE_NORMAL,
0,
-1
)
self.current_ai_message_id = message_id
return message_id
def prepare_reception(self):
self.bot_says = ""
self.full_text = ""
self.is_bot_text_started = False
def create_new_discussion(self, title):
self.current_discussion = self.db.create_discussion(title)
# Get the current timestamp
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
# Chatbot conditionning
self.condition_chatbot(self.personality.personality_conditioning)
return timestamp
def prepare_query(self, message_id=-1):
messages = self.current_discussion.get_messages()
self.full_message_list = []
for message in messages:
if message["id"]<= message_id or message_id==-1:
if message["type"]==self.db.MSG_TYPE_NORMAL:
if message["sender"]==self.personality.name:
self.full_message_list.append(self.personality.ai_message_prefix+message["content"])
else:
self.full_message_list.append(self.personality.user_message_prefix + message["content"])
link_text = self.personality.link_text
if len(self.full_message_list) > self.config["nb_messages_to_remember"]:
discussion_messages = self.personality.personality_conditioning+ link_text.join(self.full_message_list[-self.config["nb_messages_to_remember"]:])
else:
discussion_messages = self.personality.personality_conditioning+ link_text.join(self.full_message_list)
discussion_messages += link_text + self.personality.ai_message_prefix
return discussion_messages # Removes the last return
def get_discussion_to(self, message_id=-1):
messages = self.current_discussion.get_messages()
self.full_message_list = []
for message in messages:
if message["id"]<= message_id or message_id==-1:
if message["type"]!=self.db.MSG_TYPE_CONDITIONNING:
if message["sender"]==self.personality.name:
self.full_message_list.append(self.personality.ai_message_prefix+message["content"])
else:
self.full_message_list.append(self.personality.user_message_prefix + message["content"])
link_text = self.personality.link_text
if len(self.full_message_list) > self.config["nb_messages_to_remember"]:
discussion_messages = self.personality.personality_conditioning+ link_text.join(self.full_message_list[-self.config["nb_messages_to_remember"]:])
else:
discussion_messages = self.personality.personality_conditioning+ link_text.join(self.full_message_list)
return discussion_messages # Removes the last return
def remove_text_from_string(self, string, text_to_find):
"""
Removes everything from the first occurrence of the specified text in the string (case-insensitive).
Parameters:
string (str): The original string.
text_to_find (str): The text to find in the string.
Returns:
str: The updated string.
"""
index = string.lower().find(text_to_find.lower())
if index != -1:
string = string[:index]
return string
def new_text_callback(self, text: str):
if self.cancel_gen:
return False
print(text, end="")
sys.stdout.flush()
self.bot_says += text
if not self.personality.detect_antiprompt(self.bot_says):
self.socketio.emit('message', {
'data': self.bot_says,
'user_message_id':self.current_user_message_id,
'ai_message_id':self.current_ai_message_id,
'discussion_id':self.current_discussion.discussion_id
}
)
if self.cancel_gen:
print("Generation canceled")
self.cancel_gen = False
return False
else:
return True
else:
self.bot_says = self.remove_text_from_string(self.bot_says, self.personality.user_message_prefix.strip())
print("The model is halucinating")
return False
def generate_message(self):
self.generating=True
gc.collect()
total_n_predict = self.config['n_predict']
print(f"Generating {total_n_predict} outputs... ")
print(f"Input text :\n{self.discussion_messages}")
if self.config["override_personality_model_parameters"]:
self.chatbot_bindings.generate(
self.discussion_messages,
new_text_callback=self.new_text_callback,
n_predict=total_n_predict,
temp=self.config['temperature'],
top_k=self.config['top_k'],
top_p=self.config['top_p'],
repeat_penalty=self.config['repeat_penalty'],
repeat_last_n = self.config['repeat_last_n'],
seed=self.config['seed'],
n_threads=self.config['n_threads']
)
else:
self.chatbot_bindings.generate(
self.discussion_messages,
new_text_callback=self.new_text_callback,
n_predict=total_n_predict,
temp=self.personality.model_temperature,
top_k=self.personality.model_top_k,
top_p=self.personality.model_top_p,
repeat_penalty=self.personality.model_repeat_penalty,
repeat_last_n = self.personality.model_repeat_last_n,
#seed=self.config['seed'],
n_threads=self.config['n_threads']
)
self.generating=False

2
requirements.txt
View File

@ -7,7 +7,7 @@ markdown
pyllamacpp==2.1.1
gpt4all-j
pygptj
pygpt4all
gpt4all
--find-links https://download.pytorch.org/whl/cu117
torch==2.0.0
torchvision

2
requirements_dev.txt
View File

@ -6,6 +6,6 @@ pyyaml
markdown
pyllamacpp==2.0.0
gpt4all-j
pygpt4all
gpt4all
transformers
pyaipersonality>=0.0.11

12
static/js/websocket.js
View File

@ -14,7 +14,7 @@ var globals={
waitAnimation:undefined
}
var socket = io.connect('http://' + document.domain + ':' + location.port);
var socket = io.connect(location.protocol + '//' + document.domain + ':' + location.port);
socket.on('connect', function() {
});
@ -22,6 +22,8 @@ socket.on('disconnect', function() {
console.log("Disconnected")
});
socket.on('infos', function(msg) {
console.log(msg)
if(msg["status"]=="generation_started"){
if(globals.user_msg){
globals.user_msg.setSender(msg.user);
globals.user_msg.setMessage(msg.message);
@ -30,6 +32,14 @@ if(globals.user_msg){
globals.bot_msg.setSender(msg.bot);
globals.bot_msg.setID(msg.ai_message_id);
globals.bot_msg.messageTextElement.innerHTML = `Generating answer. Please stand by...`;
}
else{
globals.sendbtn.style.display="block";
globals.waitAnimation.style.display="none";
globals.stopGeneration.style.display = "none";
globals.is_generating = false
alert("It seems that no model has been loaded. Please download and install a model first, then try again.");
}
});
socket.on('waiter', function(msg) {

1
web/dist/assets/index-0f4879e0.css vendored Normal file
View File

File diff suppressed because one or more lines are too long

1
web/dist/assets/index-1c281352.css vendored
View File

File diff suppressed because one or more lines are too long

36
web/dist/assets/index-45c321b0.js vendored Normal file
View File

File diff suppressed because one or more lines are too long

36
web/dist/assets/index-a948d86d.js vendored
View File

File diff suppressed because one or more lines are too long

4
web/dist/index.html vendored
View File

@ -6,8 +6,8 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>GPT4All - WEBUI</title>
<script type="module" crossorigin src="/assets/index-a948d86d.js"></script>
<link rel="stylesheet" href="/assets/index-1c281352.css">
<script type="module" crossorigin src="/assets/index-45c321b0.js"></script>
<link rel="stylesheet" href="/assets/index-0f4879e0.css">
</head>
<body>
<div id="app"></div>

23
web/src/components/ModelEntry.vue
View File

@ -5,14 +5,9 @@
</div>
<div class="flex-1">
<h3 class="font-bold text-lg">
<input
type="radio"
:checked="selected"
:disabled="!isInstalled"
@change="handleSelection"
/>
{{ title }}
</h3>
<a :href="path" title="Download this manually (faster) and put it in the models/<your backend> folder then refresh">{{ title }}</a>
<p class="opacity-80">{{ description }}</p>
</div>
<div class="flex-shrink-0">
@ -24,13 +19,17 @@
>
<template v-if="installing">
<div class="flex items-center space-x-2">
<div class="h-2 w-20 bg-gray-300 rounded"></div>
<span>Installing...</span>
<div class="h-2 w-20 bg-gray-300 rounded">
<div :style="{ width: progress + '%'}" class="h-full bg-red-500 rounded"></div>
</div>
<span>Installing...{{ Math.floor(progress) }}%</span>
</div>
</template>
<template v-else-if="uninstalling">
<div class="flex items-center space-x-2">
<div class="h-2 w-20 bg-gray-300 rounded"></div>
<div class="h-2 w-20 bg-gray-300 rounded">
<div :style="{ width: progress + '%' }" class="h-full bg-green-500"></div>
</div>
<span>Uninstalling...</span>
</div>
</template>
@ -43,7 +42,7 @@
</template>
<script>
import { socket, state } from '@/services/websocket.js'
import socket from '@/services/websocket.js'
export default {
props: {
title: String,
@ -58,6 +57,7 @@ export default {
},
data() {
return {
progress: 0,
installing: false,
uninstalling: false
};
@ -76,8 +76,7 @@ export default {
},
handleSelection() {
if (this.isInstalled && !this.selected) {
this.selected=true;
onSelected(this);
this.onSelected(this);
}
}
}

65
web/src/components/PersonalityViewer.vue Normal file
View File

@ -0,0 +1,65 @@
<template>
<div class="p-4">
<div class="flex items-center mb-4">
<img :src="avatar" class="w-12 h-12 rounded-full mr-2" alt="Avatar">
<h2 class="text-lg font-semibold">{{ personalityName }}</h2>
</div>
<p><strong>Author:</strong> {{ personalityAuthor }}</p>
<p><strong>Description:</strong> {{ personalityDescription }}</p>
<p><strong>Language:</strong> {{ personalityLanguage }}</p>
<p><strong>Category:</strong> {{ personalityCategory }}</p>
<p v-if="disclaimer"><strong>Disclaimer:</strong> {{ disclaimer }}</p>
<p><strong>Conditioning Text:</strong> {{ conditioningText }}</p>
<p><strong>AI Prefix:</strong> {{ aiPrefix }}</p>
<p><strong>User Prefix:</strong> {{ userPrefix }}</p>
<div>
<strong>Antiprompts:</strong>
<ul>
<li v-for="antiprompt in antipromptsList" :key="antiprompt.id">
{{ antiprompt.text }}
</li>
</ul>
</div>
<button @click="editMode = true" class="mt-4 bg-blue-500 text-white px-4 py-2 rounded">
Edit
</button>
<button @click="commitChanges" v-if="editMode" class="mt-4 bg-green-500 text-white px-4 py-2 rounded">
Commit
</button>
</div>
</template>
<script>
export default {
data() {
return {
editMode: false,
avatar: 'path/to/avatar.jpg',
personalityName: 'Personality Name',
personalityAuthor: 'Author Name',
personalityDescription: 'Personality Description',
personalityLanguage: 'English',
personalityCategory: 'Category',
disclaimer: 'Disclaimer text',
conditioningText: 'Conditioning Text',
aiPrefix: 'AI Prefix',
userPrefix: 'User Prefix',
antipromptsList: [
{ id: 1, text: 'Antiprompt 1' },
{ id: 2, text: 'Antiprompt 2' },
{ id: 3, text: 'Antiprompt 3' }
]
};
},
methods: {
commitChanges() {
// Send the modified personality to the backend
// Implement your backend integration here
console.log('Personality changes committed');
this.editMode = false;
}
}
};
</script>

22
web/src/components/Toast.vue
View File

@ -5,6 +5,16 @@
role="alert">
<div class="flex flex-row">
<slot>
<div v-if="success"
class="inline-flex items-center justify-center flex-shrink-0 w-8 h-8 text-green-500 bg-green-100 rounded-lg dark:bg-green-800 dark:text-green-200">
<i data-feather="check"></i>
<span class="sr-only">Check icon</span>
</div>
<div v-if="!success" class="inline-flex items-center justify-center flex-shrink-0 w-8 h-8 text-red-500 bg-red-100 rounded-lg dark:bg-red-800 dark:text-red-200">
<i data-feather="x"></i>
<span class="sr-only">Cross icon</span>
</div>
<div class="ml-3 text-sm font-normal">{{ message }}</div>
</slot>
</div>
@ -19,6 +29,7 @@
clip-rule="evenodd"></path>
</svg>
</button>
</div>
</div>
</template>
@ -34,12 +45,23 @@ export default {
data() {
return {
show: false,
success: true,
message: ''
};
},
methods: {
close() {
this.$emit('close')
this.show = false
},
showToast(message, duration_s=3, success= true){
this.success = success;
this.message = message;
this.show = true;
setTimeout(() => {
this.$emit('close')
this.show = false
}, duration_s*1000);
}
},
watch: {

17
web/src/services/websocket.js
View File

@ -3,12 +3,9 @@
// Description :
// All websocket stuff can be found here.
// More info can be found here https://socket.io/how-to/use-with-vue
import { createApp } from 'vue';
import io from 'socket.io-client';
import { reactive } from "vue";
const state = reactive({
connected: false,
});
const socket = new io(import.meta.env.VITE_GPT4ALL_API );
@ -26,14 +23,18 @@ socket.onerror = (error) => {
};
socket.on("connect", () => {
state.connected = true;
console.log('WebSocket connected (websocket)');
});
socket.on("disconnect", () => {
state.connected = false;
console.log('WebSocket disonnected (websocket)');
});
export {socket, state};
const app = createApp(/* your root component */);
app.config.globalProperties.$socket = socket;
app.mount(/* your root element */);
export default socket;

32
web/src/views/DiscussionsView.vue
View File

@ -131,13 +131,7 @@
</div>
</div>
<Toast :showProp="isCopiedToClipboard" @close="isCopiedToClipboard = false">
<div
class="inline-flex items-center justify-center flex-shrink-0 w-8 h-8 text-green-500 bg-green-100 rounded-lg dark:bg-green-800 dark:text-green-200">
<i data-feather="check"></i>
<span class="sr-only">Check icon</span>
</div>
<div class="ml-3 text-sm font-normal">Message content copied to clipboard!</div>
<Toast ref="toast">
</Toast>
</template>
@ -148,6 +142,7 @@
</style>
<script>
export default {
setup() { },
data() {
@ -165,7 +160,7 @@ export default {
isSelectAll: false,
showConfirmation: false,
chime: new Audio("chime_aud.wav"),
isCopiedToClipboard: false,
showToast: false,
isSearch: false,
isDiscussionBottom: false,
}
@ -426,14 +421,15 @@ export default {
// Update previous message with reponse user data
//
// msgObj
//
// "status": "if the model is not ready this will inform the user that he can't promt the model"
// "type": "input_message_infos",
// "bot": self.personality.name,
// "user": self.personality.user_name,
// "message":message,#markdown.markdown(message),
// "user_message_id": self.current_user_message_id,
// "ai_message_id": self.current_ai_message_id,
console.log(msgObj);
if(msgObj["status"]=="generation_started"){
this.updateLastUserMsg(msgObj)
// Create response message
let responseMessage = {
@ -460,6 +456,13 @@ export default {
}
}
console.log("infos", msgObj)
}
else{
this.$refs.toast.showToast("It seems that no model has been loaded. Please download and install a model first, then try again.",4, false)
this.isGenerating = false
this.setDiscussionLoading(this.currentDiscussion.id, this.isGenerating)
this.chime.play()
}
},
sendMsg(msg) {
// Sends message to backend
@ -745,15 +748,14 @@ export default {
this.chime.play()
},
copyToClipBoard(content) {
this.isCopiedToClipboard = true
this.$refs.toast.showToast("Copied to clipboard successfully")
nextTick(() => {
feather.replace()
})
},
closeToast() {
this.isCopiedToClipboard = false
this.showToast = false
},
@ -831,7 +833,7 @@ export default {
},
computed: {
socketConnected() {
return state.connected
return true
},
selectedDiscussions() {
nextTick(() => {
@ -856,7 +858,7 @@ import feather from 'feather-icons'
import axios from 'axios'
import { nextTick } from 'vue'
import { socket, state } from '@/services/websocket.js'
import socket from '@/services/websocket.js'
import { onMounted } from 'vue'
import { initFlowbite } from 'flowbite'

123
web/src/views/SettingsView.vue
View File

@ -52,21 +52,32 @@
</select>
</div>
<div class="m-2">
<label for="model" class="block mb-2 text-sm font-medium text-gray-900 dark:text-white">
Model:
</label>
<select id="model" @change="update_model($event.target.value)"
class="bg-gray-50 border border-gray-300 text-gray-900 text-sm rounded-lg focus:ring-blue-500 focus:border-blue-500 block w-full p-2.5 dark:bg-gray-700 dark:border-gray-600 dark:placeholder-gray-400 dark:text-white dark:focus:ring-blue-500 dark:focus:border-blue-500">
<option v-for="item in modelsArr" :selected="item === configFile.model">{{ item }}</option>
</select>
</div>
</div>
</div>
<div
class="flex flex-col mb-2 p-3 rounded-lg bg-bg-light-tone dark:bg-bg-dark-tone hover:bg-bg-light-tone-panel hover:dark:bg-bg-dark-tone-panel duration-150 shadow-lg">
<div class="flex flex-row ">
<button @click.stop="bec_collapsed = !bec_collapsed"
<button @click.stop="mzc_collapsed = !mzc_collapsed"
class="text-2xl hover:text-primary duration-75 p-2 -m-2 w-full text-left active:translate-y-1">
<!-- <i data-feather="chevron-right"></i> -->
<h3 class="text-lg font-semibold cursor-pointer select-none "
@click.stop="bec_collapsed = !bec_collapsed">
@click.stop="mzc_collapsed = !mzc_collapsed">
Models zoo</h3>
</button>
</div>
<div :class="{ 'hidden': bec_collapsed }" class="flex flex-col mb-2 p-2">
<div :class="{ 'hidden': mzc_collapsed }" class="flex flex-col mb-2 p-2">
<div v-if="models.length > 0" class="my-2">
<label for="model" class="block ml-2 mb-2 text-sm font-medium text-gray-900 dark:text-white">
Install more models:
@ -132,7 +143,29 @@
</div>
</div>
</div>
<div
class="flex flex-col mb-2 p-3 rounded-lg bg-bg-light-tone dark:bg-bg-dark-tone hover:bg-bg-light-tone-panel hover:dark:bg-bg-dark-tone-panel duration-150 shadow-lg">
<div class="flex flex-row ">
<button @click.stop="pzc_collapsed = !pzc_collapsed"
class="text-2xl hover:text-primary duration-75 p-2 -m-2 w-full text-left active:translate-y-1">
<!-- <i data-feather="chevron-right"></i> -->
<h3 class="text-lg font-semibold cursor-pointer select-none "
@click.stop="pzc_collapsed = !pzc_collapsed">
Personalities zoo</h3>
</button>
</div>
<div :class="{ 'hidden': pzc_collapsed }" class="flex flex-col mb-2 p-2">
<div v-if="models.length > 0" class="my-2">
<label for="model" class="block ml-2 mb-2 text-sm font-medium text-gray-900 dark:text-white">
Install more models:
</label>
<div class="overflow-y-auto max-h-96 no-scrollbar p-2">
</div>
</div>
</div>
</div>
<!-- MODEL -->
<div
class="flex flex-col mb-2 p-3 rounded-lg bg-bg-light-tone dark:bg-bg-dark-tone hover:bg-bg-light-tone-panel hover:dark:bg-bg-dark-tone-panel duration-150 shadow-lg">
@ -171,7 +204,7 @@
</div>
<input id="temperature" @change="update_setting('temperature', $event.target.value)" type="range"
v-model="configFile.temp" min="0" max="5" step="0.1"
v-model="configFile.temperature" min="0" max="5" step="0.1"
class="flex-none h-2 mt-14 mb-2 w-full bg-gray-200 rounded-lg appearance-none cursor-pointer dark:bg-gray-700 focus:ring-blue-500 focus:border-blue-500 dark:border-gray-600 dark:placeholder-gray-400 dark:focus:ring-blue-500 dark:focus:border-blue-500">
</div>
</div>
@ -297,13 +330,15 @@ import { nextTick } from 'vue'
import MessageBox from "@/components/MessageBox.vue";
import YesNoDialog from "@/components/YesNoDialog.vue";
import ModelEntry from '@/components/ModelEntry.vue';
import { socket, state } from '@/services/websocket.js'
import PersonalityViewer from '@/components/PersonalityViewer.vue';
import socket from '@/services/websocket.js'
axios.defaults.baseURL = import.meta.env.VITE_GPT4ALL_API_BASEURL
export default {
components: {
MessageBox,
YesNoDialog,
ModelEntry
ModelEntry,
PersonalityViewer
},
setup() {
@ -315,16 +350,18 @@ export default {
data() {
return {
// Websocket
socket: socket,
// Models zoo installer stuff
models: [],
personalities:[],
// Accordeon stuff
bec_collapsed: false,
pc_collapsed: false,
mc_collapsed: false,
bec_collapsed: true,
mzc_collapsed: true, // models zoo
pzc_collapsed: true, // personalities zoo
pc_collapsed: true,
mc_collapsed: true,
// Settings stuff
backendsArr: [],
modelsArr: [],
persLangArr: [],
persCatgArr: [],
persArr: [],
@ -348,65 +385,64 @@ export default {
},
onSelected(model_object){
console.log("Selected model")
update_setting('model', model_object.title)
this.update_setting('model', model_object.title, (res)=>{console.log("Model selected"); })
},
// Model installation
onInstall(model_object) {
let isInstalled = model_object.isInstalled
let path = model_object.path
let path = model_object.path;
this.showProgress = true;
this.progress = 0;
console.log("installing...")
console.log("installing...");
// Use an arrow function for progressListener
const progressListener = (response) => {
console.log("received something");
if (response.status === 'progress') {
this.progress = message.progress;
console.log(`Progress = ${response.progress}`);
model_object.progress = response.progress
} else if (response.status === 'succeeded') {
// Installation completed
model_object.installing = false;
this.showProgress = false;
socket.off('install_progress', progressListener);
// Update the isInstalled property of the corresponding model
const index = this.models.findIndex((model) => model.path === path);
this.models[index].isInstalled = true;
this.showProgress = false;
this.socket.off('install_progress', progressListener);
} else if (response.status === 'failed') {
socket.off('install_progress', progressListener);
// Installation failed or encountered an error
model_object.installing = false;
this.showProgress = false;
this.socket.off('install_progress', progressListener);
console.error('Installation failed:', message.error);
console.error('Installation failed:', response.error);
}
};
this.socket.on('install_progress', progressListener);
this.socket.emit('install_model', { path: path });
socket.on('install_progress', progressListener);
socket.emit('install_model', { path: path });
console.log("Started installation, please wait");
},
onUninstall(model_object) {
console.log("uninstalling model...")
const progressListener = (response) => {
if (response.status === 'progress') {
this.progress = message.progress;
this.progress = response.progress;
} else if (response.status === 'succeeded') {
console.log(model_object)
// Installation completed
model_object.uninstalling = false;
socket.off('install_progress', progressListener);
this.showProgress = false;
// Update the isInstalled property of the corresponding model
model_object.isInstalled = false;
this.socket.off('install_progress', progressListener);
const index = this.models.findIndex((model) => model.path === model_object.path);
this.models[index].isInstalled = false;
} else if (response.status === 'failed') {
// Installation failed or encountered an error
model_object.uninstalling = false;
this.showProgress = false;
this.socket.off('install_progress', progressListener);
socket.off('install_progress', progressListener);
console.error('Installation failed:', message.error);
}
};
this.socket.on('install_progress', progressListener);
this.socket.emit('uninstall_model', { path: model_object.path });
socket.on('install_progress', progressListener);
socket.emit('uninstall_model', { path: model_object.path });
},
// messagebox ok stuff
onMessageBoxOk() {
@ -426,7 +462,7 @@ export default {
this.api_get_req("get_config").then(response => {
this.configFile = response
console.log("selecting model")
self.models.forEach(model => {
this.models.forEach(model => {
console.log(`${model} -> ${response["model"]}`)
if(model.title==response["model"]){
model.selected=true;
@ -450,13 +486,21 @@ export default {
axios.post('/update_setting', obj).then((res) => {
if (res) {
if (next !== undefined) {
next()
next(res)
}
return res.data;
}
})
.catch(error => { return { 'status': false } });
},
update_backend(value) {
console.log("Upgrading backend")
this.update_setting('backend', value, (res)=>{console.log("Backend changed"); this.fetchModels(); })
},
update_model(value) {
console.log("Upgrading model")
this.update_setting('model', value, (res)=>{console.log("Model changed"); this.fetchModels(); })
},
save_configuration() {
this.showConfirmation = false
axios.post('/save_settings', {})
@ -502,13 +546,7 @@ export default {
}
});
},
update_backend(value) {
res = update_setting('backend', value)
if (res.status) {
console.log("Backend changed")
}
},
async api_get_req(endpoint) {
try {
const res = await axios.get("/" + endpoint);
@ -534,6 +572,7 @@ export default {
this.configFile = await this.api_get_req("get_config")
this.backendsArr = await this.api_get_req("list_backends")
this.modelsArr = await this.api_get_req("list_models")
this.persLangArr = await this.api_get_req("list_personalities_languages")
this.persCatgArr = await this.api_get_req("list_personalities_categories")
this.persArr = await this.api_get_req("list_personalities")

40
webui.bat
View File

@ -262,46 +262,6 @@ if not exist \models (
md \models
)
dir ".\models\llama_cpp\*.bin" /b 2>&1
if errorlevel 1 (
echo.
choice /C YNB /M "The default model file (gpt4all-lora-quantized-ggml.bin) does not exist. Do you want to download it? Press B to download it with a browser (faster)."
if errorlevel 3 goto DOWNLOAD_WITH_BROWSER
if errorlevel 2 goto DOWNLOAD_SKIP
if errorlevel 1 goto MODEL_DOWNLOAD
) ELSE (
echo Model already installed
goto CONTINUE
)
:DOWNLOAD_WITH_BROWSER
start https://huggingface.co/ParisNeo/GPT4All/resolve/main/gpt4all-lora-quantized-ggml.bin
echo Link has been opened with the default web browser, make sure to save it into the models/llama_cpp folder before continuing. Press any key to continue...
pause
goto :CONTINUE
:MODEL_DOWNLOAD
echo.
echo Downloading latest model...
set clone_dir=%cd%
powershell -Command "Invoke-WebRequest -Uri 'https://huggingface.co/ParisNeo/GPT4All/resolve/main/gpt4all-lora-quantized-ggml.bin' -OutFile %clone_dir%'/models/llama_cpp/gpt4all-lora-quantized-ggml.bin'"
if errorlevel 1 (
echo Failed to download model. Please check your internet connection.
choice /C YN /M "Do you want to try downloading again?"
if errorlevel 2 goto DOWNLOAD_SKIP
if errorlevel 1 goto MODEL_DOWNLOAD
) else (
echo Model successfully downloaded.
)
goto :CONTINUE
:DOWNLOAD_SKIP
echo.
echo Skipping download of model file...
goto :CONTINUE
:CONTINUE
:END
if exist "./tmp" (
echo Cleaning tmp folder

48
webui.sh
View File

@ -110,53 +110,9 @@ if ping -q -c 1 google.com >/dev/null 2>&1; then
else
echo "is created"
fi
# Checking model
MODEL="./models/llama_cpp/gpt4all-lora-quantized-ggml.bin"
MODEL_URL="https://huggingface.co/ParisNeo/GPT4All/resolve/main/gpt4all-lora-quantized-ggml.bin"
fi
if [ -f "$MODEL" ]; then
echo "File $MODEL already exists. Skipping download."
else
echo "File $MODEL does not exist."
echo "What would you like to do?"
select option in "Download" "Download using browser" "Skip"; do
case $option in
Download)
if [ -x "$(command -v wget)" ]; then
wget "$MODEL_URL" -P ./models/llama_cpp/
elif [ -x "$(command -v curl)" ]; then
curl -o "$MODEL" "$MODEL_URL"
else
echo "Error: neither wget nor curl is installed. Please install one of them and try again."
exit 1
fi
break
;;
"Download using browser")
if [ -x "$(command -v xdg-open)" ]; then
xdg-open "$MODEL_URL"
elif [ -x "$(command -v gnome-open)" ]; then
gnome-open "$MODEL_URL"
elif [ -x "$(command -v kde-open)" ]; then
kde-open "$MODEL_URL"
elif [ -x "$(command -v open)" ]; then
open "$MODEL_URL"
else
echo "Error: could not detect a default browser. Please open the link in your web browser manually and press any key to continue."
read -n 1 -s -r -p "Press any key to continue"$'\n'
fi
break
;;
Skip)
echo "Skipping downloading $MODEL"
break
;;
esac
done
fi
else
echo "Internet connection not available"
fi
# Activate the virtual environment
echo -n "Activating virtual environment..."
source env/bin/activate