ExternalVendorCode/lollms
1
0
Fork 0
mirror of https://github.com/ParisNeo/lollms.git synced 2024-12-18 12:26:29 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

removed lollms personalities

Browse Source
This commit is contained in:
Saifeddine ALOUI 2023-08-18 01:29:10 +02:00
parent 72a2466d3e
commit 424e1fd9ac
11 changed files with 70 additions and 117 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
LICENSE
View File

@ -186,7 +186,7 @@
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Copyright Copyright 2023 Saifeddine ALOUI (aka: ParisNeo)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.

46
docs/server_endpoints.md
View File

@ -45,22 +45,18 @@ Events generated:
- Actions:
- Retrieves the path to the personalities folder from the server (`self.personalities_path`).
- Initializes an empty dictionary to store the available personalities.
- Iterates over each language folder in the personalities folder.
- Iterates over each category folder within the language folder.
- Checks if the current item is a directory.
- Initializes an empty dictionary to store the personalities within the language.
- Iterates over each category folder within the language folder.
- Initializes an empty list to store the personalities within the category.
- Iterates over each personality folder within the category folder.
- Checks if the current item is a directory.
- Initializes an empty list to store the personalities within the category.
- Iterates over each personality folder within the category folder.
- Checks if the current item is a directory.
- Tries to load personality information from the config file (`config.yaml`) within the personality folder.
- Retrieves the name, description, author, and version from the config data.
- Checks if the `scripts` folder exists within the personality folder to determine if the personality has scripts.
- Checks for the existence of logo files named `logo.gif` or `logo.webp` or `logo.png` or `logo.jpg` or `logo.jpeg` or `logo.bmp` within the `assets` folder to determine if the personality has a logo.
- Sets the `avatar` field of the personality info based on the available logo file.
- Appends the personality info to the list of personalities within the category.
- Adds the list of personalities to the dictionary of the current category within the language.
- Adds the dictionary of categories to the dictionary of the current language.
- Tries to load personality information from the config file (`config.yaml`) within the personality folder.
- Retrieves the name, description, author, and version from the config data.
- Checks if the `scripts` folder exists within the personality folder to determine if the personality has scripts.
- Checks for the existence of logo files named `logo.gif` or `logo.webp` or `logo.png` or `logo.jpg` or `logo.jpeg` or `logo.bmp` within the `assets` folder to determine if the personality has a logo.
- Sets the `avatar` field of the personality info based on the available logo file.
- Appends the personality info to the list of personalities within the category.
- Adds the list of personalities to the dictionary of the current category within the language.
- Sends a response to the client containing the dictionary of available personalities.
Events generated:
@ -84,32 +80,14 @@ Events generated:
#### `list_available_personalities_languages`
- Event: `'list_available_personalities_languages'`
- Description: This event is triggered when a client requests a list of available personality languages.
- Actions:
- Attempts to retrieve a list of available personality languages by iterating over the `self.personalities_path` directory.
- Sends a response to the client containing the success status and the list of available personality languages.
Parameters: None
Events:
- `'available_personalities_languages_list'`: This event is emitted as a response to the client after listing the available personality languages.
- Data:
- `'success'` (boolean): Indicates whether the operation was successful or not.
- `'available_personalities_languages'` (list): Contains the available personality languages as a list of strings.
#### `list_available_personalities_categories`
- Event: `'list_available_personalities_categories'`
- Description: This event is triggered when a client requests a list of available personality categories based on a specified language.
- Parameters:
- `data`: A dictionary containing the following parameter:
- `language`: The language for which to retrieve available personality categories.
- `data`: An empty dictionary:
- Actions:
- Extracts the `language` parameter from the request data.
- Attempts to retrieve the available personality categories for the specified language.
- Attempts to retrieve the available personality categories.
- Emits an event `'available_personalities_categories_list'` to the client.
- If successful, sends a response with a list of available personality categories in the `'available_personalities_categories'` field of the event data.
- If an error occurs, sends a response with an error message in the `'error'` field of the event data.

9
lollms/app.py
View File

@ -150,8 +150,13 @@ class LollmsApplication:
for i in to_remove:
self.unmount_personality(i)
self.personality = self.mounted_personalities[self.config.active_personality_id]
if self.config.active_personality_id>=0 and self.config.active_personality_id<len(self.mounted_personalities):
self.personality = self.mounted_personalities[self.config.active_personality_id]
else:
self.config["personalities"].insert(0, "generic/lollms")
self.mount_personality(0, callback = None)
self.config.active_personality_id = 0
self.personality = self.mounted_personalities[self.config.active_personality_id]
def set_personalities_callbacks(self, callback: Callable[[str, int, dict], bool]=None):
for personality in self.mount_personalities:
personality.setCallback(callback)

15
lollms/apps/server/__init__.py
View File

@ -236,19 +236,11 @@ class LoLLMsServer(LollmsApplication):
print(f"Problem with model : {model}")
emit('available_models_list', {'success':True, 'available_models': models}, room=request.sid)
@self.socketio.on('list_available_personalities_languages')
def handle_list_available_personalities_languages():
try:
languages = [l for l in self.personalities_path.iterdir()]
emit('available_personalities_languages_list', {'success': True, 'available_personalities_languages': languages})
except Exception as ex:
emit('available_personalities_languages_list', {'success': False, 'error':str(ex)})
@self.socketio.on('list_available_personalities_categories')
def handle_list_available_personalities_categories(data):
try:
language = data["language"]
categories = [l for l in (self.personalities_path/language).iterdir()]
categories = [l for l in (self.personalities_path).iterdir()]
emit('available_personalities_categories_list', {'success': True, 'available_personalities_categories': categories})
except Exception as ex:
emit('available_personalities_categories_list', {'success': False, 'error':str(ex)})
@ -256,9 +248,8 @@ class LoLLMsServer(LollmsApplication):
@self.socketio.on('list_available_personalities_names')
def handle_list_available_personalities_names(data):
try:
language = data["language"]
category = data["category"]
personalities = [l for l in (self.personalities_path/language/category).iterdir()]
personalities = [l for l in (self.personalities_path/category).iterdir()]
emit('list_available_personalities_names_list', {'success': True, 'list_available_personalities_names': personalities})
except Exception as ex:
emit('list_available_personalities_names_list', {'success': False, 'error':str(ex)})
@ -625,7 +616,7 @@ class LoLLMsServer(LollmsApplication):
print(f"{ASCIIColors.color_red}Mounted personalities : {ASCIIColors.color_reset}{self.config.personalities}")
if len(self.config.personalities)==0:
ASCIIColors.warning("No personality selected. Selecting lollms. You can mount other personalities using lollms-settings application")
self.config.personalities = ["english/generic/lollms"]
self.config.personalities = ["generic/lollms"]
self.config.save_config()
if self.config.active_personality_id>=len(self.config.personalities):

3
lollms/binding.py
View File

@ -51,7 +51,8 @@ class LLMBinding:
self.file_extension = file_extension
self.seed = config["seed"]
self.configuration_file_path = lollms_paths.personal_configuration_path/f"binding_{self.binding_folder_name}.yaml"
self.configuration_file_path = lollms_paths.personal_configuration_path/self.binding_folder_name/f"config.yaml"
self.configuration_file_path.parent.mkdir(parents=True, exist_ok=True)
self.binding_config.config.file_path = self.configuration_file_path
# Installation

2
lollms/configs/config.yaml
View File

@ -24,7 +24,7 @@ repeat_penalty: 1.2
n_threads: 8
#Personality parameters
personalities: ["english/generic/lollms"]
personalities: ["generic/lollms"]
active_personality_id: 0
override_personality_model_parameters: false #if true the personality parameters are overriden by those of the configuration (may affect personality behaviour)

2
lollms/main_config.py
View File

@ -37,7 +37,7 @@ DEFAULT_CONFIG = {
"n_threads": 8,
#Personality parameters
"personalities": ["english/generic/lollms"],
"personalities": ["generic/lollms"],
"active_personality_id": 0,
"override_personality_model_parameters": False, #if true the personality parameters are overriden by those of the configuration (may affect personality behaviour)

32
lollms/personality.py
View File

@ -86,7 +86,6 @@ class AIPersonality:
self._author: str = "ParisNeo"
self._name: str = "lollms"
self._user_name: str = "user"
self._language: str = "english"
self._category: str = "General"
# Conditionning
@ -162,7 +161,7 @@ Date: {{date}}
def __str__(self):
return f"{self.language}/{self.category}/{self.name}"
return f"{self.category}/{self.name}"
def load_personality(self, package_path=None):
@ -201,7 +200,6 @@ Date: {{date}}
self._author = config.get("author", self._author)
self._name = config.get("name", self._name)
self._user_name = config.get("user_name", self._user_name)
self._language = config.get("language", self._language)
self._category = config.get("category", self._category)
self._personality_description = config.get("personality_description", self._personality_description)
self._personality_conditioning = config.get("personality_conditioning", self._personality_conditioning)
@ -294,7 +292,6 @@ Date: {{date}}
"version": self._version,
"name": self._name,
"user_name": self._user_name,
"language": self._language,
"category": self._category,
"personality_description": self._personality_description,
"personality_conditioning": self._personality_conditioning,
@ -334,7 +331,6 @@ Date: {{date}}
"version": self._version,
"name": self._name,
"user_name": self._user_name,
"language": self._language,
"category": self._category,
"personality_description": self._personality_description,
"personality_conditioning": self._personality_conditioning,
@ -418,16 +414,6 @@ Date: {{date}}
"""Set the user name."""
self._user_name = value
@property
def language(self) -> str:
"""Get the language."""
return self._language
@language.setter
def language(self, value: str):
"""Set the language."""
self._language = value
@property
def category(self) -> str:
"""Get the category."""
@ -939,7 +925,9 @@ class APScript(StateMachine):
self.personality = personality
self.personality_config = personality_config
self.installation_option = personality.installation_option
self.configuration_file_path = self.personality.lollms_paths.personal_configuration_path/f"personality_{self.personality.personality_folder_name}.yaml"
self.configuration_file_path = self.personality.lollms_paths.personal_configuration_path/self.personality.personality_folder_name/f"config.yaml"
self.configuration_file_path.parent.mkdir(parents=True, exist_ok=True)
self.personality_config.config.file_path = self.configuration_file_path
self.callback = callback
@ -1146,7 +1134,7 @@ class APScript(StateMachine):
if callback:
callback(step_text, MSG_TYPE.MSG_TYPE_STEP_START)
def step_end(self, step_text, status=True, callback: Callable[[str, int, dict], bool]=None):
def step_end(self, step_text, status=True, callback: Callable[[str, int, dict, list], bool]=None):
"""This triggers a step end
Args:
@ -1211,7 +1199,7 @@ class APScript(StateMachine):
if callback:
callback(info, MSG_TYPE.MSG_TYPE_INFO)
def json(self, json_infos:dict, callback: Callable[[str, int, dict], bool]=None, indent=4):
def json(self, title:str, json_infos:dict, callback: Callable[[str, int, dict, list], bool]=None, indent=4):
"""This sends json data to front end
Args:
@ -1222,7 +1210,7 @@ class APScript(StateMachine):
callback = self.callback
if callback:
callback(json.dumps(json_infos, indent=indent), MSG_TYPE.MSG_TYPE_JSON_INFOS)
callback("", MSG_TYPE.MSG_TYPE_JSON_INFOS, metadata = [{"title":title, "content":json.dumps(json_infos, indent=indent)}])
def ui(self, html_ui:str, callback: Callable[[str, int, dict], bool]=None):
"""This sends ui elements to front end
@ -1316,7 +1304,7 @@ class APScript(StateMachine):
if callback:
callback(step_text, MSG_TYPE.MSG_TYPE_STEP_PROGRESS, {'progress':progress})
def new_message(self, message_text:str, message_type:MSG_TYPE, metadata=None, callback: Callable[[str, int, dict], bool]=None):
def new_message(self, message_text:str, message_type:MSG_TYPE, metadata=[], callback: Callable[[str, int, dict, list], bool]=None):
"""This sends step rogress to front end
Args:
@ -1327,7 +1315,7 @@ class APScript(StateMachine):
callback = self.callback
if callback:
callback(message_text, MSG_TYPE.MSG_TYPE_NEW_MESSAGE, {'type':message_type.value,'metadata':metadata})
callback(message_text, MSG_TYPE.MSG_TYPE_NEW_MESSAGE, parameters={'type':message_type.value,'metadata':metadata})
def finished_message(self, message_text:str="", callback: Callable[[str, int, dict], bool]=None):
"""This sends step rogress to front end
@ -1383,7 +1371,7 @@ class PersonalityBuilder:
else:
if id>len(self.config["personalities"]):
id = len(self.config["personalities"])-1
if len(self.config["personalities"][id].split("/"))==3:
if len(self.config["personalities"][id].split("/"))==2:
self.personality = AIPersonality(
self.lollms_paths.personalities_zoo_path / self.config["personalities"][id],
self.lollms_paths,

46
lollms/terminal.py
View File

@ -298,37 +298,27 @@ class MainMenu(Menu):
ASCIIColors.green(personality)
else:
ASCIIColors.yellow(personality)
personality_languages = [p.stem for p in self.lollms_app.lollms_paths.personalities_zoo_path.iterdir() if p.is_dir() and not p.name.startswith(".")] + ["Back"]
print("Select language")
choice = self.show_menu(personality_languages)
if 1 <= choice <= len(personality_languages)-1:
language = personality_languages[choice - 1]
print(f"You selected language: {ASCIIColors.color_green}{language}{ASCIIColors.color_reset}")
personality_categories = [p.stem for p in (self.lollms_app.lollms_paths.personalities_zoo_path/language).iterdir() if p.is_dir() and not p.name.startswith(".")]+["Back"]
print("Select category")
choice = self.show_menu(personality_categories)
if 1 <= choice <= len(personality_categories)-1:
category = personality_categories[choice - 1]
print(f"You selected category: {ASCIIColors.color_green}{category}{ASCIIColors.color_reset}")
personality_names = [p.stem for p in (self.lollms_app.lollms_paths.personalities_zoo_path/language/category).iterdir() if p.is_dir() and not p.name.startswith(".")]+["Back"]
print("Select personality")
choice = self.show_menu(personality_names)
if 1 <= choice <= len(personality_names)-1:
name = personality_names[choice - 1]
print(f"You selected personality: {ASCIIColors.color_green}{name}{ASCIIColors.color_reset}")
self.lollms_app.config["personalities"].append(f"{language}/{category}/{name}")
self.lollms_app.mount_personality(len(self.lollms_app.config["personalities"])-1, callback = self.callback)
self.lollms_app.config.save_config()
print("Personality mounted successfully!")
elif 1 <= choice <= len(personality_names):
return
else:
print("Invalid choice!")
elif 1 <= choice <= len(personality_categories):
personality_categories = [p.stem for p in (self.lollms_app.lollms_paths.personalities_zoo_path).iterdir() if p.is_dir() and not p.name.startswith(".")]+["Back"]
print("Select category")
choice = self.show_menu(personality_categories)
if 1 <= choice <= len(personality_categories)-1:
category = personality_categories[choice - 1]
print(f"You selected category: {ASCIIColors.color_green}{category}{ASCIIColors.color_reset}")
personality_names = [p.stem for p in (self.lollms_app.lollms_paths.personalities_zoo_path/category).iterdir() if p.is_dir() and not p.name.startswith(".")]+["Back"]
print("Select personality")
choice = self.show_menu(personality_names)
if 1 <= choice <= len(personality_names)-1:
name = personality_names[choice - 1]
print(f"You selected personality: {ASCIIColors.color_green}{name}{ASCIIColors.color_reset}")
self.lollms_app.config["personalities"].append(f"{category}/{name}")
self.lollms_app.mount_personality(len(self.lollms_app.config["personalities"])-1, callback = self.callback)
self.lollms_app.config.save_config()
print("Personality mounted successfully!")
elif 1 <= choice <= len(personality_names):
return
else:
print("Invalid choice!")
elif 1 <= choice <= len(personality_languages):
elif 1 <= choice <= len(personality_categories):
return
else:
print("Invalid choice!")

28
lollms/utilities.py
View File

@ -778,20 +778,20 @@ class PromptReshaper:
placeholders[placeholder]=detokenize(text_tokens)
return fill_template(self.template, placeholders)
if __name__=="__main__":
def tokenize(text):
return text.split() # Simple tokenization by splitting on spaces
def detokenize(tokens):
return ' '.join(tokens)
template = "Hello, {{name}}! How are you feeling {{emotion}} really"
placeholders = {
"name": "Alice",
"emotion": "happy and very happy"
}
max_nb_tokens = 10
class LOLLMSLocalizer:
def __init__(self, dictionary):
self.dictionary = dictionary
def localize(self, input_string):
def replace(match):
key = match.group(1)
return self.dictionary.get(key, match.group(0))
import re
pattern = r'@<([^>]+)>@'
localized_string = re.sub(pattern, replace, input_string)
return localized_string
reshaper = PromptReshaper(template)
final_text = reshaper.build(placeholders, tokenize, detokenize, max_nb_tokens,["emotion"])
print(final_text)

2
setup.py
View File

@ -26,7 +26,7 @@ def get_all_files(path):
setuptools.setup(
name="lollms",
version="2.3.4",
version="3.0.0",
author="Saifeddine ALOUI",
author_email="aloui.saifeddine@gmail.com",
description="A python library for AI personality definition",