ExternalVendorCode/lollms
1
0
Fork 0
mirror of https://github.com/ParisNeo/lollms.git synced 2024-12-19 20:57:58 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

upgraded

Browse Source
This commit is contained in:
Saifeddine ALOUI 2023-06-19 00:27:18 +02:00
parent 4cb313c8fa
commit 8f32d26309
9 changed files with 385 additions and 36 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
doc/server_endpoints.md
View File

@ -308,3 +308,4 @@ Events generated:
- `'buzzy'`: when the server is buzzy and can't process the request, it sends this event and returns. This event have parameter `message` containing a string. - `'buzzy'`: when the server is buzzy and can't process the request, it sends this event and returns. This event have parameter `message` containing a string.
- `'text_chunk'`: Generated text chunks are emitted to the client through this event during the text generation process. The event has two parameters `chunk` and `type`. - `'text_chunk'`: Generated text chunks are emitted to the client through this event during the text generation process. The event has two parameters `chunk` and `type`.
- `'text_generated'`: Once the text generation process is complete, the final generated text is emitted to the client through this event. The event has one parameter `text` containing the full generated text. - `'text_generated'`: Once the text generation process is complete, the final generated text is emitted to the client through this event. The event has one parameter `text` containing the full generated text.
- `'generation_canceled'`: Answer to `cancel_generation` endpoint call

79
examples/lollms_playground/README.md Normal file
View File

@ -0,0 +1,79 @@
```
# LoLLMs Endpoint Test Tool
This tool provides a web-based interface to test LoLLMs endpoints and generate text using a LoLLMs server.
## Prerequisites
To use this tool, you need to have [Node.js](https://nodejs.org) installed on your machine.
## Installation
1. Clone this repository or download the source code.
```bash
git clone https://github.com/ParisNeo/lollms-playground.git
```
2. Navigate to the project directory.
```bash
cd socketio-endpoint-test-tool
```
3. Install the dependencies.
```bash
npm install
```
## Usage
1. Start the LoLLMs server. You can use `lollms-server` to run the server with the desired configuration. Here are a few examples:
- To run the server on `localhost` and port `9600`:
```bash
lollms-server --host localhost --port 9600
```
- To run the server on a different host and port:
```bash
lollms-server --host mydomain.com --port 8080
```
- For more information on the available options, you can use the `--help` flag:
```bash
lollms-server --help
```
2. Start the web server for the LoLLMs Endpoint Test Tool.
```bash
npm start
```
3. Open your web browser and visit `http://localhost:8080/lollms_playground.html` (or the appropriate URL) to access the LoLLMs Endpoint Test Tool.
4. Fill in the host and port fields with the appropriate values for your LoLLMs server.
5. Click the "Connect" button to establish a connection with the LoLLMs server.
6. Once connected, you can enter a prompt and click the "Generate Text" button to initiate text generation.
7. The generated text will be displayed in the output section of the page.
## Customization
You can customize the appearance and behavior of the tool by modifying the HTML, CSS, and JavaScript code in the `test_generation.html` file.
## Contributing
Contributions are welcome! If you find any issues or want to add new features, feel free to open an issue or submit a pull request.
## License
This project is licensed under the [MIT License](LICENSE).
```

154
examples/lollms_playground/lollms_playground.html Normal file
View File

@ -0,0 +1,154 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>LoLLMs Endpoint Test</title>
<link href="https://cdn.jsdelivr.net/npm/tailwindcss@2.2.19/dist/tailwind.min.css" rel="stylesheet">
<style>
.hover\:bg-blue-600:hover {
background-color: #2563eb;
}
.active\:bg-blue-700:active {
background-color: #1d4ed8;
}
</style>
</head>
<body class="bg-gray-100 w-full">
<div class="flex items-center justify-center min-h-screen w-full">
<div class="w-full bg-blue-300 p-6 rounded shadow m-4">
<h1 class="text-2xl font-bold mb-4">LoLLMs Playground</h1>
<div id="connection-section">
<div class="mb-4">
<label for="host" class="block text-sm font-medium text-gray-700">Host:</label>
<input id="host" type="text" class="mt-1 p-2 border border-gray-300 rounded-md w-full"
value="localhost" />
</div>
<div class="mb-4">
<label for="port" class="block text-sm font-medium text-gray-700">Port:</label>
<input id="port" type="number" class="mt-1 p-2 border border-gray-300 rounded-md w-full" value="9600" />
</div>
<button id="connect-btn"
class="bg-blue-500 hover:bg-blue-600 active:bg-blue-700 text-white font-bold py-2 px-4 rounded">Connect</button>
<label class="hidden" id = "connecting">connecting ...</label>
</div>
<div id="generation-section" class="hidden w-full">
<div>
<button id="generate-btn"
class="bg-blue-500 hover:bg-blue-600 active:bg-blue-700 text-white font-bold py-2 px-4 rounded">Generate
Text</button>
</div>
<div>
<button id="stop-btn"
class="bg-red-500 hover:bg-red-600 active:bg-red-700 text-white font-bold py-2 px-4 rounded hidden">Stop
Generation</button>
</div>
<div>
<textarea id="text" class="mt-4 p-2 border border-gray-300 rounded-md h-64 overflow-y-scroll w-full" type="text"></textarea>
</div>
</div>
</div>
</div>
<script src="https://cdn.socket.io/4.4.1/socket.io.min.js"></script>
<script>
const socket = io();
const connectButton = document.getElementById('connect-btn');
const generateButton = document.getElementById('generate-btn');
const stopButton = document.getElementById('stop-btn');
const connectionSection = document.getElementById('connection-section');
const generationSection = document.getElementById('generation-section');
const connectingText = document.getElementById('connecting');
// Append the received chunks to the text div
function appendToOutput(chunk) {
const outputDiv = document.getElementById('text');
outputDiv.value += chunk;
outputDiv.scrollTop = outputDiv.scrollHeight;
}
// Event handler for receiving generated text chunks
socket.on('text_chunk', data => {
console.log('Received chunk:', data.chunk);
appendToOutput(data.chunk);
});
// Event handler for receiving generated text chunks
socket.on('text_generated', data => {
console.log('text generated:', data.text);
// Toggle button visibility
generateButton.classList.remove('hidden');
stopButton.classList.add('hidden');
});
// Event handler for successful connection
socket.on('connect', () => {
console.log('Connected to LoLLMs server');
connectButton.disabled = true;
connectingText.classList.add("hidden")
connectionSection.classList.add('hidden');
generationSection.classList.remove('hidden');
});
// Event handler for error during text generation
socket.on('buzzy', error => {
console.error('Server is busy. Wait for your turn', error);
const outputDiv = document.getElementById('text');
outputDiv.value += `<p class="text-red-500">Error: ${error.message}</p>`;
outputDiv.scrollTop = outputDiv.scrollHeight;
// Toggle button visibility
generateButton.classList.remove('hidden');
stopButton.classList.add('hidden');
});
// Event handler for error during text generation
socket.on('generation_canceled', error => {
// Toggle button visibility
generateButton.classList.remove('hidden');
stopButton.classList.add('hidden');
});
// Triggered when the "Connect" button is clicked
connectButton.addEventListener('click', () => {
const hostInput = document.getElementById('host');
const portInput = document.getElementById('port');
const host = hostInput.value.trim();
const port = parseInt(portInput.value);
if (host && port) {
socket.io.uri = `http://${host}:${port}`;
socket.connect();
connectingText.classList.remove("hidden")
}
});
// Triggered when the "Generate Text" button is clicked
generateButton.addEventListener('click', () => {
const outputDiv = document.getElementById('text');
var prompt = outputDiv.value
console.log(prompt)
// Trigger the 'generate_text' event with the prompt
socket.emit('generate_text', { prompt, personality: -1, n_predicts: 1024 });
// Toggle button visibility
generateButton.classList.add('hidden');
stopButton.classList.remove('hidden');
});
// Triggered when the "Stop Generation" button is clicked
stopButton.addEventListener('click', () => {
// Trigger the 'cancel_generation' event
socket.emit('cancel_generation',{});
});
</script>
</body>
</html>

8
lollms/server.py
View File

@ -324,6 +324,8 @@ class LoLLMsServer:
def cancel_generation(data): def cancel_generation(data):
client_id = request.sid client_id = request.sid
self.clients[client_id]["requested_stop"]=False self.clients[client_id]["requested_stop"]=False
print(f"Client {client_id} requested canceling generation")
emit("generation_canceled", {"message":"Generation is canceled."})
@self.socketio.on('generate_text') @self.socketio.on('generate_text')
@ -335,9 +337,9 @@ class LoLLMsServer:
client_id = request.sid client_id = request.sid
self.clients[client_id]["is_generating"]=True self.clients[client_id]["is_generating"]=True
self.clients[client_id]["requested_stop"]=False self.clients[client_id]["requested_stop"]=False
prompt = data['prompt'] prompt = data['prompt']
personality_id = data['personality'] personality_id = data['personality']
n_predicts = data["n_predicts"] n_predicts = data["n_predicts"]
if personality_id==-1: if personality_id==-1:
# Raw text generation # Raw text generation
print(f"Text generation requested by client: {client_id}") print(f"Text generation requested by client: {client_id}")

2
setup.py
View File

@ -26,7 +26,7 @@ def get_all_files(path):
setuptools.setup( setuptools.setup(
name="lollms", name="lollms",
version="1.2.5", version="1.2.6",
author="Saifeddine ALOUI", author="Saifeddine ALOUI",
author_email="aloui.saifeddine@gmail.com", author_email="aloui.saifeddine@gmail.com",
description="A python library for AI personality definition", description="A python library for AI personality definition",

79
tests/endoints_unit_tests/node/README.md Normal file
View File

@ -0,0 +1,79 @@
```
# LoLLMs Endpoint Test Tool
This tool provides a web-based interface to test LoLLMs endpoints and generate text using a LoLLMs server.
## Prerequisites
To use this tool, you need to have [Node.js](https://nodejs.org) installed on your machine.
## Installation
1. Clone this repository or download the source code.
```bash
git clone https://github.com/ParisNeo/lollms-playground.git
```
2. Navigate to the project directory.
```bash
cd socketio-endpoint-test-tool
```
3. Install the dependencies.
```bash
npm install
```
## Usage
1. Start the LoLLMs server. You can use `lollms-server` to run the server with the desired configuration. Here are a few examples:
- To run the server on `localhost` and port `9600`:
```bash
lollms-server --host localhost --port 9600
```
- To run the server on a different host and port:
```bash
lollms-server --host mydomain.com --port 8080
```
- For more information on the available options, you can use the `--help` flag:
```bash
lollms-server --help
```
2. Start the web server for the LoLLMs Endpoint Test Tool.
```bash
npm start
```
3. Open your web browser and visit `http://localhost:8080/lollms_playground.html` (or the appropriate URL) to access the LoLLMs Endpoint Test Tool.
4. Fill in the host and port fields with the appropriate values for your LoLLMs server.
5. Click the "Connect" button to establish a connection with the LoLLMs server.
6. Once connected, you can enter a prompt and click the "Generate Text" button to initiate text generation.
7. The generated text will be displayed in the output section of the page.
## Customization
You can customize the appearance and behavior of the tool by modifying the HTML, CSS, and JavaScript code in the `test_generation.html` file.
## Contributing
Contributions are welcome! If you find any issues or want to add new features, feel free to open an issue or submit a pull request.
## License
This project is licensed under the [Apache 2.0](LICENSE).
```

98
tests/endoints_unit_tests/test_generation.html → tests/endoints_unit_tests/node/test_generation.html
View File

@ -4,7 +4,7 @@
<head> <head>
<meta charset="UTF-8"> <meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Socket.IO Endpoint Test</title> <title>LoLLMs Endpoint Test</title>
<link href="https://cdn.jsdelivr.net/npm/tailwindcss@2.2.19/dist/tailwind.min.css" rel="stylesheet"> <link href="https://cdn.jsdelivr.net/npm/tailwindcss@2.2.19/dist/tailwind.min.css" rel="stylesheet">
<style> <style>
.hover\:bg-blue-600:hover { .hover\:bg-blue-600:hover {
@ -17,10 +17,10 @@
</style> </style>
</head> </head>
<body class="bg-gray-100"> <body class="bg-gray-100 w-full">
<div class="container mx-auto mt-8"> <div class="flex items-center justify-center min-h-screen w-full">
<div class="max-w-lg mx-auto bg-white p-6 rounded shadow"> <div class="w-full bg-blue-300 p-6 rounded shadow m-4">
<h1 class="text-2xl font-bold mb-4">Socket.IO Endpoint Test</h1> <h1 class="text-2xl font-bold mb-4">LoLLMs Playground</h1>
<div id="connection-section"> <div id="connection-section">
<div class="mb-4"> <div class="mb-4">
<label for="host" class="block text-sm font-medium text-gray-700">Host:</label> <label for="host" class="block text-sm font-medium text-gray-700">Host:</label>
@ -33,16 +33,22 @@
</div> </div>
<button id="connect-btn" <button id="connect-btn"
class="bg-blue-500 hover:bg-blue-600 active:bg-blue-700 text-white font-bold py-2 px-4 rounded">Connect</button> class="bg-blue-500 hover:bg-blue-600 active:bg-blue-700 text-white font-bold py-2 px-4 rounded">Connect</button>
<label class="hidden" id = "connecting">connecting ...</label>
</div> </div>
<div id="generation-section" class="hidden"> <div id="generation-section" class="hidden w-full">
<div class="mb-4"> <div>
<label for="prompt" class="block text-sm font-medium text-gray-700">Enter Prompt:</label> <button id="generate-btn"
<input id="prompt" type="text" class="mt-1 p-2 border border-gray-300 rounded-md w-full" />
</div>
<button id="generate-btn"
class="bg-blue-500 hover:bg-blue-600 active:bg-blue-700 text-white font-bold py-2 px-4 rounded">Generate class="bg-blue-500 hover:bg-blue-600 active:bg-blue-700 text-white font-bold py-2 px-4 rounded">Generate
Text</button> Text</button>
<div id="output" class="mt-4 p-2 border border-gray-300 rounded-md h-64 overflow-y-scroll"></div> </div>
<div>
<button id="stop-btn"
class="bg-red-500 hover:bg-red-600 active:bg-red-700 text-white font-bold py-2 px-4 rounded hidden">Stop
Generation</button>
</div>
<div>
<textarea id="text" class="mt-4 p-2 border border-gray-300 rounded-md h-64 overflow-y-scroll w-full" type="text"></textarea>
</div>
</div> </div>
</div> </div>
</div> </div>
@ -52,13 +58,15 @@
const socket = io(); const socket = io();
const connectButton = document.getElementById('connect-btn'); const connectButton = document.getElementById('connect-btn');
const generateButton = document.getElementById('generate-btn'); const generateButton = document.getElementById('generate-btn');
const stopButton = document.getElementById('stop-btn');
const connectionSection = document.getElementById('connection-section'); const connectionSection = document.getElementById('connection-section');
const generationSection = document.getElementById('generation-section'); const generationSection = document.getElementById('generation-section');
const connectingText = document.getElementById('connecting');
// Append the received chunks to the output div // Append the received chunks to the text div
function appendToOutput(chunk) { function appendToOutput(chunk) {
const outputDiv = document.getElementById('output'); const outputDiv = document.getElementById('text');
outputDiv.innerHTML += chunk; outputDiv.value += chunk;
outputDiv.scrollTop = outputDiv.scrollHeight; outputDiv.scrollTop = outputDiv.scrollHeight;
} }
@ -68,22 +76,45 @@
appendToOutput(data.chunk); appendToOutput(data.chunk);
}); });
// Event handler for receiving generated text chunks
socket.on('text_generated', data => {
console.log('text generated:', data.text);
// Toggle button visibility
generateButton.classList.remove('hidden');
stopButton.classList.add('hidden');
});
// Event handler for successful connection // Event handler for successful connection
socket.on('connect', () => { socket.on('connect', () => {
console.log('Connected to Socket.IO server'); console.log('Connected to LoLLMs server');
connectButton.disabled = true; connectButton.disabled = true;
connectingText.classList.add("hidden")
connectionSection.classList.add('hidden'); connectionSection.classList.add('hidden');
generationSection.classList.remove('hidden'); generationSection.classList.remove('hidden');
}); });
// Event handler for error during text generation // Event handler for error during text generation
socket.on('text_generated_error', error => { socket.on('buzzy', error => {
console.error('Text generation error:', error); console.error('Server is busy. Wait for your turn', error);
const outputDiv = document.getElementById('output'); const outputDiv = document.getElementById('text');
outputDiv.innerHTML += `<p class="text-red-500">Error: ${error.message}</p>`; outputDiv.value += `<p class="text-red-500">Error: ${error.message}</p>`;
outputDiv.scrollTop = outputDiv.scrollHeight; outputDiv.scrollTop = outputDiv.scrollHeight;
// Toggle button visibility
generateButton.classList.remove('hidden');
stopButton.classList.add('hidden');
}); });
// Event handler for error during text generation
socket.on('generation_canceled', error => {
// Toggle button visibility
generateButton.classList.remove('hidden');
stopButton.classList.add('hidden');
});
// Triggered when the "Connect" button is clicked // Triggered when the "Connect" button is clicked
connectButton.addEventListener('click', () => { connectButton.addEventListener('click', () => {
const hostInput = document.getElementById('host'); const hostInput = document.getElementById('host');
@ -94,25 +125,28 @@
if (host && port) { if (host && port) {
socket.io.uri = `http://${host}:${port}`; socket.io.uri = `http://${host}:${port}`;
socket.connect(); socket.connect();
connectButton.disabled = true; connectingText.classList.remove("hidden")
connectionSection.classList.add('hidden');
generationSection.classList.remove('hidden');
} }
}); });
// Triggered when the "Generate Text" button is clicked // Triggered when the "Generate Text" button is clicked
generateButton.addEventListener('click', () => { generateButton.addEventListener('click', () => {
const promptInput = document.getElementById('prompt'); const outputDiv = document.getElementById('text');
const prompt = promptInput.value.trim(); var prompt = outputDiv.value
console.log(prompt)
// Trigger the 'generate_text' event with the prompt
socket.emit('generate_text', { prompt, personality: -1, n_predicts: 1024 });
if (prompt) { // Toggle button visibility
// Clear output div generateButton.classList.add('hidden');
document.getElementById('output').innerHTML = ''; stopButton.classList.remove('hidden');
});
// Triggered when the "Stop Generation" button is clicked
stopButton.addEventListener('click', () => {
// Trigger the 'cancel_generation' event
socket.emit('cancel_generation',{});
// Trigger the 'generate_text' event with the prompt
socket.emit('generate_text', { prompt, personality: -1, n_predicts: 1024 });
promptInput.value = '';
}
}); });
</script> </script>
</body> </body>

0
tests/endoints_unit_tests/example_text_gen.txt → tests/endoints_unit_tests/python/example_text_gen.txt
View File

0
tests/endoints_unit_tests/test_generation.py → tests/endoints_unit_tests/python/test_generation.py
View File