ExternalVendorCode/lollms-webui
1
0
Fork 0
mirror of https://github.com/ParisNeo/lollms-webui.git synced 2024-12-18 20:17:50 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

add lollms markdown renderer

Browse Source
This commit is contained in:
Saifeddine ALOUI 2024-08-27 21:05:20 +02:00
parent 52888ba13c
commit 350d1ce320
5 changed files with 326 additions and 26 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

70
Dockerfile
View File

@ -1,30 +1,50 @@
FROM python:3.10
# Use an official Python runtime as a parent image
FROM python:3.11-slim
WORKDIR /srv
COPY ./requirements.txt .
# Set environment variables
ENV PYTHONNOUSERSITE=1 \
PYTHONPATH="" \
PYTHONHOME="" \
TEMP="/installer_files/temp" \
TMP="/installer_files/temp" \
MINICONDA_DIR="/installer_files/miniconda3" \
INSTALL_ENV_DIR="/installer_files/lollms_env" \
PACKAGES_TO_INSTALL="python=3.11 git pip"
COPY ./app.py /srv/app.py
COPY ./api /srv/api
COPY ./static /srv/static
COPY ./templates /srv/templates
COPY ./web /srv/web
COPY ./assets /srv/assets
# Create necessary directories
RUN mkdir -p /installer_files/temp /installer_files/miniconda3 /installer_files/lollms_env
# TODO: this is monkey-patch for check_update() function, should be disabled in Docker
COPY ./.git /srv/.git
# Install dependencies
RUN apt-get update && apt-get install -y \
curl \
git \
&& rm -rf /var/lib/apt/lists/*
VOLUME [ "/data" ]
# Download and install Miniconda
RUN curl -LO https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh && \
bash Miniconda3-latest-Linux-x86_64.sh -b -p $MINICONDA_DIR && \
rm Miniconda3-latest-Linux-x86_64.sh
# Monkey-patch (1): because "binding_zoo" install the packets inside venv, we cannot do pip install on build time
# Monkey-patch (2): send a "enter" keystroke to python to confirm the first launch process
CMD ["/bin/bash", "-c", " \
python -m venv /data/venv; \
source /data/venv/bin/activate; \
python -m pip install --no-cache-dir -r requirements.txt --upgrade pip; \
echo -ne '\n' | \
python app.py \
--host 0.0.0.0 \
--port 9600 \
--discussion_db_name /data/Documents/databases/database.db \
--config /configs/config.yaml \
"]
# Initialize conda
RUN $MINICONDA_DIR/bin/conda init bash
# Create and activate the conda environment
RUN $MINICONDA_DIR/bin/conda create -y -p $INSTALL_ENV_DIR $PACKAGES_TO_INSTALL && \
$MINICONDA_DIR/bin/conda install -y conda
# Clone the repository and install dependencies
RUN git clone --depth 1 --recurse-submodules https://github.com/ParisNeo/lollms-webui.git && \
cd lollms-webui && \
git submodule update --init --recursive && \
cd lollms_core && \
pip install -e . && \
cd ../utilities/pipmaster && \
pip install -e . && \
cd ../.. && \
pip install -r requirements.txt
# Set the working directory
WORKDIR /lollms-webui
# Default command
CMD ["bash"]

90
endpoints/docs/lollms_markdown_renderer/DOC.md Normal file
View File

@ -0,0 +1,90 @@
Certainly! Here's the updated version of the quick guide using `await` for rendering the Markdown content:
## MarkdownRenderer Library - Quick Guide
The `MarkdownRenderer` library is a JavaScript class for converting Markdown content into HTML with features like code highlighting, LaTeX math rendering, and Mermaid diagrams.
### Key Features
1. **Mermaid Diagrams**: Render diagrams from Markdown code blocks.
2. **Code Highlighting**: Use Prism.js for syntax highlighting.
3. **Math Equations**: Support LaTeX-style math equations.
4. **Tables**: Convert Markdown tables into HTML.
5. **Text Formatting**: Handle headers, bold, italic, links, lists, blockquotes, and horizontal rules.
6. **Copy Code**: Enable copying of code blocks with line numbers.
### Integration Steps
Include the following in your HTML file:
```html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Markdown Renderer Example</title>
<!-- External Libraries -->
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.5.1/styles/default.min.css">
<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.5.1/highlight.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/3.2.0/es5/tex-mml-chtml.js"></script>
<script src="https://cdn.jsdelivr.net/npm/mermaid/dist/mermaid.min.js"></script>
<script src="https://cdn.tailwindcss.com"></script>
<script src="https://cdn.jsdelivr.net/npm/axios/dist/axios.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/animejs@3.2.1/lib/anime.min.js"></script>
<link href="https://cdnjs.cloudflare.com/ajax/libs/prism/1.24.1/themes/prism.min.css" rel="stylesheet">
<script src="https://cdnjs.cloudflare.com/ajax/libs/prism/1.24.1/prism.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/prism/1.24.1/components/prism-javascript.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/prism/1.24.1/components/prism-python.min.js"></script>
<!-- MarkdownRenderer -->
<script src="/lollms_assets/js/lollms_markdown_renderer"></script>
<link rel="stylesheet" href="/lollms_assets/css/lollms_markdown_renderer">
</head>
<body>
<div id="markdown-content"></div>
<script>
(async () => {
const mr = new MarkdownRenderer();
const markdownText = `
# Sample Markdown
**Bold** text, *italic* text.
## Code Block
\`\`\`javascript
console.log('Hello, World!');
\`\`\`
## Mermaid Diagram
\`\`\`mermaid
graph TD;
A-->B;
A-->C;
B-->D;
C-->D;
\`\`\`
## Math Equation
$$E = mc^2$$
## Table
| Name | Age |
|-------|-----|
| Alice | 24 |
| Bob | 30 |
`;
const renderedHtml = await mr.renderMarkdown(markdownText);
document.getElementById('markdown-content').innerHTML = renderedHtml;
})();
</script>
</body>
</html>
```
### Usage
- **Code Blocks**: Use Prism.js for syntax highlighting by specifying the language (e.g., `javascript`).
- **Mermaid Diagrams**: Use `mermaid` identifier in code blocks.
- **Math Equations**: Use `$...$` for inline and `$$...$$` for block equations.
- **Tables**: Automatically convert Markdown tables to HTML.
This guide provides a concise overview for integrating and using the `MarkdownRenderer` library in web applications.

187
endpoints/docs/lollms_markdown_renderer/README.md Normal file
View File

@ -0,0 +1,187 @@
## Documentation for `MarkdownRenderer` Library
The `MarkdownRenderer` library is a JavaScript class designed to render Markdown content into HTML. It supports various features such as code highlighting, LaTeX math rendering, Mermaid diagrams, and more. Below is a detailed explanation of how the library works, how to integrate it into an HTML file, and examples of its usage.
### Features
1. **Mermaid Diagrams**: Renders Mermaid diagrams from Markdown code blocks.
2. **Code Highlighting**: Uses Prism.js for syntax highlighting of code blocks.
3. **Math Equations**: Supports LaTeX-style math equations.
4. **Tables**: Converts Markdown tables into HTML tables.
5. **Text Formatting**: Handles headers, bold, italic, links, lists, blockquotes, and horizontal rules.
6. **Copy Code Functionality**: Allows users to copy code blocks with line numbers.
### Integration into an HTML File
To use the `MarkdownRenderer` library, you need to include several external libraries and stylesheets in your HTML file. Below is an example of how to set up your HTML file:
```html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Markdown Renderer Example</title>
<!-- Highlight.js for code highlighting -->
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.5.1/styles/default.min.css">
<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.5.1/highlight.min.js"></script>
<!-- MathJax for LaTeX math rendering -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/3.2.0/es5/tex-mml-chtml.js"></script>
<!-- Mermaid for graph rendering -->
<script src="https://cdn.jsdelivr.net/npm/mermaid/dist/mermaid.min.js"></script>
<!-- Tailwind CSS -->
<script src="https://cdn.tailwindcss.com"></script>
<!-- Axios for HTTP requests -->
<script src="https://cdn.jsdelivr.net/npm/axios/dist/axios.min.js"></script>
<!-- Anime.js for animations -->
<script src="https://cdn.jsdelivr.net/npm/animejs@3.2.1/lib/anime.min.js"></script>
<!-- Prism CSS and JS for code highlighting -->
<link href="https://cdnjs.cloudflare.com/ajax/libs/prism/1.24.1/themes/prism.min.css" rel="stylesheet">
<script src="https://cdnjs.cloudflare.com/ajax/libs/prism/1.24.1/prism.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/prism/1.24.1/components/prism-javascript.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/prism/1.24.1/components/prism-python.min.js"></script>
<!-- Add more languages as needed -->
<!-- Include the MarkdownRenderer script -->
<script src="/lollms_assets/js/lollms_markdown_renderer"></script>
<link rel="stylesheet" href="/lollms_assets/css/lollms_markdown_renderer">
</head>
<body>
<div id="markdown-content">
<!-- Markdown content will be rendered here -->
</div>
<script>
// Initialize the MarkdownRenderer
const mr = new MarkdownRenderer();
// Example Markdown text
const markdownText = `
# Sample Markdown
This is a **bold** text and this is *italic* text.
## Code Block
\`\`\`javascript
console.log('Hello, World!');
\`\`\`
## Mermaid Diagram
\`\`\`mermaid
graph TD;
A-->B;
A-->C;
B-->D;
C-->D;
\`\`\`
## Math Equation
$$E = mc^2$$
## Table
| Name | Age |
|-------|-----|
| Alice | 24 |
| Bob | 30 |
`;
// Render the Markdown text
mr.renderMarkdown(markdownText).then(renderedHtml => {
document.getElementById('markdown-content').innerHTML = renderedHtml;
});
</script>
</body>
</html>
```
Or using async
```html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Markdown Renderer Example</title>
<!-- External Libraries -->
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.5.1/styles/default.min.css">
<script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.5.1/highlight.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/3.2.0/es5/tex-mml-chtml.js"></script>
<script src="https://cdn.jsdelivr.net/npm/mermaid/dist/mermaid.min.js"></script>
<script src="https://cdn.tailwindcss.com"></script>
<script src="https://cdn.jsdelivr.net/npm/axios/dist/axios.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/animejs@3.2.1/lib/anime.min.js"></script>
<link href="https://cdnjs.cloudflare.com/ajax/libs/prism/1.24.1/themes/prism.min.css" rel="stylesheet">
<script src="https://cdnjs.cloudflare.com/ajax/libs/prism/1.24.1/prism.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/prism/1.24.1/components/prism-javascript.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/prism/1.24.1/components/prism-python.min.js"></script>
<!-- MarkdownRenderer -->
<script src="/lollms_assets/js/lollms_markdown_renderer"></script>
<link rel="stylesheet" href="/lollms_assets/css/lollms_markdown_renderer">
</head>
<body>
<div id="markdown-content"></div>
<script>
(async () => {
const mr = new MarkdownRenderer();
const markdownText = `
# Sample Markdown
**Bold** text, *italic* text.
## Code Block
\`\`\`javascript
console.log('Hello, World!');
\`\`\`
## Mermaid Diagram
\`\`\`mermaid
graph TD;
A-->B;
A-->C;
B-->D;
C-->D;
\`\`\`
## Math Equation
$$E = mc^2$$
## Table
| Name | Age |
|-------|-----|
| Alice | 24 |
| Bob | 30 |
`;
const renderedHtml = await mr.renderMarkdown(markdownText);
document.getElementById('markdown-content').innerHTML = renderedHtml;
})();
</script>
</body>
</html>
```
### Explanation of the Code
- **External Libraries**: The HTML file includes several external libraries such as Highlight.js, MathJax, Mermaid, Tailwind CSS, Axios, Anime.js, and Prism.js. These libraries are necessary for rendering code, math equations, diagrams, and styling.
- **MarkdownRenderer Initialization**: An instance of `MarkdownRenderer` is created and used to render Markdown text into HTML.
- **Rendering Markdown**: The `renderMarkdown` method is called with the Markdown text. It processes the text and converts it into HTML, which is then inserted into the `#markdown-content` div.
### Usage Examples
1. **Rendering Code Blocks**: The library uses Prism.js to highlight code blocks. You can specify the language for syntax highlighting by using the appropriate language identifier (e.g., `javascript`, `python`).
2. **Rendering Mermaid Diagrams**: Wrap your Mermaid code in a code block with the `mermaid` identifier to render diagrams.
3. **Rendering Math Equations**: Use LaTeX-style syntax for math equations. Inline equations can be wrapped in `$...$`, and block equations can be wrapped in `$$...$$`.
4. **Rendering Tables**: Markdown tables are automatically converted into HTML tables with proper styling.
5. **Copy Code Functionality**: Each code block includes a "Copy" button that allows users to copy the code with line numbers.
This documentation provides a comprehensive guide to using the `MarkdownRenderer` library. By following the integration steps and examples, you can easily render Markdown content in your web applications.

3
endpoints/lollms_apps.py
View File

@ -452,6 +452,9 @@ async def lollms_assets(asset_type: str, file_name: str):
# Construct the full file path
file_path = directory / f"{safe_file_name}{file_extension}"
file_path_with_entension = directory / f"{safe_file_name}"
if file_path_with_entension.is_file() and file_path_with_entension.is_relative_to(directory):
file_path = file_path_with_entension
# Check if the file exists and is within the allowed directory
if not file_path.is_file() or not file_path.is_relative_to(directory):

2
zoos/personalities_zoo

@ -1 +1 @@
Subproject commit cb3ed03ce7c6347a683973d21fb2f91e68d20605
Subproject commit 62d69436d4c9f760d1a36bad5b597bd421201662