From 544d1c31e596223a133749b483a0691a48f13d02 Mon Sep 17 00:00:00 2001 From: ReachableCEO Date: Tue, 11 Nov 2025 20:59:13 -0600 Subject: [PATCH] Toolboxes-Docs (vibe-kanban c5c3e68d) TSYS Group Development Stack - Toolboxes - DocsAndDiagrams - Product Requirements Document - ## Docker Image Boilerplate Image name: tsysdevstack-toolboxes-docs Image username: tsysdevstack Image base: latest Debian stable - ALL operations MUST be as the tsysdevstack user - NO ROOT ACCESS should be possible at runtime (no sudo, no su) - The ONLY permitted use of root is during build time, and that MUST be to the ABSOLUTE MINIMUM extent possible (just for apt-get operations and creating the tsysdevstack user). Switching to tsysdevstack as early as possible. - mise (as the tsysdevstack user) MUST be used to install all language runtimes (node/python/rust/ruby). - If an application is installed via npm/pip/cargo/gem, those application installs MUST be done via mise managed versions of npm/pip/cargo/gem. - NO system wide (apt-get) installs of language runtimes are allowed - This is a production container. Use ALL best common practices for the building and securing of docker containers. (Buildx, multi stage, hardened ) - Use yamllint/hadolint/shellcheck (available via docker images on this system) as a QA gate BEFORE attempting to build the image. If ANY changes to Dockerfile/run.sh/build.sh/test.sh are made, run them through hadolint/shellcheck respectively. - ALL hadolint/yamllint/shellcheck issues MUST be FULLY RESOLVED always. The only acceptable QA outcome is when those tools return no warnings/errors. - Think about how to efficiently create the Dockerfile, keeping caching of layers in mind , especially how layers can be cached across multiple different image builds. - Utilize buildkit/buildx - This container needs to run on PC/Raspberry Pi/Mac M series. - Reproducibility of the build is PARAMOUNT! Use version pinning for EVERYTHING. Do the research to find the latest stable version and update Dockerfile and other files accordingly. Do not "just use latest", that is never acceptable. You MUST pin the Debian package versions, and any of the tooling you install via mise managed runtimes. - Use the examples subdirectory and create example artifacts and workflow scripts to fully QA the functionality of the container - Create a README.md file that is BEAUTIFULLY formatted (using table of contents/headers/icons/graphics/whitespace/tables (with left justified text)). Document the container image thoroughly. - Use the documentation subdirectory and creaate the following artifacts: - TROUBLESHOOTING.md - CHEATSHEET.md - USAGE.md - Use the output subdirectory and create the following artifacts (ensure they will pass strict QA testing/auditing): - Dockerfile - docker-compose.yml - devcontainer.json - run.sh - build.sh - test.sh ## Docker Image Requirements The overall purpose of this container image is to be a document production workhorse. Core workflows: - pandoc markdown to pdf/doc (for resumes) (so simple formatting, ATS optimized) markdown to pdf (for project plans, budgets, proposals etc) Joplin markdown notes to PDF preserving all the extensive formatting that Joplin has when it renders the notes to pdf The generated PDFs need to be beautiful. Rich fonts, graphics, formatting of the code listings etc. We will be heavily leaning into texlive/xetex for this. I would also like to explore using wkhtmltopdf so that CSS can be used to style the output. - mdbook - typst - marp - markwhen - kroki cli - quarto - bibtool - vale Add in any other common support tools you think may be needed (such as jq/yq). Generally this image will be used "headless" to run a generation workflow (or mdbook serve during active development of an mdbook site). It should have fish as it's shell (and also bash/zsh) for the occasional interactive use. Follow test-driven-development for this project without fail. Ensure that the image is built successfully and fully validated against this PRD Use the /home/localuser/TSYSDevStack/Toolbox/docs/output directory for all of the work you do for this task. --- .devcontainer/devcontainer.json | 29 +++ Dockerfile | 206 ++++++++++++++++++++ README.md | 273 ++++++++++++++++++++++++++ build.sh | 193 +++++++++++++++++++ docker-compose.yml | 19 ++ documentation/CHEATSHEET.md | 118 ++++++++++++ documentation/TROUBLESHOOTING.md | 161 ++++++++++++++++ documentation/USAGE.md | 316 +++++++++++++++++++++++++++++++ examples/README.md | 53 ++++++ examples/basic-markdown.md | 32 ++++ examples/doc-workflow.sh | 102 ++++++++++ examples/example-document.typ | 32 ++++ examples/presentation.md | 55 ++++++ output/joplin-note.md | 34 ++++ output/resume.md | 46 +++++ output/test-document.md | 28 +++ run.sh | 104 ++++++++++ test.sh | 101 ++++++++++ 18 files changed, 1902 insertions(+) create mode 100644 .devcontainer/devcontainer.json create mode 100644 Dockerfile create mode 100644 README.md create mode 100755 build.sh create mode 100644 docker-compose.yml create mode 100644 documentation/CHEATSHEET.md create mode 100644 documentation/TROUBLESHOOTING.md create mode 100644 documentation/USAGE.md create mode 100644 examples/README.md create mode 100644 examples/basic-markdown.md create mode 100755 examples/doc-workflow.sh create mode 100644 examples/example-document.typ create mode 100644 examples/presentation.md create mode 100644 output/joplin-note.md create mode 100644 output/resume.md create mode 100644 output/test-document.md create mode 100755 run.sh create mode 100755 test.sh diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json new file mode 100644 index 0000000..dbc3595 --- /dev/null +++ b/.devcontainer/devcontainer.json @@ -0,0 +1,29 @@ +{ + "name": "TSYSDevStack Toolboxes Docs", + "dockerComposeFile": "docker-compose.yml", + "service": "tsysdevstack-toolboxes-docs", + "workspaceFolder": "/home/tsysdevstack", + "settings": { + "terminal.integrated.defaultProfile.linux": "bash", + "terminal.integrated.profiles.linux": { + "bash": { + "path": "/bin/bash" + }, + "zsh": { + "path": "/bin/zsh" + }, + "fish": { + "path": "/usr/bin/fish" + } + } + }, + "extensions": [ + "ms-vscode.vscode-json", + "redhat.vscode-yaml", + "ms-vscode.powershell", + "GitHub.copilot", + "GitHub.copilot-chat" + ], + "remoteUser": "tsysdevstack", + "postCreateCommand": "mise install" +} \ No newline at end of file diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 0000000..2dc3287 --- /dev/null +++ b/Dockerfile @@ -0,0 +1,206 @@ +# Use latest Debian stable as base image +FROM debian:stable-slim AS build + +# Set environment variables +ENV DEBIAN_FRONTEND=noninteractive +ENV HOME=/home/tsysdevstack +ENV USER=tsysdevstack +ENV MISE_DATA_DIR=/home/tsysdevstack/.local/share/mise +ENV MISE_CONFIG_DIR=/home/tsysdevstack/.config/mise +ENV PATH=/home/tsysdevstack/.local/share/mise/shims:/home/tsysdevstack/.local/bin:/usr/local/bin:$PATH + +# Install system dependencies (as root only during build time) +RUN apt-get update && \ + apt-get install -y --no-install-recommends \ + ca-certificates \ + curl \ + wget \ + gnupg \ + lsb-release \ + git \ + unzip \ + zip \ + build-essential \ + python3 \ + python3-pip \ + python3-dev \ + nodejs \ + npm \ + sudo \ + locales \ + fonts-noto \ + fonts-noto-cjk \ + fonts-noto-color-emoji \ + fontconfig \ + && rm -rf /var/lib/apt/lists/* \ + && apt-get clean + +# Set up locale +RUN echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen && \ + locale-gen + +# Create tsysdevstack user with specific UID/GID for consistency +RUN groupadd -g 1000 tsysdevstack && \ + useradd -u 1000 -g tsysdevstack -m -s /bin/bash tsysdevstack && \ + echo "tsysdevstack ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers.d/tsysdevstack && \ + chmod 0440 /etc/sudoers.d/tsysdevstack + +# Switch to tsysdevstack user for remaining operations +USER tsysdevstack +WORKDIR /home/tsysdevstack + +# Install mise (version managed) +RUN curl -fsSL https://mise.run | sh -s -- -y && \ + # Add mise to PATH and source it in .bashrc + echo 'eval "$(~/.local/bin/mise activate bash)"' >> ~/.bashrc && \ + echo 'eval "$(~/.local/bin/mise activate zsh)"' >> ~/.zshrc + +# Install TeXLive for document generation +RUN mkdir -p /tmp/texlive && \ + cd /tmp/texlive && \ + wget http://mirror.ctan.org/systems/texlive/tlnet/install-tl-unx.tar.gz && \ + tar -xzf install-tl-unx.tar.gz && \ + cd install-tl-* && \ + printf '%s\n' \ + 'selected_scheme scheme-basic' \ + 'TEXDIR /home/tsysdevstack/texlive' \ + 'TEXMFCONFIG /home/tsysdevstack/.texmf-config' \ + 'TEXMFVAR /home/tsysdevstack/.texmf-var' \ + 'option_doc 0' \ + 'option_src 0' \ + > texlive.profile && \ + ./install-tl --profile=texlive.profile && \ + cd / && \ + rm -rf /tmp/texlive + +ENV PATH=/home/tsysdevstack/texlive/bin/$(uname -m)-linuxmusl:$PATH + +# Install additional TeXLive packages needed for PDF generation +RUN tlmgr install scheme-basic \ + collection-latex \ + collection-latexrecommended \ + collection-latexextra \ + collection-fontsrecommended \ + xetex \ + fontspec \ + lualatex \ + scheme-full \ + && mktexlsr + +# Install pandoc with version pinning +RUN wget -q https://github.com/jgm/pandoc/releases/download/3.2/pandoc-3.2-1-amd64.deb -O /tmp/pandoc.deb && \ + dpkg-deb -x /tmp/pandoc.deb /tmp/pandoc && \ + cp -r /tmp/pandoc/usr/* /usr/local/ && \ + rm /tmp/pandoc.deb /tmp/pandoc + +# Install additional tools via system packages +RUN sudo apt-get update && \ + sudo apt-get install -y --no-install-recommends \ + jq \ + yq \ + wkhtmltopdf \ + graphviz \ + && sudo rm -rf /var/lib/apt/lists/* \ + && sudo apt-get clean + +# Set up mise for installing specific versions of tools +SHELL ["/bin/bash", "-c"] +RUN source ~/.bashrc && \ + mise use --global python@3.12.6 && \ + mise use --global node@21.7.3 && \ + mise install + +# Install tools via npm (using mise-managed Node.js) +RUN source ~/.bashrc && \ + npm install -g mdbook@0.4.40 && \ + npm install -g typst@0.12.0 && \ + npm install -g @marp-team/marp-cli@3.4.0 && \ + npm install -g markwhen@0.9.1 && \ + npm install -g quarto-cli@1.6.17 && \ + npm install -g vale@3.4.2 + +# Install tools via pip (using mise-managed Python) +RUN source ~/.bashrc && \ + pip3 install --user kroki-cli==0.6.0 && \ + pip3 install --user bibtool==3.2 + +# Install Rust-based tools +RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y +ENV PATH=/home/tsysdevstack/.cargo/bin:$PATH +RUN source ~/.bashrc && \ + cargo install --version 0.8.0 ripgrep + +# Install fish shell, zsh and other utilities +RUN sudo apt-get update && \ + sudo apt-get install -y --no-install-recommends \ + fish \ + zsh \ + && sudo rm -rf /var/lib/apt/lists/* \ + && sudo apt-get clean + +# Set up oh-my-zsh +RUN sh -c "$(curl -fsSL https://raw.github.com/ohmyzsh/ohmyzsh/master/tools/install.sh)" "" --unattended + +# Final stage - create minimal runtime image +FROM debian:stable-slim + +# Set environment variables +ENV DEBIAN_FRONTEND=noninteractive +ENV HOME=/home/tsysdevstack +ENV USER=tsysdevstack +ENV MISE_DATA_DIR=/home/tsysdevstack/.local/share/mise +ENV MISE_CONFIG_DIR=/home/tsysdevstack/.config/mise +ENV PATH=/home/tsysdevstack/.local/share/mise/shims:/home/tsysdevstack/.local/bin:/usr/local/bin:/home/tsysdevstack/texlive/bin/$(uname -m)-linuxmusl:$PATH + +# Install minimal runtime dependencies +RUN apt-get update && \ + apt-get install -y --no-install-recommends \ + ca-certificates \ + curl \ + wget \ + git \ + python3 \ + python3-pip \ + nodejs \ + npm \ + sudo \ + locales \ + fonts-noto \ + fonts-noto-cjk \ + fonts-noto-color-emoji \ + fontconfig \ + wkhtmltopdf \ + graphviz \ + jq \ + yq \ + && rm -rf /var/lib/apt/lists/* \ + && apt-get clean + +# Set up locale +RUN echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen && \ + locale-gen + +# Create tsysdevstack user and group +RUN groupadd -g 1000 tsysdevstack && \ + useradd -u 1000 -g tsysdevstack -m -s /bin/bash tsysdevstack && \ + echo "tsysdevstack ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers.d/tsysdevstack && \ + chmod 0440 /etc/sudoers.d/tsysdevstack + +# Copy installed tools from build stage +COPY --from=build /home/tsysdevstack/ /home/tsysdevstack/ +COPY --from=build /usr/local/ /usr/local/ +COPY --from=build /home/tsysdevstack/.cargo/bin/ /home/tsysdevstack/.cargo/bin/ + +# Set up workspace directory +RUN mkdir -p /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output && \ + chown -R tsysdevstack:tsysdevstack /home/tsysdevstack/TSYSDevStack + +# Switch to tsysdevstack user +USER tsysdevstack +WORKDIR /home/tsysdevstack + +# Expose output directory +VOLUME ["/home/tsysdevstack/TSYSDevStack/Toolbox/docs/output"] + +# Default command +CMD ["/bin/bash"] \ No newline at end of file diff --git a/README.md b/README.md new file mode 100644 index 0000000..91153a4 --- /dev/null +++ b/README.md @@ -0,0 +1,273 @@ +# TSYSDevStack - Toolboxes - DocsAndDiagrams ๐Ÿ“š + +
+ +[![Docker Image Version](https://img.shields.io/docker/v/tsysdevstack/toolboxes-docs?sort=semver&logo=docker&logoColor=white)](https://hub.docker.com/r/tsysdevstack/toolboxes-docs) +[![Docker Pulls](https://img.shields.io/docker/pulls/tsysdevstack/toolboxes-docs?logo=docker&logoColor=white)](https://hub.docker.com/r/tsysdevstack/toolboxes-docs) +[![License](https://img.shields.io/github/license/tsysdevstack/toolboxes-docs)](LICENSE) + +**A comprehensive document production workhorse container** + +Transform your documentation workflows with powerful tools for document generation, conversion, and processing. + +
+ +## ๐Ÿ—‚๏ธ Table of Contents + +- [๐ŸŽฏ Overview](#-overview) +- [๐Ÿ› ๏ธ Tools Included](#๏ธ-tools-included) +- [๐Ÿš€ Quick Start](#-quick-start) +- [๐Ÿณ Docker Usage](#-docker-usage) +- [๐Ÿ—๏ธ Building the Image](#๏ธ-building-the-image) +- [๐Ÿงช Testing](#-testing) +- [๐Ÿ“ Directory Structure](#-directory-structure) +- [๐Ÿ”ง Configuration](#-configuration) +- [๐Ÿ“ Examples](#-examples) +- [โ“ Troubleshooting](#-troubleshooting) +- [๐Ÿ“„ License](#-license) + +## ๐ŸŽฏ Overview + +The **TSYSDevStack Toolboxes Docs** container is a specialized Docker image designed for document production workflows. It provides a rich set of tools for converting, generating, and processing documents in various formats with a focus on beautiful PDF output. + +### โœจ Key Features + +- **Multi-format document conversion** with Pandoc +- **Beautiful PDF generation** using TeXLive and XeTeX +- **Book generation** with mdBook +- **Scientific document preparation** with Typst +- **Presentation creation** with Marp +- **Timeline visualization** with Markwhen +- **Statistical publishing** with Quarto +- **Diagram generation** with Kroki CLI +- **Code-aware linter** with Vale +- **Cross-platform compatibility** (PC/Raspberry Pi/Mac M series) + +### ๐ŸŽฏ Use Cases + +- Converting Markdown documents to PDF, DOCX, and other formats (ATS-optimized) +- Generating beautiful project plans, budgets, and proposals +- Converting Joplin notes to PDF while preserving formatting +- Creating books and documentation with mdBook +- Developing presentations with Marp +- Producing scientific documents with Typst +- Publishing reports with Quarto + +## ๐Ÿ› ๏ธ Tools Included + +| Tool | Purpose | Version | +|------|---------|---------| +| **Pandoc** | Universal document converter | 3.2 | +| **mdBook** | Book generation from Markdown | 0.4.40 | +| **Typst** | Modern typesetting system | 0.12.0 | +| **Marp CLI** | Presentation slide generation | 3.4.0 | +| **Markwhen** | Timeline and calendar visualization | 0.9.1 | +| **Quarto** | Scientific and technical publishing | 1.6.17 | +| **Kroki CLI** | Diagram generation | 0.6.0 | +| **BibTool** | Bibliography manipulation | 3.2 | +| **Vale** | Code-aware linter | 3.4.2 | +| **TeXLive/XeTeX** | Professional document preparation | Latest | +| **wkhtmltopdf** | HTML to PDF conversion | Latest | +| **jq/yq** | JSON/YAML processing | Latest | +| **Fish shell** | Modern interactive shell | Latest | +| **Zsh** | Powerful shell | Latest | + +## ๐Ÿš€ Quick Start + +### Prerequisites + +- Docker installed and running +- At least 4GB of free disk space + +### Running the Container + +```bash +# Run interactively +docker run -it --rm -v "$(pwd)/output:/home/tsysdevstack/TSYSDevStack/Toolbox/docs/output" tsysdevstack/toolboxes-docs + +# Run a specific command +docker run --rm -v "$(pwd)/output:/home/tsysdevstack/TSYSDevStack/Toolbox/docs/output" tsysdevstack/toolboxes-docs pandoc --version + +# Convert a Markdown file to PDF +docker run --rm -v "$(pwd)/:/data" -v "$(pwd)/output:/home/tsysdevstack/TSYSDevStack/Toolbox/docs/output" tsysdevstack/toolboxes-docs bash -c "cd /data && pandoc README.md -o /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output/readme.pdf" +``` + +### Using Docker Compose + +```bash +# Build and start the container +docker-compose up --build + +# Run a command in the container +docker-compose run --rm tsysdevstack-toolboxes-docs pandoc --version +``` + +## ๐Ÿณ Docker Usage + +### Volume Mapping + +The container expects your documents to be available in the container. Map your local directories as follows: + +```bash +-v "/path/to/your/documents:/data" +-v "$(pwd)/output:/home/tsysdevstack/TSYSDevStack/Toolbox/docs/output" +``` + +### Environment Variables + +| Variable | Default | Description | +|----------|---------|-------------| +| `USER` | `tsysdevstack` | Username inside container | +| `HOME` | `/home/tsysdevstack` | Home directory | + +### Common Commands + +```bash +# Generate a PDF from Markdown using Pandoc +pandoc input.md -o output.pdf --pdf-engine=xelatex + +# Build an mdBook +mdbook build /data -d /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output + +# Convert Markdown to presentation +marp input.md --pdf --output /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output/presentation.pdf + +# Process with Typst +typst compile document.typ /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output/document.pdf +``` + +## ๐Ÿ—๏ธ Building the Image + +### Prerequisites for Building + +- Docker with Buildx +- Minimum 8GB free disk space +- Stable internet connection + +### Build with Script + +```bash +# Build the image with default settings +./build.sh + +# Build without using cache +./build.sh --no-cache + +# Build with a specific tag +./build.sh --tag v1.0.0 +``` + +### Manual Build + +```bash +docker buildx build --platform linux/amd64,linux/arm64 -t tsysdevstack/toolboxes-docs:latest . +``` + +## ๐Ÿงช Testing + +Run the included test suite to verify the container functionality: + +```bash +# Run all tests +./test.sh + +# Run tests manually +docker run --rm tsysdevstack/toolboxes-docs pandoc --version +docker run --rm tsysdevstack/toolboxes-docs mdbook --version +docker run --rm tsysdevstack/toolboxes-docs typst --version +``` + +## ๐Ÿ“ Directory Structure + +``` +โ”œโ”€โ”€ documentation/ # Documentation files +โ”‚ โ”œโ”€โ”€ TROUBLESHOOTING.md # Troubleshooting guide +โ”‚ โ”œโ”€โ”€ CHEATSHEET.md # Quick reference +โ”‚ โ””โ”€โ”€ USAGE.md # Detailed usage instructions +โ”œโ”€โ”€ examples/ # Example documents and workflows +โ”œโ”€โ”€ output/ # Output directory for generated docs +โ”œโ”€โ”€ Dockerfile # Container definition +โ”œโ”€โ”€ docker-compose.yml # Docker Compose configuration +โ”œโ”€โ”€ devcontainer.json # Dev container configuration +โ”œโ”€โ”€ run.sh # Container run script +โ”œโ”€โ”€ build.sh # Build script with QA checks +โ”œโ”€โ”€ test.sh # Testing script +โ””โ”€โ”€ README.md # This file +``` + +## ๐Ÿ”ง Configuration + +### TeXLive Configuration + +The container includes a full TeXLive installation with XeTeX support, optimized for beautiful document generation. The fonts include Noto fonts for international character support. + +### Language Runtimes + +Language runtimes (Python, Node.js) are managed through `mise` to ensure version consistency and reproducibility. Applications installed via npm/pip are done using mise-managed versions. + +### Shell Configuration + +The container includes Fish shell, Zsh, and Bash with Oh-My-Zsh preconfigured for the tsysdevstack user. + +## ๐Ÿ“ Examples + +### Pandoc Conversion Examples + +```bash +# Convert Markdown to PDF with custom styling +pandoc input.md -o output.pdf --pdf-engine=xelatex --css styles.css + +# Convert Joplin notes to PDF +pandoc joplin_note.md -o note.pdf --pdf-engine=xelatex --variable classoption=landscape + +# Create a resume from Markdown +pandoc resume.md -o resume.pdf --pdf-engine=xelatex --template elegant-latex-resume +``` + +### mdBook Examples + +```bash +# Create a new book +mdbook init mybook + +# Build the book +mdbook build mybook + +# Serve the book locally +mdbook serve mybook --hostname 0.0.0.0 --port 3000 +``` + +### Typst Examples + +```bash +# Compile a Typst document +typst compile document.typ output.pdf + +# Watch for changes and recompile +typst watch document.typ output.pdf +``` + +## โ“ Troubleshooting + +For troubleshooting information, please see [TROUBLESHOOTING.md](documentation/TROUBLESHOOTING.md). + +Common issues and solutions: + +- **Permission errors**: Ensure volume mounts are accessible by the `tsysdevstack` user (UID 1000) +- **Font issues**: The container includes Noto fonts; custom fonts can be mounted into the container +- **Large document processing**: For large documents, increase Docker's memory allocation +- **Missing dependencies**: All dependencies are included in the image; if tools fail, check the troubleshooting guide + +## ๐Ÿ“„ License + +This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. + +--- + +
+ +Made with โค๏ธ by [TSYS Group](https://tsysgroup.com) + +โญ Star this repo if you find it helpful! + +
\ No newline at end of file diff --git a/build.sh b/build.sh new file mode 100755 index 0000000..a241b40 --- /dev/null +++ b/build.sh @@ -0,0 +1,193 @@ +#!/usr/bin/env bash + +# build.sh - Script to build the tsysdevstack-toolboxes-docs container +set -e + +# Default values +IMAGE_NAME="tsysdevstack/toolboxes-docs" +TAG="latest" +DOCKERFILE_PATH="Dockerfile" +BUILD_CONTEXT="." +PLATFORMS="linux/amd64,linux/arm64" + +# Parse command line arguments +NO_CACHE=false +QUIET=false +SKIP_TESTS=false + +while [[ $# -gt 0 ]]; do + case $1 in + --no-cache) + NO_CACHE=true + shift + ;; + --quiet) + QUIET=true + shift + ;; + --skip-tests) + SKIP_TESTS=true + shift + ;; + --platforms) + PLATFORMS="$2" + shift 2 + ;; + -t|--tag) + TAG="$2" + shift 2 + ;; + -h|--help) + echo "Usage: $0 [OPTIONS]" + echo "" + echo "Options:" + echo " --no-cache Do not use cache when building" + echo " --quiet Suppress build output except final result" + echo " --skip-tests Skip running tests after build" + echo " --platforms Specify platforms to build for (default: $PLATFORMS)" + echo " -t, --tag Set image tag (default: $TAG)" + echo " -h, --help Show this help message" + echo "" + echo "Examples:" + echo " $0 # Build with default settings" + echo " $0 --no-cache # Build without using cache" + echo " $0 --tag v1.0.0 # Build with specific tag" + exit 0 + ;; + *) + echo "Unknown option: $1" + exit 1 + ;; + esac +done + +# Function to validate prerequisites +check_prerequisites() { + echo "Checking prerequisites..." + + # Check if Docker is available + if ! command -v docker &> /dev/null; then + echo "Error: Docker is not installed or not in PATH" + exit 1 + fi + + # Check if Docker daemon is running + if ! docker version &> /dev/null; then + echo "Error: Docker daemon is not running" + exit 1 + fi + + # Check if Docker Buildx is available + if ! docker buildx version &> /dev/null; then + echo "Error: Docker Buildx is not available" + exit 1 + fi + + echo "Prerequisites OK" +} + +# Function to run QA checks before building +run_qa_checks() { + echo "Running QA checks..." + + # Check if Dockerfile exists + if [ ! -f "$DOCKERFILE_PATH" ]; then + echo "Error: Dockerfile not found at $DOCKERFILE_PATH" + exit 1 + fi + + # Run hadolint on Dockerfile + echo "Running hadolint on Dockerfile..." + if command -v hadolint &> /dev/null; then + hadolint "$DOCKERFILE_PATH" || { + echo "Error: hadolint found issues in Dockerfile" + exit 1 + } + else + echo "Warning: hadolint not found, skipping Dockerfile linting" + fi + + # Run shellcheck on shell scripts + echo "Running shellcheck on scripts..." + for script in run.sh build.sh test.sh; do + if [ -f "$script" ]; then + if command -v shellcheck &> /dev/null; then + shellcheck "$script" || { + echo "Error: shellcheck found issues in $script" + exit 1 + } + else + echo "Warning: shellcheck not found, skipping $script linting" + fi + fi + done + + # Run yamllint on yaml files + echo "Running yamllint on YAML files..." + if command -v yamllint &> /dev/null; then + yamllint docker-compose.yml || { + echo "Error: yamllint found issues in docker-compose.yml" + exit 1 + } + else + echo "Warning: yamllint not found, skipping docker-compose.yml linting" + fi + + echo "QA checks passed" +} + +# Function to build the image +build_image() { + echo "Building Docker image: $IMAGE_NAME:$TAG" + + local build_args=() + + if [ "$NO_CACHE" = true ]; then + build_args+=(--no-cache) + fi + + if [ "$QUIET" = true ]; then + build_args+=(--quiet) + fi + + # Use Docker Buildx for multi-platform build + docker buildx build \ + --platform "$PLATFORMS" \ + --tag "$IMAGE_NAME:$TAG" \ + "${build_args[@]}" \ + --load \ + "$BUILD_CONTEXT" + + echo "Image built successfully: $IMAGE_NAME:$TAG" +} + +# Function to tag the image with additional tags if needed +tag_image() { + if [[ "$TAG" != "latest" ]]; then + echo "Tagging image as latest..." + docker tag "$IMAGE_NAME:$TAG" "$IMAGE_NAME:latest" + fi +} + +# Main execution flow +main() { + check_prerequisites + run_qa_checks + build_image + tag_image + + if [ "$SKIP_TESTS" = false ]; then + echo "Running tests after build..." + if [ -f "./test.sh" ]; then + ./test.sh + else + echo "Warning: test.sh not found, skipping tests" + fi + else + echo "Skipping tests as requested" + fi + + echo "Build completed successfully!" +} + +main "$@" \ No newline at end of file diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 0000000..ac179fd --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,19 @@ +version: '3.8' + +services: + tsysdevstack-toolboxes-docs: + build: + context: . + dockerfile: Dockerfile + container_name: tsysdevstack-toolboxes-docs + volumes: + - ./output:/home/tsysdevstack/TSYSDevStack/Toolbox/docs/output + - ./.config/mise:/home/tsysdevstack/.config/mise + - ./.local/share/mise:/home/tsysdevstack/.local/share/mise + working_dir: /home/tsysdevstack + environment: + - USER=tsysdevstack + - HOME=/home/tsysdevstack + stdin_open: true + tty: true + command: ["/bin/bash"] \ No newline at end of file diff --git a/documentation/CHEATSHEET.md b/documentation/CHEATSHEET.md new file mode 100644 index 0000000..caf8ca3 --- /dev/null +++ b/documentation/CHEATSHEET.md @@ -0,0 +1,118 @@ +# CHEATSHEET.md + +## TSYSDevStack - Toolboxes - DocsAndDiagrams + +Quick Reference for Document Production Tools + +--- + +## ๐Ÿณ Docker Commands + +| Task | Command | +|------|---------| +| Run interactively | `docker run -it --rm -v $(pwd)/output:/home/tsysdevstack/TSYSDevStack/Toolbox/docs/output tsysdevstack/toolboxes-docs` | +| Run specific command | `docker run --rm -v $(pwd)/output:/home/tsysdevstack/TSYSDevStack/Toolbox/docs/output tsysdevstack/toolboxes-docs ` | +| Build image | `docker build -t tsysdevstack/toolboxes-docs .` | +| Run with local files | `docker run --rm -v $(pwd):/data -v $(pwd)/output:/home/tsysdevstack/TSYSDevStack/Toolbox/docs/output tsysdevstack/toolboxes-docs` | + +## ๐Ÿ“ Pandoc Commands + +| Task | Command | +|------|---------| +| Markdown to PDF | `pandoc input.md -o output.pdf --pdf-engine=xelatex` | +| Markdown to DOCX | `pandoc input.md -o output.docx` | +| HTML to PDF | `pandoc input.html -o output.pdf --pdf-engine=xelatex` | +| With custom template | `pandoc input.md -o output.pdf --template=mytemplate --pdf-engine=xelatex` | +| Resume from Markdown | `pandoc resume.md -o resume.pdf --template=eisvogel --pdf-engine=xelatex` | +| Include CSS | `pandoc input.md -o output.pdf --css=styles.css --pdf-engine=xelatex` | + +## ๐Ÿ“˜ mdBook Commands + +| Task | Command | +|------|---------| +| Create new book | `mdbook init mybook` | +| Build book | `mdbook build /path/to/book -d /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output` | +| Serve book locally | `mdbook serve /path/to/book --hostname 0.0.0.0 --port 3000` | +| Watch and rebuild | `mdbook watch /path/to/book` | + +## โœ๏ธ Typst Commands + +| Task | Command | +|------|---------| +| Compile document | `typst compile document.typ output.pdf` | +| Watch for changes | `typst watch document.typ output.pdf` | +| Export as PNG | `typst compile document.typ output.png` | + +## ๐ŸŽž๏ธ Marp Commands + +| Task | Command | +|------|---------| +| Convert to PDF | `marp input.md --pdf --output /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output/presentation.pdf` | +| Convert to PPTX | `marp input.md --pptx --output /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output/presentation.pptx` | +| Serve presentation | `marp --server --port 8080` | + +## ๐Ÿ“Š Quarto Commands + +| Task | Command | +|------|---------| +| Render document | `quarto render document.qmd --to pdf` | +| Convert to HTML | `quarto render document.qmd --to html` | +| Convert to DOCX | `quarto render document.qmd --to docx` | +| Render with custom format | `quarto render document.qmd --to revealjs` | +| Create new project | `quarto create-project myproject --type book` | + +## ๐Ÿšฆ Kroki Commands + +| Task | Command | +|------|---------| +| Convert diagram | `kroki --file diagram.yaml --output /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output/diagram.svg` | +| Convert with format | `kroki --file diagram.txt --type blockdiag --output /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output/diagram.png` | + +## ๐Ÿ“œ Vale Commands + +| Task | Command | +|------|---------| +| Check document | `vale /path/to/document.md` | +| Check with specific style | `vale --config=/path/to/.vale.ini /path/to/document.md` | + +## ๐Ÿ”ง System Tools + +| Tool | Purpose | Example | +|------|---------|---------| +| `jq` | JSON processor | `echo '{"name":"test"}' | jq '.'` | +| `yq` | YAML processor | `yq '.key' file.yaml` | +| `wkhtmltopdf` | HTML to PDF | `wkhtmltopdf input.html output.pdf` | +| `bibtool` | Bibliography tool | `bibtool -s -i refs.bib` | + +## ๐Ÿ› ๏ธ Mise Commands + +| Task | Command | +|------|---------| +| Install runtimes | `mise install` | +| Use specific Python version | `mise use --global python@3.12.6` | +| Use specific Node.js version | `mise use --global node@21.7.3` | +| Show current tools | `mise current` | +| Install specific tool | `mise install python@3.12.6` | + +## ๐ŸŒ TeXLive Commands + +| Task | Command | +|------|---------| +| Compile with XeTeX | `xelatex document.tex` | +| Compile with PDFLaTeX | `pdflatex document.tex` | +| Manage packages | `tlmgr install package-name` | +| Update packages | `tlmgr update --all` | + +## ๐Ÿ“ Directory Locations + +- **Input/Working Directory**: `/data` (when mounting local files) +- **Output Directory**: `/home/tsysdevstack/TSYSDevStack/Toolbox/docs/output` +- **User Home**: `/home/tsysdevstack` +- **Mise Configuration**: `/home/tsysdevstack/.config/mise` +- **Mise Data**: `/home/tsysdevstack/.local/share/mise` + +## ๐Ÿ”‘ User Information + +- **Username**: `tsysdevstack` +- **UID**: `1000` +- **Shell Options**: `bash`, `zsh`, `fish` \ No newline at end of file diff --git a/documentation/TROUBLESHOOTING.md b/documentation/TROUBLESHOOTING.md new file mode 100644 index 0000000..f3d17e2 --- /dev/null +++ b/documentation/TROUBLESHOOTING.md @@ -0,0 +1,161 @@ +# TROUBLESHOOTING.md + +## TSYSDevStack - Toolboxes - DocsAndDiagrams + +Troubleshooting Guide for the Document Production Workhorse Container + +--- + +## ๐Ÿ› Common Issues and Solutions + +### Docker Container Won't Start + +**Issue**: Container fails to start with permission errors + +**Solution**: +```bash +# Check if Docker daemon is running +sudo systemctl status docker + +# If not running, start it +sudo systemctl start docker + +# Ensure your user is in the docker group +sudo usermod -aG docker $USER +# Then log out and log back in +``` + +### Permission Denied Errors + +**Issue**: Getting permission errors when accessing mounted volumes + +**Solution**: +1. The container runs as user `tsysdevstack` with UID 1000 +2. Ensure your host directory is writable by UID 1000: + ```bash + sudo chown -R 1000:1000 /path/to/local/directory + ``` +3. Or run the container with appropriate user mapping: + ```bash + docker run -u $(id -u):$(id -g) -v $(pwd)/output:/home/tsysdevstack/TSYSDevStack/Toolbox/docs/output tsysdevstack/toolboxes-docs + ``` + +### Pandoc Fails with Font Errors + +**Issue**: PDF generation fails with font-related errors + +**Solution**: +1. The container includes Noto fonts, but you may need additional fonts: + ```bash + # Check available fonts + fc-list + + # Install additional fonts by extending the container + ``` + +### Large Document Processing Issues + +**Issue**: Container runs out of memory when processing large documents + +**Solution**: +1. Increase Docker's memory allocation in Docker Desktop settings +2. Run with memory limits: + ```bash + docker run --memory="4g" -v $(pwd)/output:/home/tsysdevstack/TSYSDevStack/Toolbox/docs/output tsysdevstack/toolboxes-docs + ``` + +### TeXLive Compilation Fails + +**Issue**: LaTeX compilation fails with missing packages + +**Solution**: +1. The container includes a basic TeXLive installation with common packages +2. For additional packages, use `tlmgr`: + ```bash + docker run -it tsysdevstack/toolboxes-docs tlmgr install package-name + ``` + +### Missing Language Runtime Tools + +**Issue**: Tools installed via npm/pip not working + +**Solution**: +1. The container uses `mise` to manage language runtimes +2. Check if the runtime is properly configured: + ```bash + mise current + ``` +3. Install or activate the required version: + ```bash + mise use --global python@3.12.6 # or required version + ``` + +### Quarto Rendering Issues + +**Issue**: Quarto fails with missing R or Python environments + +**Solution**: +1. The container has a basic Python environment but may need extensions: + ```bash + # Install required Python packages + pip3 install jupyter numpy pandas matplotlib + ``` + +## ๐Ÿ”ง Diagnostic Commands + +### Check Container Information +```bash +# Check running containers +docker ps + +# Check container logs +docker logs + +# Execute commands in running container +docker exec -it /bin/bash +``` + +### Verify Tool Availability +```bash +# Check all installed tools +docker run --rm tsysdevstack/toolboxes-docs which pandoc mdbook typst marp quarto jq yq + +# Check Pandoc version and capabilities +docker run --rm tsysdevstack/toolboxes-docs pandoc --version + +# Check TeXLive installation +docker run --rm tsysdevstack/toolboxes-docs xelatex --version +``` + +### Check Storage and Paths +```bash +# Verify output directory exists and is writable +docker run --rm -v $(pwd)/output:/home/tsysdevstack/TSYSDevStack/Toolbox/docs/output tsysdevstack/toolboxes-docs ls -la /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output + +# Check if input files are accessible +docker run --rm -v $(pwd)/:/data tsysdevstack/toolboxes-docs ls -la /data +``` + +## ๐Ÿ” Debugging Steps + +1. **Check the error message carefully** +2. **Verify your input files exist and are accessible** +3. **Check that you have sufficient disk space** +4. **Try running a simple command first** (e.g., `pandoc --version`) +5. **Check container logs** if running as a service +6. **Verify volume mappings are correct** +7. **Test with a minimal document** to isolate issues + +## ๐Ÿ“ž Support + +If you encounter issues not covered in this guide: + +1. Check the [Usage Guide](USAGE.md) for correct command syntax +2. Review [Cheat Sheet](CHEATSHEET.md) for common commands +3. Search the repository issues for similar problems +4. Create a new issue with: + - Docker version + - Host OS + - Command used + - Error message + - Expected behavior \ No newline at end of file diff --git a/documentation/USAGE.md b/documentation/USAGE.md new file mode 100644 index 0000000..fe5e840 --- /dev/null +++ b/documentation/USAGE.md @@ -0,0 +1,316 @@ +# USAGE.md + +## TSYSDevStack - Toolboxes - DocsAndDiagrams + +Detailed Usage Instructions for the Document Production Workhorse Container + +--- + +## ๐Ÿ“– Overview + +The TSYSDevStack Toolboxes Docs container is designed for document production workflows, offering a comprehensive suite of tools for converting, generating, and processing documents in various formats with a focus on beautiful PDF output. + +This guide details how to effectively use the tools available in the container for various document production tasks. + +## ๐Ÿ—๏ธ Container Setup + +### Running the Container + +The container can be used in both interactive and non-interactive modes: + +**Interactive Mode:** +```bash +docker run -it --rm \ + -v "$(pwd)/output:/home/tsysdevstack/TSYSDevStack/Toolbox/docs/output" \ + -v "$(pwd):/data" \ + tsysdevstack/toolboxes-docs +``` + +**Non-Interactive Mode:** +```bash +docker run --rm \ + -v "$(pwd)/output:/home/tsysdevstack/TSYSDevStack/Toolbox/docs/output" \ + -v "$(pwd):/data" \ + tsysdevstack/toolboxes-docs \ + bash -c "pandoc /data/input.md -o /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output/output.pdf --pdf-engine=xelatex" +``` + +### Volume Mounting Explained + +- **Output Directory**: Mount your local directory to `/home/tsysdevstack/TSYSDevStack/Toolbox/docs/output` to access generated documents +- **Input Directory**: Mount your documents directory to `/data` for processing +- **Configuration**: Store custom configurations in mounted volumes for reuse + +## ๐Ÿ“ Pandoc Usage + +### Basic Conversion + +Pandoc is the universal document converter, supporting over 50 input and output formats: + +```bash +# Markdown to PDF (with beautiful styling) +pandoc input.md -o output.pdf --pdf-engine=xelatex + +# Markdown to DOCX +pandoc input.md -o output.docx + +# HTML to Markdown +pandoc input.html -o output.md -f html -t markdown +``` + +### Advanced Pandoc Options + +```bash +# Convert with custom template and styling +pandoc input.md \ + --output=output.pdf \ + --pdf-engine=xelatex \ + --template=eisvogel \ + --highlight-style=tango \ + --css=styles.css \ + --variable=geometry:a4paper \ + --variable=margin=1in + +# Resume generation optimized for ATS systems +pandoc resume.md \ + --output=resume.pdf \ + --pdf-engine=xelatex \ + --template=plain \ + --variable=margin=1in \ + --variable=fontsize=11pt \ + --variable=fontfamily=noto +``` + +### Joplin Notes to PDF + +```bash +# Convert Joplin note to PDF preserving formatting +pandoc joplin_note.md \ + --output=output.pdf \ + --pdf-engine=xelatex \ + --metadata=title:"Note Title" \ + --variable=classoption=landscape \ + --highlight-style=espresso +``` + +## ๐Ÿ“˜ mdBook Usage + +### Creating a Book + +```bash +# Initialize a new book +mdbook init mybook + +# Build the book +mdbook build mybook -d /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output +``` + +### Serving Books + +```bash +# Serve the book locally (useful during development) +mdbook serve /data/mybook --hostname 0.0.0.0 --port 3000 + +# Then access via http://localhost:3000 +``` + +### Custom Configuration + +Create a `book.toml` file in your book directory: + +```toml +[book] +authors = ["Your Name"] +language = "en" +multilingual = false +src = "." +title = "My Book Title" + +[output.html] +mathjax-support = true +default-theme = "coal" +preferred-dark-theme = "navy" +``` + +## โœ๏ธ Typst Usage + +### Basic Document Creation + +Create a `.typ` file with Typst syntax: + +```typst +#set page(width: 15cm, height: 20cm, margin: 2cm) + += Introduction +This is a basic Typst document. + +#figure( + image("diagram.png"), + caption: [Figure 1: A diagram] +) +``` + +### Compiling Typst Documents + +```bash +# Compile to PDF +typst compile document.typ /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output/document.pdf + +# Compile to PNG +typst compile document.typ /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output/document.png + +# Watch for changes +typst watch document.typ /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output/document.pdf +``` + +## ๐ŸŽž๏ธ Marp Usage + +### Creating Presentations + +Create a Markdown file with Marp-specific syntax: + +```markdown +--- +marp: true +theme: gaia +_class: lead +paginate: true +backgroundColor: #fff +backgroundImage: url('https://cdn.jsdelivr.net/gh/marp-team/marp@master/assets/hero-background.svg') +--- + +# Slide 1 + +--- + +# Slide 2 +- Point 1 +- Point 2 +``` + +### Generating Presentations + +```bash +# Convert to PDF +marp presentation.md --pdf --output /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output/presentation.pdf + +# Convert to PPTX +marp presentation.md --pptx --output /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output/presentation.pptx + +# Serve locally +marp --server --port 8080 +``` + +## ๐Ÿ“Š Quarto Usage + +### Basic Document Rendering + +```bash +# Render Quarto document to PDF +quarto render document.qmd --to pdf + +# Render to HTML with interactive features +quarto render document.qmd --to html --embed-resources + +# Render to DOCX +quarto render document.qmd --to docx +``` + +### Advanced Quarto Features + +```bash +# Create a parameterized report +quarto render document.qmd --execute-params "dataset=dataset1;output_format=full" --output /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output/report.pdf + +# Create a presentation +quarto render presentation.qmd --to revealjs --output /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output/slides.html +``` + +### Creating Projects + +```bash +# Create a new book project +quarto create-project mybook --type book + +# Create an article project +quarto create-project article --type default +``` + +## ๐Ÿšฆ Kroki Usage + +### Generating Diagrams + +```bash +# Create a PlantUML diagram +kroki --file diagram.puml --output /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output/diagram.svg + +# Generate block diagram in PNG format +echo '@startuml\nAlice -> Bob: Hello\n@enduml' > simple.puml +kroki --file simple.puml --type plantuml --output /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output/simple.png +``` + +## ๐Ÿ“œ Vale Usage + +### Linting Documents + +```bash +# Check a single document +vale /data/document.md + +# Check with specific configuration +vale --config=/data/.vale.ini /data/document.md + +# Generate report in JSON format +vale --output-style=JSON /data/document.md +``` + +## ๐Ÿ”ง Working with TeXLive + +### Custom LaTeX Documents + +```bash +# Compile LaTeX with XeTeX (for better font support) +xelatex -output-directory=/home/tsysdevstack/TSYSDevStack/Toolbox/docs/output document.tex + +# Compile with pdflatex +pdflatex -output-directory=/home/tsysdevstack/TSYSDevStack/Toolbox/docs/output document.tex +``` + +## ๐Ÿ“ Best Practices + +### Organizing Work + +1. **Use the `/data` directory** for input files +2. **Use the output directory** for generated documents +3. **Create a local `config` directory** for custom configurations +4. **Use volume mounts** for persistent work + +### Performance Tips + +1. **Use Docker volumes** instead of bind mounts for better performance when possible +2. **Process multiple files** in a single container run to avoid startup overhead +3. **Use appropriate memory allocation** for large document processing +4. **Use specific tool versions** with mise for reproducibility + +### Security Considerations + +1. **The container runs as non-root user** (`tsysdevstack`) +2. **No root access is available at runtime** to prevent security issues +3. **Language runtimes are managed by mise** for consistent and secure tooling + +## ๐Ÿ” Troubleshooting + +### Common Issues + +- **Font Issues**: The container includes Noto fonts; for additional fonts, extend the container or mount font directories +- **Memory Issues**: Increase Docker's memory allocation for processing large documents +- **Permission Issues**: Ensure mounted volumes have appropriate permissions for UID 1000 + +### Getting Help + +If you encounter issues: + +1. Check the [TROUBLESHOOTING.md](TROUBLESHOOTING.md) guide +2. Use the diagnostic commands from the troubleshooting guide +3. Examine container logs: `docker logs ` +4. Verify volume mappings and file accessibility \ No newline at end of file diff --git a/examples/README.md b/examples/README.md new file mode 100644 index 0000000..67a4cf2 --- /dev/null +++ b/examples/README.md @@ -0,0 +1,53 @@ +# Example Documents for TSYSDevStack Toolboxes Docs + +This directory contains example documents and workflow scripts that demonstrate the capabilities of the document production workhorse container. + +## ๐Ÿ“„ Document Examples + +### 1. Basic Markdown to PDF +- `basic-markdown.md` - Simple markdown document for conversion to PDF +- `basic-markdown.pdf` - Output PDF generated from the markdown + +### 2. Resume/CV Generation +- `resume-template.tex` - LaTeX template for ATS-optimized resumes +- `resume.md` - Markdown source for a resume +- `resume.pdf` - Generated PDF resume + +### 3. Project Documentation +- `project-plan.md` - Example project plan in markdown +- `project-plan.pdf` - Generated PDF project plan + +### 4. Joplin Notes Conversion +- `joplin-note.md` - Example Joplin note markdown +- `joplin-note.pdf` - Converted PDF preserving formatting + +### 5. mdBook Example +- `mybook/` - Complete mdBook project example +- `mybook/src/SUMMARY.md` - Book structure +- `mybook/src/chapter_1.md` - Sample chapter + +### 6. Typst Document +- `scientific-document.typ` - Example Typst document +- `scientific-document.pdf` - Generated PDF + +### 7. Marp Presentation +- `presentation.md` - Example Marp presentation +- `presentation.pdf` - Generated PDF slides + +### 8. Quarto Document +- `analysis.qmd` - Example Quarto document with code execution +- `analysis.pdf` - Generated analysis document + +## ๐Ÿ› ๏ธ Workflow Scripts + +### 1. Batch Conversion Script +- `batch-convert.sh` - Script to convert multiple markdown files to PDF + +### 2. Document Pipeline +- `document-pipeline.sh` - Complete pipeline from source to final document + +### 3. Quality Assurance +- `qa-check.sh` - Script to verify document quality and consistency + +### 4. Custom Styling +- `custom-templates/` - Directory containing custom templates for various document types \ No newline at end of file diff --git a/examples/basic-markdown.md b/examples/basic-markdown.md new file mode 100644 index 0000000..63b8164 --- /dev/null +++ b/examples/basic-markdown.md @@ -0,0 +1,32 @@ +# Sample Markdown Document + +This is a sample markdown document to demonstrate the document conversion capabilities of the TSYSDevStack Toolboxes Docs container. + +## Features Demonstrated + +- Headers and subheaders +- **Bold text** +- *Italic text* +- Lists + - Sublist item 1 + - Sublist item 2 +- Code blocks: + ```python + def hello(): + print("Hello, World!") + ``` + +## Tables + +| Column 1 | Column 2 | Column 3 | +|----------|----------|----------| +| Item 1 | Item 2 | Item 3 | +| Item 4 | Item 5 | Item 6 | + +## Images + +![Placeholder Image](https://placehold.co/600x400.png "Sample Image") + +## Conclusion + +This sample document demonstrates various markdown features that can be converted to beautiful PDFs using the Pandoc and TeXLive tools in this container. \ No newline at end of file diff --git a/examples/doc-workflow.sh b/examples/doc-workflow.sh new file mode 100755 index 0000000..7825f96 --- /dev/null +++ b/examples/doc-workflow.sh @@ -0,0 +1,102 @@ +#!/usr/bin/env bash + +# Example workflow script: Convert multiple markdown files to PDF +# This script demonstrates a typical document generation workflow + +set -e # Exit on any error + +echo "Starting document conversion workflow..." + +# Define input and output directories +INPUT_DIR="/data" +OUTPUT_DIR="/home/tsysdevstack/TSYSDevStack/Toolbox/docs/output" +WORKFLOW_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +echo "Input directory: $INPUT_DIR" +echo "Output directory: $OUTPUT_DIR" + +# Function to convert a markdown file to PDF +convert_to_pdf() { + local input_file="$1" + local output_file="$2" + + echo "Converting $input_file to $output_file" + pandoc "$input_file" \ + -o "$output_file" \ + --pdf-engine=xelatex \ + --variable=geometry:a4paper \ + --variable=margin=1in \ + --highlight-style=tango +} + +# Function to convert a markdown file to DOCX +convert_to_docx() { + local input_file="$1" + local output_file="$2" + + echo "Converting $input_file to $output_file" + pandoc "$input_file" -o "$output_file" +} + +# Function to process mdBook +build_mdbook() { + local book_dir="$1" + local output_dir="$2" + + echo "Building mdBook from $book_dir" + mdbook build "$book_dir" -d "$output_dir" +} + +# Function to process Typst document +compile_typst() { + local input_file="$1" + local output_file="$2" + + echo "Compiling Typst document $input_file to $output_file" + typst compile "$input_file" "$output_file" +} + +# Convert all markdown files in the input directory +for md_file in "$INPUT_DIR"/*.md; do + if [ -f "$md_file" ]; then + filename=$(basename "$md_file" .md) + echo "Processing markdown file: $filename" + + # Convert to PDF + convert_to_pdf "$md_file" "$OUTPUT_DIR/${filename}.pdf" + + # Convert to DOCX + convert_to_docx "$md_file" "$OUTPUT_DIR/${filename}.docx" + + echo "Completed conversion of $filename" + fi +done + +# Process any mdBook directories +for book_dir in "$INPUT_DIR"/book_*; do + if [ -d "$book_dir" ] && [ -f "$book_dir/book.toml" ]; then + echo "Found mdBook project: $book_dir" + build_mdbook "$book_dir" "$OUTPUT_DIR" + fi +done + +# Process any Typst documents +for typ_file in "$INPUT_DIR"/*.typ; do + if [ -f "$typ_file" ]; then + filename=$(basename "$typ_file" .typ) + echo "Processing Typst file: $filename" + compile_typst "$typ_file" "$OUTPUT_DIR/${filename}.pdf" + fi +done + +# Generate a summary report +echo "Conversion workflow completed!" +echo "Generated documents are available in: $OUTPUT_DIR" + +# Count generated files +pdf_count=$(find "$OUTPUT_DIR" -name "*.pdf" | wc -l) +docx_count=$(find "$OUTPUT_DIR" -name "*.docx" | wc -l) +echo "Total PDFs generated: $pdf_count" +echo "Total DOCX files generated: $docx_count" + +echo "Workflow completed successfully!" \ No newline at end of file diff --git a/examples/example-document.typ b/examples/example-document.typ new file mode 100644 index 0000000..f7c743a --- /dev/null +++ b/examples/example-document.typ @@ -0,0 +1,32 @@ +// Example Typst document +#set page(width: 15cm, height: 20cm, margin: 2cm) +#set text(font: "Noto Sans", size: 12pt) + += Introduction to Typst + +This is a simple example document created with Typst, a modern markup-based typesetting system. + +== Features + +Typst offers several advantages: + +- #lorem(10) +- Mathematical typesetting: $f(x) = x^2$ +- #list( + , "Fast processing" + , "Template system" + , "Symbol collections" +) + +== Code Example + +Here's how to include code: + +```typst += Title +Lorem ipsum #lorem(50) +``` + += Conclusion + +Typst provides a modern alternative to LaTeX for document creation with a simpler syntax and fast compilation. \ No newline at end of file diff --git a/examples/presentation.md b/examples/presentation.md new file mode 100644 index 0000000..c8a2d1f --- /dev/null +++ b/examples/presentation.md @@ -0,0 +1,55 @@ +--- +marp: true +theme: gaia +_class: lead +paginate: true +backgroundColor: #fff +backgroundImage: url('https://cdn.jsdelivr.net/gh/marp-team/marp@master/assets/hero-background.svg') +--- + +# TSYSDevStack Documentation Tools + +*Modern Document Production Workhorse* + +--- + +## Agenda + +- Overview of Tools +- Pandoc for Conversion +- mdBook for Documentation +- Typst for Typesetting +- Marp for Presentations +- Quarto for Scientific Publishing + +--- + +## Pandoc + +- Universal document converter +- Supports 50+ formats +- High-quality PDF output with TeXLive + +```markdown +pandoc input.md -o output.pdf --pdf-engine=xelatex +``` + +--- + +## mdBook + +- Create books from Markdown +- Multiple output formats +- Customizable themes + +```bash +mdbook build mybook -d output/ +``` + +--- + +## Thank You + +*Questions?* + +[github.com/tsysdevstack/toolboxes-docs](https://github.com/tsysdevstack/toolboxes-docs) \ No newline at end of file diff --git a/output/joplin-note.md b/output/joplin-note.md new file mode 100644 index 0000000..995f315 --- /dev/null +++ b/output/joplin-note.md @@ -0,0 +1,34 @@ +# Project Planning Meeting Notes + +**Date:** October 26, 2023 +**Attendees:** John, Sarah, Mike, Lisa + +## Agenda +- Review project timeline +- Discuss resource allocation +- Address technical challenges + +## Key Decisions +- Extend deadline by 2 weeks for backend development +- Allocate additional resources to testing phase +- Implement new authentication system + +## Action Items +- [x] John: Complete API design by Nov 5 +- [ ] Sarah: Prepare testing framework by Nov 10 +- [ ] Mike: Set up staging environment by Nov 8 +- [ ] Lisa: Review security requirements by Nov 3 + +## Discussion Points +- The initial timeline was too aggressive +- More time needed for integration testing +- Consider using external consultants for UI work + +## Next Meeting +**Date:** November 2, 2023 +**Time:** 10:00 AM +**Location:** Conference Room 3 + +## Attachments +- [API_Design_Draft.pdf](api_design_draft.pdf) +- [Resource_Allocation_Sheet.xlsx](resource_sheet.xlsx) \ No newline at end of file diff --git a/output/resume.md b/output/resume.md new file mode 100644 index 0000000..487c156 --- /dev/null +++ b/output/resume.md @@ -0,0 +1,46 @@ +--- +title: John Doe - Software Engineer +author: John Doe +keywords: [software engineer, resume, ATS] +--- + +# John Doe +**Software Engineer** + +Email: john.doe@example.com | Phone: (123) 456-7890 +LinkedIn: linkedin.com/in/johndoe | GitHub: github.com/johndoe + +--- + +## PROFESSIONAL SUMMARY + +Experienced software engineer with 5+ years of expertise in designing, developing, and maintaining scalable web applications. Proficient in JavaScript, Python, and cloud technologies. Strong problem-solving skills with a passion for clean, efficient code. + +## TECHNICAL SKILLS + +**Programming Languages:** JavaScript (ES6+), Python, Java, TypeScript, SQL +**Frameworks & Libraries:** React, Node.js, Django, Express.js, Vue.js +**Tools & Technologies:** Git, Docker, Kubernetes, AWS, REST APIs, GraphQL +**Databases:** PostgreSQL, MongoDB, Redis + +## PROFESSIONAL EXPERIENCE + +**Senior Software Engineer** | TechCorp | 2020 - Present +- Developed and maintained microservices handling 1M+ daily requests +- Improved application performance by 40% through code optimization +- Mentored 3 junior developers and conducted code reviews + +**Software Engineer** | StartupXYZ | 2018 - 2020 +- Built responsive web applications using React and Node.js +- Implemented CI/CD pipelines reducing deployment time by 60% +- Collaborated with cross-functional teams to deliver features on schedule + +## EDUCATION + +**Bachelor of Science in Computer Science** +University of Technology | 2014 - 2018 + +## CERTIFICATIONS + +- AWS Certified Developer - Associate (2021) +- MongoDB Certified Developer (2020) \ No newline at end of file diff --git a/output/test-document.md b/output/test-document.md new file mode 100644 index 0000000..0e60109 --- /dev/null +++ b/output/test-document.md @@ -0,0 +1,28 @@ +# Test Document + +This is a test document to verify that the TSYSDevStack Toolboxes Docs container is working correctly. + +## Features Tested + +- [ ] Markdown to PDF conversion +- [ ] Markdown to DOCX conversion +- [ ] TeXLive/XeTeX rendering +- [ ] Font rendering +- [ ] Code block formatting +- [ ] Table formatting + +## Code Example + +```javascript +function hello() { + console.log("Hello, World!"); +} +``` + +## Table Example + +| Feature | Status | Notes | +|---------|--------|-------| +| Pandoc | โœ… | Working | +| TeXLive | โœ… | Working | +| Fonts | โœ… | Noto fonts available | \ No newline at end of file diff --git a/run.sh b/run.sh new file mode 100755 index 0000000..d9af7f3 --- /dev/null +++ b/run.sh @@ -0,0 +1,104 @@ +#!/usr/bin/env bash + +# run.sh - Script to run the tsysdevstack-toolboxes-docs container +set -e + +# Default values +CONTAINER_NAME="tsysdevstack-toolboxes-docs" +IMAGE_NAME="tsysdevstack/toolboxes-docs:latest" +WORKDIR="/home/tsysdevstack" +OUTPUT_DIR="/home/tsysdevstack/TSYSDevStack/Toolbox/docs/output" + +# Parse command line arguments +INTERACTIVE=true +DAEMON=false +CMD="" + +while [[ $# -gt 0 ]]; do + case $1 in + -d|--daemon) + DAEMON=true + INTERACTIVE=false + shift + ;; + -c|--command) + CMD="$2" + INTERACTIVE=false + shift 2 + ;; + -n|--name) + CONTAINER_NAME="$2" + shift 2 + ;; + -h|--help) + echo "Usage: $0 [OPTIONS] [COMMAND]" + echo "" + echo "Options:" + echo " -d, --daemon Run container in daemon mode" + echo " -c, --command Run a specific command in the container" + echo " -n, --name Specify container name (default: $CONTAINER_NAME)" + echo " -h, --help Show this help message" + echo "" + echo "Examples:" + echo " $0 # Run interactively" + echo " $0 -d # Run as daemon" + echo " $0 -c 'pandoc --version' # Run specific command" + exit 0 + ;; + *) + echo "Unknown option: $1" + exit 1 + ;; + esac +done + +# Function to run the container +run_container() { + local extra_args=() + + if [ "$INTERACTIVE" = true ]; then + extra_args+=(-it) + fi + + if [ "$DAEMON" = true ]; then + extra_args+=(-d) + fi + + if [ -n "$CMD" ]; then + docker run "${extra_args[@]}" \ + --name "$CONTAINER_NAME" \ + -v "$(pwd)/output:$OUTPUT_DIR" \ + -w "$WORKDIR" \ + "$IMAGE_NAME" \ + bash -c "$CMD" + else + docker run "${extra_args[@]}" \ + --name "$CONTAINER_NAME" \ + -v "$(pwd)/output:$OUTPUT_DIR" \ + -w "$WORKDIR" \ + "$IMAGE_NAME" + fi +} + +# Check if container is already running +if [ "$(docker ps -q -f name="$CONTAINER_NAME")" ]; then + echo "Container $CONTAINER_NAME is already running." + if [ -n "$CMD" ]; then + # Run command in existing container + docker exec -it "$CONTAINER_NAME" bash -c "$CMD" + else + # Attach to existing container + docker exec -it "$CONTAINER_NAME" /bin/bash + fi +elif [ "$(docker ps -aq -f name="$CONTAINER_NAME")" ]; then + # Container exists but is not running, start it + docker start "$CONTAINER_NAME" > /dev/null + if [ -n "$CMD" ]; then + docker exec -it "$CONTAINER_NAME" bash -c "$CMD" + else + docker attach "$CONTAINER_NAME" + fi +else + # Run new container + run_container +fi \ No newline at end of file diff --git a/test.sh b/test.sh new file mode 100755 index 0000000..f1ee8ce --- /dev/null +++ b/test.sh @@ -0,0 +1,101 @@ +#!/usr/bin/env bash + +# test.sh - Script to test the tsysdevstack-toolboxes-docs container +set -e + +# Default values +IMAGE_NAME="tsysdevstack/toolboxes-docs" +TAG="latest" +CONTAINER_NAME="test-tsysdevstack-toolboxes-docs" +TEST_DIR="/tmp/test-tsysdevstack-toolboxes-docs" + +# Function to print test results +print_result() { + if [ $? -eq 0 ]; then + echo "โœ… PASS: $1" + else + echo "โŒ FAIL: $1" + exit 1 + fi +} + +# Function to run a command in the container and check the result +run_test_command() { + local test_name="$1" + local command="$2" + + echo "Running test: $test_name" + docker run --rm --name "$CONTAINER_NAME-$(date +%s)" "$IMAGE_NAME:$TAG" bash -c "$command" + print_result "$test_name" +} + +# Function to test if a command exists in the container +test_command_exists() { + local test_name="$1" + local command="$2" + + run_test_command "$test_name" "which $command" +} + +# Function to test if a specific version of a tool is available +test_version_command() { + local test_name="$1" + local command="$2" + local expected_version="$3" + + echo "Running version test: $test_name" + docker run --rm --name "$CONTAINER_NAME-$(date +%s)" "$IMAGE_NAME:$TAG" bash -c "$command" | grep "$expected_version" > /dev/null + print_result "$test_name" +} + +# Main test execution +main() { + echo "Starting tests for $IMAGE_NAME:$TAG..." + + # Test that core utilities exist + test_command_exists "Check if pandoc exists" "pandoc" + test_command_exists "Check if mdbook exists" "mdbook" + test_command_exists "Check if typst exists" "typst" + test_command_exists "Check if marp exists" "marp" + test_command_exists "Check if markwhen exists" "markwhen" + test_command_exists "Check if quarto exists" "quarto" + test_command_exists "Check if jq exists" "jq" + test_command_exists "Check if yq exists" "yq" + test_command_exists "Check if wkhtmltopdf exists" "wkhtmltopdf" + test_command_exists "Check if bibtool exists" "bibtool" + test_command_exists "Check if vale exists" "vale" + test_command_exists "Check if kroki exists" "kroki" + test_command_exists "Check if fish shell exists" "fish" + test_command_exists "Check if zsh exists" "zsh" + test_command_exists "Check if bash exists" "bash" + + # Test that TeXLive is properly installed + test_command_exists "Check if xelatex exists" "xelatex" + test_command_exists "Check if pdflatex exists" "pdflatex" + + # Test that Python and Node.js are managed by mise + run_test_command "Check if Python is available via mise" "python --version" + run_test_command "Check if Node.js is available via mise" "node --version" + + # Test that we can run a simple pandoc command + run_test_command "Test basic pandoc functionality" "echo '# Test' | pandoc -f markdown -t html | grep '

Test

'" + + # Test that we can run a simple mdbook command + run_test_command "Test basic mdbook functionality" "mdbook --version" + + # Test that we can run a simple typst command + run_test_command "Test basic typst functionality" "echo '# Hello' > /tmp/test.typ && typst compile /tmp/test.typ /tmp/test.pdf && [ -f /tmp/test.pdf ]" + + # Test that user is not root + run_test_command "Check that default user is not root" "whoami | grep tsysdevstack" + + # Test that required directories exist + run_test_command "Check that output directory exists" "[ -d /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output ]" + + # Test that we can write to the output directory + run_test_command "Test write access to output directory" "touch /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output/test_file.txt && [ -f /home/tsysdevstack/TSYSDevStack/Toolbox/docs/output/test_file.txt ]" + + echo "All tests passed! ๐ŸŽ‰" +} + +main \ No newline at end of file