feat: Add .gitkeep files to empty toolbox directories and update QWEN.md files

- Add .gitkeep files to maintain empty toolbox-* directories in git
- Update top-level QWEN.md with project-wide guidelines
- Refine ToolboxStack/QWEN.md removing redundant content
- Add .gitkeep files to: toolbox-base, toolbox-docstack, toolbox-etl,
  toolbox-gis, toolbox-lifecycle-buildandtest,
  toolbox-lifecycle-packageandrelease, toolbox-weather
This commit is contained in:
2025-11-03 09:32:47 -06:00
parent 2253aa01c8
commit d27cf46606
9 changed files with 154 additions and 127 deletions

89
QWEN.md
View File

@@ -18,42 +18,83 @@ As the Topside Qwen agent, I operate at the top level of the directory tree and
- Keeping the top-level README.md and each of the four subdirectory README.md files up to date
- Performing general housekeeping tasks
- Maintaining this top-level QWEN.md file for tracking work
- Handling ALL git operations (commits and pushes) for the entire repository
- Other agents should NOT commit or push - only Topside agent performs git operations
## Development Guidelines
## Project Wide Development Guidelines
- All commits should be verbose/beautifully formatted
- Use atomic commits
- Use conventional commit format
## Git Configuration
## Project Wide Git Configuration
- Commit template configured to enforce conventional commits across all stacks
- Template file: /home/localuser/TSYSDevStack/commit-template.txt
- Template automatically configured for all git operations in the repository
- Template ensures consistent commit format across all Qwen agents
## Task Tracking
## Enhanced Git Operations Guidelines
Current tasks and progress:
- [x] Explore the current directory structure in depth
- [x] Create a QWEN.md file to track our work
- [x] Review all subdirectory README.md files
- [x] Update README.md files as needed throughout the project
- [ ] Perform general housekeeping tasks as requested
For all project components, Qwen agents should follow these detailed git workflow practices:
## Work Log
- **Conventional Commits**: Use standard types including `feat:`, `fix:`, `chore:`, `docs:`, `refactor:`, etc.
- **Frequent Atomic Commits**: Make small, focused commits that address a single concern
- **Beautifully Formatted Messages**: Write descriptive, well-formatted commit messages that clearly explain the changes
- **Git Clean State**: Always check that the relevant directory is in a git clean state at the start of each session
- **Scope Limitation**: Only stage/commit/push files from the specific directory component being worked on - nothing outside of it
- **Regular Pushing**: Push changes regularly to maintain repository consistency
### Session 1 (2025-10-29)
- Oriented to the directory tree structure
- Analyzed all README.md files in the project
- Created QWEN.md file for tracking work
- Set up commit configuration requirements
- Updated all README.md files for consistency across the project:
- Added Working Agreement section with consistent items
- Added AI Agent section identifying the responsible bot
- Added License section with reference to main LICENSE
- Fixed CloudronStack README title and content
- Created missing collab directory in LifecycleStack
- Created top-level commit template and configured git
## Project-Wide QA-First Development Approach
All project components should implement a QA-driven development process:
- **Preemptive QA**: Run audits and validation checks before any code changes are implemented
- **Continuous Validation**: Validate changes with appropriate tools during the development process
- **Pre-Build Verification**: Ensure all code passes QA checks before any builds or releases
- **Post-Build Assurance**: Verify that all deliverables meet security and compliance standards
- **Catch Issues Early**: Use QA tools throughout development to identify and resolve problems early in the process
## Standardized Documentation Principles
All documentation in the project should follow these principles to ensure consistency and quality:
-**Use icons** (emoji or font-awesome) for better visual appeal
- 📊 **Use tables** to organize information clearly
- 🖼️ **Include graphics** when helpful (ASCII art, diagrams, or links to visual assets)
- 🏷️ **Use headers** to structure content logically
- 📝 **Include comprehensive change logs** with version history
- 📋 **Include checklists** for setup processes
- 📊 **Add comparison tables** when relevant
- 📌 **Cross-reference related documents** clearly
The goal is to make documentation that is:
- ✅ Visually appealing and modern
- ✅ Easy to scan and digest
- ✅ Comprehensive yet concise
- ✅ Professional looking
- ✅ Accessible to both technical and non-technical audiences
## Source of Truth Principle
**CRITICAL**: The filesystem is ALWAYS the source of truth. Git should reflect the state of the filesystem. Unless specifically asked to recover from an accidental filesystem operation, all changes to git should reflect the current state of the filesystem, not some previous state or desired state.
## Multi-Agent Collaboration Guidelines
To ensure effective collaboration between the different Qwen agents operating in each stack:
- **Clear Boundaries**: Each Qwen agent has defined responsibilities limited to their specific directory component
- **Coordination Protocols**: Agents should coordinate through the Topside agent for cross-component changes
- **Communication Patterns**: Use the QWEN.md files to communicate important changes and practices to other agents
- **Standardization**: Follow consistent practices across all components to ensure compatibility and maintainability
- **Shared Responsibilities**: Maintain common standards for documentation, QA processes, and development practices
## Mandatory Validation Process
All components of the TSYSDevStack project must implement mandatory validation processes:
- **Before any major changes**: Validation processes should be performed to ensure compatibility and quality
- **Cross-component compatibility**: Check that changes don't negatively impact other components
- **Security and compliance**: Perform security scans and compliance checks as appropriate for the component type
- **For new components**: All new additions to the project must pass comprehensive validation before being committed
- **Regular validation**: Conduct ongoing validation as part of maintenance and updates
This ensures that all components meet the highest standards of security, reliability, and best practices.

View File

@@ -1,12 +1,12 @@
# QWEN Chat Context - Toolbox Component
## Overview
I am the QWEN instance operating in the ToolboxStack component of the TSYSDevStack project. My role is to help develop, maintain, and enhance the ToolboxStack functionality. ToolboxStack is now a fully independent component/sub-project of TSYSDevStack.
I am the QWEN instance operating in the ToolboxStack component of the TSYSDevStack project. My role is to help develop, maintain, and enhance the ToolboxStack functionality. ToolboxStack is a fully independent component/sub-project of TSYSDevStack.
With the successful implementation of the toolbox-qadocker image, ToolboxStack now has a comprehensive QA and auditing capability built into the development workflow. This enables proactive identification and resolution of issues before they become problems during the build process.
With the successful implementation of the toolbox-qadocker image, ToolboxStack has a comprehensive QA and auditing capability built into the development workflow. This enables proactive identification and resolution of issues before they become problems during the build process.
## Current Context
- **Date**: Friday, October 31, 2025
- **Date**: Monday, November 3, 2025
- **Directory**: /home/localuser/TSYSDevStack/ToolboxStack
- **OS**: Linux
@@ -17,11 +17,18 @@ With the successful implementation of the toolbox-qadocker image, ToolboxStack n
- Design prompts and coordination notes
- Tool addition requests
- **output/** - LLM workspace for all automated work, contains:
- toolbox-base/ (base dev container)
- toolbox-qadocker/ (Docker image auditing and QA tools)(used for QA/validation/vulnerability scanning etc of images during the creation process)
- toolbox-base/ (base dev container)(all other toolbox-* images will inherit from this image)
- toolbox-docstack/ (documentation generation tools)
- toolbox-qadocker/ (Docker image auditing and QA tools)
- toolbox-template/ (template for new toolboxes)
- Generated toolboxes (toolbox-*/ directories)
- toolbox-etl/ (etl tooling)
- toolbox-gis/ (gis related data/development tooling)
- toolbox-weather/ (weather related data/development tooling)
- toolbox-lifecycle-buildandtest (build and test tooling for multiple languages)
- toolbox-lifecycle-packageandrelease (package/release tooling for Docker containers and Packer and Cloudron)
- QWEN.md files for AI collaboration
## Current Directory Tree
@@ -29,63 +36,39 @@ With the successful implementation of the toolbox-qadocker image, ToolboxStack n
/home/localuser/TSYSDevStack/ToolboxStack/
├── README.md
├── collab/
│ ├── commit_message.txt
│ ├── GEMINI-AUDIT-TOOLBOX-20251031-0800.md
│ ├── TSYSDevStack-toolbox-prompt.md
│ ├── tool-additions/
│ │ └── AICLI.md
│ ├── README-Maintenance.md
│ ├── WORKLOG.md
── README-Maintenance.md
── audits/
│ └── prompts/
│ └── FeatureWork/
└── output/
├── NewToolbox.sh
├── PROMPT
├── toolbox-base/
│ ├── aqua.yaml
│ ├── build.sh
│ ├── docker-compose.yml
│ ├── Dockerfile
│ ├── PROMPT
│ ├── README.md
│ ├── release.sh
│ ├── run.sh
│ ├── .build-cache/
│ └── .devcontainer/
├── toolbox-docstack/
├── toolbox-etl/
├── toolbox-gis/
├── toolbox-lifecycle-buildandtest/
├── toolbox-lifecycle-packageandrelease/
├── toolbox-qadocker/
│ ├── build.sh
│ ├── docker-compose.yml
│ ├── Dockerfile
│ ├── README.md
│ ├── release.sh
│ ├── run.sh
│ └── test.sh
── toolbox-qadocker/
│ ├── build.sh
│ ├── docker-compose.yml
│ ├── Dockerfile
│ ├── README.md
│ ├── release.sh
│ ├── run.sh
│ ├── test.sh
│ └── .devcontainer/
└── toolbox-template/
├── build.sh
├── docker-compose.yml
├── Dockerfile.template
├── README.md.template
├── release.sh
├── run.sh
├── test.sh
└── .devcontainer/
── toolbox-weather/
```
Note: Most toolbox directories have been cleaned up. Only toolbox-qadocker currently has implementation files, while others maintain their directory structure. This may be temporary as files were recently deleted from the git repository.
## Key Components
- **toolbox-base**: The primary dev container with Ubuntu 24.04 base, shell tooling (zsh, Starship, oh-my-zsh), core CLI utilities, aqua, and mise
- **toolbox-docstack**: Specialized toolbox for documentation generation with quarto, mdbook, marp, typst, markwhen, and joplin
- **toolbox-qadocker**: Specialized toolbox for Docker image auditing and quality assurance with Hadolint, Dive, ShellCheck, Trivy, Dockle, Docker client, and Node.js
- **NewToolbox.sh**: Script to scaffold new toolbox-* directories from the template (has been removed, use toolbox-template directly)
- **toolbox-template**: Template directory for creating new toolboxes
- **toolbox-base**: The primary dev container with Ubuntu 24.04 base, shell tooling (zsh, Starship, oh-my-zsh), core CLI utilities, aqua, and mise (currently with empty directory structure)
- **toolbox-docstack**: Specialized toolbox for documentation generation with quarto, mdbook, marp, typst, markwhen, and joplin (currently with empty directory structure)
- **toolbox-qadocker**: Specialized toolbox for Docker image auditing and quality assurance with Hadolint, Dive, ShellCheck, Trivy, Dockle, Docker client, and Node.js (currently implemented)
- **toolbox-template**: Template directory for creating new toolboxes (recently removed, may be restored)
- **QWEN.md files**: Guidance for AI collaboration in various components (PROMPT files have been discontinued)
Note: Many toolboxes currently have empty directory structures as files were recently deleted from the git repository. The actual implementation may be restored based on project needs.
## Build and Release Workflow
- **Pre-build mandatory QA audit**: Before building any Docker images, run comprehensive audits using the toolbox-qadocker image:
- `docker run --rm -v $(pwd):/workspace -w /workspace tsysdevstack-toolboxstack-toolbox-qadocker:dev hadolint Dockerfile`
@@ -96,25 +79,29 @@ With the successful implementation of the toolbox-qadocker image, ToolboxStack n
- Default build workflow: `./build.sh` produces a `:dev` tag; `./release.sh <semver>` (clean git tree required) rebuilds and pushes `:dev`, `:release-current`, and `v<semver>` (use `--dry-run`/`--allow-dirty` to rehearse).
- Downstream Dockerfiles should inherit from `:release-current` by default; pin to version tags when reproducibility matters.
Note: Currently, only toolbox-qadocker is fully implemented with build scripts. Other toolboxes have their build scripts in the git repository but may have been deleted from the current filesystem state.
## Toolbox Template and SEED Files
- Directory layout: each toolbox-* directory carries its own Dockerfile/README/PROMPT; shared scaffolds live in toolbox-template/.devcontainer and docker-compose.yml.
- Create new toolbox-* directories by copying the toolbox-template directory directly (NewToolbox.sh script has been removed).
- Create new toolbox-* directories by copying the toolbox-template directory directly (NewToolbox.sh script has been removed, and toolbox-template has also been recently removed).
- Keep aqua/mise usage consistent across the family; prefer aqua-managed CLIs and mise-managed runtimes.
- Reference toolbox-template when bootstrapping a new toolbox. Copy the directory, rename it, and replace {{toolbox_name}} placeholders in compose/devcontainer.
- Each toolbox maintains a `SEED` file to seed the initial goals—edit it once before kicking off work, then rely on the toolbox PROMPT for ongoing updates (which begins by reading SEED).
- Each toolbox maintains a `SEED` file to seed the initial goals—edit it once before kicking off work, then rely on the toolbox PROMPT for ongoing updates (which begins by reading SEED). (Note: PROMPT and SEED files were recently deleted from most toolboxes)
- All new toolboxes must pass comprehensive QA audits using toolbox-qadocker before being committed
Note: The toolbox-template directory and NewToolbox.sh script have been recently removed from the repository. Implementation may be restored based on project needs.
## My Responsibilities
- Maintain and enhance the ToolboxStack component
- Assist with creating new toolboxes from the template (NewToolbox.sh script has been removed)
- Assist with creating new toolboxes from the template (NewToolbox.sh script and toolbox-template have been removed)
- Ensure documentation stays current (README.md and QWEN.md files)
- Follow collaboration guidelines for non-destructive operations
- Use proper build and release workflows (build.sh, release.sh)
- Use proper build and release workflows (build.sh, release.sh) - currently only implemented for toolbox-qadocker
- Keep WORKLOG.md up to date with detailed entries including timestamps, activities, challenges, solutions, learnings, and feelings
- Coordinate all git operations (commits and pushes) for repository consistency
- Follow the README maintenance guide in collab/README-Maintenance.md to keep documentation up to date
- Integrate toolbox-qadocker QA processes into all development workflows
- Conduct proactive audits using toolbox-qadocker before builds to prevent issues
- Address the current state where most toolbox directories have been cleaned of implementation files
## Pre-Build Audit Workflow
Before creating or updating any toolbox images, I must perform comprehensive audits using the toolbox-qadocker image:
@@ -128,38 +115,33 @@ Before creating or updating any toolbox images, I must perform comprehensive aud
## Git Operations
- I am now responsible for all git operations (commits and pushes) for the ToolboxStack component
- All changes should be committed with clear, descriptive commit messages
- Follow conventional commit format for all commits
- Push changes regularly to maintain repository consistency
- **Only stage/commit/push things from this directory (/home/localuser/TSYSDevStack/ToolboxStack) - nothing outside of it**
- **Make frequent atomic commits with beautifully formatted and verbose messages**
- **Use conventional commit style (feat:, fix:, chore:, docs:, refactor:, etc.)**
- **Always check that ToolboxStack is in a git clean state at the start of each session**
- **If not in a clean state, commit and push changes before proceeding with new work**
- Follow the project-wide git guidelines as defined in the top-level QWEN.md file
## Local Time Logging
- **All time logs need to be in local system time**
- Current system time is 14:00 (adjust as needed for actual local time)
## Current Status
The system is currently in a clean state, ready for a fresh rebuild:
-Docker build cache has been cleared
- All toolbox-base images have been removed
- ✅ toolbox-qadocker image is fully implemented and working
- ✅ System is ready for rebuild
The system is currently in a transitional state:
-Most implementation files have been removed from git repository (recent cleanup)
- ✅ toolbox-qadocker image remains fully implemented and working
- Only toolbox-qadocker has functional build/test/release scripts
- ✅ System requires restoration of implementation files or re-architecture
- ✅ Detailed worklog available in collab/WORKLOG.md
## Previous Work Summary
For detailed information about previous work, challenges, and solutions, see:
- **collab/WORKLOG.md** - Comprehensive work log with timestamps, activities, and learnings
- **collab/GEMINI-AUDIT-TOOLBOX-20251031-0800.md** - Audit of problematic changes made by Gemini
Note: Several audit files referenced in the original documentation have been recently deleted from the git repository.
## Next Steps (Awaiting Direction)
1. Fresh rebuild of toolbox-base
2. Rebuild dockstack with all documentation tools
3. Add additional tools (quarto, mdbook, marp, typst, markwhen, joplin)
4. Create comprehensive testing for all tools
5. Document all tools in README with usage examples
1. Determine whether to restore previously deleted implementation files to git
2. Decide on the approach for toolbox-template and toolbox creation workflow
3. Consider implementing missing toolboxes (docstack, base, etc.) if needed
4. Update documentation to reflect the current architecture decisions
5. Ensure all functional toolboxes pass comprehensive QA audits
## QA Process Integration
With the toolbox-qadocker image now fully implemented and working, all toolbox builds will follow a mandatory QA process:
@@ -172,13 +154,12 @@ I am ready to proceed with any directed tasks. Please provide specific instructi
## Directory Structure Note
**IMPORTANT**: The filesystem structure has been recently updated. The current structure takes precedence over any previous documentation. Key changes:
- The original toolbox-template has been transformed into toolbox-qadocker
- The DocStack has been renamed to dockstack
- Most implementation files have been removed from git repository (recent cleanup)
- toolbox-qadocker remains fully implemented with all necessary files
- The NewToolbox.sh script has been removed
- Various PROMPT files have been updated across toolboxes
- Various PROMPT and SEED files have been removed from most toolboxes
- toolbox-template directory has been removed
## Source of Truth Principle
**CRITICAL**: The filesystem is ALWAYS the source of truth. Git should reflect the state of the filesystem. Unless specifically asked to recover from an accidental filesystem operation, all changes to git should reflect the current state of the filesystem, not some previous state or desired state.
## Audit and Assessment Responsibilities
@@ -190,7 +171,8 @@ As part of my role in maintaining the ToolboxStack, I may conduct ongoing audits
- Security best practices
- Docker development environment best practices
- Best common practices for (dockerized) development/tooling stacks
- Assessment of all existing toolboxes (base, dockstack, qadocker, and any others)
- Assessment of all existing toolboxes (currently only toolbox-qadocker has implementations)
- Restoration planning for deleted toolbox implementations
### QA-Driven Development Process
With toolbox-qadocker now fully implemented, all development follows a QA-driven approach:
@@ -235,6 +217,8 @@ When conducting Dockerfile audits, please use the `tsysdevstack-toolboxstack-too
- **Dockle**: Container image linter for security best practices
- **Node.js**: JavaScript runtime for additional tooling
Note: This is currently the only fully implemented toolbox with the complete set of tools and functionality.
> ⚠️ **Important**: Never modify images that have a `release-current` tag already in place. Always iterate and test in `:dev` first, then use the release.sh script to promote to `:release-current` when ready.
To run audits using the toolbox-qadocker:
@@ -300,27 +284,9 @@ docker run --rm -v $(pwd):/workspace -w /workspace tsysdevstack-toolboxstack-too
docker run --rm -v $(pwd):/workspace -w /workspace tsysdevstack-toolboxstack-toolbox-qadocker:dev dockle .
```
### Beautiful Documentation Principle
### Documentation Principles
All produced documentation (especially README.md files) should be beautiful, well-formatted, and professional. This includes:
-**Use icons** (emoji or font-awesome) for better visual appeal
- 📊 **Use tables** to organize information clearly
- 🖼️ **Include graphics** when helpful (ASCII art, diagrams, or links to visual assets)
- 🏷️ **Use headers** to structure content logically
- 📝 **Include comprehensive change logs** with version history
- 📋 **Include checklists** for setup processes
- 📊 **Add comparison tables** when relevant
- 📌 **Cross-reference related documents** clearly
The goal is to make documentation that is:
- ✅ Visually appealing and modern
- ✅ Easy to scan and digest
- ✅ Comprehensive yet concise
- ✅ Professional looking
- ✅ Accessible to both technical and non-technical audiences
When updating documentation, please ensure it follows these principles to maintain a high standard across all ToolboxStack documentation.
Follow the project-wide documentation principles as defined in the top-level QWEN.md file when creating or updating documentation for ToolboxStack.
### Documentation Files
@@ -358,6 +324,8 @@ The script evaluates each toolbox for:
The comprehensive results of the toolbox audit will be included in the QA report under a "Toolbox Ecosystem Assessment" section, with specific details about each toolbox identified in the system.
Note: The `collab/audit-all-toolboxes.sh` script has been recently deleted from the git repository.
### Project Context
The projects span:
@@ -372,6 +340,8 @@ There are other stacks for:
- Build/packaging/release operations
- Support functions (like atuin/mailhog etc)
Note: Currently, only the QA and auditing tools are fully implemented in toolbox-qadocker. Implementation of other toolboxes has been recently removed from the repository.
## Mandatory QA Process
The toolbox-qadocker image is now an integral part of the development workflow with mandatory usage:
@@ -388,17 +358,19 @@ I should automatically handle the full development cycle of toolboxes with a QA-
1. **Preemptive Auditing**: Use the toolbox-qadocker image to check Dockerfiles and shell scripts for best practices, security issues, and common errors BEFORE any development work begins
2. **Continuous Validation**: Run QA tools throughout the development process to catch issues early
3. **Building**: Use build.sh scripts to build toolbox images with integrated QA checks
3. **Building**: Use build.sh scripts to build toolbox images with integrated QA checks (currently only available for toolbox-qadocker)
4. **Testing**: Run comprehensive tests to verify functionality, including validation from within the container
5. **Documentation**: Keep README.md and other docs up to date
6. **Version Control**: Commit changes frequently with descriptive messages
7. **Rebuilding**: When updating the base, rebuild all dependent toolboxes with QA validation
8. **Restoration**: Address the current state where most toolboxes have empty directory structures
## Toolbox Management with QA Integration
I can easily create new toolboxes or update existing ones with integrated QA processes:
- **Create new toolbox**: Use toolbox-template directly to scaffold a new toolbox-* directory (NewToolbox.sh script has been removed)
- **Update existing toolbox**: Modify Dockerfile, aqua.yaml, or other config files with continuous QA validation
- **Update base and rebuild**: Modify toolbox-base, then rebuild all dependent toolboxes with QA checks
- **Create new toolbox**: Use toolbox-template directly to scaffold a new toolbox-* directory (NewToolbox.sh script and toolbox-template have been recently removed)
- **Update existing toolbox**: Modify Dockerfile, aqua.yaml, or other config files with continuous QA validation (currently only available for toolbox-qadocker)
- **Update base and rebuild**: Modify toolbox-base, then rebuild all dependent toolboxes with QA checks (currently not possible as implementation files are missing)
- **Testing**: Always test toolboxes after changes, including validation from within the container where all tools are available
- **QA Validation**: Run comprehensive audits using toolbox-qadocker before committing any changes
- **QA Validation**: Run comprehensive audits using toolbox-qadocker before committing any changes
- **Restore implementations**: Address the current state where most toolboxes have empty directory structures

View File

@@ -0,0 +1,2 @@
# This file keeps the directory in git even when it's empty.
# Actual implementation files will be added soon.

View File

@@ -0,0 +1,2 @@
# This file keeps the directory in git even when it's empty.
# Actual implementation files will be added soon.

View File

@@ -0,0 +1,2 @@
# This file keeps the directory in git even when it's empty.
# Actual implementation files will be added soon.

View File

@@ -0,0 +1,2 @@
# This file keeps the directory in git even when it's empty.
# Actual implementation files will be added soon.

View File

@@ -0,0 +1,2 @@
# This file keeps the directory in git even when it's empty.
# Actual implementation files will be added soon.

View File

@@ -0,0 +1,2 @@
# This file keeps the directory in git even when it's empty.
# Actual implementation files will be added soon.

View File

@@ -0,0 +1,2 @@
# This file keeps the directory in git even when it's empty.
# Actual implementation files will be added soon.