chore: apply local filesystem changes (remove legacy dirs; add new structure)

2025-09-12 14:05:02 -05:00
parent d34e1760c8
commit 59fc7fedf4
36 changed files with 0 additions and 0 deletions
--- a/PackagingForCloudronWorkspace/README.md
+++ b/PackagingForCloudronWorkspace/README.md
@@ -0,0 +1,170 @@
+# Cloudron Packaging Workspace
+
+This workspace contains development tools and upstream source repositories for Cloudron application packaging.
+
+## 🏗️ Workspace Structure
+
+```
+CloudronPackagingWorkspace/
+├── README.md                    # This file
+├── Docker/ (gitignored)         # Upstream application sources (~56 apps)
+├── NonDocker/ (gitignored)      # Non-Docker application sources  
+├── UpstreamVendor-Clone.sh     # Clone all upstream repositories
+└── UpstreamVendor-Update.sh    # Update existing repositories
+```
+
+## 🚀 Setup Instructions
+
+### Initial Setup
+```bash
+cd CloudronPackagingWorkspace/
+
+# Create Docker directory for upstream sources
+mkdir -p Docker
+
+# Make scripts executable
+chmod +x *.sh
+
+# Clone all upstream vendor repositories
+./UpstreamVendor-Clone.sh
+```
+
+This will perform a clone of all upstream vendor software/Docker repositories for every application that KNEL is packaging for Cloudron deployment.
+
+### Keeping Sources Updated
+```bash
+# Update all existing checkouts to latest versions
+./UpstreamVendor-Update.sh
+```
+
+## 📦 Available Applications
+
+The workspace contains ~56 upstream application repositories including:
+
+### High Priority Applications
+- **apisix** - Apache APISIX API Gateway
+- **jenkins** - Jenkins CI/CD Platform
+- **grist-core** - Grist Database/Spreadsheet
+- **rundeck** - Rundeck Job Scheduler
+- **reviewboard** - ReviewBoard Code Review
+- **consuldemocracy** - Consul Democracy Platform
+
+### Development & Infrastructure Tools
+- **InvenTree** - Inventory Management System
+- **elabftw** - Laboratory Management
+- **netbox-docker** - Network Documentation
+- **signoz** - Observability Platform
+- **healthchecks** - Health Monitoring
+- **fleet** - Device Management
+
+### Productivity & Specialized Applications
+- **huginn** - Web Automation
+- **windmill** - Workflow Automation
+- **docassemble** - Document Assembly
+- **jamovi** - Statistical Analysis
+- And many more...
+
+## 🛠️ Development Workflow
+
+### Using the Workspace
+
+1. **Source Access**: All upstream sources are available in `Docker/[appname]/`
+2. **Development**: Use the `tsys-cloudron-packaging` container for all work
+3. **Package Creation**: Create packages in separate temporary directories
+4. **Git Exclusion**: All upstream sources are gitignored to keep repository clean
+
+### Container Development
+```bash
+# Access development container
+docker exec -it tsys-cloudron-packaging bash
+
+# Navigate to workspace
+cd /workspace
+
+# Access application source
+cd CloudronPackagingWorkspace/Docker/[appname]/
+
+# Create new package (outside of workspace)
+cd /workspace
+mkdir -p [appname]_package_new
+```
+
+## 📋 Workspace Management
+
+### Adding New Applications
+1. Update `UpstreamVendor-Clone.sh` with new repository URL
+2. Run the clone script to fetch the new application
+3. Add application to priority list in `TASKS.md`
+
+### Removing Applications  
+1. Remove directory from `Docker/`
+2. Update clone script to prevent future re-cloning
+3. Update task lists and documentation
+
+### Repository Updates
+- **Frequency**: Weekly or before starting new package development
+- **Method**: Run `./UpstreamVendor-Update.sh`
+- **Verification**: Check for breaking changes in upstream
+
+## ⚠️ Important Notes
+
+### Git Exclusions
+- **Docker/**: All contents are gitignored
+- **NonDocker/**: All contents are gitignored
+- This keeps the main repository clean while preserving access to sources
+
+### Repository Integrity
+- Never commit upstream sources to the main repository
+- Use temporary directories for package development
+- Move final packages to `CloudronPackages/` when complete
+
+### Source Licenses
+- Each upstream repository maintains its own license
+- Review license compatibility before packaging
+- Include appropriate license information in final packages
+
+## 🔧 Script Maintenance
+
+### UpstreamVendor-Clone.sh
+- Contains git clone commands for all upstream repositories
+- Handles both GitHub and other git hosting platforms
+- Includes error handling for failed clones
+
+### UpstreamVendor-Update.sh
+- Updates existing repositories to latest versions
+- Skips missing directories gracefully
+- Provides summary of update status
+
+### Customization
+Edit scripts as needed to:
+- Add new repository sources
+- Change clone depth or branch targets
+- Modify update behavior
+- Handle special cases
+
+## 📊 Workspace Statistics
+
+- **Total Applications**: 56 repositories
+- **Repository Size**: ~2-3 GB total (varies by application)
+- **Update Frequency**: Weekly recommended
+- **Clone Time**: ~15-30 minutes for full clone
+
+## 🤝 Team Usage
+
+### For Developers
+1. Use `./UpstreamVendor-Clone.sh` on first setup
+2. Run `./UpstreamVendor-Update.sh` weekly or before new package work
+3. Always work in the containerized environment
+4. Never commit workspace contents to git
+
+### For DevOps
+1. Monitor disk space usage of workspace
+2. Ensure container environment has access to workspace
+3. Backup workspace if needed for disaster recovery
+4. Update scripts when adding/removing applications
+
+---
+
+**Last Updated**: 2025-01-04  
+**Maintained By**: KNEL/TSYS Development Team  
+**Part of**: [KNEL Production Containers](../README.md) packaging project
--- a/PackagingForCloudronWorkspace/UpstreamVendor-Clone.sh
+++ b/PackagingForCloudronWorkspace/UpstreamVendor-Clone.sh
@@ -0,0 +1,214 @@
+#!/bin/bash
+
+export PS4='(${BASH_SOURCE}:${LINENO}): - [${SHLVL},${BASH_SUBSHELL},$?] $ '
+
+function error_out()
+{
+        echo "Bailing out. See above for reason...."
+        exit 1
+}
+
+function handle_failure() {
+  local lineno=$1
+  local fn=$2
+  local exitstatus=$3
+  local msg=$4
+  local lineno_fns=${0% 0}
+  if [[ "$lineno_fns" != "-1" ]] ; then
+    lineno="${lineno} ${lineno_fns}"
+  fi
+  echo "${BASH_SOURCE[0]}: Function: ${fn} Line Number : [${lineno}] Failed with status ${exitstatus}: $msg"
+}
+
+trap 'handle_failure "${BASH_LINENO[*]}" "$LINENO" "${FUNCNAME[*]:-script}" "$?" "$BASH_COMMAND"' ERR
+
+set -o errexit
+set -o nounset
+set -o pipefail
+set -o functrace
+
+
+export GIT_REPO_LIST
+GIT_REPO_LIST=(
+
+####################
+# Vp techops stuff
+####################
+
+#https://projects.knownelement.com/issues/179
+https://github.com/apache/apisix.git
+
+#https://projects.knownelement.com/issues/204
+https://github.com/target/goalert.git
+
+#https://projects.knownelement.com/issues/189
+https://github.com/consuldemocracy/consuldemocracy.git
+
+#https://projects.knownelement.com/issues/195
+https://github.com/fleetdm/fleet.git
+
+#https://projects.knownelement.com/issues/227
+https://github.com/fonoster/fonoster.git
+
+#https://projects.knownelement.com/issues/192
+https://github.com/healthchecks/healthchecks.git
+
+#https://projects.knownelement.com/issues/209
+https://github.com/juspay/hyperswitch
+
+#https://projects.knownelement.com/issues/201
+https://github.com/netbox-community/netbox-docker.git
+
+# https://projects.knownelement.com/issues/205 
+https://github.com/openboxes/openboxes-docker.git
+
+#https://projects.knownelement.com/issues/316
+https://github.com/openfiletax/openfile.git
+
+#https://projects.knownelement.com/issues/211
+https://github.com/GemGeorge/SniperPhish-Docker.git
+
+#https://projects.knownelement.com/issues/309
+https://github.com/datahub-project/datahub.git
+
+#https://projects.knownelement.com/issues/54
+https://github.com/wiredlush/easy-gate.git
+
+#https://projects.knownelement.com/issues/208
+https://github.com/Payroll-Engine/PayrollEngine.git
+
+#https://projects.knownelement.com/issues/194
+https://github.com/huginn/huginn.git
+
+#https://projects.knownelement.com/issues/191
+https://github.com/gristlabs/grist-core
+
+#https://projects.knownelement.com/issues/277
+https://github.com/jhpyle/docassemble.git
+
+#https://projects.knownelement.com/issues/273
+https://github.com/kazhuravlev/database-gateway.git
+
+#https://projects.knownelement.com/issues/217
+https://github.com/rundeck/rundeck.git
+
+#https://projects.knownelement.com/issues/222
+https://github.com/SchedMD/slurm.git
+https://github.com/giovtorres/slurm-docker-cluster.git
+
+#https://projects.knownelement.com/issues/225
+https://github.com/rathole-org/rathole.git
+
+#https://projects.knownelement.com/issues/234
+https://github.com/jenkinsci/jenkins.git
+
+#https://projects.knownelement.com/issues/322
+https://github.com/runmedev/runme.git
+
+#https://projects.knownelement.com/issues/301
+https://github.com/apache/seatunnel
+
+#https://projects.knownelement.com/issues/271
+https://github.com/thecatlady/docker-webhook
+
+####################
+# CTO Stuff
+####################
+
+#https://projects.knownelement.com/issues/173
+https://github.com/inventree/InvenTree.git
+
+#https://projects.knownelement.com/issues/180
+https://github.com/Cloud-RF/tak-server
+
+#https://projects.knownelement.com/issues/178
+https://github.com/midday-ai/midday.git
+
+#https://projects.knownelement.com/issues/181
+https://github.com/killbill/killbill.git
+
+#https://projects.knownelement.com/issues/184
+https://github.com/chirpstack/chirpstack.git
+
+#https://projects.knownelement.com/issues/185
+https://github.com/CraigChat/craig.git
+
+#https://projects.knownelement.com/issues/188
+https://github.com/elabftw/elabftw.git
+
+#https://projects.knownelement.com/issues/196
+https://github.com/jamovi/jamovi.git
+
+#https://projects.knownelement.com/issues/197
+https://github.com/INTI-CMNB/KiBot.git
+
+#https://projects.knownelement.com/issues/214
+https://github.com/Resgrid/Core
+
+#https://projects.knownelement.com/issues/216
+https://github.com/reviewboard/reviewboard.git
+
+#https://projects.knownelement.com/issues/218
+https://gitlab.com/librespacefoundation/satnogs/docker-kaitai.git
+https://gitlab.com/librespacefoundation/satnogs/docker-satnogs-webgui.git
+
+#https://projects.knownelement.com/issues/219
+https://github.com/f4exb/sdrangel-docker
+
+#https://projects.knownelement.com/issues/221
+https://github.com/SigNoz/signoz.git
+
+#https://projects.knownelement.com/issues/228
+https://github.com/sebo-b/warp.git
+
+#https://projects.knownelement.com/issues/272
+https://github.com/jgraph/docker-drawio
+
+#https://projects.knownelement.com/issues/274
+https://github.com/openblocks-dev/openblocks.git
+
+#https://projects.knownelement.com/issues/276
+https://github.com/wireviz/wireviz-web.git
+
+#https://projects.knownelement.com/issues/278
+https://github.com/opulo-inc/autobom.git
+
+#https://projects.knownelement.com/issues/279
+https://github.com/PLMore/PLMore
+
+#https://projects.knownelement.com/issues/282
+https://github.com/manyfold3d/manyfold.git
+
+#https://projects.knownelement.com/issues/283
+https://github.com/langfuse/oss-llmops-stack.git
+
+#https://projects.knownelement.com/issues/286
+https://github.com/HeyPuter/puter.git
+
+#https://projects.knownelement.com/issues/285
+https://github.com/windmill-labs/windmill.git
+
+#https://projects.knownelement.com/issues/326
+https://github.com/sbabic/swupdate.git
+
+#https://projects.knownelement.com/issues/300
+https://github.com/mendersoftware/mender-server.git
+
+#https://projects.knownelement.com/issues/50
+https://github.com/vanila-io/wireflow.git
+
+#https://projects.knownelement.com/issues/226
+https://github.com/nautechsystems/nautilus_trader.git
+
+#TBD
+https://github.com/funmusicplace/mirlo.git
+
+)
+
+cd Docker
+
+IFS=$'\n\t'
+
+for GIT_REPO in ${GIT_REPO_LIST[@]};do
+	git clone --depth 1 $GIT_REPO || true
+done
--- a/PackagingForCloudronWorkspace/UpstreamVendor-Update.sh
+++ b/PackagingForCloudronWorkspace/UpstreamVendor-Update.sh
@@ -0,0 +1,44 @@
+#!/bin/bash
+
+export PS4='(${BASH_SOURCE}:${LINENO}): - [${SHLVL},${BASH_SUBSHELL},$?] $ '
+
+function error_out()
+{
+        echo "Bailing out. See above for reason...."
+        exit 1
+}
+
+function handle_failure() {
+  local lineno=$1
+  local fn=$2
+  local exitstatus=$3
+  local msg=$4
+  local lineno_fns=${0% 0}
+  if [[ "$lineno_fns" != "-1" ]] ; then
+    lineno="${lineno} ${lineno_fns}"
+  fi
+  echo "${BASH_SOURCE[0]}: Function: ${fn} Line Number : [${lineno}] Failed with status ${exitstatus}: $msg"
+}
+
+trap 'handle_failure "${BASH_LINENO[*]}" "$LINENO" "${FUNCNAME[*]:-script}" "$?" "$BASH_COMMAND"' ERR
+
+set -o errexit
+set -o nounset
+set -o pipefail
+set -o functrace
+
+
+cd Docker
+
+GIT_REPO_LIST="$(ls -d */)"
+
+IFS=$'\n\t'
+
+for GIT_REPO in ${GIT_REPO_LIST[@]};
+do
+  CURRENT_DIR=$(realpath $PWD)
+  echo "Updating from $GIT_REPO..."
+  cd $GIT_REPO 
+  git pull
+  cd $CURRENT_DIR
+done