Package or bust!

This commit is contained in:
2025-10-17 15:09:16 -05:00
parent 1fe434ea3e
commit 8918d89b45
61 changed files with 891 additions and 1471 deletions

View File

@@ -0,0 +1,104 @@
# AI Agents - Cloudron Packaging Project
**Purpose**: Document all AI agents, their roles, configurations, and interactions for this project
---
## Active Agents
### Agent: Claude Sonnet 4.5 (Cursor)
- **Role**: Senior DevOps/SRE Engineer - Cloudron Packaging Specialist
- **Activation Date**: 2025-10-17 14:30
- **Primary Responsibilities**:
- Package 56 applications for Cloudron
- Maintain project documentation
- Create and test Cloudron packages
- Smoke test all implementations
- Provide proof of completion for all work
- **Communication Protocol**:
- ALL communication via markdown files in Workspace/ or collab/ directories
- Chat ONLY used to notify when markdown files are ready for review
- No questions or substantive discussion in chat
- **Operating Principles** (from SEED_PROMPT.md):
- Expert-level work (senior engineer, not junior)
- Accuracy over speed - think carefully before acting
- Defensive thinking from first principles
- Brutal self-assessment and compliance with instructions
- Act as own QA tester
- Provide hard proof (logs, curl output, etc.) for all completions
- Never claim completion without proof
- **Workflow**:
- questions → proposals → plans → prompts → implementation
- One application at a time
- All work done in containers, not on host (except git, docker, bash commands)
- Use KNELCloudronDev-[package-name] prefix for all Docker artifacts
- **Constraints**:
- Must not deviate from SEED_PROMPT.md instructions or approved plans
- Must update TheSoverignCloud.md continuously
- Must follow questions/proposals/plans/prompts/implementation workflow
- Must wait for approval before implementation
- **Current Task**: Initial planning and question gathering
- **Current Status**: Awaiting user answers to questions in TheSoverignCloud.md
---
## Agent Interactions Log
### 2025-10-17 14:30 - Initial Engagement
- **Event**: Agent activated and provided SEED_PROMPT.md
- **Agent Action**: Failed initial communication protocol (used chat instead of markdown)
- **Corrective Action**: Created TheSoverignCloud.md and AGENTS.md per instructions
- **Lesson Learned**: MUST communicate via markdown files only, chat only for notifications
---
## Configuration & Context
### Key Project Files
- **SEED_PROMPT.md**: Master instructions and requirements
- **TheSoverignCloud.md**: Central planning, status tracking, questions, lessons learned
- **AGENTS.md**: This file - agent documentation
- **PackageList.txt**: All 56 applications to package with priorities
### Environment Configuration
- **Working Directory**: `/home/localuser/TSYSPublic/ChiefOperationsOfficer/VpTechops/KNELProductionContainers/CloudronPackages`
- **Container Prefix**: KNELCloudronDev-
- **Cloudron Instance**: https://my.knownelement.com
- **Builder Service**: https://cloudron-app-build.knownelement.com/
- **Registry Service**: https://cloudron-docker-registry.knownelement.com/
### Agent Capabilities
- File operations (read, write, search)
- Terminal command execution
- Docker operations
- Git operations
- Web search for latest Cloudron documentation
- Code analysis and generation
- Testing and validation
---
## Prompt Templates
*This section will be populated with reusable prompt templates as we establish patterns*
---
## Performance Metrics
*This section will track agent performance and efficiency*
- **Applications Packaged**: 0/56
- **Success Rate**: N/A
- **Average Time per Package**: N/A
- **Errors/Rework Required**: 1 (initial communication protocol failure)
---
*Last Updated: 2025-10-17 14:30*

View File

@@ -0,0 +1,52 @@
# Cloudron Packaging Helper Container
# Based on Ubuntu 22.04 as requested
FROM ubuntu:22.04
# Set environment variables
ENV DEBIAN_FRONTEND=noninteractive
ENV TZ=UTC
# Install system dependencies
RUN apt-get update && apt-get install -y \
curl \
wget \
git \
docker.io \
docker-compose \
jq \
vim \
nano \
build-essential \
python3 \
python3-pip \
nodejs \
npm \
golang-go \
&& rm -rf /var/lib/apt/lists/*
# Install yq separately
RUN wget -qO /usr/local/bin/yq https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 && \
chmod +x /usr/local/bin/yq
# Install Cloudron CLI tools
# Note: These may need to be updated based on latest Cloudron documentation
RUN curl -fsSL https://cloudron.io/cloudron-cli/install.sh | bash
# Create workspace directory
WORKDIR /workspace
# Copy the entire CloudronPackages directory
COPY . /workspace/
# Set up git configuration (will be overridden by user's config)
RUN git config --global user.name "Cloudron Packager" && \
git config --global user.email "packager@cloudron.local"
# Create directories for package development
RUN mkdir -p /workspace/packages /workspace/temp
# Set working directory to the main project
WORKDIR /workspace
# Default command
CMD ["/bin/bash"]

View File

@@ -0,0 +1,56 @@
https://github.com/apache/apisix.git
https://github.com/jenkinsci/jenkins.git
https://github.com/rundeck/rundeck.git
https://github.com/SigNoz/signoz.git
https://github.com/netbox-community/netbox-docker.git
https://github.com/juspay/hyperswitch
https://github.com/gristlabs/grist-core
https://github.com/inventree/InvenTree.git
https://github.com/consuldemocracy/consuldemocracy.git
https://github.com/reviewboard/reviewboard.git
https://github.com/healthchecks/healthchecks.git
https://github.com/fleetdm/fleet.git
https://github.com/huginn/huginn.git
https://github.com/windmill-labs/windmill.git
https://github.com/target/goalert.git
https://github.com/datahub-project/datahub.git
https://github.com/elabftw/elabftw.git
https://github.com/jhpyle/docassemble.git
https://github.com/killbill/killbill.git
https://github.com/openboxes/openboxes-docker.git
https://github.com/rathole-org/rathole.git
https://github.com/fonoster/fonoster.git
https://github.com/chirpstack/chirpstack.git
https://github.com/wiredlush/easy-gate.git
https://github.com/GemGeorge/SniperPhish-Docker.git
https://github.com/SchedMD/slurm.git
https://github.com/giovtorres/slurm-docker-cluster.git
https://github.com/runmedev/runme.git
https://github.com/apache/seatunnel
https://github.com/thecatlady/docker-webhook
https://github.com/Cloud-RF/tak-server
https://github.com/midday-ai/midday.git
https://github.com/CraigChat/craig.git
https://github.com/jamovi/jamovi.git
https://github.com/INTI-CMNB/KiBot.git
https://github.com/Resgrid/Core
https://gitlab.com/librespacefoundation/satnogs/docker-kaitai.git
https://gitlab.com/librespacefoundation/satnogs/docker-satnogs-webgui.git
https://github.com/f4exb/sdrangel-docker
https://github.com/sebo-b/warp.git
https://github.com/jgraph/docker-drawio
https://github.com/openblocks-dev/openblocks.git
https://github.com/wireviz/wireviz-web.git
https://github.com/opulo-inc/autobom.git
https://github.com/PLMore/PLMore
https://github.com/manyfold3d/manyfold.git
https://github.com/langfuse/oss-llmops-stack.git
https://github.com/HeyPuter/puter.git
https://github.com/sbabic/swupdate.git
https://github.com/mendersoftware/mender-server.git
https://github.com/vanila-io/wireflow.git
https://github.com/nautechsystems/nautilus_trader.git
https://github.com/openfiletax/openfile.git
https://github.com/kazhuravlev/database-gateway.git
https://github.com/Payroll-Engine/PayrollEngine.git
https://github.com/funmusicplace/mirlo.git

View File

@@ -0,0 +1,160 @@
# Cloudron Packaging Workspace
This workspace contains development tools and upstream source repositories for Cloudron application packaging.
## 🏗️ Workspace Structure
```
PackagingForCloudronWorkspace/
├── README.md # This file
├── Docker/ (gitignored) # Upstream application sources (many apps)
├── NonDocker/ (gitignored) # Non-Docker application sources
├── UpstreamVendor-Clone.sh # Clone all upstream repositories
└── UpstreamVendor-Update.sh # Update existing repositories
```
## 🚀 Setup Instructions
### Initial Setup
```bash
cd PackagingForCloudronWorkspace/
# Create Docker directory for upstream sources
mkdir -p Docker
# Make scripts executable
chmod +x *.sh
# Clone all upstream vendor repositories
./UpstreamVendor-Clone.sh
```
This clones upstream vendor repositories used when packaging applications for Cloudron.
### Keeping Sources Updated
```bash
# Update all existing checkouts to latest versions
./UpstreamVendor-Update.sh
```
## 📦 Available Applications
The workspace contains ~56 upstream application repositories including:
### High Priority Applications
- **apisix** - Apache APISIX API Gateway
- **jenkins** - Jenkins CI/CD Platform
- **grist-core** - Grist Database/Spreadsheet
- **rundeck** - Rundeck Job Scheduler
- **reviewboard** - ReviewBoard Code Review
- **consuldemocracy** - Consul Democracy Platform
### Development & Infrastructure Tools
- **InvenTree** - Inventory Management System
- **elabftw** - Laboratory Management
- **netbox-docker** - Network Documentation
- **signoz** - Observability Platform
- **healthchecks** - Health Monitoring
- **fleet** - Device Management
### Productivity & Specialized Applications
- **huginn** - Web Automation
- **windmill** - Workflow Automation
- **docassemble** - Document Assembly
- **jamovi** - Statistical Analysis
- And many more...
## 🛠️ Development Workflow
### Using the Workspace
1. **Source Access**: All upstream sources are available in `Docker/[appname]/`
2. **Development**: Use the `KNELCloudron-packaging` container for all work
3. **Package Creation**: Create packages in separate temporary directories
4. **Git Exclusion**: All upstream sources are gitignored to keep repository clean
### Container Development
```bash
# Access development container
docker exec -it KNELCloudron-packaging bash
# Navigate to workspace
cd /workspace
# Access application source
cd CloudronPackagingWorkspace/Docker/[appname]/
# Create new package (outside of workspace)
cd /workspace
mkdir -p [appname]_package_new
```
## 📋 Workspace Management
### Adding New Applications
1. Update `UpstreamVendor-Clone.sh` with the new repository URL
2. Run the clone script to fetch the new application
### Removing Applications
1. Remove directory from `Docker/`
2. Update clone script to prevent future re-cloning
### Repository Updates
- Run `./UpstreamVendor-Update.sh` periodically or before starting packaging work
- Check for breaking changes in upstream before building
## ⚠️ Important Notes
### Git Exclusions
- `Docker/` and `NonDocker/` are gitignored (see repo `.gitignore`)
- Keeps the repo slim while preserving local sources
### Repository Integrity
- Never commit upstream sources to the repository
- Develop packages outside of `Docker/` (e.g., directly under `CloudronPackages/<AppName>`)
### Source Licenses
- Each upstream repository maintains its own license
- Review license compatibility before packaging
- Include appropriate license information in final packages
## 🔧 Script Maintenance
### UpstreamVendor-Clone.sh
- Contains git clone commands for all upstream repositories
- Handles both GitHub and other git hosting platforms
- Includes error handling for failed clones
### UpstreamVendor-Update.sh
- Updates existing repositories to latest versions
- Skips missing directories gracefully
- Provides summary of update status
### Customization
Edit scripts as needed to:
- Add new repository sources
- Change clone depth or branch targets
- Modify update behavior
- Handle special cases
## 📊 Notes
- Number of upstream repositories and size vary over time.
## 🤝 Team Usage
### For Developers
1. Use `./UpstreamVendor-Clone.sh` on first setup
2. Run `./UpstreamVendor-Update.sh` weekly or before new package work
3. Always work in the containerized environment
4. Never commit workspace contents to git
### For DevOps
1. Monitor disk space usage of workspace
2. Ensure container environment has access to workspace
3. Backup workspace if needed for disaster recovery
4. Update scripts when adding/removing applications
---
**Maintained By**: KNEL/TSYS Development Team

View File

@@ -0,0 +1,347 @@
# The Sovereign Cloud - Cloudron Packaging Project
**Mission**: Package 56 production-critical applications for Cloudron within 48 hours
**Status**: Initial Planning Phase
---
## Questions: 2025-10-17 14:30
### Authentication & Access
1. Do you currently have the Cloudron API token/credentials ready to place in `secret.txt`, or should I wait for you to provide them?
Yes I do. I have an api key from the super admin account. I also have a token from the cloudron builder service.
This host has performed a succesful docker login to cloudron-docker-registry.knownelement.com.
Also the builder service is configured to use cloudron-docker-registry.knownelement.com.
2. Should I verify connectivity to https://my.knownelement.com and https://cloudron-app-build.knownelement.com/ before proceeding?
Sure.
### Application Prioritization
3. PackageList.txt shows 56 applications in Tiers A-E. Should I start with apisix (first in Tier A - API Gateway), or do you want to specify a different starting application?
Please remove everything from that file except for the URLs.
Go ahead and pick your own order of packaging based on what you think is most important in terms of core technical and business infrastructure.
4. Would you prefer I start with a simpler application first to establish and validate the workflow before tackling critical infrastructure?
Yes. Lets do rathole and then inventree. I think those are good representative samples.
### Existing Work Assessment
5. I see several applications already cloned in Workspace/Docker/ (apisix, grist-core, hyperswitch, InvenTree, jenkins, killbill, openboxes, PayrollEngine, rundeck). Have you already attempted any packaging work that I should audit first?
6. Are there any existing CloudronManifest.json files or Dockerfiles I should review?
Start from scratch please. I blew all of that away. Be advised that Workspace/Docker is where git clones from upstream happen and should serve as input material. Please use CloudronPackages/Packages for actual cloudron package bits.
### Development Environment
7. For the helper container with Cloudron bits (Ubuntu 22.04 base): Do you want me to research and create the Dockerfile/build instructions, or do you have specific requirements/existing setup?
I leave it to you.
8. Should the helper container include specific Cloudron CLI tools? Which versions?
Whatever works? I don't have any specific needs
9. What should be mounted into the helper container? (I assume the entire CloudronPackages directory at minimum)
Yes , I think that should cover it.
### Workflow & Structure
10. You mentioned using collab/ (audit, questions, proposals, plans, prompts) and output/ directories. Should I create this structure, or only TheSoverignCloud.md and AGENTS.md as mentioned in lines 11 and 35?
11. If using collab/ structure: Should TheSoverignCloud.md live in collab/ or Workspace/?
12. What should go in AGENTS.md? (Documentation of AI agents used, their roles, prompts, etc.?)
For now, lets try just AGENTS.md and TheSoverignCloud.md and see how that works for the two test packages?
### Technical Research Needs
13. Should I perform web research on latest Cloudron packaging standards before creating the detailed plan?
Yes
14. Do you have any existing Cloudron packages you've created that I should study as reference implementations?
No
15. Are there specific Cloudron base images or addons I should be aware of?
Not that i'm aware of
### Approval & Communication Workflow
16. After you answer these questions inline, should I then create a detailed proposal document (in collab/proposals/) before creating the implementation plan?
17. For the questions -> proposals -> plans -> prompts -> implementation workflow: Should each stage have its own timestamped markdown file?
18. When I complete a stage (e.g., questions answered), should I notify you in chat that the next document is ready?
Lets table 16/17/18 for now, we can come back to it if we need it. This project may be contained enough, and this file (as it is and as it will be updated by me and you) may be sufficient for successful outcomes (working installs of new packages on Cloudron)
### Secret Management
19. Should I create the secret.txt file now with placeholder comments, or wait for you to create it?
Yes please
20. What secrets beyond Cloudron tokens will be needed? (Docker registry credentials, Git tokens, etc.?)
I don't think anything else will go in it. i already have ssh keys setup and tea is logged in to gitea already and docker login has been done to the cloudron registry.
### Testing & Validation
21. For smoke testing: What constitutes "proof of completion"? (Container starts, health check passes, specific functionality works, Cloudron install succeeds?)
Yes all of that.
Pass 1: The whole stack (app/web/db/worker whatever) comes up on my workstation and I can access it at localhost:12000 (use that port for all the testing on my workstation ok?). YOu will tell me to test and wait for me to give you feedback.
Pass 2: Cloudron install succeeds
22. Should I document the smoke test procedure for each application in a standardized format?
Sure
### Repository Management
23. I see Packages/ directory is empty. Should packaged applications go there with structure like Packages/[AppName]/CloudronManifest.json?
Yes. Use AppName and put all the bits for each app in the AppName sub directory exactly
24. Should I create a standardized directory structure for each package?
Sure. Deviate as needed, but i feel they will have much overlap.
---
## Status Tracker
### Applications to Package (56 Total)
#### Tier A: Critical Infrastructure (5 apps)
- [ ] apisix - API Gateway (Issue #179)
- [ ] jenkins - CI/CD Platform (Issue #234)
- [ ] rundeck - Job Scheduler/Automation (Issue #217)
- [ ] signoz - Observability Platform (Issue #221)
- [ ] netbox-docker - Network Documentation (Issue #201)
#### Tier B: Core Business Services (7 apps)
- [ ] hyperswitch - Payment Infrastructure (Issue #209)
- [ ] grist-core - Database/Spreadsheet (Issue #191)
- [ ] InvenTree - Inventory Management (Issue #173)
- [ ] consuldemocracy - Democracy Platform (Issue #189)
- [ ] reviewboard - Code Review (Issue #216)
- [ ] healthchecks - Monitoring (Issue #192)
- [ ] fleet - Device Management (Issue #195)
#### Tier C: Development & Automation Tools (6 apps)
- [ ] huginn - Web Automation (Issue #194)
- [ ] windmill - Workflow Automation (Issue #285)
- [ ] goalert - On-Call Management (Issue #204)
- [ ] datahub - Data Catalog (Issue #309)
- [ ] elabftw - Lab Management (Issue #188)
- [ ] docassemble - Document Assembly (Issue #277)
#### Tier D: Specialized Infrastructure (7 apps)
- [ ] killbill - Billing Platform (Issue #181)
- [ ] openboxes-docker - Supply Chain (Issue #205)
- [ ] rathole - Tunneling (Issue #225)
- [ ] fonoster - Telephony (Issue #227)
- [ ] chirpstack - LoRaWAN Server (Issue #184)
- [ ] easy-gate - Access Control (Issue #54)
- [ ] SniperPhish-Docker - Phishing Simulation (Issue #211)
#### Tier E: Remaining Applications (31 apps)
- [ ] slurm - HPC Job Scheduler (Issue #222)
- [ ] runme - Development Environment (Issue #322)
- [ ] seatunnel - Data Processing (Issue #301)
- [ ] docker-webhook - Webhook Handler (Issue #271)
- [ ] tak-server - TAK Server (Issue #180)
- [ ] midday - AI Assistant (Issue #178)
- [ ] craig - Chat Platform (Issue #185)
- [ ] jamovi - Statistical Analysis (Issue #196)
- [ ] KiBot - PCB Design (Issue #197)
- [ ] Resgrid Core - Emergency Management (Issue #214)
- [ ] satnogs - Satellite Ground Station (Issue #218)
- [ ] sdrangel-docker - SDR Software (Issue #219)
- [ ] warp - File Sharing (Issue #228)
- [ ] docker-drawio - Diagram Tool (Issue #272)
- [ ] openblocks - Low-Code Platform (Issue #274)
- [ ] wireviz-web - Network Documentation (Issue #276)
- [ ] autobom - BOM Management (Issue #278)
- [ ] PLMore - Project Management (Issue #279)
- [ ] manyfold - 3D Modeling (Issue #282)
- [ ] oss-llmops-stack - LLM Ops Stack (Issue #283)
- [ ] puter - Cloud Desktop (Issue #286)
- [ ] swupdate - Firmware Updates (Issue #326)
- [ ] mender-server - IoT Device Management (Issue #300)
- [ ] wireflow - Wireframing (Issue #50)
- [ ] nautilus_trader - Trading Platform (Issue #226)
- [ ] openfile - Tax Filing (Issue #316)
- [ ] database-gateway - Database Gateway (Issue #273)
- [ ] PayrollEngine - Payroll Engine (Issue #208)
- [ ] mirlo - Music Platform (TBD)
---
## Lessons Learned
*This section will be populated as we progress through packaging applications*
---
## Best Practices
*This section will be populated as we establish proven patterns*
---
## Technical Notes
### Current Environment
- Workspace: `/home/localuser/TSYSPublic/ChiefOperationsOfficer/VpTechops/KNELProductionContainers/CloudronPackages`
- Cloudron Instance: https://my.knownelement.com
- Builder Service: https://cloudron-app-build.knownelement.com/
- Docker Registry: https://cloudron-docker-registry.knownelement.com/
### Upstream Sources Status
- Location: `Workspace/Docker/`
- Currently Cloned: apisix, grist-core, hyperswitch, InvenTree, jenkins, jenkins-docker, killbill, openboxes, openboxes-docker, PayrollEngine, rundeck
- Remaining to Clone: 45+ applications
### Naming Convention
All Docker artifacts for testing: `KNELCloudronDev-[package-name]-[artifact-type]`
---
## Next Steps
### Completed Actions:
1.**Clean PackageList.txt** - Removed everything except URLs
2.**Create secret.txt** - Created with placeholder comments for Cloudron tokens
3.**Create helper container Dockerfile** - Created Dockerfile.helper with Ubuntu 22.04 base
4. 🔄 **Research Cloudron packaging standards** - In progress (limited web search results)
### Immediate Next Actions:
1. **Build helper container** and test it
2. **Start with rathole** as first test package
3. **Follow with InvenTree** as second test package
### Test Package Plan:
#### Package 1: rathole (Tunneling - Issue #225)
- **Repository**: https://github.com/rathole-org/rathole.git
- **Type**: Network tunneling tool (no auth)
- **Complexity**: Low (good for workflow validation)
- **Port**: 12000 (as specified)
- **Dependencies**: Minimal
#### Package 2: InvenTree (Inventory Management - Issue #173)
- **Repository**: https://github.com/inventree/InvenTree.git
- **Type**: Web application with database
- **Complexity**: Medium (good for testing database integration)
- **Port**: 12000 (as specified)
- **Dependencies**: PostgreSQL, Redis (likely)
### Cloudron Packaging Structure:
```
Packages/[AppName]/
├── CloudronManifest.json
├── Dockerfile
├── README.md
└── (other app-specific files)
```
### Testing Protocol:
1. **Pass 1**: Local testing on localhost:12000
- Container starts successfully
- Health check passes
- Application accessible via browser
- User provides feedback
2. **Pass 2**: Cloudron installation
- Package uploads to Cloudron
- Installation succeeds
- Application accessible via Cloudron
## Implementation Plan: rathole Package
### Phase 1: Environment Setup
1. **Build helper container**:
```bash
docker build -f Dockerfile.helper -t KNELCloudronDev-helper .
```
2. **Start helper container**:
```bash
docker run -it --name KNELCloudronDev-helper-container \
-v $(pwd):/workspace \
-p 12000:12000 \
KNELCloudronDev-helper
```
### Phase 2: rathole Package Development
1. **Clone rathole repository**:
```bash
cd /workspace/Workspace/Docker
git clone https://github.com/rathole-org/rathole.git
```
2. **Analyze rathole structure**:
- Review existing Dockerfile (if any)
- Identify dependencies and configuration
- Determine port requirements
3. **Create Cloudron package structure**:
```
Packages/rathole/
├── CloudronManifest.json
├── Dockerfile
├── README.md
└── config/ (if needed)
```
4. **Create CloudronManifest.json**:
- Define app metadata
- Set port configuration (12000)
- Configure health checks
5. **Create optimized Dockerfile**:
- Based on rathole's requirements
- Cloudron-compatible
- Health check enabled
### Phase 3: Local Testing
1. **Build rathole package**:
```bash
cd Packages/rathole
docker build -t KNELCloudronDev-rathole .
```
2. **Test locally**:
```bash
docker run -d --name KNELCloudronDev-rathole-test \
-p 12000:12000 \
KNELCloudronDev-rathole
```
3. **Verify functionality**:
- Check container logs
- Test port accessibility
- Validate health endpoint
4. **User testing**: Wait for user feedback on localhost:12000
### Phase 4: Cloudron Integration
1. **Package for Cloudron**:
- Create package archive
- Validate CloudronManifest.json
2. **Upload to Cloudron**:
- Use Cloudron CLI tools
- Test installation process
3. **Verify Cloudron deployment**:
- Confirm successful installation
- Test application access via Cloudron
### Phase 5: Documentation & Cleanup
1. **Update TheSoverignCloud.md** with lessons learned
2. **Document rathole-specific patterns** for reuse
3. **Clean up test containers**:
```bash
docker rm KNELCloudronDev-rathole-test
docker rmi KNELCloudronDev-rathole
```
---
*Last Updated: 2025-10-17 15:15*

View File

@@ -0,0 +1,182 @@
#!/bin/bash
# Ship The Sovereign Cloud - Repository Management Script
# Manages upstream repository cloning and updating for Cloudron packaging
# Runs on HOST (not in container) - uses host git commands
set -euo pipefail
# Script configuration
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PACKAGE_LIST="${SCRIPT_DIR}/PackageList.txt"
TARGET_DIR="${SCRIPT_DIR}/Docker"
WORKSPACE_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Logging functions
log_info() {
echo -e "${BLUE}[INFO]${NC} $1"
}
log_success() {
echo -e "${GREEN}[SUCCESS]${NC} $1"
}
log_warning() {
echo -e "${YELLOW}[WARNING]${NC} $1"
}
log_error() {
echo -e "${RED}[ERROR]${NC} $1"
}
# Check prerequisites
check_prerequisites() {
log_info "Checking prerequisites..."
if ! command -v git &> /dev/null; then
log_error "git is not installed or not in PATH"
exit 1
fi
if [[ ! -f "$PACKAGE_LIST" ]]; then
log_error "PackageList.txt not found at: $PACKAGE_LIST"
exit 1
fi
log_success "Prerequisites check passed"
}
# Create target directory
setup_workspace() {
log_info "Setting up workspace..."
mkdir -p "$TARGET_DIR"
log_success "Workspace ready at: $TARGET_DIR"
}
# Extract repository URLs from PackageList.txt
get_repo_urls() {
# Extract URLs, ignoring comments and empty lines
grep -v '^#' "$PACKAGE_LIST" | grep -v '^$' | grep -v '^[[:space:]]*$' || true
}
# Clone or update a single repository
manage_single_repo() {
local repo_url="$1"
local repo_name
local repo_path
# Extract repository name from URL
repo_name=$(basename "$repo_url" .git)
repo_path="$TARGET_DIR/$repo_name"
log_info "Processing: $repo_name"
if [[ -d "$repo_path/.git" ]]; then
# Repository exists, update it
log_info "Updating existing repository: $repo_name"
cd "$repo_path"
# Fetch latest changes
if git fetch --all --prune; then
# Try to fast-forward merge
if git pull --ff-only; then
log_success "Updated: $repo_name"
else
log_warning "Could not fast-forward $repo_name (may have local changes)"
log_info "Repository $repo_name is up to date but has diverged"
fi
else
log_warning "Failed to fetch updates for $repo_name"
fi
else
# Repository doesn't exist, clone it
log_info "Cloning new repository: $repo_name"
cd "$TARGET_DIR"
if git clone --depth 1 "$repo_url" "$repo_name"; then
log_success "Cloned: $repo_name"
else
log_error "Failed to clone: $repo_url"
return 1
fi
fi
}
# Main execution function
main() {
log_info "Ship The Sovereign Cloud - Repository Manager"
log_info "=============================================="
check_prerequisites
setup_workspace
# Get all repository URLs
local repo_urls
mapfile -t repo_urls < <(get_repo_urls)
if [[ ${#repo_urls[@]} -eq 0 ]]; then
log_warning "No repositories found in PackageList.txt"
exit 0
fi
log_info "Found ${#repo_urls[@]} repositories to manage"
# Process each repository
local success_count=0
local error_count=0
for repo_url in "${repo_urls[@]}"; do
if [[ -n "$repo_url" ]]; then
if manage_single_repo "$repo_url"; then
((success_count++))
else
((error_count++))
fi
fi
done
# Summary
log_info "=============================================="
log_info "Repository management complete"
log_success "Successfully processed: $success_count"
if [[ $error_count -gt 0 ]]; then
log_error "Failed: $error_count"
fi
log_info "Total repositories: ${#repo_urls[@]}"
# Show workspace status
log_info "Workspace location: $TARGET_DIR"
log_info "Repository count: $(find "$TARGET_DIR" -maxdepth 1 -type d -name ".git" | wc -l)"
}
# Handle script arguments
case "${1:-}" in
"help"|"-h"|"--help")
echo "Ship The Sovereign Cloud - Repository Manager"
echo ""
echo "Usage: $0 [command]"
echo ""
echo "Commands:"
echo " (no args) - Clone missing repos and update existing ones"
echo " help - Show this help message"
echo ""
echo "This script reads from PackageList.txt and manages repositories"
echo "in the Docker/ subdirectory."
exit 0
;;
"")
main
;;
*)
log_error "Unknown command: $1"
log_info "Use '$0 help' for usage information"
exit 1
;;
esac