diff --git a/JOURNAL.md b/JOURNAL.md index 15d875e..03ea6a2 100644 --- a/JOURNAL.md +++ b/JOURNAL.md @@ -1304,6 +1304,69 @@ terminal); SSO users register WITHOUT privileges. --- +### 17. ConsulDemocracy (Collaboration) ✅ +**Date**: 2026-09-06 +**Application**: ConsulDemocracy — citizen participation platform behind +Madrid's Consul (proposals, participatory budgeting, debates, polls, +collaborative legislation). Upstream: +https://github.com/consuldemocracy/consuldemocracy (MIT). Rails 8 + +PostgreSQL + delayed_job + memcached. First package verified through the +grind lifecycle (`scripts/grind-stack.sh`) end to end. + +**Ticket**: [#653](https://projects.knownelement.com/issues/653) + +**Pattern**: build-from-source on ruby:3.4.10-trixie, trimmed from the +upstream dev Dockerfile (Chromium/E2E and sudo dropped, fixed non-root +uid 1000). No published upstream image exists. + +**Auth gate verdict**: ✅ OIDC preferred. Generic OpenID Connect via +`omniauth_openid_connect` (devise.rb:289-296, discovery: true), secrets +contract `oidc_client_id/secret/issuer` in config/secrets.yml — wired +from `CLOUDRON_OIDC_*` by start.sh. SAML also available. + +**Key decisions / lessons (four build-breaking gotchas, all fixed)**: + +1. **`ruby file: ".ruby-version"`**: the Gemfile resolves its ruby + version from `.ruby-version` BY NAME — copying it under a different + name kills `bundle install` with a bare backtrace. +2. **`eval_gemfile "./Gemfile_custom"`**: the Gemfile expects a + developer-local `Gemfile_custom` that isn't in git — `touch` an empty + one before `bundle install`. +3. **assets:precompile cannot run at build time**: this app's Rails + environment touches the DB (apartment), so precompile moved to + start.sh after the postgres wait (idempotent; slow only first boot). + Related: Rails 8's `regexp_timeout` default killed the graphiql + minified-JS regexes under CPU contention — disabled via a baked + initializer (`Regexp.timeout = nil`). +4. **Named-volume ownership**: a fresh docker volume is root-owned while + the container runs as uid 1000 — declare `VOLUME /app/data` AFTER + `chown` in the Dockerfile so volume initialization inherits the + right owner (Cloudron chowns its own mounts, but the grind stack and + any plain docker run need this). +5. **Seed verification over marker trust**: one boot observed + `db:create db:migrate db:seed` exit 0 while persisting nothing; a + standalone re-seed landed fine. start.sh now checks `Setting.count` + and re-seeds an empty database regardless of the marker file. +6. **`feature.oidc_login` defaults to false** upstream — start.sh + enables it ONCE on first boot when platform OIDC vars are present; + operators can toggle it afterwards in Admin → Settings → Features. + +**Verification** (grind-stack, ephemeral postgres): build green; first +boot ~5 min — migrate → seed (123 settings + admin@consul.dev) → OIDC +enable → precompile → gated delayed_job worker → Rails on :3000; +homepage 200; login page renders the `/users/auth/oidc` SSO button. + +**Files Created**: Dockerfile, CloudronManifest.json (port 3000, +localstorage + postgresql, 2048MB), start.sh (config generation, seed +verify, memcached, worker gate), README.md, CHANGELOG.md, +.dockerignore (repo/.git only — the source IS the build context), +logo.png (upstream public/consul_logo.png). + +**Commit**: `feat(consuldemocracy): add initial Cloudron package` (7c65dc1, +landed mid-flight) + verification hardening follow-up + +--- + ## Packaging Pattern: Download Pre-Compiled Binaries ### When to Use diff --git a/Package-Workspace/Collaboration/consuldemocracy/CHANGELOG.md b/Package-Workspace/Collaboration/consuldemocracy/CHANGELOG.md index 3d306d6..9f38d7d 100644 --- a/Package-Workspace/Collaboration/consuldemocracy/CHANGELOG.md +++ b/Package-Workspace/Collaboration/consuldemocracy/CHANGELOG.md @@ -7,7 +7,17 @@ - postgresql + localstorage addons; Rails server on :3000 - Generated configs (database.yml / secrets.yml) under /app/data, regenerated each boot; persisted secret_key_base -- First-boot db:create/migrate/seed with marker; gated delayed_job - worker; in-container memcached -- Platform OIDC wired into secrets.yml; local admin seeded +- First-boot db:create/migrate/seed with marker + empty-DB verification + (a stale marker against a fresh database re-seeds); gated delayed_job + worker; in-container memcached; assets:precompile at start (the app's + environment needs the DB, so precompile cannot run at build time) +- Platform OIDC wired into secrets.yml; `feature.oidc_login` auto-enabled + on first boot when the provider is present; local admin seeded (admin@consul.dev — password change required on first login) + +## Verification (2026-09-06) + +Full-stack grind-stack test against ephemeral postgres: build green, +first boot ~5 min (migrate → seed [123 settings, admin] → OIDC enable → +precompile → delayed_job → Rails), homepage 200, login page renders the +`/users/auth/oidc` SSO button. diff --git a/Package-Workspace/Collaboration/consuldemocracy/README.md b/Package-Workspace/Collaboration/consuldemocracy/README.md index 5cdfb87..862f11f 100644 --- a/Package-Workspace/Collaboration/consuldemocracy/README.md +++ b/Package-Workspace/Collaboration/consuldemocracy/README.md @@ -18,6 +18,9 @@ regenerates secrets.yml on every boot with the Cloudron platform provider values (`CLOUDRON_OIDC_*`). - SSO redirect: `https:///users/auth/oidc/callback` +- The SSO button (`feature.oidc_login`, default off upstream) is + auto-enabled on FIRST boot when the platform provider is present — + after that it's yours to toggle in Admin → Settings → Features. - Local login stays available alongside SSO. - SAML is also supported (saml_* keys in secrets.yml) if an external IdP is preferred. diff --git a/Package-Workspace/Collaboration/consuldemocracy/logo.png b/Package-Workspace/Collaboration/consuldemocracy/logo.png new file mode 100644 index 0000000..0261d8c Binary files /dev/null and b/Package-Workspace/Collaboration/consuldemocracy/logo.png differ diff --git a/Package-Workspace/Collaboration/consuldemocracy/start.sh b/Package-Workspace/Collaboration/consuldemocracy/start.sh index 9c38c55..d27021a 100755 --- a/Package-Workspace/Collaboration/consuldemocracy/start.sh +++ b/Package-Workspace/Collaboration/consuldemocracy/start.sh @@ -97,11 +97,30 @@ memcached -d -p 11211 -u consul -m 64 # --- 3. database lifecycle ------------------------------------------------------- wait_tcp "${DB_HOST}" "${DB_PORT}" "PostgreSQL" -if [[ ! -f "${MARKER}" ]]; then - echo "First boot: creating + migrating + seeding the database ..." +# seed decision: the marker alone is not trusted - a marker left by a +# previous database (restore, migration, fresh test DB) must not skip +# seeding an empty one (observed: seed chain exits 0 yet persists +# nothing when re-run in the same first-boot process; a standalone +# re-seed lands fine, so verify + retry instead of debugging ghosts) +SEED_COUNT="$(bundle exec rails runner 'print Setting.count' 2>/dev/null || echo 0)" +if [[ ! -f "${MARKER}" || "${SEED_COUNT}" = "0" ]]; then + echo "First boot (or empty database): creating + migrating + seeding ..." bundle exec rails db:create db:migrate db:seed + SEED_COUNT="$(bundle exec rails runner 'print Setting.count' 2>/dev/null || echo 0)" + if [[ "${SEED_COUNT}" = "0" ]]; then + echo "seed pass 1 did not persist - retrying standalone seed" + bundle exec rails db:seed + fi touch "${MARKER}" echo "Database seeded (local admin: admin@consul.dev - change the password!)" + + # first-boot only: expose the SSO button when the platform provider is + # wired (feature.oidc_login defaults to false; operators can toggle it + # later in Admin > Settings > Features without us re-enabling it) + if [[ -n "${CLOUDRON_OIDC_CLIENT_ID:-}" ]]; then + bundle exec rails runner 'Setting["feature.oidc_login"] = true' || true + echo "OIDC login enabled (admin panel can toggle it)" + fi else bundle exec rails db:migrate fi diff --git a/README.md b/README.md index 8dde7e4..92e76cd 100644 --- a/README.md +++ b/README.md @@ -9,9 +9,9 @@ The Cloudron component focuses on packaging upstream free/libre/open application ### 📊 Current Progress - **Total Applications**: ~57 (see [GitUrlList.txt](GitUrlList.txt)) -- **Completed Packages**: 16/~57 (~28%) +- **Completed Packages**: 17/~57 (~30%) - **Packaging Templates**: Created ✅ -- **Packages Committed & Pushed**: 16 ✅ +- **Packages Committed & Pushed**: 17 ✅ - **Build Tickets**: 46 filed (#633-#678, umbrella [#632](https://projects.knownelement.com/issues/632), Redmine project 55); grist-core excluded (packaged upstream) @@ -38,6 +38,7 @@ The Cloudron component focuses on packaging upstream free/libre/open application | 14 | ChirpStack | Infrastructure | 83.4MB | 8080 | localstorage, postgresql, redis | ✅ Committed | | 15 | eLabFTW | Business-Apps | ~209MB (compressed) | 443 | localstorage, mysql, ldap | ✅ Committed | | 16 | NetBox | Infrastructure | ~1GB | 8080 | localstorage, postgresql, redis | ✅ Committed | +| 17 | ConsulDemocracy | Collaboration | ~1.6GB | 3000 | localstorage, postgresql | ✅ Committed | ### 📦 Packages in Development diff --git a/STATUS.md b/STATUS.md index 72aae0d..a60b229 100644 --- a/STATUS.md +++ b/STATUS.md @@ -3,7 +3,7 @@ > **Human read-only. Agents maintain this file automatically after each work > session.** Do not edit by hand — the next agent run will overwrite it. > -> **Last updated:** 2026-09-06 by ZCode (GLM-5.3) — NetBox packaged +> **Last updated:** 2026-09-06 by ZCode (GLM-5.3) — ConsulDemocracy packaged (17th; verified via grind lifecycle) > (#648, Infrastructure, 16th package); auth gate verdict: native OIDC via > python-social-auth (OpenIdConnectAuth + SOCIAL_AUTH_OIDC_*) wired to the > platform provider; official-image wrapper of netboxcommunity/netbox @@ -13,14 +13,14 @@ ## Current State: STABLE (packaging phase, ongoing) -Cloudron packaging pipeline is operational. 16 of ~57 upstream applications are +Cloudron packaging pipeline is operational. 17 of ~57 upstream applications are packaged, committed, and pushed. Packaging templates exist for the core patterns. The gardening protocol (this file + AGENTS.md) keeps docs in sync. All remaining apps now carry build tickets (#633-#678) under umbrella [#632](https://projects.knownelement.com/issues/632) in Redmine project 55 — ready for the sequential grind-driver pattern. -## Completed Packages (16) +## Completed Packages (17) | # | Application | Category | Pattern | Port(s) | Addons | |---|-------------|----------|---------|---------|--------| @@ -45,7 +45,7 @@ Each package lives in `Package-Workspace///` and contains a `Dockerfile`, `CloudronManifest.json`, `README.md`, `CHANGELOG.md`, `logo.png`, and (where relevant) `start.sh` + `.env.example`. -## Build Ticket Map (46 filed 2026-09-01; 39 open — 7 landed: #639, #640, #648, #650, #651, #668, #669) +## Build Ticket Map (46 filed 2026-09-01; 38 open — 8 landed: #639, #640, #648, #650, #651, #653, #668, #669) All under umbrella [#632](https://projects.knownelement.com/issues/632) (Redmine project 55, tracker Feature). Excluded: **grist-core** (a Cloudron @@ -68,12 +68,12 @@ package already exists upstream — do not package). | #645 | OpenBoxes | Business-Apps | needs LDAP runtime verification (note 2026-09-06) | | #646 | Nautilus Trader | Financial-Trading | disposition (headless lib) | | #647 | Fleet | Monitoring | build (research queue) | -| #648 | NetBox | Infrastructure | build (redis addon; strong candidate) | +| #648 | NetBox | Infrastructure | DONE 2026-09-06 (16th package) | | #649 | SeaTunnel | Data-Management | disposition (service vs tool) | | #650 | Rathole | Infrastructure | build (proxy) | | #651 | Easy-Gate | Infrastructure | build (proxy) | | #652 | Huginn | Automation | blocked-on-auth (local-only Devise; note 2026-09-06) | -| #653 | ConsulDemocracy | Collaboration | build | +| #653 | ConsulDemocracy | Collaboration | DONE 2026-09-06 (17th, grind-verified) | | #654 | BOINC | Scientific-Computing | disposition (server packaging heavy) | | #655 | Slurm | System-Administration | disposition (likely out of scope) | | #656 | Gophish | Security | blocked-on-auth | @@ -122,7 +122,7 @@ Full write-ups of each pattern + challenges are in [`JOURNAL.md`](JOURNAL.md). | Monitoring | 6 | 1/6 | healthchecks done | | Automation | 4 | 1/4 | windmill done | | Business-Apps | 8 | 1/8 | elabftw done | -| Collaboration | 2 | 0/2 | | +| Collaboration | 2 | 1/2 | consuldemocracy done | | Communication | 1 | 0/1 | | | Data-Management | 2 | 0/2 | | | DevOps-Tools | 1 | 1/1 (100%) ✅ | fx done | @@ -142,7 +142,7 @@ Auth capability is a hard gate before packaging (see LDAP acceptable (risk flag), 🔄 = auth-proxy (no users), ❌ = local-only (unacceptable / blocked-on-auth). -### Completed packages (16) +### Completed packages (17) | App | OIDC | LDAP | Verdict | Note | |-----|------|------|---------|------| @@ -162,6 +162,7 @@ LDAP acceptable (risk flag), 🔄 = auth-proxy (no users), ❌ = local-only | ChirpStack | yes | no | ✅ preferred | **Packaged**; native `[user_authentication.openid_connect]` wired to `CLOUDRON_OIDC_*`; OIDC-registered users are non-admin — one-time `CHIRPSTACK_AUTH_MODE=internal` bootstrap links the seeded `admin` to your SSO email (README) | | eLabFTW | no | yes | ⚠️ risk (LDAP) | **Packaged**; no OIDC upstream (auth = local/SAML/LDAP per `src/Enums/AuthMethod.php`); `ldap` addon enabled, README maps `CLOUDRON_LDAP_*` → sysconfig LDAP panel — must be validated on the live Cloudron before production; SAML possible via external IdP | | NetBox | yes | plugin | ✅ preferred | **Packaged**; python-social-auth OIDC (`REMOTE_AUTH_BACKEND=OpenIdConnectAuth` + `SOCIAL_AUTH_OIDC_*`) wired from `CLOUDRON_OIDC_*`; local Django login retained for admin bootstrap; SSO users register without privileges (README) | +| ConsulDemocracy | yes | no | ✅ preferred | **Packaged**; omniauth_openid_connect (discovery) wired into generated secrets.yml; `feature.oidc_login` auto-enabled on first boot when the provider is present (admin-panel toggle afterwards); SAML also available; local admin seeded | ### Candidates researched