fix(consuldemocracy): verify end-to-end + harden seed/OIDC [#653]
Seed verification over marker trust (observed exit-0 no-op seed), first-boot-only OIDC auto-enable, logo added, docs synced (JOURNAL s17, counts 17/~57). grind-stack verified: homepage 200, OIDC SSO button live. Detail: https://projects.knownelement.com/issues/653#note-5044
This commit is contained in:
+63
@@ -1304,6 +1304,69 @@ terminal); SSO users register WITHOUT privileges.
|
||||
|
||||
---
|
||||
|
||||
### 17. ConsulDemocracy (Collaboration) ✅
|
||||
**Date**: 2026-09-06
|
||||
**Application**: ConsulDemocracy — citizen participation platform behind
|
||||
Madrid's Consul (proposals, participatory budgeting, debates, polls,
|
||||
collaborative legislation). Upstream:
|
||||
https://github.com/consuldemocracy/consuldemocracy (MIT). Rails 8 +
|
||||
PostgreSQL + delayed_job + memcached. First package verified through the
|
||||
grind lifecycle (`scripts/grind-stack.sh`) end to end.
|
||||
|
||||
**Ticket**: [#653](https://projects.knownelement.com/issues/653)
|
||||
|
||||
**Pattern**: build-from-source on ruby:3.4.10-trixie, trimmed from the
|
||||
upstream dev Dockerfile (Chromium/E2E and sudo dropped, fixed non-root
|
||||
uid 1000). No published upstream image exists.
|
||||
|
||||
**Auth gate verdict**: ✅ OIDC preferred. Generic OpenID Connect via
|
||||
`omniauth_openid_connect` (devise.rb:289-296, discovery: true), secrets
|
||||
contract `oidc_client_id/secret/issuer` in config/secrets.yml — wired
|
||||
from `CLOUDRON_OIDC_*` by start.sh. SAML also available.
|
||||
|
||||
**Key decisions / lessons (four build-breaking gotchas, all fixed)**:
|
||||
|
||||
1. **`ruby file: ".ruby-version"`**: the Gemfile resolves its ruby
|
||||
version from `.ruby-version` BY NAME — copying it under a different
|
||||
name kills `bundle install` with a bare backtrace.
|
||||
2. **`eval_gemfile "./Gemfile_custom"`**: the Gemfile expects a
|
||||
developer-local `Gemfile_custom` that isn't in git — `touch` an empty
|
||||
one before `bundle install`.
|
||||
3. **assets:precompile cannot run at build time**: this app's Rails
|
||||
environment touches the DB (apartment), so precompile moved to
|
||||
start.sh after the postgres wait (idempotent; slow only first boot).
|
||||
Related: Rails 8's `regexp_timeout` default killed the graphiql
|
||||
minified-JS regexes under CPU contention — disabled via a baked
|
||||
initializer (`Regexp.timeout = nil`).
|
||||
4. **Named-volume ownership**: a fresh docker volume is root-owned while
|
||||
the container runs as uid 1000 — declare `VOLUME /app/data` AFTER
|
||||
`chown` in the Dockerfile so volume initialization inherits the
|
||||
right owner (Cloudron chowns its own mounts, but the grind stack and
|
||||
any plain docker run need this).
|
||||
5. **Seed verification over marker trust**: one boot observed
|
||||
`db:create db:migrate db:seed` exit 0 while persisting nothing; a
|
||||
standalone re-seed landed fine. start.sh now checks `Setting.count`
|
||||
and re-seeds an empty database regardless of the marker file.
|
||||
6. **`feature.oidc_login` defaults to false** upstream — start.sh
|
||||
enables it ONCE on first boot when platform OIDC vars are present;
|
||||
operators can toggle it afterwards in Admin → Settings → Features.
|
||||
|
||||
**Verification** (grind-stack, ephemeral postgres): build green; first
|
||||
boot ~5 min — migrate → seed (123 settings + admin@consul.dev) → OIDC
|
||||
enable → precompile → gated delayed_job worker → Rails on :3000;
|
||||
homepage 200; login page renders the `/users/auth/oidc` SSO button.
|
||||
|
||||
**Files Created**: Dockerfile, CloudronManifest.json (port 3000,
|
||||
localstorage + postgresql, 2048MB), start.sh (config generation, seed
|
||||
verify, memcached, worker gate), README.md, CHANGELOG.md,
|
||||
.dockerignore (repo/.git only — the source IS the build context),
|
||||
logo.png (upstream public/consul_logo.png).
|
||||
|
||||
**Commit**: `feat(consuldemocracy): add initial Cloudron package` (7c65dc1,
|
||||
landed mid-flight) + verification hardening follow-up
|
||||
|
||||
---
|
||||
|
||||
## Packaging Pattern: Download Pre-Compiled Binaries
|
||||
|
||||
### When to Use
|
||||
|
||||
@@ -7,7 +7,17 @@
|
||||
- postgresql + localstorage addons; Rails server on :3000
|
||||
- Generated configs (database.yml / secrets.yml) under /app/data,
|
||||
regenerated each boot; persisted secret_key_base
|
||||
- First-boot db:create/migrate/seed with marker; gated delayed_job
|
||||
worker; in-container memcached
|
||||
- Platform OIDC wired into secrets.yml; local admin seeded
|
||||
- First-boot db:create/migrate/seed with marker + empty-DB verification
|
||||
(a stale marker against a fresh database re-seeds); gated delayed_job
|
||||
worker; in-container memcached; assets:precompile at start (the app's
|
||||
environment needs the DB, so precompile cannot run at build time)
|
||||
- Platform OIDC wired into secrets.yml; `feature.oidc_login` auto-enabled
|
||||
on first boot when the provider is present; local admin seeded
|
||||
(admin@consul.dev — password change required on first login)
|
||||
|
||||
## Verification (2026-09-06)
|
||||
|
||||
Full-stack grind-stack test against ephemeral postgres: build green,
|
||||
first boot ~5 min (migrate → seed [123 settings, admin] → OIDC enable →
|
||||
precompile → delayed_job → Rails), homepage 200, login page renders the
|
||||
`/users/auth/oidc` SSO button.
|
||||
|
||||
@@ -18,6 +18,9 @@ regenerates secrets.yml on every boot with the Cloudron platform
|
||||
provider values (`CLOUDRON_OIDC_*`).
|
||||
|
||||
- SSO redirect: `https://<app-domain>/users/auth/oidc/callback`
|
||||
- The SSO button (`feature.oidc_login`, default off upstream) is
|
||||
auto-enabled on FIRST boot when the platform provider is present —
|
||||
after that it's yours to toggle in Admin → Settings → Features.
|
||||
- Local login stays available alongside SSO.
|
||||
- SAML is also supported (saml_* keys in secrets.yml) if an external
|
||||
IdP is preferred.
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 12 KiB |
@@ -97,11 +97,30 @@ memcached -d -p 11211 -u consul -m 64
|
||||
# --- 3. database lifecycle -------------------------------------------------------
|
||||
wait_tcp "${DB_HOST}" "${DB_PORT}" "PostgreSQL"
|
||||
|
||||
if [[ ! -f "${MARKER}" ]]; then
|
||||
echo "First boot: creating + migrating + seeding the database ..."
|
||||
# seed decision: the marker alone is not trusted - a marker left by a
|
||||
# previous database (restore, migration, fresh test DB) must not skip
|
||||
# seeding an empty one (observed: seed chain exits 0 yet persists
|
||||
# nothing when re-run in the same first-boot process; a standalone
|
||||
# re-seed lands fine, so verify + retry instead of debugging ghosts)
|
||||
SEED_COUNT="$(bundle exec rails runner 'print Setting.count' 2>/dev/null || echo 0)"
|
||||
if [[ ! -f "${MARKER}" || "${SEED_COUNT}" = "0" ]]; then
|
||||
echo "First boot (or empty database): creating + migrating + seeding ..."
|
||||
bundle exec rails db:create db:migrate db:seed
|
||||
SEED_COUNT="$(bundle exec rails runner 'print Setting.count' 2>/dev/null || echo 0)"
|
||||
if [[ "${SEED_COUNT}" = "0" ]]; then
|
||||
echo "seed pass 1 did not persist - retrying standalone seed"
|
||||
bundle exec rails db:seed
|
||||
fi
|
||||
touch "${MARKER}"
|
||||
echo "Database seeded (local admin: admin@consul.dev - change the password!)"
|
||||
|
||||
# first-boot only: expose the SSO button when the platform provider is
|
||||
# wired (feature.oidc_login defaults to false; operators can toggle it
|
||||
# later in Admin > Settings > Features without us re-enabling it)
|
||||
if [[ -n "${CLOUDRON_OIDC_CLIENT_ID:-}" ]]; then
|
||||
bundle exec rails runner 'Setting["feature.oidc_login"] = true' || true
|
||||
echo "OIDC login enabled (admin panel can toggle it)"
|
||||
fi
|
||||
else
|
||||
bundle exec rails db:migrate
|
||||
fi
|
||||
|
||||
@@ -9,9 +9,9 @@ The Cloudron component focuses on packaging upstream free/libre/open application
|
||||
### 📊 Current Progress
|
||||
|
||||
- **Total Applications**: ~57 (see [GitUrlList.txt](GitUrlList.txt))
|
||||
- **Completed Packages**: 16/~57 (~28%)
|
||||
- **Completed Packages**: 17/~57 (~30%)
|
||||
- **Packaging Templates**: Created ✅
|
||||
- **Packages Committed & Pushed**: 16 ✅
|
||||
- **Packages Committed & Pushed**: 17 ✅
|
||||
- **Build Tickets**: 46 filed (#633-#678, umbrella
|
||||
[#632](https://projects.knownelement.com/issues/632), Redmine project 55);
|
||||
grist-core excluded (packaged upstream)
|
||||
@@ -38,6 +38,7 @@ The Cloudron component focuses on packaging upstream free/libre/open application
|
||||
| 14 | ChirpStack | Infrastructure | 83.4MB | 8080 | localstorage, postgresql, redis | ✅ Committed |
|
||||
| 15 | eLabFTW | Business-Apps | ~209MB (compressed) | 443 | localstorage, mysql, ldap | ✅ Committed |
|
||||
| 16 | NetBox | Infrastructure | ~1GB | 8080 | localstorage, postgresql, redis | ✅ Committed |
|
||||
| 17 | ConsulDemocracy | Collaboration | ~1.6GB | 3000 | localstorage, postgresql | ✅ Committed |
|
||||
|
||||
### 📦 Packages in Development
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
> **Human read-only. Agents maintain this file automatically after each work
|
||||
> session.** Do not edit by hand — the next agent run will overwrite it.
|
||||
>
|
||||
> **Last updated:** 2026-09-06 by ZCode (GLM-5.3) — NetBox packaged
|
||||
> **Last updated:** 2026-09-06 by ZCode (GLM-5.3) — ConsulDemocracy packaged (17th; verified via grind lifecycle)
|
||||
> (#648, Infrastructure, 16th package); auth gate verdict: native OIDC via
|
||||
> python-social-auth (OpenIdConnectAuth + SOCIAL_AUTH_OIDC_*) wired to the
|
||||
> platform provider; official-image wrapper of netboxcommunity/netbox
|
||||
@@ -13,14 +13,14 @@
|
||||
|
||||
## Current State: STABLE (packaging phase, ongoing)
|
||||
|
||||
Cloudron packaging pipeline is operational. 16 of ~57 upstream applications are
|
||||
Cloudron packaging pipeline is operational. 17 of ~57 upstream applications are
|
||||
packaged, committed, and pushed. Packaging templates exist for the core
|
||||
patterns. The gardening protocol (this file + AGENTS.md) keeps docs in sync.
|
||||
All remaining apps now carry build tickets (#633-#678) under umbrella
|
||||
[#632](https://projects.knownelement.com/issues/632) in Redmine project 55 —
|
||||
ready for the sequential grind-driver pattern.
|
||||
|
||||
## Completed Packages (16)
|
||||
## Completed Packages (17)
|
||||
|
||||
| # | Application | Category | Pattern | Port(s) | Addons |
|
||||
|---|-------------|----------|---------|---------|--------|
|
||||
@@ -45,7 +45,7 @@ Each package lives in `Package-Workspace/<Category>/<app>/` and contains a
|
||||
`Dockerfile`, `CloudronManifest.json`, `README.md`, `CHANGELOG.md`, `logo.png`,
|
||||
and (where relevant) `start.sh` + `.env.example`.
|
||||
|
||||
## Build Ticket Map (46 filed 2026-09-01; 39 open — 7 landed: #639, #640, #648, #650, #651, #668, #669)
|
||||
## Build Ticket Map (46 filed 2026-09-01; 38 open — 8 landed: #639, #640, #648, #650, #651, #653, #668, #669)
|
||||
|
||||
All under umbrella [#632](https://projects.knownelement.com/issues/632)
|
||||
(Redmine project 55, tracker Feature). Excluded: **grist-core** (a Cloudron
|
||||
@@ -68,12 +68,12 @@ package already exists upstream — do not package).
|
||||
| #645 | OpenBoxes | Business-Apps | needs LDAP runtime verification (note 2026-09-06) |
|
||||
| #646 | Nautilus Trader | Financial-Trading | disposition (headless lib) |
|
||||
| #647 | Fleet | Monitoring | build (research queue) |
|
||||
| #648 | NetBox | Infrastructure | build (redis addon; strong candidate) |
|
||||
| #648 | NetBox | Infrastructure | DONE 2026-09-06 (16th package) |
|
||||
| #649 | SeaTunnel | Data-Management | disposition (service vs tool) |
|
||||
| #650 | Rathole | Infrastructure | build (proxy) |
|
||||
| #651 | Easy-Gate | Infrastructure | build (proxy) |
|
||||
| #652 | Huginn | Automation | blocked-on-auth (local-only Devise; note 2026-09-06) |
|
||||
| #653 | ConsulDemocracy | Collaboration | build |
|
||||
| #653 | ConsulDemocracy | Collaboration | DONE 2026-09-06 (17th, grind-verified) |
|
||||
| #654 | BOINC | Scientific-Computing | disposition (server packaging heavy) |
|
||||
| #655 | Slurm | System-Administration | disposition (likely out of scope) |
|
||||
| #656 | Gophish | Security | blocked-on-auth |
|
||||
@@ -122,7 +122,7 @@ Full write-ups of each pattern + challenges are in [`JOURNAL.md`](JOURNAL.md).
|
||||
| Monitoring | 6 | 1/6 | healthchecks done |
|
||||
| Automation | 4 | 1/4 | windmill done |
|
||||
| Business-Apps | 8 | 1/8 | elabftw done |
|
||||
| Collaboration | 2 | 0/2 | |
|
||||
| Collaboration | 2 | 1/2 | consuldemocracy done |
|
||||
| Communication | 1 | 0/1 | |
|
||||
| Data-Management | 2 | 0/2 | |
|
||||
| DevOps-Tools | 1 | 1/1 (100%) ✅ | fx done |
|
||||
@@ -142,7 +142,7 @@ Auth capability is a hard gate before packaging (see
|
||||
LDAP acceptable (risk flag), 🔄 = auth-proxy (no users), ❌ = local-only
|
||||
(unacceptable / blocked-on-auth).
|
||||
|
||||
### Completed packages (16)
|
||||
### Completed packages (17)
|
||||
|
||||
| App | OIDC | LDAP | Verdict | Note |
|
||||
|-----|------|------|---------|------|
|
||||
@@ -162,6 +162,7 @@ LDAP acceptable (risk flag), 🔄 = auth-proxy (no users), ❌ = local-only
|
||||
| ChirpStack | yes | no | ✅ preferred | **Packaged**; native `[user_authentication.openid_connect]` wired to `CLOUDRON_OIDC_*`; OIDC-registered users are non-admin — one-time `CHIRPSTACK_AUTH_MODE=internal` bootstrap links the seeded `admin` to your SSO email (README) |
|
||||
| eLabFTW | no | yes | ⚠️ risk (LDAP) | **Packaged**; no OIDC upstream (auth = local/SAML/LDAP per `src/Enums/AuthMethod.php`); `ldap` addon enabled, README maps `CLOUDRON_LDAP_*` → sysconfig LDAP panel — must be validated on the live Cloudron before production; SAML possible via external IdP |
|
||||
| NetBox | yes | plugin | ✅ preferred | **Packaged**; python-social-auth OIDC (`REMOTE_AUTH_BACKEND=OpenIdConnectAuth` + `SOCIAL_AUTH_OIDC_*`) wired from `CLOUDRON_OIDC_*`; local Django login retained for admin bootstrap; SSO users register without privileges (README) |
|
||||
| ConsulDemocracy | yes | no | ✅ preferred | **Packaged**; omniauth_openid_connect (discovery) wired into generated secrets.yml; `feature.oidc_login` auto-enabled on first boot when the provider is present (admin-panel toggle afterwards); SAML also available; local admin seeded |
|
||||
|
||||
### Candidates researched
|
||||
|
||||
|
||||
Reference in New Issue
Block a user