fix(consuldemocracy): verify end-to-end + harden seed/OIDC [#653]
Seed verification over marker trust (observed exit-0 no-op seed), first-boot-only OIDC auto-enable, logo added, docs synced (JOURNAL s17, counts 17/~57). grind-stack verified: homepage 200, OIDC SSO button live. Detail: https://projects.knownelement.com/issues/653#note-5044
This commit is contained in:
+63
@@ -1304,6 +1304,69 @@ terminal); SSO users register WITHOUT privileges.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
### 17. ConsulDemocracy (Collaboration) ✅
|
||||||
|
**Date**: 2026-09-06
|
||||||
|
**Application**: ConsulDemocracy — citizen participation platform behind
|
||||||
|
Madrid's Consul (proposals, participatory budgeting, debates, polls,
|
||||||
|
collaborative legislation). Upstream:
|
||||||
|
https://github.com/consuldemocracy/consuldemocracy (MIT). Rails 8 +
|
||||||
|
PostgreSQL + delayed_job + memcached. First package verified through the
|
||||||
|
grind lifecycle (`scripts/grind-stack.sh`) end to end.
|
||||||
|
|
||||||
|
**Ticket**: [#653](https://projects.knownelement.com/issues/653)
|
||||||
|
|
||||||
|
**Pattern**: build-from-source on ruby:3.4.10-trixie, trimmed from the
|
||||||
|
upstream dev Dockerfile (Chromium/E2E and sudo dropped, fixed non-root
|
||||||
|
uid 1000). No published upstream image exists.
|
||||||
|
|
||||||
|
**Auth gate verdict**: ✅ OIDC preferred. Generic OpenID Connect via
|
||||||
|
`omniauth_openid_connect` (devise.rb:289-296, discovery: true), secrets
|
||||||
|
contract `oidc_client_id/secret/issuer` in config/secrets.yml — wired
|
||||||
|
from `CLOUDRON_OIDC_*` by start.sh. SAML also available.
|
||||||
|
|
||||||
|
**Key decisions / lessons (four build-breaking gotchas, all fixed)**:
|
||||||
|
|
||||||
|
1. **`ruby file: ".ruby-version"`**: the Gemfile resolves its ruby
|
||||||
|
version from `.ruby-version` BY NAME — copying it under a different
|
||||||
|
name kills `bundle install` with a bare backtrace.
|
||||||
|
2. **`eval_gemfile "./Gemfile_custom"`**: the Gemfile expects a
|
||||||
|
developer-local `Gemfile_custom` that isn't in git — `touch` an empty
|
||||||
|
one before `bundle install`.
|
||||||
|
3. **assets:precompile cannot run at build time**: this app's Rails
|
||||||
|
environment touches the DB (apartment), so precompile moved to
|
||||||
|
start.sh after the postgres wait (idempotent; slow only first boot).
|
||||||
|
Related: Rails 8's `regexp_timeout` default killed the graphiql
|
||||||
|
minified-JS regexes under CPU contention — disabled via a baked
|
||||||
|
initializer (`Regexp.timeout = nil`).
|
||||||
|
4. **Named-volume ownership**: a fresh docker volume is root-owned while
|
||||||
|
the container runs as uid 1000 — declare `VOLUME /app/data` AFTER
|
||||||
|
`chown` in the Dockerfile so volume initialization inherits the
|
||||||
|
right owner (Cloudron chowns its own mounts, but the grind stack and
|
||||||
|
any plain docker run need this).
|
||||||
|
5. **Seed verification over marker trust**: one boot observed
|
||||||
|
`db:create db:migrate db:seed` exit 0 while persisting nothing; a
|
||||||
|
standalone re-seed landed fine. start.sh now checks `Setting.count`
|
||||||
|
and re-seeds an empty database regardless of the marker file.
|
||||||
|
6. **`feature.oidc_login` defaults to false** upstream — start.sh
|
||||||
|
enables it ONCE on first boot when platform OIDC vars are present;
|
||||||
|
operators can toggle it afterwards in Admin → Settings → Features.
|
||||||
|
|
||||||
|
**Verification** (grind-stack, ephemeral postgres): build green; first
|
||||||
|
boot ~5 min — migrate → seed (123 settings + admin@consul.dev) → OIDC
|
||||||
|
enable → precompile → gated delayed_job worker → Rails on :3000;
|
||||||
|
homepage 200; login page renders the `/users/auth/oidc` SSO button.
|
||||||
|
|
||||||
|
**Files Created**: Dockerfile, CloudronManifest.json (port 3000,
|
||||||
|
localstorage + postgresql, 2048MB), start.sh (config generation, seed
|
||||||
|
verify, memcached, worker gate), README.md, CHANGELOG.md,
|
||||||
|
.dockerignore (repo/.git only — the source IS the build context),
|
||||||
|
logo.png (upstream public/consul_logo.png).
|
||||||
|
|
||||||
|
**Commit**: `feat(consuldemocracy): add initial Cloudron package` (7c65dc1,
|
||||||
|
landed mid-flight) + verification hardening follow-up
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Packaging Pattern: Download Pre-Compiled Binaries
|
## Packaging Pattern: Download Pre-Compiled Binaries
|
||||||
|
|
||||||
### When to Use
|
### When to Use
|
||||||
|
|||||||
@@ -7,7 +7,17 @@
|
|||||||
- postgresql + localstorage addons; Rails server on :3000
|
- postgresql + localstorage addons; Rails server on :3000
|
||||||
- Generated configs (database.yml / secrets.yml) under /app/data,
|
- Generated configs (database.yml / secrets.yml) under /app/data,
|
||||||
regenerated each boot; persisted secret_key_base
|
regenerated each boot; persisted secret_key_base
|
||||||
- First-boot db:create/migrate/seed with marker; gated delayed_job
|
- First-boot db:create/migrate/seed with marker + empty-DB verification
|
||||||
worker; in-container memcached
|
(a stale marker against a fresh database re-seeds); gated delayed_job
|
||||||
- Platform OIDC wired into secrets.yml; local admin seeded
|
worker; in-container memcached; assets:precompile at start (the app's
|
||||||
|
environment needs the DB, so precompile cannot run at build time)
|
||||||
|
- Platform OIDC wired into secrets.yml; `feature.oidc_login` auto-enabled
|
||||||
|
on first boot when the provider is present; local admin seeded
|
||||||
(admin@consul.dev — password change required on first login)
|
(admin@consul.dev — password change required on first login)
|
||||||
|
|
||||||
|
## Verification (2026-09-06)
|
||||||
|
|
||||||
|
Full-stack grind-stack test against ephemeral postgres: build green,
|
||||||
|
first boot ~5 min (migrate → seed [123 settings, admin] → OIDC enable →
|
||||||
|
precompile → delayed_job → Rails), homepage 200, login page renders the
|
||||||
|
`/users/auth/oidc` SSO button.
|
||||||
|
|||||||
@@ -18,6 +18,9 @@ regenerates secrets.yml on every boot with the Cloudron platform
|
|||||||
provider values (`CLOUDRON_OIDC_*`).
|
provider values (`CLOUDRON_OIDC_*`).
|
||||||
|
|
||||||
- SSO redirect: `https://<app-domain>/users/auth/oidc/callback`
|
- SSO redirect: `https://<app-domain>/users/auth/oidc/callback`
|
||||||
|
- The SSO button (`feature.oidc_login`, default off upstream) is
|
||||||
|
auto-enabled on FIRST boot when the platform provider is present —
|
||||||
|
after that it's yours to toggle in Admin → Settings → Features.
|
||||||
- Local login stays available alongside SSO.
|
- Local login stays available alongside SSO.
|
||||||
- SAML is also supported (saml_* keys in secrets.yml) if an external
|
- SAML is also supported (saml_* keys in secrets.yml) if an external
|
||||||
IdP is preferred.
|
IdP is preferred.
|
||||||
|
|||||||
Binary file not shown.
|
After Width: | Height: | Size: 12 KiB |
@@ -97,11 +97,30 @@ memcached -d -p 11211 -u consul -m 64
|
|||||||
# --- 3. database lifecycle -------------------------------------------------------
|
# --- 3. database lifecycle -------------------------------------------------------
|
||||||
wait_tcp "${DB_HOST}" "${DB_PORT}" "PostgreSQL"
|
wait_tcp "${DB_HOST}" "${DB_PORT}" "PostgreSQL"
|
||||||
|
|
||||||
if [[ ! -f "${MARKER}" ]]; then
|
# seed decision: the marker alone is not trusted - a marker left by a
|
||||||
echo "First boot: creating + migrating + seeding the database ..."
|
# previous database (restore, migration, fresh test DB) must not skip
|
||||||
|
# seeding an empty one (observed: seed chain exits 0 yet persists
|
||||||
|
# nothing when re-run in the same first-boot process; a standalone
|
||||||
|
# re-seed lands fine, so verify + retry instead of debugging ghosts)
|
||||||
|
SEED_COUNT="$(bundle exec rails runner 'print Setting.count' 2>/dev/null || echo 0)"
|
||||||
|
if [[ ! -f "${MARKER}" || "${SEED_COUNT}" = "0" ]]; then
|
||||||
|
echo "First boot (or empty database): creating + migrating + seeding ..."
|
||||||
bundle exec rails db:create db:migrate db:seed
|
bundle exec rails db:create db:migrate db:seed
|
||||||
|
SEED_COUNT="$(bundle exec rails runner 'print Setting.count' 2>/dev/null || echo 0)"
|
||||||
|
if [[ "${SEED_COUNT}" = "0" ]]; then
|
||||||
|
echo "seed pass 1 did not persist - retrying standalone seed"
|
||||||
|
bundle exec rails db:seed
|
||||||
|
fi
|
||||||
touch "${MARKER}"
|
touch "${MARKER}"
|
||||||
echo "Database seeded (local admin: admin@consul.dev - change the password!)"
|
echo "Database seeded (local admin: admin@consul.dev - change the password!)"
|
||||||
|
|
||||||
|
# first-boot only: expose the SSO button when the platform provider is
|
||||||
|
# wired (feature.oidc_login defaults to false; operators can toggle it
|
||||||
|
# later in Admin > Settings > Features without us re-enabling it)
|
||||||
|
if [[ -n "${CLOUDRON_OIDC_CLIENT_ID:-}" ]]; then
|
||||||
|
bundle exec rails runner 'Setting["feature.oidc_login"] = true' || true
|
||||||
|
echo "OIDC login enabled (admin panel can toggle it)"
|
||||||
|
fi
|
||||||
else
|
else
|
||||||
bundle exec rails db:migrate
|
bundle exec rails db:migrate
|
||||||
fi
|
fi
|
||||||
|
|||||||
@@ -9,9 +9,9 @@ The Cloudron component focuses on packaging upstream free/libre/open application
|
|||||||
### 📊 Current Progress
|
### 📊 Current Progress
|
||||||
|
|
||||||
- **Total Applications**: ~57 (see [GitUrlList.txt](GitUrlList.txt))
|
- **Total Applications**: ~57 (see [GitUrlList.txt](GitUrlList.txt))
|
||||||
- **Completed Packages**: 16/~57 (~28%)
|
- **Completed Packages**: 17/~57 (~30%)
|
||||||
- **Packaging Templates**: Created ✅
|
- **Packaging Templates**: Created ✅
|
||||||
- **Packages Committed & Pushed**: 16 ✅
|
- **Packages Committed & Pushed**: 17 ✅
|
||||||
- **Build Tickets**: 46 filed (#633-#678, umbrella
|
- **Build Tickets**: 46 filed (#633-#678, umbrella
|
||||||
[#632](https://projects.knownelement.com/issues/632), Redmine project 55);
|
[#632](https://projects.knownelement.com/issues/632), Redmine project 55);
|
||||||
grist-core excluded (packaged upstream)
|
grist-core excluded (packaged upstream)
|
||||||
@@ -38,6 +38,7 @@ The Cloudron component focuses on packaging upstream free/libre/open application
|
|||||||
| 14 | ChirpStack | Infrastructure | 83.4MB | 8080 | localstorage, postgresql, redis | ✅ Committed |
|
| 14 | ChirpStack | Infrastructure | 83.4MB | 8080 | localstorage, postgresql, redis | ✅ Committed |
|
||||||
| 15 | eLabFTW | Business-Apps | ~209MB (compressed) | 443 | localstorage, mysql, ldap | ✅ Committed |
|
| 15 | eLabFTW | Business-Apps | ~209MB (compressed) | 443 | localstorage, mysql, ldap | ✅ Committed |
|
||||||
| 16 | NetBox | Infrastructure | ~1GB | 8080 | localstorage, postgresql, redis | ✅ Committed |
|
| 16 | NetBox | Infrastructure | ~1GB | 8080 | localstorage, postgresql, redis | ✅ Committed |
|
||||||
|
| 17 | ConsulDemocracy | Collaboration | ~1.6GB | 3000 | localstorage, postgresql | ✅ Committed |
|
||||||
|
|
||||||
### 📦 Packages in Development
|
### 📦 Packages in Development
|
||||||
|
|
||||||
|
|||||||
@@ -3,7 +3,7 @@
|
|||||||
> **Human read-only. Agents maintain this file automatically after each work
|
> **Human read-only. Agents maintain this file automatically after each work
|
||||||
> session.** Do not edit by hand — the next agent run will overwrite it.
|
> session.** Do not edit by hand — the next agent run will overwrite it.
|
||||||
>
|
>
|
||||||
> **Last updated:** 2026-09-06 by ZCode (GLM-5.3) — NetBox packaged
|
> **Last updated:** 2026-09-06 by ZCode (GLM-5.3) — ConsulDemocracy packaged (17th; verified via grind lifecycle)
|
||||||
> (#648, Infrastructure, 16th package); auth gate verdict: native OIDC via
|
> (#648, Infrastructure, 16th package); auth gate verdict: native OIDC via
|
||||||
> python-social-auth (OpenIdConnectAuth + SOCIAL_AUTH_OIDC_*) wired to the
|
> python-social-auth (OpenIdConnectAuth + SOCIAL_AUTH_OIDC_*) wired to the
|
||||||
> platform provider; official-image wrapper of netboxcommunity/netbox
|
> platform provider; official-image wrapper of netboxcommunity/netbox
|
||||||
@@ -13,14 +13,14 @@
|
|||||||
|
|
||||||
## Current State: STABLE (packaging phase, ongoing)
|
## Current State: STABLE (packaging phase, ongoing)
|
||||||
|
|
||||||
Cloudron packaging pipeline is operational. 16 of ~57 upstream applications are
|
Cloudron packaging pipeline is operational. 17 of ~57 upstream applications are
|
||||||
packaged, committed, and pushed. Packaging templates exist for the core
|
packaged, committed, and pushed. Packaging templates exist for the core
|
||||||
patterns. The gardening protocol (this file + AGENTS.md) keeps docs in sync.
|
patterns. The gardening protocol (this file + AGENTS.md) keeps docs in sync.
|
||||||
All remaining apps now carry build tickets (#633-#678) under umbrella
|
All remaining apps now carry build tickets (#633-#678) under umbrella
|
||||||
[#632](https://projects.knownelement.com/issues/632) in Redmine project 55 —
|
[#632](https://projects.knownelement.com/issues/632) in Redmine project 55 —
|
||||||
ready for the sequential grind-driver pattern.
|
ready for the sequential grind-driver pattern.
|
||||||
|
|
||||||
## Completed Packages (16)
|
## Completed Packages (17)
|
||||||
|
|
||||||
| # | Application | Category | Pattern | Port(s) | Addons |
|
| # | Application | Category | Pattern | Port(s) | Addons |
|
||||||
|---|-------------|----------|---------|---------|--------|
|
|---|-------------|----------|---------|---------|--------|
|
||||||
@@ -45,7 +45,7 @@ Each package lives in `Package-Workspace/<Category>/<app>/` and contains a
|
|||||||
`Dockerfile`, `CloudronManifest.json`, `README.md`, `CHANGELOG.md`, `logo.png`,
|
`Dockerfile`, `CloudronManifest.json`, `README.md`, `CHANGELOG.md`, `logo.png`,
|
||||||
and (where relevant) `start.sh` + `.env.example`.
|
and (where relevant) `start.sh` + `.env.example`.
|
||||||
|
|
||||||
## Build Ticket Map (46 filed 2026-09-01; 39 open — 7 landed: #639, #640, #648, #650, #651, #668, #669)
|
## Build Ticket Map (46 filed 2026-09-01; 38 open — 8 landed: #639, #640, #648, #650, #651, #653, #668, #669)
|
||||||
|
|
||||||
All under umbrella [#632](https://projects.knownelement.com/issues/632)
|
All under umbrella [#632](https://projects.knownelement.com/issues/632)
|
||||||
(Redmine project 55, tracker Feature). Excluded: **grist-core** (a Cloudron
|
(Redmine project 55, tracker Feature). Excluded: **grist-core** (a Cloudron
|
||||||
@@ -68,12 +68,12 @@ package already exists upstream — do not package).
|
|||||||
| #645 | OpenBoxes | Business-Apps | needs LDAP runtime verification (note 2026-09-06) |
|
| #645 | OpenBoxes | Business-Apps | needs LDAP runtime verification (note 2026-09-06) |
|
||||||
| #646 | Nautilus Trader | Financial-Trading | disposition (headless lib) |
|
| #646 | Nautilus Trader | Financial-Trading | disposition (headless lib) |
|
||||||
| #647 | Fleet | Monitoring | build (research queue) |
|
| #647 | Fleet | Monitoring | build (research queue) |
|
||||||
| #648 | NetBox | Infrastructure | build (redis addon; strong candidate) |
|
| #648 | NetBox | Infrastructure | DONE 2026-09-06 (16th package) |
|
||||||
| #649 | SeaTunnel | Data-Management | disposition (service vs tool) |
|
| #649 | SeaTunnel | Data-Management | disposition (service vs tool) |
|
||||||
| #650 | Rathole | Infrastructure | build (proxy) |
|
| #650 | Rathole | Infrastructure | build (proxy) |
|
||||||
| #651 | Easy-Gate | Infrastructure | build (proxy) |
|
| #651 | Easy-Gate | Infrastructure | build (proxy) |
|
||||||
| #652 | Huginn | Automation | blocked-on-auth (local-only Devise; note 2026-09-06) |
|
| #652 | Huginn | Automation | blocked-on-auth (local-only Devise; note 2026-09-06) |
|
||||||
| #653 | ConsulDemocracy | Collaboration | build |
|
| #653 | ConsulDemocracy | Collaboration | DONE 2026-09-06 (17th, grind-verified) |
|
||||||
| #654 | BOINC | Scientific-Computing | disposition (server packaging heavy) |
|
| #654 | BOINC | Scientific-Computing | disposition (server packaging heavy) |
|
||||||
| #655 | Slurm | System-Administration | disposition (likely out of scope) |
|
| #655 | Slurm | System-Administration | disposition (likely out of scope) |
|
||||||
| #656 | Gophish | Security | blocked-on-auth |
|
| #656 | Gophish | Security | blocked-on-auth |
|
||||||
@@ -122,7 +122,7 @@ Full write-ups of each pattern + challenges are in [`JOURNAL.md`](JOURNAL.md).
|
|||||||
| Monitoring | 6 | 1/6 | healthchecks done |
|
| Monitoring | 6 | 1/6 | healthchecks done |
|
||||||
| Automation | 4 | 1/4 | windmill done |
|
| Automation | 4 | 1/4 | windmill done |
|
||||||
| Business-Apps | 8 | 1/8 | elabftw done |
|
| Business-Apps | 8 | 1/8 | elabftw done |
|
||||||
| Collaboration | 2 | 0/2 | |
|
| Collaboration | 2 | 1/2 | consuldemocracy done |
|
||||||
| Communication | 1 | 0/1 | |
|
| Communication | 1 | 0/1 | |
|
||||||
| Data-Management | 2 | 0/2 | |
|
| Data-Management | 2 | 0/2 | |
|
||||||
| DevOps-Tools | 1 | 1/1 (100%) ✅ | fx done |
|
| DevOps-Tools | 1 | 1/1 (100%) ✅ | fx done |
|
||||||
@@ -142,7 +142,7 @@ Auth capability is a hard gate before packaging (see
|
|||||||
LDAP acceptable (risk flag), 🔄 = auth-proxy (no users), ❌ = local-only
|
LDAP acceptable (risk flag), 🔄 = auth-proxy (no users), ❌ = local-only
|
||||||
(unacceptable / blocked-on-auth).
|
(unacceptable / blocked-on-auth).
|
||||||
|
|
||||||
### Completed packages (16)
|
### Completed packages (17)
|
||||||
|
|
||||||
| App | OIDC | LDAP | Verdict | Note |
|
| App | OIDC | LDAP | Verdict | Note |
|
||||||
|-----|------|------|---------|------|
|
|-----|------|------|---------|------|
|
||||||
@@ -162,6 +162,7 @@ LDAP acceptable (risk flag), 🔄 = auth-proxy (no users), ❌ = local-only
|
|||||||
| ChirpStack | yes | no | ✅ preferred | **Packaged**; native `[user_authentication.openid_connect]` wired to `CLOUDRON_OIDC_*`; OIDC-registered users are non-admin — one-time `CHIRPSTACK_AUTH_MODE=internal` bootstrap links the seeded `admin` to your SSO email (README) |
|
| ChirpStack | yes | no | ✅ preferred | **Packaged**; native `[user_authentication.openid_connect]` wired to `CLOUDRON_OIDC_*`; OIDC-registered users are non-admin — one-time `CHIRPSTACK_AUTH_MODE=internal` bootstrap links the seeded `admin` to your SSO email (README) |
|
||||||
| eLabFTW | no | yes | ⚠️ risk (LDAP) | **Packaged**; no OIDC upstream (auth = local/SAML/LDAP per `src/Enums/AuthMethod.php`); `ldap` addon enabled, README maps `CLOUDRON_LDAP_*` → sysconfig LDAP panel — must be validated on the live Cloudron before production; SAML possible via external IdP |
|
| eLabFTW | no | yes | ⚠️ risk (LDAP) | **Packaged**; no OIDC upstream (auth = local/SAML/LDAP per `src/Enums/AuthMethod.php`); `ldap` addon enabled, README maps `CLOUDRON_LDAP_*` → sysconfig LDAP panel — must be validated on the live Cloudron before production; SAML possible via external IdP |
|
||||||
| NetBox | yes | plugin | ✅ preferred | **Packaged**; python-social-auth OIDC (`REMOTE_AUTH_BACKEND=OpenIdConnectAuth` + `SOCIAL_AUTH_OIDC_*`) wired from `CLOUDRON_OIDC_*`; local Django login retained for admin bootstrap; SSO users register without privileges (README) |
|
| NetBox | yes | plugin | ✅ preferred | **Packaged**; python-social-auth OIDC (`REMOTE_AUTH_BACKEND=OpenIdConnectAuth` + `SOCIAL_AUTH_OIDC_*`) wired from `CLOUDRON_OIDC_*`; local Django login retained for admin bootstrap; SSO users register without privileges (README) |
|
||||||
|
| ConsulDemocracy | yes | no | ✅ preferred | **Packaged**; omniauth_openid_connect (discovery) wired into generated secrets.yml; `feature.oidc_login` auto-enabled on first boot when the provider is present (admin-panel toggle afterwards); SAML also available; local admin seeded |
|
||||||
|
|
||||||
### Candidates researched
|
### Candidates researched
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user