From cead1523ad16154dc0d540c1998ee7c7472dd2d8 Mon Sep 17 00:00:00 2001 From: reachableceo Date: Sun, 6 Sep 2026 06:26:54 -0500 Subject: [PATCH] feat: add NetBox Cloudron package (Infrastructure) [#648] Official-image wrapper of netboxcommunity/netbox v4.6.10 (netbox-docker 5.0.2, Granian), digest-pinned. OIDC via python-social-auth wired to the platform provider; RQ worker backgrounded behind a web-port gate; addon waits cover the 30s upstream DB timeout. Full-stack verified: migrations, Granian, worker, login page + SSO button. 16/~57. Detail + verification: https://projects.knownelement.com/issues/648#note-4976 --- JOURNAL.md | 95 +++++++++++++++- .../Infrastructure/netbox/.dockerignore | 1 + .../Infrastructure/netbox/.env.example | 11 ++ .../Infrastructure/netbox/CHANGELOG.md | 13 +++ .../netbox/CloudronManifest.json | 23 ++++ .../Infrastructure/netbox/Dockerfile | 60 ++++++++++ .../Infrastructure/netbox/README.md | 67 +++++++++++ .../Infrastructure/netbox/logo.png | Bin 0 -> 6832 bytes .../Infrastructure/netbox/start.sh | 106 ++++++++++++++++++ README.md | 5 +- STATUS.md | 25 +++-- 11 files changed, 392 insertions(+), 14 deletions(-) create mode 100644 Package-Workspace/Infrastructure/netbox/.dockerignore create mode 100644 Package-Workspace/Infrastructure/netbox/.env.example create mode 100644 Package-Workspace/Infrastructure/netbox/CHANGELOG.md create mode 100644 Package-Workspace/Infrastructure/netbox/CloudronManifest.json create mode 100644 Package-Workspace/Infrastructure/netbox/Dockerfile create mode 100644 Package-Workspace/Infrastructure/netbox/README.md create mode 100644 Package-Workspace/Infrastructure/netbox/logo.png create mode 100755 Package-Workspace/Infrastructure/netbox/start.sh diff --git a/JOURNAL.md b/JOURNAL.md index 50617df..15d875e 100644 --- a/JOURNAL.md +++ b/JOURNAL.md @@ -4,7 +4,7 @@ **Project**: TSYSDevStack-SupportStack-Cloudron **Goal**: Package ~57 applications for Cloudron PaaS platform **Start Date**: 2025-01-24 -**Current Status**: 15/~57 packages completed (~26%) +**Current Status**: 16/~57 packages completed (~28%) ## Completed Packages @@ -1211,6 +1211,99 @@ IdP if the directory path disappoints. --- +### 16. NetBox (Infrastructure) ✅ +**Date**: 2026-09-06 +**Application**: NetBox — open-source IPAM/DCIM (IP addresses, prefixes, +VLANs, sites, racks, devices, circuits, virtualization, REST + GraphQL +APIs). Upstream: https://github.com/netbox-community/netbox +(Apache-2.0), v4.6.10. First package in the workspace verified with a +**full-stack local test** (ephemeral PG16 + Redis 7 → migrations → +Granian bind → RQ worker → login page 200 + OIDC button). + +**Ticket**: [#648](https://projects.knownelement.com/issues/648) + +**Pattern**: official-image wrapper. NetBox itself has no Docker +tooling — images come from the separate netbox-docker repo; v4.6.10 is +built from netbox-docker **5.0.2** (beware stale docs claiming 3.x: +3.4.2 was the last nginx-unit release; 4.0+ serves via **Granian** on +:8080). The image's configuration is env-driven +(`DB_*`, `REDIS[_CACHE]_*`, `SECRET_KEY`, `ALLOWED_HOSTS`, +`CSRF_TRUSTED_ORIGINS`, `REMOTE_AUTH_BACKEND`, `SOCIAL_AUTH_OIDC_*`), +so no config-file generation is needed — start.sh only maps Cloudron +env onto it. + +**Auth gate verdict**: ✅ OIDC preferred. NetBox 4.6 uses +python-social-auth directly; there is NO `SOCIAL_AUTH_TYPE` (legacy +netbox-docker 1.x/2.x mechanism). Minimum viable OIDC: +`REMOTE_AUTH_BACKEND='social_core.backends.open_id_connect.OpenIdConnectAuth'` ++ `SOCIAL_AUTH_OIDC_{OIDC_ENDPOINT,KEY,SECRET}` — wired from +`CLOUDRON_OIDC_*`, with the issuer normalized to a trailing slash +(social-core discovery requirement). Local Django login stays available +for admin bootstrap (`manage.py createsuperuser` via the Cloudron +terminal); SSO users register WITHOUT privileges. + +**Key decisions**: + +- **One-container problem**: upstream compose runs the RQ worker as a + separate service; Cloudron is one container per app. start.sh + backgrounds `manage.py rqworker` behind a gate that waits for + 127.0.0.1:8080 to answer — Granian binding is the signal that + docker-entrypoint.sh finished migrating, so the worker never races + the schema. When Granian exits, the container dies and takes the + worker with it. +- **Addon waits BEFORE the entrypoint**: netbox-docker's own DB wait is + only `DB_WAIT_TIMEOUT=30s` — too tight against a cold Cloudron + postgres addon. start.sh does the repo-standard bash `/dev/tcp` wait + for PostgreSQL AND Redis first (same pattern as ChirpStack/eLabFTW). +- **Persistence via build-time symlinks**: the container runs as + unprivileged `netbox` (uid 999, gid 0), so `/opt/netbox/netbox/ + {media,reports,scripts}` are replaced by symlinks into `/app/data` + at BUILD time (root), not in start.sh. +- **One Redis instance, two logical DBs**: tasks=0, caching=1 + (upstream convention; Cloudron's single redis addon suffices). +- **SKIP_SUPERUSER=true**: no baked-in admin password; README covers + `createsuperuser` via the terminal (mirrors the ChirpStack bootstrap + story). `SECRET_KEY` generated as 64 hex chars (NetBox enforces + >= 50), persisted under /app/data. +- **PG floor**: 4.6 requires PostgreSQL 14+ (15+ from 4.7) — the + Cloudron postgresql addon satisfies it. + +**Verification** (the deepest in the workspace so far): + +- `docker build --cgroup-parent ukrrs-batch.slice` green (image + pulled by digest, ~1GB). +- No-addons smoke: secret generated, entrypoint reached its DB wait — + i.e. Django configuration parsed cleanly. +- Full-stack test on a scratch docker network: ephemeral postgres:16 + + redis:7 (requirepass), mapped through the same `CLOUDRON_*` env + the platform injects. Results: all ~230 migrations applied; config + banner confirmed every `/etc/netbox/config/*.py` loaded; Granian + `[INFO] Listening at: http://:::8080`; worker gate fired ("web port + is up" → RQ `Listening on high, default, low`); `GET /login/` = 200 + with the SSO button linking `/oauth/login/oidc/` — the OIDC backend + is registered. (A `Host: 127.0.0.1` curl returns 400 — ALLOWED_HOSTS + working as designed; the Cloudron proxy sends the real domain.) +- First boot took ~24 min under a load-19 contended dev box (migrate + ~15 min + reindex); minutes on an idle host — noted in the README. + +**Files Created**: + +- Dockerfile (official-image wrapper, tag+amd64-manifest-digest + pinned, build-time persistence symlinks, tini ENTRYPOINT preserved) +- CloudronManifest.json (manifestVersion 2, port 8080, healthCheckPath + /login/, localstorage + postgresql + redis, memoryLimit 2048) +- start.sh (secret persistence, env mapping, OIDC wiring, addon waits, + gated RQ worker, entrypoint handoff) — committed executable +- README.md (auth story, admin bootstrap, runtime layout, knobs) +- CHANGELOG.md +- .env.example (TIME_ZONE / GRANIAN_WORKERS / METRICS_ENABLED) +- .dockerignore (excludes the cloned repo/ from the build context) +- logo.png (from upstream netbox/project-static/img/netbox_touch-icon-180.png) + +**Commit**: `feat: add NetBox Cloudron package (Infrastructure) [#648]` + +--- + ## Packaging Pattern: Download Pre-Compiled Binaries ### When to Use diff --git a/Package-Workspace/Infrastructure/netbox/.dockerignore b/Package-Workspace/Infrastructure/netbox/.dockerignore new file mode 100644 index 0000000..f606d5e --- /dev/null +++ b/Package-Workspace/Infrastructure/netbox/.dockerignore @@ -0,0 +1 @@ +repo diff --git a/Package-Workspace/Infrastructure/netbox/.env.example b/Package-Workspace/Infrastructure/netbox/.env.example new file mode 100644 index 0000000..90b43f8 --- /dev/null +++ b/Package-Workspace/Infrastructure/netbox/.env.example @@ -0,0 +1,11 @@ +# NetBox operator knobs (all optional; defaults shown) +# Copy values into the Cloudron app's custom env config as needed. + +# Django TIME_ZONE (IANA name) +NETBOX_TIME_ZONE=UTC + +# Granian web workers (default 4; lower on small installs) +NETBOX_GRANIAN_WORKERS=4 + +# Expose Prometheus metrics at /metrics +NETBOX_METRICS_ENABLED=false diff --git a/Package-Workspace/Infrastructure/netbox/CHANGELOG.md b/Package-Workspace/Infrastructure/netbox/CHANGELOG.md new file mode 100644 index 0000000..a9a44d1 --- /dev/null +++ b/Package-Workspace/Infrastructure/netbox/CHANGELOG.md @@ -0,0 +1,13 @@ +# Changelog — NetBox Cloudron Package + +## 1.0.0 — 2026-09-06 + +- Initial package: NetBox 4.6.10 (netboxcommunity/netbox v4.6.10, from + netbox-docker 5.0.2), digest-pinned amd64 manifest + `sha256:5ee780bd…` +- Official-image wrapper; Granian web on :8080, RQ worker backgrounded by + start.sh after migrations complete +- postgresql + redis + localstorage addons; tasks DB 0 / caching DB 1 +- Platform OIDC wired via `REMOTE_AUTH_BACKEND` (OpenIdConnectAuth) + + `SOCIAL_AUTH_OIDC_*`; local Django login retained for admin bootstrap +- Persistence: media / reports / scripts / .secret_key under /app/data diff --git a/Package-Workspace/Infrastructure/netbox/CloudronManifest.json b/Package-Workspace/Infrastructure/netbox/CloudronManifest.json new file mode 100644 index 0000000..ad0cd40 --- /dev/null +++ b/Package-Workspace/Infrastructure/netbox/CloudronManifest.json @@ -0,0 +1,23 @@ +{ + "manifestVersion": 2, + "type": "app", + "id": "io.cloudron.netbox", + "title": "NetBox", + "description": "Open-source IPAM and DCIM web application: IP address and prefix management, VLANs, circuits, racks, devices, virtualization, tenancy, plus REST and GraphQL APIs. PostgreSQL stores all state; Redis backs caching and background jobs (webhooks, scripts, reports, scheduled tasks). SSO via the Cloudron OIDC provider; local Django login remains available for admin bootstrap.", + "author": "NetBox Community / NetBox-Docker contributors", + "website": "https://netbox.readthedocs.io/", + "contactEmail": "cloudron@tsys.dev", + "tagline": "IP address management (IPAM) and data center infrastructure management (DCIM)", + "version": "4.6.10", + "healthCheckPath": "/login/", + "httpPort": 8080, + "memoryLimit": 2048, + "addons": { + "localstorage": {}, + "postgresql": {}, + "redis": {} + }, + "mediaLinks": [], + "changelog": "Initial Cloudron package for NetBox 4.6.10 (official-image wrapper of netboxcommunity/netbox v4.6.10, built from netbox-docker 5.0.2, digest pinned). start.sh maps the Cloudron postgresql/redis addons and the platform OIDC provider onto the image's env-driven configuration, persists the Django SECRET_KEY under /app/data, rebinds media/reports/scripts into localstorage, and backgrounds the RQ worker (gated on the web port so migrations finish first). First admin is created via the Cloudron terminal (manage.py createsuperuser); SSO users register without privileges.", + "icon": "file://logo.png" +} diff --git a/Package-Workspace/Infrastructure/netbox/Dockerfile b/Package-Workspace/Infrastructure/netbox/Dockerfile new file mode 100644 index 0000000..15b3946 --- /dev/null +++ b/Package-Workspace/Infrastructure/netbox/Dockerfile @@ -0,0 +1,60 @@ +# NetBox Cloudron Package +# +# NetBox is an open-source IPAM / DCIM web application (Django): circuits, +# devices, racks, IP address management, prefixes, VLANs, virtualization, +# with a REST + GraphQL API. PostgreSQL for storage, Redis for caching and +# background tasks (RQ webhooks/scripts/jobs). +# +# Upstream: https://github.com/netbox-community/netbox (Apache-2.0), v4.6.10 +# - Built/published by the separate netbox-docker repo; v4.6.10 images +# come from netbox-docker 5.0.2 (tag alias v4.6.10-5.0.2, pushed +# 2026-09-02). 5.x serves HTTP via Granian on port 8080 (nginx-unit is +# gone since 4.0), runs as user netbox (uid 999, gid 0), and the +# default container is WEB-ONLY: upstream compose runs the RQ worker +# as a separate service. Cloudron is one-container-per-app, so +# start.sh backgrounds `manage.py rqworker` (gated on the web port so +# it only starts once the entrypoint's migrations have finished). +# - The image configuration is env-driven (netbox-docker +# configuration/configuration.py): DB_*, REDIS[_CACHE]_*, SECRET_KEY, +# ALLOWED_HOSTS, CSRF_TRUSTED_ORIGINS, REMOTE_AUTH_BACKEND + +# SOCIAL_AUTH_OIDC_* all map straight onto Cloudron addon/platform env. +# - docker-entrypoint.sh waits for the DB, migrates, and creates the +# superuser unless SKIP_SUPERUSER=true (our default; the README covers +# creating the admin via the Cloudron terminal). +# +# Authentication: NATIVE OIDC (preferred). NetBox 4.6 uses python-social- +# auth directly: REMOTE_AUTH_BACKEND=OpenIdConnectAuth + the +# SOCIAL_AUTH_OIDC_{OIDC_ENDPOINT,KEY,SECRET} trio, wired in start.sh from +# the Cloudron platform provider (CLOUDRON_OIDC_*). The local Django login +# form remains available alongside SSO for admin bootstrap; SSO-registered +# users arrive with no privileges (README). +# +# Pattern: official-image wrapper. Image pinned by tag AND digest (amd64 +# manifest digest of the v4.6.10 tag, from docker manifest inspect — the +# Hub API's index digest is NOT usable as tag@digest, see JOURNAL #668). +FROM netboxcommunity/netbox:v4.6.10@sha256:5ee780bdc63b5a5f7ad5073e05e265f74d1a1c52ca0057facc9e4279fcfdb8cc + +# Persistence rebinds must happen at BUILD time: the container runs as the +# unprivileged netbox user (999) and could not replace these directories +# later. Symlink targets are created by start.sh under /app/data. +USER root +RUN rm -rf /opt/netbox/netbox/media /opt/netbox/netbox/reports /opt/netbox/netbox/scripts \ + && ln -s /app/data/media /opt/netbox/netbox/media \ + && ln -s /app/data/reports /opt/netbox/netbox/reports \ + && ln -s /app/data/scripts /opt/netbox/netbox/scripts + +# start.sh (runs as the netbox user): persists the SECRET_KEY, maps the +# Cloudron addon/platform env onto the image's env-driven configuration, +# backgrounds the RQ worker, then execs the upstream entrypoint chain +# (docker-entrypoint.sh launch-netbox.sh) under tini. +# Made executable on the host, not at build time (Cloudron builds hit +# permission errors on RUN chmod). +COPY start.sh /app/start.sh + +# Granian binds :: (dual-stack) on 8080; Cloudron proxies TLS here. +EXPOSE 8080 + +USER 999:0 + +# Keep upstream's tini as PID 1; start.sh execs the upstream CMD chain. +ENTRYPOINT ["/usr/bin/tini", "--", "/bin/bash", "/app/start.sh"] diff --git a/Package-Workspace/Infrastructure/netbox/README.md b/Package-Workspace/Infrastructure/netbox/README.md new file mode 100644 index 0000000..b136967 --- /dev/null +++ b/Package-Workspace/Infrastructure/netbox/README.md @@ -0,0 +1,67 @@ +# NetBox — Cloudron Package + +[NetBox](https://github.com/netbox-community/netbox) v4.6.10 — open-source +IPAM/DCIM: IP addresses, prefixes, VLANs, sites, racks, devices, circuits, +virtualization, tenancy, REST + GraphQL APIs. Packaged as an +**official-image wrapper** around `netboxcommunity/netbox:v4.6.10` +(built from netbox-docker 5.0.2, served by Granian on :8080), digest-pinned. + +## Authentication (auth gate: ✅ OIDC preferred) + +NetBox 4.6 authenticates via **python-social-auth** with no legacy +`SOCIAL_AUTH_TYPE` involved: + +- `REMOTE_AUTH_BACKEND = social_core.backends.open_id_connect.OpenIdConnectAuth` +- `SOCIAL_AUTH_OIDC_OIDC_ENDPOINT / KEY / SECRET` ← wired by `start.sh` + from the Cloudron platform OIDC provider (`CLOUDRON_OIDC_*`, + manifestVersion 2), scopes `openid profile email`. + +Two consequences that matter on first boot: + +1. **SSO-registered users are created without privileges.** An admin must + exist first — create it from the Cloudron **terminal**: + + ```sh + cd /opt/netbox/netbox + python3 manage.py createsuperuser + ``` + + Then log in on `/login/` with the local form (it stays available + alongside the SSO button) and grant groups/permissions as needed. + +2. The platform OIDC client must register the standard redirect: + `https:///complete/oidc/` (the Cloudron provider handles + this when the app is installed). + +## Runtime layout + +| Concern | How | +|---------|-----| +| Database | Cloudron `postgresql` addon (PG 16 ≥ NetBox 4.6's PG 14 floor; PG 15+ required from 4.7) | +| Cache + tasks | Cloudron `redis` addon — one instance, two logical DBs (tasks=0, caching=1, upstream convention) | +| Migrations | Run automatically by the upstream `docker-entrypoint.sh` (waits for PostgreSQL first) | +| RQ worker | Backgrounded by `start.sh`, started only after the web port answers (i.e. post-migration) — webhooks, scripts, reports, scheduled/housekeeping jobs | +| Persistence | `/app/data` (localstorage): `media/` (attachments), `reports/`, `scripts/` (custom code), `.secret_key` | +| SECRET_KEY | Generated once (64 hex chars), persisted; rotating it invalidates sessions | +| First boot | Allow up to ~10-15 min — `migrate` + `reindex --lazy` run before Granian binds :8080 (observed: ~24 min on a 3x-oversubscribed dev box, minutes on an idle host; subsequent boots are fast) | + +## Operator knobs (`.env.example`) + +| Variable | Default | Purpose | +|----------|---------|---------| +| `NETBOX_TIME_ZONE` | `UTC` | Django `TIME_ZONE` | +| `NETBOX_GRANIAN_WORKERS` | `4` | Web workers (lower on small installs) | +| `NETBOX_METRICS_ENABLED` | `false` | Prometheus endpoint at `/metrics` | + +## Upgrades + +The image is pinned by tag **and** digest in the `Dockerfile`. To upgrade, +bump both (verify the new amd64 digest with +`docker manifest inspect netboxcommunity/netbox:`) and rebuild — +migrations run automatically on first boot. + +## Package pattern + +Official-image wrapper + `start.sh` env mapping — see the repo +`JOURNAL.md` NetBox section for the full write-up (worker-gating trick, +build-time persistence symlinks, OIDC wiring). diff --git a/Package-Workspace/Infrastructure/netbox/logo.png b/Package-Workspace/Infrastructure/netbox/logo.png new file mode 100644 index 0000000000000000000000000000000000000000..a3a1f266e7b1dff3b1292a16d0baf3946cf4f8cc GIT binary patch literal 6832 zcmeI1=RaIe*#Fn=B05X3N{Hws%8D)_dV&zWcdJ`9O4MjU2+`|`UZQS9FNqq96}_(L zU9_m--tUv&@6r7q+z+mE&77IJ&TGz`nb-CHocCIqDrCeA!~g(*OifksHO`~{vwH-% ztL!j=2+rYoy;gY+s2*Y3##I#U4Aty4GyuG~@;v}vv?BogUl*J);0yo&A@KkpoX7j` z9uoNfT2V;Q|1ba7QGYVcA2*0WO;KLg4{zU$Foi~M>eezwp-&zU^p0KLGnt*usGb;- z!V&XlRWO#XI_4lm%aV*_v#Pq<3SUVsoDIp1TtSwxQuwHK_pgPS$-bhc zwdFrsk}Y-In0$HOQDd?A1{-v~4nz*BBqht4s#H_|PK7SdM42{dvO@0%^P7dRx0lCecKg4IUmCXh3QT=5i9Xw%UR~~f zXOLfhr)UVYa|iII6FM1yoOJOZklVek^`Wks>@XjbP=Ru}IbmU=j0U5%Od!(qx~00s~$HO9zWW4!iFK-ysO8C z&ydcH%W3s;1&XT#BRTlK`=F<0fVRI+5o)0i*qUiE&0d?z>pxS4{f})$U1o$7eeckX zLD#;gyVDev`d@%>gc7+LYa%zEk2~SgEumIj&ZTnWKJY=%1H(&aRXa&+A~_G7)b3}t zcnw9oPKmmFD(~?e@w@8Q11ay_DYEqSC*K0jobY#qZ0TPo-7bcS_XEUZE2zlL41f-c zq!NL6h7H0FWYu8rY%$ljFt7KJ?^XCO)YTQ^Xax{kn1?qvC!@4Zua$f_5bn~;%$b=a zw3r|!Pirb|gE=fdlO!$8L^a5}Q2B$9O&_TtzE_D5wXOEACJ!4df4kS_2-^0{RSDRv z_{o(EaDxD{Rus7Eu8LXq8ZTRBS*2G0Dk$fCd%2!&wT+H~T&#BA{<|J4Qls9R|0Zat zee!O31VVxreQ_bIdt_6Il6ZYV8t zO`>ft$%(6Nsn@T393JX@sTi&qr;so%^O1$zzkReMo zvNxJkm}C#_M{*pWSMWNM6TNL_S4rL{>dXcH-MT~ILZeWk{H9G9-&xx)h|A-3RSk{d ztfE_YeIU=X%U4+$0&vb}xp(6G7=hmmat-G?rJB9CPum+>mo+f!ztrY05vXq=-eBCx z^>dt}#6vc8$yf8&MDyp${3+LNy-@R7%P^bZq0`3IZMtF0vnoE>RKG7^hWBK5U+TKf zrp>b5HZ6wqLE~TyK~H{i2TPuOm;;wVjX5(950C1L`caEe&hAFn`KtsQrR--0jiZ^( zB^SIQ$7v0gPZ9K6LRuq?2Ty1|RoXjv)>-`^kh%Hu1F7jsSX^3a(-T8QjMk5s{hWl( zs*xnI?~u%AWSNLvX@N@V9GJEG_Es4+R1+C|yyTxajH460(u{wVBOyG&#%tkA#2&Xf z|BVIO;$=VjU_+SGkaU6$n3*6MgJI6_E%hT9xypEnhOj-FUep1@3Y+~Cc(O4}FB^aw1^Zo{ zI)1$;nxgxWkdy}Y&qs3DOiHznKhe>x5I>?JK--5(8!p+}P771H_{nbei z>;GeIZ4F(-gq=t)ui!VTXYYrlQho-%EopAK1=SX-8^Xj~2_N{=&Tzwl+gG#oJcvLJ zWNi+m^&a_EXwKkArDT~`RN{oju$nWD38ROjO=-!1CE;0FnaL;9{EheC+D;$_zn$_ljp%BXfo&bfvcTpS2#@7|Bgx<1dCQKXEFzr z89g|3xz~0cD=Fk<&za%qp1@WPm5TkI*j(Z`yzE0o{R80DA2uTWH=0%Gi9$4MQPWSh zj7g60%ns-YS7vtRn}hq~3uPGq*d=jDREPOZ5}|jxUhTaCC;Si@f(OY9Z3=BQ!N85T z_%3%ga8-kFUST#!Sqkat4se0PsmJyHza?q!E2DP23wiP^%{$T=!Qejhd4Ao-%CG&I zRSH+2h|<*?d}es)FAp1P;; zjn@xt?_l3XQk|L#P|$XhhX}95SyWFI+}fA}-^BL+o}mKT9!0JC`Urit(Sq^VL~Ugs zg{m92+MTfXaW|1`M480FCsaJy3fz2RRN5tJkJnRK!Y%-4f5I(+nx{%5s-Yg4fj)nC zyL@_rnTRufY!m7I%&Q}^ucxNRpj2L~%9z{qJyd$7MY?ug(e|0(6cbqStLg<3&Sm-i zmTDS-JMrU zkb4ZT&=en>2?A`G(6(55jOd_;{3ckKUadb(b7Jp{6p?jB#QM6%4vh!i%!Yht=4iQ? zOO=7eI=8kuO$b$rP6})1qmf^2MGGahtaP?o-#AUXxB2?WSD{TY@t$`7R6=rA1d`8a zKaM|GoRjB4j^xONU>StGmqPRc(IGtveLs^oQP# zil<61fj+jM`Z$Ux9E>Gsc!+FYAUhQL!q8W)H&cYD4`gK(D{{cz1siL<&;(GM8`q#N2}DAo%gliU?Y~_1V7w0SumATkrsu#&=3gn5cN_O>(U=|eL8O4 zLX9Qi+D<`PQLk&N%Kd&G*~4CW$7lzOAIVQjqS8mb&TesZf3m=1>(T^}F>(7SyvZ42 z-JGSWhoo?jazC6qd_(@Gf~&PSSSlu(>f@`Lm*3dAY;G=&xOGb6muxH;x^KSG-M;_o zJVn3ZF$tQBvAD{{%1)Oj2pVyYJzFdc-vM1Ux%eCx5xtU@P_nt(f5u~_^ACSq5V=$g zX6G7+x0Bm$idZpO+i$S`B2P}oIcm;ra+#Xzp_v@|C+qfk@z$38rRqh^(drO~qm-z;B znJZR)IYe1aVs7ZK{)lrz{#)UVfB6kCS5k+KBJqWc>2=m34JjIC(_1Nm3-g?7*I&5} zu(iQX|AEx8gMEfl>e8EC#+HPZcpxYXnJq>^e;R+*M8Y*5mQ*tk>)$w}xR!hn%)TnR zlYgsvq2Ge$zLLSTIh3awf7TZhLe2^(>*otv-3z#_$vM716ftX%y@1&{Dc={lmvQVk zLI2@8YI{S{krs0xc>BmLzTGNHLep6s@2fPxi|(?2QT>vBu+n`Uv&%#mBM^Wm8}a+` zm#~sE(EPJwX+v=crt%{`^R2cXk;6i>@=nPxU5R(k`-M2@dy&vd!NJuULM zL06Lo7GFs>&>q+S&apwO>;ZnY9tb+*VcUtOiT#GUyj}3gSNdf0osu)<6u@w1Gki2} zt~ZVj*6TcUj+@h0+}sG5xD1RWf2Q!fBckREog!}CKAW~sDSnZs0CQW$H7za{26hcs^wP;N7P(RsO%eS}eJ(#Xx z*_kL`RUNI=tBkVx5e1!cJ$yA#3k`fQ@SW9V-nRAuV(M_IT^g5kZ#2Xa3>`*oI({?g z63OHc#NTo$c>dmDu2CR%N`jWvU@MVZ#pqs?Ict25Y`|$(;9+|-3WZ`SzC0+=gq(Pf zYlvHo=E)}DWdSLbMhWeM`$0j4xeED1juP+qqV`gTQVG4Ne#zQ>Sr4yX#TIbxiPV40x|bSTF+2&ObdGkrGXrh4`0Vb8AR+BMVb3Fpw?z&Zy}x5UpRagz&4cyCemFi$4zd^Yd@)s}wbFh(+PZnwigm>^Fdu=QP2LJ>I~o!&+iPGkKwl5Lqhy|uIpQ0e5B(| z=YHe7rETEEI@*>k5jdP74DvkgwH>4%3ARxi*yXe&?0rNo{bnE7ctDylivF>gAM&8V zei)c&I;*kJ8?>4GhV|e_i{AWs-q@F=0JiQ0{kLQi(pba~vMD~%GAkU|W=MLKbnB$L zC<84*{ypiq++c($Js-Kg+|0L`LperW+C0F#^$GgZO^Dnr)A8OwL%sKAs8-x}b*b|z z(mQBz>jIo+_x<(Q{}U<8Q>OIC&{mUPex!d@JC1EH{?5B=w|F~BuX}2k^(V6Q<0#zY z&vLi(uSu3E_FA0Mak?}4a{AHZoUCTbw=3_tLvewG79ukTRTXq5X z4Qp9H#!oZ|aR|Xjn6y+jCDM5Hc+tkoC-r;3ib=|7qn@Ak#%3m>(H%Bo(G)C=>oW=F z9M-)U#s2ZNh!^NT`vdff+xPT+&28mt2DA04U#2c0=u3_k%C=VR`+0o3)a3`tXi)DvNc#Y*XmS&i!&DWt%;t!69z-Ft3_Bc?PK3`p zEEzS}e>jBM{iKJ3pxV{VEyb0o%2 z(F-U?9aTz5QYsqy~{1VQ25 zXmTXjq$lWnk?>%X?=@uLaab833kF*iLv&(EGqyiI543f1a`JsTfqemdN~EZ?Zuxc@ zCmv)0Veqw%pleb8)bQVy7gH{7Zbd!KJ( zzPQi?Tdq^eDx6*;Hkg_DcO~)J?M%uD8lScOYty?TRW(*!q6mD4yu(ksOm7czzb$yi zxyNPBkAOvgkO{>+V6JQa&N0Ab2I_`J5E3 z1xdaFMGe6{pSUb@SJKz6QGVs|F7*);Tu@M|U@NH~)$2iPhP(j9A4~c1f+Rl!k}!=l zWS{^C?w#98$%}Dz?r_L{r{_G)4(@Cgb#Hajmdb98T-1HXKD8KTKFe%iJZv`h) zS^Z-z)p=)2{wJe9J1>|hV{7Cj6>r$r|74^0<3k0Zpet{cZuQ(unAcRVEg-2=Kq zB~z66M5w{s{E={g1g5gVoKHBP77=AX19oZ7$3?%ajmo)QK9x)jyv`7(Z6ZNV!Ls}rTPQW;(eYqdFK z+y`?v>W4RbGm_>=Kv{KMWJPo(AB5a^sC1*pTic3M!e)W2J$;GME%H^=w7f)GC?_SP z*$Dt@eVFtZ2^9-f$;@G|5T3~WPxZ5wN9yp^J=!HNjEueG&U(gzX@7SXz-IWan%yrX z(vAEAZPqeiN4!DST6#>V(tLO0A1;GFS^aviSH?3lwV zY1FcY%ElBiKJ< zmHK2GAXk`1r#gO7M3?tcl@0?oW)E)?;_YN3fm|RGB<^B& zezR+Tz3h?w9p!_kVW3$GYQ{hZ%biJbn6McY*D(u|-;NF7t)ho~%X|4MJ2|fCfc=(G zd)>gkEU(x#Fv)*eiTOm~Ksb10(A6w|$c$5SP;m+I9EPzLMLDx zaoBPF_;{huBu-bnB!qxT)hi0tlJNL;EVG5VVv-V*qvQ`~nWT_@um7RWra$~B{QfK} zUe*M`PqPT!XM08eqQkf>z!EQ8WBl~tDD4pc0F&yd+ToJ~+7E#T&4k#HyLRnQys}J0 zh^VD@76&YFAe$?Pp<2Mp^d|q11?ZyLF1Cmtf)5tGeicW^d<1or#p#JIX@3tE2b_6(h>BtFXYL>Vl0ICqOBU8b~|ltFd<3F zSy3)rhz`moQjoHqauNu#T)yw)dCX-8)hZ-L*f@H9K$t%Lgun6VQCR*Xl$A|&^uzU% z@hF~%qZA^zakKnSIwG~v!*|H_TrTN12R*Ha9lO%DjOB~{ADa@+)h~ORt3oJ{Z|IP1 z;AFvv)m;I%lHjAURk@u9bwD&EE@~!QaERhZuPp;+Ew$qp*gKN=KBmpz%VLdPia}eY zWJzOp0C%9)daNpc{mQI?pd3j3NmAxcz!1y(y1t2JW`SRlJMO8maGABs{CZ1oev;u2 zAWGYg*G!Fd4l=;JPhR#n6}*=c3Pk1p()Cd?!|{?f3vd!onga=NEfv)>(@yWY1N9Q+ WA#|1gSpE06NKHvovHH1X`2PSj&J~vc literal 0 HcmV?d00001 diff --git a/Package-Workspace/Infrastructure/netbox/start.sh b/Package-Workspace/Infrastructure/netbox/start.sh new file mode 100755 index 0000000..4b0fda8 --- /dev/null +++ b/Package-Workspace/Infrastructure/netbox/start.sh @@ -0,0 +1,106 @@ +#!/bin/bash +set -euo pipefail + +# NetBox runtime setup for Cloudron: +# 1. create the persistent data dirs (media / reports / scripts — the +# image paths are build-time symlinks into /app/data) +# 2. persist the Django SECRET_KEY (>= 50 chars; rotating it invalidates +# sessions and encrypted values) +# 3. map the Cloudron addon + platform env onto the image's env-driven +# configuration (netbox-docker configuration/configuration.py): +# DB_* from postgresql, REDIS[_CACHE]_* from redis, OIDC from the +# platform OIDC provider +# 4. background the RQ worker (gated on the web port so it only starts +# after the entrypoint's migrations), then exec the upstream +# entrypoint chain, which waits for the DB, migrates, and hands over +# to Granian on :8080 +# +# Authentication: SSO via the Cloudron platform OIDC provider +# (REMOTE_AUTH_BACKEND = OpenIdConnectAuth + SOCIAL_AUTH_OIDC_*). The +# local Django login stays available on /login/ for admin bootstrap; +# SSO-registered users are created WITHOUT privileges (see README). + +DATA_DIR="/app/data" +SECRET_FILE="${DATA_DIR}/.secret_key" + +# --- 1. persistent data directories ------------------------------------------- +mkdir -p "${DATA_DIR}/media" "${DATA_DIR}/reports" "${DATA_DIR}/scripts" + +# --- 2. persistent SECRET_KEY -------------------------------------------------- +# hex on purpose: 64 chars, comfortably over NetBox's 50-char minimum +if [[ ! -s "${SECRET_FILE}" ]]; then + ( umask 077; head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n' > "${SECRET_FILE}" ) + echo "Generated new NetBox secret key" +fi +export SECRET_KEY="$(cat "${SECRET_FILE}")" + +# --- 3. Cloudron -> netbox-docker environment ----------------------------------- +export DB_HOST="${CLOUDRON_POSTGRESQL_HOST:-127.0.0.1}" +export DB_PORT="${CLOUDRON_POSTGRESQL_PORT:-5432}" +export DB_NAME="${CLOUDRON_POSTGRESQL_DATABASE:-netbox}" +export DB_USER="${CLOUDRON_POSTGRESQL_USERNAME:-netbox}" +export DB_PASSWORD="${CLOUDRON_POSTGRESQL_PASSWORD:-}" + +# one Cloudron redis instance, two logical databases (upstream convention: +# tasks on 0, caching on 1) +export REDIS_HOST="${CLOUDRON_REDIS_HOST:-127.0.0.1}" +export REDIS_PORT="${CLOUDRON_REDIS_PORT:-6379}" +export REDIS_PASSWORD="${CLOUDRON_REDIS_PASSWORD:-}" +export REDIS_DATABASE=0 +export REDIS_CACHE_HOST="${CLOUDRON_REDIS_HOST:-127.0.0.1}" +export REDIS_CACHE_PORT="${CLOUDRON_REDIS_PORT:-6379}" +export REDIS_CACHE_PASSWORD="${CLOUDRON_REDIS_PASSWORD:-}" +export REDIS_CACHE_DATABASE=1 + +# hosts / CSRF: Cloudron terminates TLS at the platform proxy +export ALLOWED_HOSTS="${CLOUDRON_APP_DOMAIN:-localhost} localhost" +export CSRF_TRUSTED_ORIGINS="${CLOUDRON_APP_ORIGIN:-http://localhost}" + +# first admin is created by the operator via the Cloudron terminal +# (README): the entrypoint's own superuser seeding is skipped +export SKIP_SUPERUSER=true + +# operator knobs (see .env.example) +export TIME_ZONE="${NETBOX_TIME_ZONE:-UTC}" +export GRANIAN_WORKERS="${NETBOX_GRANIAN_WORKERS:-4}" +export METRICS_ENABLED="${NETBOX_METRICS_ENABLED:-false}" + +# --- 3b. platform OIDC provider -> python-social-auth --------------------------- +# social-core needs the issuer base URL WITH a trailing slash for +# .well-known/openid-configuration discovery +OIDC_ISSUER="${CLOUDRON_OIDC_ISSUER:-}" +OIDC_ISSUER="${OIDC_ISSUER%/}/" +export REMOTE_AUTH_BACKEND='social_core.backends.open_id_connect.OpenIdConnectAuth' +export SOCIAL_AUTH_OIDC_OIDC_ENDPOINT="${OIDC_ISSUER}" +export SOCIAL_AUTH_OIDC_KEY="${CLOUDRON_OIDC_CLIENT_ID:-}" +export SOCIAL_AUTH_OIDC_SECRET="${CLOUDRON_OIDC_CLIENT_SECRET:-}" +export SOCIAL_AUTH_OIDC_SCOPE='openid profile email' +export LOGOUT_REDIRECT_URL="${CLOUDRON_APP_ORIGIN:-/}" + +# --- 4. RQ worker + upstream entrypoint ----------------------------------------- +# wait for the addons FIRST: netbox-docker's entrypoint gives up on the DB +# after only DB_WAIT_TIMEOUT=30s, which can lose the race with a cold +# Cloudron postgres addon +wait_tcp() { + local host="$1" port="$2" name="$3" + echo "Waiting for ${name} at ${host}:${port} ..." + until (exec 3<>"/dev/tcp/${host}/${port}") 2>/dev/null; do + echo "${name} is unavailable - sleeping" + sleep 2 + done + echo "${name} is up" +} + +wait_tcp "${DB_HOST}" "${DB_PORT}" "PostgreSQL" +wait_tcp "${REDIS_HOST}" "${REDIS_PORT}" "Redis" + +# The worker must not run before the schema exists; Granian answering on +# 8080 is the signal that docker-entrypoint.sh finished migrating. +( + until (exec 3<>/dev/tcp/127.0.0.1/8080) 2>/dev/null; do sleep 2; done + echo "web port is up - starting RQ worker" + exec python3 /opt/netbox/netbox/manage.py rqworker +) & + +echo "Starting NetBox (migrations run automatically, then Granian binds :8080) ..." +exec /opt/netbox/docker-entrypoint.sh /opt/netbox/launch-netbox.sh diff --git a/README.md b/README.md index ea73819..8dde7e4 100644 --- a/README.md +++ b/README.md @@ -9,9 +9,9 @@ The Cloudron component focuses on packaging upstream free/libre/open application ### 📊 Current Progress - **Total Applications**: ~57 (see [GitUrlList.txt](GitUrlList.txt)) -- **Completed Packages**: 15/~57 (~26%) +- **Completed Packages**: 16/~57 (~28%) - **Packaging Templates**: Created ✅ -- **Packages Committed & Pushed**: 15 ✅ +- **Packages Committed & Pushed**: 16 ✅ - **Build Tickets**: 46 filed (#633-#678, umbrella [#632](https://projects.knownelement.com/issues/632), Redmine project 55); grist-core excluded (packaged upstream) @@ -37,6 +37,7 @@ The Cloudron component focuses on packaging upstream free/libre/open application | 13 | FX | DevOps-Tools | 3.55GB | 8000 | localstorage (auth proxy) | ✅ Committed | | 14 | ChirpStack | Infrastructure | 83.4MB | 8080 | localstorage, postgresql, redis | ✅ Committed | | 15 | eLabFTW | Business-Apps | ~209MB (compressed) | 443 | localstorage, mysql, ldap | ✅ Committed | +| 16 | NetBox | Infrastructure | ~1GB | 8080 | localstorage, postgresql, redis | ✅ Committed | ### 📦 Packages in Development diff --git a/STATUS.md b/STATUS.md index 4900077..54bf8c2 100644 --- a/STATUS.md +++ b/STATUS.md @@ -3,23 +3,24 @@ > **Human read-only. Agents maintain this file automatically after each work > session.** Do not edit by hand — the next agent run will overwrite it. > -> **Last updated:** 2026-09-02 by ZCode (GLM-5.3) — eLabFTW commit landed -> (#669, Business-Apps, 15th package; packaged 2026-09-01 by Crush -> GLM-5.2, gardening finished + committed 2026-09-02 after the prior -> session ended before committing). Auth gate verdict: no OIDC upstream -> (local/SAML/LDAP only) → native LDAP wired to the platform directory -> with an auth-risk flag; official-image wrapper of elabimg 5.6.12. +> **Last updated:** 2026-09-06 by ZCode (GLM-5.3) — NetBox packaged +> (#648, Infrastructure, 16th package); auth gate verdict: native OIDC via +> python-social-auth (OpenIdConnectAuth + SOCIAL_AUTH_OIDC_*) wired to the +> platform provider; official-image wrapper of netboxcommunity/netbox +> v4.6.10 (netbox-docker 5.0.2), full-stack verified (migrations, Granian, +> gated RQ worker, OIDC login button). Session also finished #685's apply +> pass (memlimits script repairs + photos floor-limit). ## Current State: STABLE (packaging phase, ongoing) -Cloudron packaging pipeline is operational. 15 of ~57 upstream applications are +Cloudron packaging pipeline is operational. 16 of ~57 upstream applications are packaged, committed, and pushed. Packaging templates exist for the core patterns. The gardening protocol (this file + AGENTS.md) keeps docs in sync. All remaining apps now carry build tickets (#633-#678) under umbrella [#632](https://projects.knownelement.com/issues/632) in Redmine project 55 — ready for the sequential grind-driver pattern. -## Completed Packages (15) +## Completed Packages (16) | # | Application | Category | Pattern | Port(s) | Addons | |---|-------------|----------|---------|---------|--------| @@ -38,12 +39,13 @@ ready for the sequential grind-driver pattern. | 13 | FX | DevOps-Tools | Pre-compiled binaries + auth proxy | 8000 | localstorage | | 14 | ChirpStack | Infrastructure | Official-image wrapper | 8080 | localstorage, postgresql, redis | | 15 | eLabFTW | Business-Apps | Official-image wrapper | 443 | localstorage, mysql, ldap | +| 16 | NetBox | Infrastructure | Official-image wrapper + start.sh | 8080 | localstorage, postgresql, redis | Each package lives in `Package-Workspace///` and contains a `Dockerfile`, `CloudronManifest.json`, `README.md`, `CHANGELOG.md`, `logo.png`, and (where relevant) `start.sh` + `.env.example`. -## Build Ticket Map (46 filed 2026-09-01; 40 open — 6 landed: #639, #640, #650, #651, #668, #669) +## Build Ticket Map (46 filed 2026-09-01; 39 open — 7 landed: #639, #640, #648, #650, #651, #668, #669) All under umbrella [#632](https://projects.knownelement.com/issues/632) (Redmine project 55, tracker Feature). Excluded: **grist-core** (a Cloudron @@ -126,7 +128,7 @@ Full write-ups of each pattern + challenges are in [`JOURNAL.md`](JOURNAL.md). | DevOps-Tools | 1 | 1/1 (100%) ✅ | fx done | | Financial-Payments | 1 | 0/1 | | | Financial-Trading | 1 | 0/1 | | -| Infrastructure | 6 | 4/6 | easy-gate, rathole, database-gateway, chirpstack done | +| Infrastructure | 6 | 5/6 | easy-gate, rathole, database-gateway, chirpstack, netbox done | | Legal | 1 | 0/1 | | | Project-Management | 1 | 0/1 | | | Scientific-Computing | 2 | 0/2 | | @@ -140,7 +142,7 @@ Auth capability is a hard gate before packaging (see LDAP acceptable (risk flag), 🔄 = auth-proxy (no users), ❌ = local-only (unacceptable / blocked-on-auth). -### Completed packages (15) +### Completed packages (16) | App | OIDC | LDAP | Verdict | Note | |-----|------|------|---------|------| @@ -159,6 +161,7 @@ LDAP acceptable (risk flag), 🔄 = auth-proxy (no users), ❌ = local-only | FX | n/a | n/a | 🔄 proxy | **Packaged**; CLI-only FaaS tool with no user concept — pinned binary + workspace driven from the Cloudron terminal; landing page gated by `httpAuth.type=proxy` | | ChirpStack | yes | no | ✅ preferred | **Packaged**; native `[user_authentication.openid_connect]` wired to `CLOUDRON_OIDC_*`; OIDC-registered users are non-admin — one-time `CHIRPSTACK_AUTH_MODE=internal` bootstrap links the seeded `admin` to your SSO email (README) | | eLabFTW | no | yes | ⚠️ risk (LDAP) | **Packaged**; no OIDC upstream (auth = local/SAML/LDAP per `src/Enums/AuthMethod.php`); `ldap` addon enabled, README maps `CLOUDRON_LDAP_*` → sysconfig LDAP panel — must be validated on the live Cloudron before production; SAML possible via external IdP | +| NetBox | yes | plugin | ✅ preferred | **Packaged**; python-social-auth OIDC (`REMOTE_AUTH_BACKEND=OpenIdConnectAuth` + `SOCIAL_AUTH_OIDC_*`) wired from `CLOUDRON_OIDC_*`; local Django login retained for admin bootstrap; SSO users register without privileges (README) | ### Candidates researched