mrcharles 4806e6f24d feat: absorb cloudron kuma monitoring from PFVCluster [#727]
monitoring/ = cloudron-apps.txt (55-app inventory, #435) +
kuma-cloudron-sync.py (coverage checker, path re-rooted).
Removed from PFVCluster in the paired commit there.
Meat: https://projects.knownelement.com/issues/727#note-3913
2026-09-02 18:51:04 -05:00

Knowledge docs live on Discourse — perf write-up: t/316 · VP TechOps category · Work tracking: Redmine — Cloudron project · Infra audit log: t/298

KNEL/cloudron — tsys-cloudron fleet-machine repo

Production Cloudron host my.knownelement.com (tailnet name tsys-cloudron.knel.net). ~55 apps behind the Cloudron panel, platform Postgres/MySQL/mail containers, monitored by Uptime Kuma (Cloudron group).

Path What
perf/ #685 perf work: memory census, pg role-cap SQL, applied sysctl, planned docker daemon.json
monitoring/ Uptime Kuma Cloudron app coverage — 55-app inventory + sync checker (moved from PFVCluster, #435/#685/#727)
ansible/ Reserved — AWX/Ansible IaC convergence (#454); playbooks land here as manual changes are codified
scripts/ Rule engine + git hooks (enforcement layer, ADOPTING pattern)
tests/ shellcheck wrapper (Docker — host stays clean)

Ground truth (2026-09-02, #685 — narrative + measurements on t/316)

  • Platform Postgres slot exhaustion root-caused to 7 booking instances + Penpot pinning ~61 idle connections each. Role caps applied (booking 10, Penpot 15, every other app role 20), max_connections 500→700, shared_buffers 2GB, idle-in-transaction timeout 15min. Victim too many clients FATALs → zero.
  • Host sysctl: vm.swappiness=10 (perf/sysctl/). THP already never, virtio scheduler none — optimal, left alone.
  • /etc/docker/daemon.json is still ABSENT on the host (live-restore=false, containers default to nofile=1024 unless bridged). The postgres fd bridge (prlimit 262144) is EPHEMERAL — lost on any postgres container recreate. Applying perf/docker-daemon.json.planned needs ONE planned dockerd bounce (restarts all ~300 containers) in a human-chosen window.
  • Memory sliders: shrink table on t/316 (~28GB headroom). Do NOT shrink bi.knownelement.com or platform services (Cloudron auto-sizes those).

Access

SSH root@my.knownelement.com is the only access path — routed through the KNEL/PFVCluster tests/remote.sh chokepoint (see AGENTS.md). Cloudron API token pending; once issued, refs live in ~/.creds/cloudron.env, never here.

S
Description
tsys-cloudron (my.knownelement.com) fleet machine repo — perf tuning, config-as-code, AWX/Ansible IaC [#685]
Readme AGPL-3.0
95 KiB
Languages
Shell 89.3%
Python 10.7%