reachableceo/ultix
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Open Terminal fleet for all accounts + OpenWebUI UAT pass (#610)

Browse Source 
open-terminal 0.11.34 bare-metal (human-authorized host install) with a
systemd template unit: reachableceo on tailscale-only :30000 (cwd
~/projects) and all 8 other accounts on :30001-30008, each with its own
key kept out of ps in per-user TOMLs, all users added to the docker group.
Keys live in ~/.creds + /etc/ukrrs/open-terminal, never in this repo.

UAT: throwaway OpenWebUI v0.11.1 (matched to the human's Cloudron prod)
driven purely over its HTTP API against the beta gateway lane - connection
verify/config PASS, direct exec as uid 1001 PASS, and the chat round trip
PASS: the model emitted run_command, we executed it through OpenWebUI's
terminal proxy (server holds the key), and the final answer named 9/9 real
project directories; earlier negative runs prove the test catches
hallucinated output. Ops note OPEN-TERMINAL.md documents production wiring
(prod openwebui container v0.3.10 is too old and needs an upgrade first).

Also tonight: agent-stack relaunchers removed (6-, backups kept) so no
screen/crush sessions auto-start on reboot; 22:00 night-profile flip
observed live; docs synced for the earlier teardown.

💘 Generated with Crush

Assisted-by: Crush:glm-5.2
[#610]
This commit is contained in:
mrcharles
2026-08-31 22:50:53 -05:00
parent 74934b1095
commit c810942239
21 changed files with 940 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
TRACKING.md
+4
View File
@@ -14,10 +14,14 @@ One table, updated in place at each checkpoint. States: done ✅ / doing 🔄 /
| OPT-8 | repos | ✅ | PMO backpressure design note in ukrrs/docs/harness; code = ticketed |
| OPT-9 | integ | ⏳ | #604: node_exporter svc + gateway pressure scrape (Q10) |
| OPT-10 | integ | ⏳ | #605: k8s join (Q11, blocked by #601) + proxmox-ctl token (Q12); GPUs #606 |
| #610 | fleet | ✅ | Open Terminal 9-account fleet (tailscale-only :30000-30008) + OpenWebUI UAT PASS |
Inbox (mid-task interrupts): none.
Decisions log (latest wins):
- 2026-08-31 (night, con handed to crush): #610 DONE in one session: open-terminal 0.11.34 bare-metal (HUMAN EXCEPTION to no-host-install rule, recorded), template unit open-terminal@.service, per-user TOML keys (never in ps/repo; ~/.creds holds copies), bind TAILSCALE-ONLY 100.101.187.119 per human ruling (0.0.0.0 rejected); all 8 other accounts + docker group; UAT via throwaway openwebui v0.11.1 (matches Cloudron prod) ALL PASS incl. hallucination-guarded chat round trip (model ran ls through OpenWebUI's terminal proxy, answered 9/9 real names). NOTE: prod openwebui container (v0.3.10, :3000) is too old for Open Terminal — upgrade to 0.11.x before wiring; Cloudron 0.11.1 is fine. Learned: raw API callers execute tool_calls client-side; the UI path runs the server-side loop.
- 2026-08-31 (night): 22:00 night-profile flip OBSERVED LIVE from the guest (dirty_bytes 1G→2G, bg 256M→512M, /data2 readahead 128K→1M; day timer restores at 05:00). First runtime-only flip since install.
- 2026-08-31 (night): agent-stack relaunchers REMOVED by 6-remove-agent-stacks.sh (reachableceo-agent-stack.service, tsg-supervisor.timer/.service 5-min self-heal, dead tsg-agent-stacks.service; launchers + units backed up to removed-agent-stacks/). No screen/crush sessions auto-start on reboot anymore; human: stacks no longer used for pmo/work.
- 2026-08-31 (night): docs synced post-pass: REPORT v1.1 (status banner, per-item STATUS tags, §6/§8 truth), RUNBOOK amended for the Oct replan, all paths ~/optimize → ~/projects/ultix (repo moved). Human ruling: system idle tonight → multiqueue bounce approved for this session, after remaining tasks. Agent-stack teardown prepped as 6-remove-agent-stacks.sh (kills boot-time RCEO/TSG screen respawn + the 5-min supervisor self-heal; unit/launcher backups to removed-agent-stacks/; root also quits leftover root screen ukrrs-gateway-final, leaves 'boot' alone).
- 2026-08-31 (late): Redmine replan complete: #601 Oct window (due 10-31, High, version october-2026-window; Redmine API refuses start_date on update — Oct-1 start recorded in the note instead), NEW #608 cgroup_parent wiring (High; THE starvation fix on 8 vCPU), NEW #609 Fri 09-04 troubleshooting-only outage (USB SDR reseat + socket/PSU/BIOS checks; NO upgrades; host down → this VM auto-starts via onboot=1, gateway auto-recovers). #605/#606 noted. docs repo pushed (main 9eb324e).
- 2026-08-31 (eve): HARDWARE WINDOW → ~OCT 2026; presume 8 vCPU/48G operating values; cpuset reservation deferred → cgroup_parent wiring = the starvation fix; one VM-5111 reboot owed for multiqueue (5-activate-multiqueue.sh, human-timed; crush may roll OTHER VMs on pfv-tsys5 anytime).