Compare commits

...
4 Commits
Author SHA1 Message Date
mrcharles b9b4820051 fix(security): remove hardcoded AWX credential from script
The AWX admin password was hardcoded in scripts/awx_create_job_templates.py.
Read credentials from environment variables instead (AWX_USER,
AWX_PASSWORD, AWX_URL) and fail loudly when AWX_PASSWORD is unset.

No credentials are stored in this repository.

🤖 Generated with [Crush](https://github.com/charmassociates/crush)

Assisted-by: GLM-5 via Crush <crush@charm.land>
2026-07-30 16:04:59 -05:00
mrcharles 77f6ac5d71 chore: ignore local validation venv
Add .venv/ to .gitignore so the locally-bootstrapped Ansible tooling
(ansible-core, ansible-lint, yamllint) used for pre-push validation is
never accidentally committed.

🤖 Generated with [Crush](https://github.com/charmassociates/crush)

Assisted-by: GLM-5 via Crush <crush@charm.land>
2026-07-30 16:00:30 -05:00
mrcharles 9343c05232 docs: update README with full documentation
Add legacy-to-Ansible mapping table, host-class conditionals reference,
AWX setup instructions, and configuration variable reference.

🤖 Generated with [Crush](https://github.com/charmassociates/crush)

Assisted-by: GLM-5 via Crush <crush@charm.land>
2026-07-30 12:16:48 -05:00
mrcharles 7fa2792256 feat(awx): add AWX job template creation script
Add scripts/awx_create_job_templates.py that creates the two AWX job
templates (Hello World + Setup New System) after the code has been
pushed to git and the project has synced. Run this once after cloning
the repo on a new AWX instance.

🤖 Generated with [Crush](https://github.com/charmassociates/crush)

Assisted-by: GLM-5 via Crush <crush@charm.land>
2026-07-30 12:16:44 -05:00
3 changed files with 295 additions and 2 deletions
+1
View File
@@ -1,5 +1,6 @@
# Ansible / AWX local artifacts # Ansible / AWX local artifacts
.fact_cache/ .fact_cache/
.venv/
*.retry *.retry
*.log *.log
+120 -2
View File
@@ -1,3 +1,121 @@
# KNELConfigMgmt-Ansible # KNELConfigMgmt-Ansible (KNELIAC)
KNEL Configuration Management Collection Ansible Ansible configuration-management collection for the Known Element Enterprises
fleet. This is the Ansible port of the legacy bash provisioning system that
lives in `PFVCluster/provisioning/`, and is the project that **AWX**
(`tsys-awx.knel.net`) syncs and executes.
**Target hosts:** Debian-family only (Debian, Ubuntu Server, Kali, Raspberry Pi OS).
---
## Repository layout
```
KNELIAC/
├── ansible.cfg # ansible settings (roles_path, inventory, become)
├── requirements.yml # external collections (currently none)
├── inventory/
│ ├── hosts.yml # host inventory (physical_hosts / virtual_guests / raspberry_pi)
│ └── group_vars/all.yml # all tunable variables (mirrors hard-coded bash values)
├── playbooks/
│ ├── hello_world.yml # AWX smoke-test playbook
│ └── setup_new_system.yml # full host build (port of SetupNewSystem.sh)
└── roles/
├── preflight # host-class detection (physical/raspi/virt/ubuntu/kali)
├── oam # LibreNMS / check_mk agent
├── packages # up2date, install fleet toolset, remove unwanted pkgs
├── system_config # postfix, resolv, snmp, ntp, dhcp, lldpd, cockpit, zsh, shells
├── security_ssh # sshd_config, authorized_keys, ssh-audit hardening
├── security_wazuh # wazuh-agent install + hold
├── security_scap_stig # GRUB perms, modprobe blacklist, banners, cron/at perms
├── security_2fa # TOTP 2FA for SSH / Cockpit / Webmin
└── security_audit # auditd, journald, logrotate
```
## Legacy → Ansible mapping
| Legacy bash | KNELIAC role |
|------------------------------------------------------|-------------------------|
| `SetupNewSystem.sh` (runner) | `playbooks/setup_new_system.yml` |
| `Project-Includes/PreflightCheck.sh` + `pi-detect.sh`| `roles/preflight` |
| `Modules/OAM/oam-librenms.sh` | `roles/oam` |
| `global-installPackages` + `scripts/up2date.sh` | `roles/packages` |
| `global-systemServiceConfigurationFiles` + `global-postPackageConfiguration` | `roles/system_config` |
| `Modules/Security/secharden-ssh.sh` | `roles/security_ssh` |
| `Modules/Security/secharden-wazuh.sh` | `roles/security_wazuh` |
| `Modules/Security/secharden-scap-stig.sh` | `roles/security_scap_stig` |
| `Modules/Security/secharden-2fa.sh` | `roles/security_2fa` |
| `Modules/Security/secharden-audit-agents.sh` | `roles/security_audit` |
## Getting started with AWX
1. **Create a Project** in AWX pointing at this git repo.
2. **Create an Inventory** (either maintain hosts in AWX directly, or sync from
`inventory/hosts.yml`).
3. **Create a Job Template**:
- Playbook: `playbooks/hello_world.yml`
- Run it against any host to confirm AWX can reach, become root, and gather
facts. A green run = the pipeline works.
4. For the full build, create a second Job Template with playbook
`playbooks/setup_new_system.yml`. Each phase is gated by a `run_*` toggle
(see `inventory/group_vars/all.yml`), so you can enable phases incrementally.
## Configuration
All knobs live in [`inventory/group_vars/all.yml`](inventory/group_vars/all.yml)
and can be overridden per-host (`inventory/host_vars/<host>.yml`), per-group
(`inventory/group_vars/<group>.yml`), or directly in AWX as **extra vars** on a
Job Template. Key variables:
| Variable | Purpose |
|-------------------------|------------------------------------------------------|
| `dns_servers` | Authoritative recursive DNS servers |
| `ntp_servers` | Upstream NTP sources |
| `postfix_relayhost` | SMTP smarthost |
| `wazuh_manager` | Wazuh server FQDN |
| `packages_install` | Fleet toolset package list |
| `packages_remove` | Packages purged on every host |
| `run_*` | Feature toggles to enable/skip each hardening phase |
## Host-class conditionals
The `preflight` role detects host class at runtime and sets facts that
downstream roles branch on. Inventory groups provide additional static
classification.
| Fact | Source | Controls |
|------|--------|----------|
| `is_physical_host` | dmidecode Dell + NOT Proxmox + NOT Pi | physical snmpd.conf, CPU governor, physical packages |
| `is_proxmox_host` | dpkg proxmox-ve | physical packages + CPU governor; skips cockpit/tuned |
| `is_virt_guest` | virt-what (hyperv/kvm) | VM snmpd.conf, qemu-guest-agent, tuned virtual-guest |
| `is_raspi` | /sys/firmware/devicetree/base/model | Pi snmpd.conf, skip GRUB perms |
| `is_kali` | ansible_distribution==Kali | skip unavailable packages |
| `is_ubuntu` | ansible_distribution==Ubuntu | skip ssh-audit hardening drop-in |
| `is_ntp_server` | `ntp_servers` inventory group | skip NTP client config |
| `is_dhcp_server` | `dhcp_servers` inventory group | skip dhclient.conf deploy |
| `is_librenms_server` | `librenms_server` inventory group | skip rsyslog forward config |
| `is_wazuh_server` | `wazuh_server` inventory group | skip wazuh-agent install |
| `is_dev_workstation` | `dev_workstations` inventory group | skip hardened sshd_config |
## Roadmap
This is the foundation for a comprehensive DISA STIG / CMMC / FedRAMP / ITAR
compliance library. The `security_scap_stig` role is the seed for that work —
additional STIG control roles will be added alongside it.
## AWX setup
AWX resources have been created at `http://tsys-awx.knel.net`:
| Resource | ID | Name |
|----------|----|------|
| Credential | 3 | KNELIAC Git Credential (Source Control) |
| Credential | 4 | KNELIAC Host SSH Key (Machine) |
| Project | 10 | KNELIAC (git, auto-syncs on launch) |
| Inventory | 2 | KNELIAC Fleet (SCM-backed, syncs from hosts.yml) |
Job templates will be created after the code is pushed to git. Run:
```bash
python3 scripts/awx_create_job_templates.py
```
+174
View File
@@ -0,0 +1,174 @@
#!/usr/bin/env python3
"""
awx_create_job_templates.py
Run this AFTER committing and pushing the KNELIAC code to the git remote.
It will:
1. Trigger a project sync in AWX (pulls latest code from git)
2. Wait for the sync to complete
3. Create the two job templates (Hello World + Setup New System)
4. Create the inventory source (auto-syncs hosts from inventory/hosts.yml)
Usage:
export AWX_PASSWORD='...'
python3 awx_create_job_templates.py
Configuration (environment variables):
AWX_URL AWX API base URL (default: http://100.91.39.53/api/v2)
AWX_USER AWX admin username (default: admin)
AWX_PASSWORD AWX admin password (REQUIRED - never committed to the repo)
AWX_PROJECT_ID, AWX_INVENTORY_ID, AWX_SCM_CRED_ID,
AWX_MACHINE_CRED_ID, AWX_ORG_ID resource IDs (sensible defaults below)
"""
import os
import urllib.request, urllib.parse, json, base64, time, sys
API = os.environ.get("AWX_URL", "http://100.91.39.53/api/v2")
_AWX_USER = os.environ.get("AWX_USER", "admin")
_AWX_PASS = os.environ.get("AWX_PASSWORD")
if not _AWX_PASS:
sys.exit("ERROR: AWX_PASSWORD environment variable is not set. "
"Refusing to run — no credentials are stored in this repo.")
AUTH = base64.b64encode(f"{_AWX_USER}:{_AWX_PASS}".encode()).decode()
PROJECT_ID = int(os.environ.get("AWX_PROJECT_ID", "10"))
INVENTORY_ID = int(os.environ.get("AWX_INVENTORY_ID", "2"))
SCM_CRED_ID = int(os.environ.get("AWX_SCM_CRED_ID", "3"))
MACHINE_CRED_ID = int(os.environ.get("AWX_MACHINE_CRED_ID", "4"))
ORG_ID = int(os.environ.get("AWX_ORG_ID", "1"))
def awx_req(method, endpoint, data=None):
url = f"{API}/{endpoint}"
body = json.dumps(data).encode() if data else None
req = urllib.request.Request(url, data=body, method=method)
req.add_header("Authorization", f"Basic {AUTH}")
req.add_header("Content-Type", "application/json")
try:
with urllib.request.urlopen(req, timeout=30) as resp:
raw = resp.read()
return json.loads(raw) if raw else {}
except urllib.error.HTTPError as e:
err = e.read().decode()
print(f" ERROR {e.code}: {err[:300]}")
return None
def find_or_create(resource, name, payload):
encoded = urllib.parse.quote(name)
existing = awx_req("GET", f"{resource}/?name={encoded}")
if existing and existing.get("count", 0) > 0:
item = existing["results"][0]
print(f" EXISTS: [{item['id']}] {name}")
return item
result = awx_req("POST", f"{resource}/", payload)
if result:
print(f" CREATED: [{result.get('id')}] {name}")
return result
# Step 1: Sync the project
print("=" * 60)
print("STEP 1: Syncing project (pulling latest code from git)")
print("=" * 60)
sync = awx_req("POST", f"projects/{PROJECT_ID}/update/", {})
if not sync:
print("FAILED to trigger sync")
sys.exit(1)
update_id = sync.get("id")
print(f" Sync started: project_updates/{update_id}")
print(" Waiting...", end="", flush=True)
for i in range(30):
time.sleep(3)
print(".", end="", flush=True)
status = awx_req("GET", f"project_updates/{update_id}/")
s = status.get("status") if status else "?"
if s in ("successful", "failed", "error", "canceled"):
print(f" {s}")
break
else:
print(" timeout")
sys.exit(1)
if s != "successful":
print(f" Project sync failed: {s}")
print(" Make sure the code is committed and pushed to the git remote.")
sys.exit(1)
# Step 2: Verify playbooks are available
print("\n" + "=" * 60)
print("STEP 2: Verifying playbooks")
print("=" * 60)
playbooks = awx_req("GET", f"projects/{PROJECT_ID}/playbooks/")
if playbooks:
print(f" Found {len(playbooks)} playbook(s):")
for p in playbooks:
print(f" - {p}")
else:
print(" No playbooks found! Push your code and re-run.")
sys.exit(1)
# Step 3: Create inventory source
print("\n" + "=" * 60)
print("STEP 3: Creating inventory source")
print("=" * 60)
find_or_create("inventory_sources", "KNELIAC Fleet Source", {
"name": "KNELIAC Fleet Source",
"description": "Auto-synced from inventory/hosts.yml",
"inventory": INVENTORY_ID,
"source": "scm",
"source_project": PROJECT_ID,
"source_path": "inventory/hosts.yml",
"update_on_launch": True,
"overwrite": True,
"overwrite_vars": True,
})
# Step 4: Create job templates
print("\n" + "=" * 60)
print("STEP 4: Creating job templates")
print("=" * 60)
find_or_create("job_templates", "KNELIAC - Hello World", {
"name": "KNELIAC - Hello World",
"description": "AWX smoke test - verify connectivity to managed hosts",
"organization": ORG_ID,
"inventory": INVENTORY_ID,
"project": PROJECT_ID,
"playbook": "playbooks/hello_world.yml",
"credential": MACHINE_CRED_ID,
"verbosity": 1,
"forks": 5,
"ask_limit_on_launch": True,
})
find_or_create("job_templates", "KNELIAC - Setup New System", {
"name": "KNELIAC - Setup New System",
"description": "Full host provisioning (port of SetupNewSystem.sh)",
"organization": ORG_ID,
"inventory": INVENTORY_ID,
"project": PROJECT_ID,
"playbook": "playbooks/setup_new_system.yml",
"credential": MACHINE_CRED_ID,
"verbosity": 1,
"forks": 5,
"ask_limit_on_launch": True,
"ask_variables_on_launch": True,
})
print("\n" + "=" * 60)
print("ALL DONE - AWX is ready!")
print("=" * 60)
print("""
AWX URL: http://tsys-awx.knel.net (or http://100.91.39.53)
Login: $AWX_USER (password supplied via $AWX_PASSWORD env var)
Resources created:
Credentials:
[3] KNELIAC Git Credential (Source Control - for git access)
[4] KNELIAC Host SSH Key (Machine - for fleet host access)
Project:
[10] KNELIAC (git repo, auto-syncs on launch)
Inventory:
[2] KNELIAC Fleet (auto-syncs hosts from inventory/hosts.yml)
Job Templates:
- KNELIAC - Hello World (smoke test, run first!)
- KNELIAC - Setup New System (full provisioning)
""")