Compare commits
4
Commits
95d6d6a6f7
...
b9b4820051
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b9b4820051 | ||
|
|
77f6ac5d71 | ||
|
|
9343c05232 | ||
|
|
7fa2792256 |
@@ -1,5 +1,6 @@
|
|||||||
# Ansible / AWX local artifacts
|
# Ansible / AWX local artifacts
|
||||||
.fact_cache/
|
.fact_cache/
|
||||||
|
.venv/
|
||||||
*.retry
|
*.retry
|
||||||
*.log
|
*.log
|
||||||
|
|
||||||
|
|||||||
@@ -1,3 +1,121 @@
|
|||||||
# KNELConfigMgmt-Ansible
|
# KNELConfigMgmt-Ansible (KNELIAC)
|
||||||
|
|
||||||
KNEL Configuration Management Collection Ansible
|
Ansible configuration-management collection for the Known Element Enterprises
|
||||||
|
fleet. This is the Ansible port of the legacy bash provisioning system that
|
||||||
|
lives in `PFVCluster/provisioning/`, and is the project that **AWX**
|
||||||
|
(`tsys-awx.knel.net`) syncs and executes.
|
||||||
|
|
||||||
|
**Target hosts:** Debian-family only (Debian, Ubuntu Server, Kali, Raspberry Pi OS).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Repository layout
|
||||||
|
|
||||||
|
```
|
||||||
|
KNELIAC/
|
||||||
|
├── ansible.cfg # ansible settings (roles_path, inventory, become)
|
||||||
|
├── requirements.yml # external collections (currently none)
|
||||||
|
├── inventory/
|
||||||
|
│ ├── hosts.yml # host inventory (physical_hosts / virtual_guests / raspberry_pi)
|
||||||
|
│ └── group_vars/all.yml # all tunable variables (mirrors hard-coded bash values)
|
||||||
|
├── playbooks/
|
||||||
|
│ ├── hello_world.yml # AWX smoke-test playbook
|
||||||
|
│ └── setup_new_system.yml # full host build (port of SetupNewSystem.sh)
|
||||||
|
└── roles/
|
||||||
|
├── preflight # host-class detection (physical/raspi/virt/ubuntu/kali)
|
||||||
|
├── oam # LibreNMS / check_mk agent
|
||||||
|
├── packages # up2date, install fleet toolset, remove unwanted pkgs
|
||||||
|
├── system_config # postfix, resolv, snmp, ntp, dhcp, lldpd, cockpit, zsh, shells
|
||||||
|
├── security_ssh # sshd_config, authorized_keys, ssh-audit hardening
|
||||||
|
├── security_wazuh # wazuh-agent install + hold
|
||||||
|
├── security_scap_stig # GRUB perms, modprobe blacklist, banners, cron/at perms
|
||||||
|
├── security_2fa # TOTP 2FA for SSH / Cockpit / Webmin
|
||||||
|
└── security_audit # auditd, journald, logrotate
|
||||||
|
```
|
||||||
|
|
||||||
|
## Legacy → Ansible mapping
|
||||||
|
|
||||||
|
| Legacy bash | KNELIAC role |
|
||||||
|
|------------------------------------------------------|-------------------------|
|
||||||
|
| `SetupNewSystem.sh` (runner) | `playbooks/setup_new_system.yml` |
|
||||||
|
| `Project-Includes/PreflightCheck.sh` + `pi-detect.sh`| `roles/preflight` |
|
||||||
|
| `Modules/OAM/oam-librenms.sh` | `roles/oam` |
|
||||||
|
| `global-installPackages` + `scripts/up2date.sh` | `roles/packages` |
|
||||||
|
| `global-systemServiceConfigurationFiles` + `global-postPackageConfiguration` | `roles/system_config` |
|
||||||
|
| `Modules/Security/secharden-ssh.sh` | `roles/security_ssh` |
|
||||||
|
| `Modules/Security/secharden-wazuh.sh` | `roles/security_wazuh` |
|
||||||
|
| `Modules/Security/secharden-scap-stig.sh` | `roles/security_scap_stig` |
|
||||||
|
| `Modules/Security/secharden-2fa.sh` | `roles/security_2fa` |
|
||||||
|
| `Modules/Security/secharden-audit-agents.sh` | `roles/security_audit` |
|
||||||
|
|
||||||
|
## Getting started with AWX
|
||||||
|
|
||||||
|
1. **Create a Project** in AWX pointing at this git repo.
|
||||||
|
2. **Create an Inventory** (either maintain hosts in AWX directly, or sync from
|
||||||
|
`inventory/hosts.yml`).
|
||||||
|
3. **Create a Job Template**:
|
||||||
|
- Playbook: `playbooks/hello_world.yml`
|
||||||
|
- Run it against any host to confirm AWX can reach, become root, and gather
|
||||||
|
facts. A green run = the pipeline works.
|
||||||
|
4. For the full build, create a second Job Template with playbook
|
||||||
|
`playbooks/setup_new_system.yml`. Each phase is gated by a `run_*` toggle
|
||||||
|
(see `inventory/group_vars/all.yml`), so you can enable phases incrementally.
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
All knobs live in [`inventory/group_vars/all.yml`](inventory/group_vars/all.yml)
|
||||||
|
and can be overridden per-host (`inventory/host_vars/<host>.yml`), per-group
|
||||||
|
(`inventory/group_vars/<group>.yml`), or directly in AWX as **extra vars** on a
|
||||||
|
Job Template. Key variables:
|
||||||
|
|
||||||
|
| Variable | Purpose |
|
||||||
|
|-------------------------|------------------------------------------------------|
|
||||||
|
| `dns_servers` | Authoritative recursive DNS servers |
|
||||||
|
| `ntp_servers` | Upstream NTP sources |
|
||||||
|
| `postfix_relayhost` | SMTP smarthost |
|
||||||
|
| `wazuh_manager` | Wazuh server FQDN |
|
||||||
|
| `packages_install` | Fleet toolset package list |
|
||||||
|
| `packages_remove` | Packages purged on every host |
|
||||||
|
| `run_*` | Feature toggles to enable/skip each hardening phase |
|
||||||
|
|
||||||
|
## Host-class conditionals
|
||||||
|
|
||||||
|
The `preflight` role detects host class at runtime and sets facts that
|
||||||
|
downstream roles branch on. Inventory groups provide additional static
|
||||||
|
classification.
|
||||||
|
|
||||||
|
| Fact | Source | Controls |
|
||||||
|
|------|--------|----------|
|
||||||
|
| `is_physical_host` | dmidecode Dell + NOT Proxmox + NOT Pi | physical snmpd.conf, CPU governor, physical packages |
|
||||||
|
| `is_proxmox_host` | dpkg proxmox-ve | physical packages + CPU governor; skips cockpit/tuned |
|
||||||
|
| `is_virt_guest` | virt-what (hyperv/kvm) | VM snmpd.conf, qemu-guest-agent, tuned virtual-guest |
|
||||||
|
| `is_raspi` | /sys/firmware/devicetree/base/model | Pi snmpd.conf, skip GRUB perms |
|
||||||
|
| `is_kali` | ansible_distribution==Kali | skip unavailable packages |
|
||||||
|
| `is_ubuntu` | ansible_distribution==Ubuntu | skip ssh-audit hardening drop-in |
|
||||||
|
| `is_ntp_server` | `ntp_servers` inventory group | skip NTP client config |
|
||||||
|
| `is_dhcp_server` | `dhcp_servers` inventory group | skip dhclient.conf deploy |
|
||||||
|
| `is_librenms_server` | `librenms_server` inventory group | skip rsyslog forward config |
|
||||||
|
| `is_wazuh_server` | `wazuh_server` inventory group | skip wazuh-agent install |
|
||||||
|
| `is_dev_workstation` | `dev_workstations` inventory group | skip hardened sshd_config |
|
||||||
|
|
||||||
|
## Roadmap
|
||||||
|
|
||||||
|
This is the foundation for a comprehensive DISA STIG / CMMC / FedRAMP / ITAR
|
||||||
|
compliance library. The `security_scap_stig` role is the seed for that work —
|
||||||
|
additional STIG control roles will be added alongside it.
|
||||||
|
|
||||||
|
## AWX setup
|
||||||
|
|
||||||
|
AWX resources have been created at `http://tsys-awx.knel.net`:
|
||||||
|
|
||||||
|
| Resource | ID | Name |
|
||||||
|
|----------|----|------|
|
||||||
|
| Credential | 3 | KNELIAC Git Credential (Source Control) |
|
||||||
|
| Credential | 4 | KNELIAC Host SSH Key (Machine) |
|
||||||
|
| Project | 10 | KNELIAC (git, auto-syncs on launch) |
|
||||||
|
| Inventory | 2 | KNELIAC Fleet (SCM-backed, syncs from hosts.yml) |
|
||||||
|
|
||||||
|
Job templates will be created after the code is pushed to git. Run:
|
||||||
|
```bash
|
||||||
|
python3 scripts/awx_create_job_templates.py
|
||||||
|
```
|
||||||
|
|||||||
@@ -0,0 +1,174 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
awx_create_job_templates.py
|
||||||
|
|
||||||
|
Run this AFTER committing and pushing the KNELIAC code to the git remote.
|
||||||
|
It will:
|
||||||
|
1. Trigger a project sync in AWX (pulls latest code from git)
|
||||||
|
2. Wait for the sync to complete
|
||||||
|
3. Create the two job templates (Hello World + Setup New System)
|
||||||
|
4. Create the inventory source (auto-syncs hosts from inventory/hosts.yml)
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
export AWX_PASSWORD='...'
|
||||||
|
python3 awx_create_job_templates.py
|
||||||
|
|
||||||
|
Configuration (environment variables):
|
||||||
|
AWX_URL AWX API base URL (default: http://100.91.39.53/api/v2)
|
||||||
|
AWX_USER AWX admin username (default: admin)
|
||||||
|
AWX_PASSWORD AWX admin password (REQUIRED - never committed to the repo)
|
||||||
|
AWX_PROJECT_ID, AWX_INVENTORY_ID, AWX_SCM_CRED_ID,
|
||||||
|
AWX_MACHINE_CRED_ID, AWX_ORG_ID resource IDs (sensible defaults below)
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import urllib.request, urllib.parse, json, base64, time, sys
|
||||||
|
|
||||||
|
API = os.environ.get("AWX_URL", "http://100.91.39.53/api/v2")
|
||||||
|
_AWX_USER = os.environ.get("AWX_USER", "admin")
|
||||||
|
_AWX_PASS = os.environ.get("AWX_PASSWORD")
|
||||||
|
if not _AWX_PASS:
|
||||||
|
sys.exit("ERROR: AWX_PASSWORD environment variable is not set. "
|
||||||
|
"Refusing to run — no credentials are stored in this repo.")
|
||||||
|
AUTH = base64.b64encode(f"{_AWX_USER}:{_AWX_PASS}".encode()).decode()
|
||||||
|
PROJECT_ID = int(os.environ.get("AWX_PROJECT_ID", "10"))
|
||||||
|
INVENTORY_ID = int(os.environ.get("AWX_INVENTORY_ID", "2"))
|
||||||
|
SCM_CRED_ID = int(os.environ.get("AWX_SCM_CRED_ID", "3"))
|
||||||
|
MACHINE_CRED_ID = int(os.environ.get("AWX_MACHINE_CRED_ID", "4"))
|
||||||
|
ORG_ID = int(os.environ.get("AWX_ORG_ID", "1"))
|
||||||
|
|
||||||
|
def awx_req(method, endpoint, data=None):
|
||||||
|
url = f"{API}/{endpoint}"
|
||||||
|
body = json.dumps(data).encode() if data else None
|
||||||
|
req = urllib.request.Request(url, data=body, method=method)
|
||||||
|
req.add_header("Authorization", f"Basic {AUTH}")
|
||||||
|
req.add_header("Content-Type", "application/json")
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(req, timeout=30) as resp:
|
||||||
|
raw = resp.read()
|
||||||
|
return json.loads(raw) if raw else {}
|
||||||
|
except urllib.error.HTTPError as e:
|
||||||
|
err = e.read().decode()
|
||||||
|
print(f" ERROR {e.code}: {err[:300]}")
|
||||||
|
return None
|
||||||
|
|
||||||
|
def find_or_create(resource, name, payload):
|
||||||
|
encoded = urllib.parse.quote(name)
|
||||||
|
existing = awx_req("GET", f"{resource}/?name={encoded}")
|
||||||
|
if existing and existing.get("count", 0) > 0:
|
||||||
|
item = existing["results"][0]
|
||||||
|
print(f" EXISTS: [{item['id']}] {name}")
|
||||||
|
return item
|
||||||
|
result = awx_req("POST", f"{resource}/", payload)
|
||||||
|
if result:
|
||||||
|
print(f" CREATED: [{result.get('id')}] {name}")
|
||||||
|
return result
|
||||||
|
|
||||||
|
# Step 1: Sync the project
|
||||||
|
print("=" * 60)
|
||||||
|
print("STEP 1: Syncing project (pulling latest code from git)")
|
||||||
|
print("=" * 60)
|
||||||
|
sync = awx_req("POST", f"projects/{PROJECT_ID}/update/", {})
|
||||||
|
if not sync:
|
||||||
|
print("FAILED to trigger sync")
|
||||||
|
sys.exit(1)
|
||||||
|
update_id = sync.get("id")
|
||||||
|
print(f" Sync started: project_updates/{update_id}")
|
||||||
|
print(" Waiting...", end="", flush=True)
|
||||||
|
for i in range(30):
|
||||||
|
time.sleep(3)
|
||||||
|
print(".", end="", flush=True)
|
||||||
|
status = awx_req("GET", f"project_updates/{update_id}/")
|
||||||
|
s = status.get("status") if status else "?"
|
||||||
|
if s in ("successful", "failed", "error", "canceled"):
|
||||||
|
print(f" {s}")
|
||||||
|
break
|
||||||
|
else:
|
||||||
|
print(" timeout")
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
if s != "successful":
|
||||||
|
print(f" Project sync failed: {s}")
|
||||||
|
print(" Make sure the code is committed and pushed to the git remote.")
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
# Step 2: Verify playbooks are available
|
||||||
|
print("\n" + "=" * 60)
|
||||||
|
print("STEP 2: Verifying playbooks")
|
||||||
|
print("=" * 60)
|
||||||
|
playbooks = awx_req("GET", f"projects/{PROJECT_ID}/playbooks/")
|
||||||
|
if playbooks:
|
||||||
|
print(f" Found {len(playbooks)} playbook(s):")
|
||||||
|
for p in playbooks:
|
||||||
|
print(f" - {p}")
|
||||||
|
else:
|
||||||
|
print(" No playbooks found! Push your code and re-run.")
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
# Step 3: Create inventory source
|
||||||
|
print("\n" + "=" * 60)
|
||||||
|
print("STEP 3: Creating inventory source")
|
||||||
|
print("=" * 60)
|
||||||
|
find_or_create("inventory_sources", "KNELIAC Fleet Source", {
|
||||||
|
"name": "KNELIAC Fleet Source",
|
||||||
|
"description": "Auto-synced from inventory/hosts.yml",
|
||||||
|
"inventory": INVENTORY_ID,
|
||||||
|
"source": "scm",
|
||||||
|
"source_project": PROJECT_ID,
|
||||||
|
"source_path": "inventory/hosts.yml",
|
||||||
|
"update_on_launch": True,
|
||||||
|
"overwrite": True,
|
||||||
|
"overwrite_vars": True,
|
||||||
|
})
|
||||||
|
|
||||||
|
# Step 4: Create job templates
|
||||||
|
print("\n" + "=" * 60)
|
||||||
|
print("STEP 4: Creating job templates")
|
||||||
|
print("=" * 60)
|
||||||
|
|
||||||
|
find_or_create("job_templates", "KNELIAC - Hello World", {
|
||||||
|
"name": "KNELIAC - Hello World",
|
||||||
|
"description": "AWX smoke test - verify connectivity to managed hosts",
|
||||||
|
"organization": ORG_ID,
|
||||||
|
"inventory": INVENTORY_ID,
|
||||||
|
"project": PROJECT_ID,
|
||||||
|
"playbook": "playbooks/hello_world.yml",
|
||||||
|
"credential": MACHINE_CRED_ID,
|
||||||
|
"verbosity": 1,
|
||||||
|
"forks": 5,
|
||||||
|
"ask_limit_on_launch": True,
|
||||||
|
})
|
||||||
|
|
||||||
|
find_or_create("job_templates", "KNELIAC - Setup New System", {
|
||||||
|
"name": "KNELIAC - Setup New System",
|
||||||
|
"description": "Full host provisioning (port of SetupNewSystem.sh)",
|
||||||
|
"organization": ORG_ID,
|
||||||
|
"inventory": INVENTORY_ID,
|
||||||
|
"project": PROJECT_ID,
|
||||||
|
"playbook": "playbooks/setup_new_system.yml",
|
||||||
|
"credential": MACHINE_CRED_ID,
|
||||||
|
"verbosity": 1,
|
||||||
|
"forks": 5,
|
||||||
|
"ask_limit_on_launch": True,
|
||||||
|
"ask_variables_on_launch": True,
|
||||||
|
})
|
||||||
|
|
||||||
|
print("\n" + "=" * 60)
|
||||||
|
print("ALL DONE - AWX is ready!")
|
||||||
|
print("=" * 60)
|
||||||
|
print("""
|
||||||
|
AWX URL: http://tsys-awx.knel.net (or http://100.91.39.53)
|
||||||
|
Login: $AWX_USER (password supplied via $AWX_PASSWORD env var)
|
||||||
|
|
||||||
|
Resources created:
|
||||||
|
Credentials:
|
||||||
|
[3] KNELIAC Git Credential (Source Control - for git access)
|
||||||
|
[4] KNELIAC Host SSH Key (Machine - for fleet host access)
|
||||||
|
Project:
|
||||||
|
[10] KNELIAC (git repo, auto-syncs on launch)
|
||||||
|
Inventory:
|
||||||
|
[2] KNELIAC Fleet (auto-syncs hosts from inventory/hosts.yml)
|
||||||
|
Job Templates:
|
||||||
|
- KNELIAC - Hello World (smoke test, run first!)
|
||||||
|
- KNELIAC - Setup New System (full provisioning)
|
||||||
|
""")
|
||||||
Reference in New Issue
Block a user