From bbbfa9fb9006bb69d3cca8f301b8ac8cd4faf54a Mon Sep 17 00:00:00 2001 From: reachableceo Date: Sun, 6 Sep 2026 14:03:35 -0500 Subject: [PATCH] SOC: Wazuh alert tuning + #823-pattern detections 16 level/frequency overrides (overwrite semantics documented), new 100500/100501/100510 WordPress-compromise rules, tuning doc with before/after numbers. Zero L7+ email after deploy. https://projects.knownelement.com/issues/825#note-5008 --- docs/wazuh-tuning.md | 95 ++++++++++++++++++++++++ rules/local_rules.xml | 168 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 263 insertions(+) create mode 100644 docs/wazuh-tuning.md create mode 100644 rules/local_rules.xml diff --git a/docs/wazuh-tuning.md b/docs/wazuh-tuning.md new file mode 100644 index 0000000..f93ba2d --- /dev/null +++ b/docs/wazuh-tuning.md @@ -0,0 +1,95 @@ +# Wazuh Alert Tuning — KNEL SOC + +**Date:** 2026-09-06 · **Ticket:** [Redmine #825](https://projects.knownelement.com/issues/825) · **Change:** GLPI CR 23 +**Applied on:** tsys-siem (Wazuh manager 4.14.1), `/var/ossec/etc/rules/local_rules.xml` +**Discourse:** https://community.turnsys.com/t/soc-security-operations-lane-wazuh-tuning-823-follow-through-soc-team/340 +**Rollback:** `cp /var/ossec/etc/rules/local_rules.xml.bak-20260906-soc /var/ossec/etc/rules/local_rules.xml && systemctl restart wazuh-manager` + +## Why + +The manager emails every alert of level >= 7 to coo@turnsys.com (max 12/hour). +September 2026 through 09-06: **664k alerts this month** (518k from tsys-cloudron-new +web 400s alone, ingested via journald = every Cloudron app's container stdout), and a +permanent L7+ email storm: scanner-storm composites (31151 @ L10), CVE backlog refires +(23505 High @ L10, 23506 Critical @ L13, 23504 Medium @ L7 — the whole backlog re-fires +every scan), SCA failures (19007 @ L7), ssh/PAM fail amplifiers (2502 @ L10, 5551 @ L10), +dpkg events (2902/2904 @ L7), generic syscheck churn (550 @ L7). + +Design rule going forward: **email (L7+) is reserved for genuine incidents.** Everything +else lives in dashboards and the SOC weekly loop (triage notes on #825, OpenVAS/GLPI +tickets for vulnerabilities, hardening backlog for SCA). + +## Mechanism note (learned the hard way) + +Wazuh 4.14 rule `overwrite="yes"` in local_rules.xml **REPLACES the original rule +definition — it is not a merge.** An override with only a level would match *every* +event (observed: rule 2502 flagging kernel i2c lines at 1000/min). Therefore every +override below carries the FULL original rule body with only level/frequency/timeframe +changed. Also: nested `` elements are invalid in rule files — single root group +only. Level-0 rules do not index but still count as "fired" for frequency composites +(verified: 31151 keeps firing over the L0 31101 parent). + +## Changes + +| Rule | Meaning | Was | Now | Rationale | +|------|---------|-----|-----|-----------| +| 31101 | web 400 per-line | L5 | **L0** | 492k/mo bot noise; raw logs stay in journald | +| 31120 / 31122 | web 500 per-line | L5 | **L3** | real app errors, dashboard-only | +| 31151 | web 400 storm composite | L10 freq 14/90s | **L6 freq 40/240s** | sustained scans only, no email | +| 5501 / 5502 | PAM session open/close | L3 | **L0** | session accounting | +| 23502 / 23507 | CVE solved / cleared | L3 | **L0** | lifecycle info | +| 23503 | CVE low | L5 | **L3** | | +| 23504 | CVE medium/untriaged | L7 | **L4** | was emailing; backlog refire | +| 23505 | CVE high | L10 | **L6** | vulns = weekly OpenVAS/GLPI loop, not email | +| 23506 | CVE critical | L13 | **L6** | same — backlog refire storm made email useless | +| 19007 | SCA check failed | L7 | **L5** | hardening backlog | +| 2902 / 2904 | dpkg installed / half-configured | L7 | **L3** | ops inventory, not incidents | +| 550 | syscheck checksum changed (generic) | L7 | **L5** | churn; high-signal subcases below | +| 2502 | "missed password >once" | L10 | **L5** | dup of 5503/5760 | +| 5551 | PAM multiple failed logins | L10 | **L6** | 2.3k/mo on routine churn | +| 31510 | CMS brute force | L8 freq 8/30s | **L6 freq 20/120s** | bot storms | +| 40704 | systemd unit failed | L5 | **L3** | 145k/mo churn; root-cause dig tracked on #825 | + +**Kept at L7+ deliberately:** 5712 (sshd brute, L10), 5763 (sshd max-auth), 31152/31153 +(401/403 storms, L10), 31162 (suspicious URI, L10), plus all auth *failures* at L5 +(5710/5760/5503) — cheap and real now that SSH is tailnet-only (#769). + +## New detections (the #823 pattern) + +The July 2026 ezeda.org compromise ([Redmine #823](https://projects.knownelement.com/issues/823)) +persisted via wp-config.php backdoor + 41 webshells in uploads. New rules + Cloudron +agent syscheck watchers catch a repeat: + +| Rule | Level | Fires when | +|------|-------|------------| +| 100500 | **L12 (email)** | any `wp-config.php` under Cloudron app data changes (syscheck) | +| 100501 | **L10 (email)** | a `.php` file appears/changes inside any WP `uploads/` dir (syscheck) | +| 100510 | L8 | web request reaches `.php` inside an `uploads/` path (access-log) | + +Agent 001 (tsys-cloudron-new) `ossec.conf` additions: syscheck +`/home/yellowtent/appsdata/*/data/wp-config.php` + `*/data/wp-content/uploads` +(frequency 12h); quarantine dirs excluded (`#823` evidence holds known malware). + +## Verification (2026-09-06, clean window after deploy) + +| Metric | Before (per hour) | After (per hour) | +|--------|-------------------|------------------| +| Total alerts | ~19,000+ | ~2,500 | +| L5 web-400 flood | ~25,000 | **0** | +| 2502 misfire flood | ~60,000 at peak | **0** (7 per 10 min = genuine) | +| L7+ (email to coo@) | constant (12/h cap hit) | **0** | +| L10 scanner storms (31151) | continuous | L6, sustained scans only | + +Remaining L3 volume = systemd churn (40704) + apparmor denials (52002) + web 500s + +sshd successes: visible for triage, below every notification path. + +## Week-loop plan (adjustment cadence) + +1. Daily: triage new L6+ in the Wazuh dashboard; note dispositions on #825. +2. Watch 5710/5760/5503 — should stay near-zero with tailnet-only SSH; a spike = someone + knocking from inside the tailnet. +3. 100500/100501/100510 must stay at zero; any fire = possible #823 recurrence → incident. +4. Investigate 40704 root cause on tsys-siem (which unit churns) as an ops item. +5. Re-check email volume at coo@ weekly; tune further only on evidence. +6. Vulnerability backlog: OpenVAS scans (GLPI-fed targets) → tickets per the triage + standard; Wazuh vuln alerts feed the same weekly review. diff --git a/rules/local_rules.xml b/rules/local_rules.xml new file mode 100644 index 0000000..9ccf66c --- /dev/null +++ b/rules/local_rules.xml @@ -0,0 +1,168 @@ + + + + + + + + 31100 + ^4 + Web server 400 error code. + attack,pci_dss_6.5,pci_dss_11.4,gdpr_IV_35.7.d,nist_800_53_SA.11,nist_800_53_SI.4,tsc_CC6.6,tsc_CC7.1,tsc_CC8.1,tsc_CC6.1,tsc_CC6.8,tsc_CC7.2,tsc_CC7.3, + + + 31100 + ^50 + Web server 500 error code (server error). + + + 31120 + ^500 + Web server 500 error code (Internal Error). + system_error, + + + 31101 + + Multiple web server 400 error codes + from same source ip. + + T1595.002 + + web_scan,recon,pci_dss_6.5,pci_dss_11.4,gdpr_IV_35.7.d,nist_800_53_SA.11,nist_800_53_SI.4,tsc_CC6.6,tsc_CC7.1,tsc_CC8.1,tsc_CC6.1,tsc_CC6.8,tsc_CC7.2,tsc_CC7.3, + + + 5500 + session opened for user + PAM: Login session opened. + + T1078 + + authentication_success,pci_dss_10.2.5,gpg13_7.8,gpg13_7.9,gdpr_IV_32.2,hipaa_164.312.b,nist_800_53_AU.14,nist_800_53_AC.7,tsc_CC6.8,tsc_CC7.2,tsc_CC7.3, + + + 5500 + session closed for user + PAM: Login session closed. + pci_dss_10.2.5,gpg13_7.8,gpg13_7.9,gdpr_IV_32.2,hipaa_164.312.b,nist_800_53_AU.14,nist_800_53_AC.7,tsc_CC6.8,tsc_CC7.2,tsc_CC7.3, + + + 23501 + no_full_log + Solved + The $(vulnerability.cve) that affected $(vulnerability.package.name) was solved due to an update in the agent or feed. + + + 23501 + no_full_log + Active + Low + $(vulnerability.cve) affects $(vulnerability.package.name) + + + 23501 + no_full_log + Active + Medium|Untriaged + $(vulnerability.cve) affects $(vulnerability.package.name) + + + 23501 + no_full_log + Active + High + $(vulnerability.cve) affects $(vulnerability.package.name) + + + 23501 + no_full_log + Active + Critical + $(vulnerability.cve) affects $(vulnerability.package.name) + + + json + no_full_log + Clear + Vulnerabilities cleared + + + 19006 + ^failed + no_full_log + $(sca.policy): $(sca.check.title) + + + 2900 + ^status installed$ + New dpkg (Debian Package) installed. + config_changed,pci_dss_10.6.1,pci_dss_10.2.7,gpg13_4.10,gdpr_IV_35.7.d,hipaa_164.312.b,nist_800_53_AU.6,nist_800_53_AU.14,tsc_CC7.2,tsc_CC7.3,tsc_CC6.8,tsc_CC8.1, + + + 2900 + ^status half-configured$ + Dpkg (Debian Package) half configured. + config_changed,pci_dss_10.6.1,pci_dss_10.2.7,gpg13_4.10,gdpr_IV_35.7.d,hipaa_164.312.b,nist_800_53_AU.6,nist_800_53_AU.14,tsc_CC7.2,tsc_CC7.3,tsc_CC6.8,tsc_CC8.1, + + + ossec + syscheck_integrity_changed + Integrity checksum changed. + + T1565.001 + + syscheck,syscheck_entry_modified,syscheck_file,pci_dss_11.5,gpg13_4.11,gdpr_II_5.1.f,hipaa_164.312.c.1,hipaa_164.312.c.2,nist_800_53_SI.7,tsc_PI1.4,tsc_PI1.5,tsc_CC6.1,tsc_CC6.8,tsc_CC7.2,tsc_CC7.3, + + + more authentication failures;|REPEATED login failures + syslog: User missed the password more than one time + + T1110 + + authentication_failed,pci_dss_10.2.4,pci_dss_10.2.5,gpg13_7.8,gdpr_IV_35.7.d,gdpr_IV_32.2,hipaa_164.312.b,nist_800_53_AU.14,nist_800_53_AC.7,tsc_CC6.1,tsc_CC6.8,tsc_CC7.2,tsc_CC7.3, + + + 5503 + + PAM: Multiple failed logins in a small period of time. + + T1110 + + authentication_failures,pci_dss_10.2.4,pci_dss_10.2.5,pci_dss_11.4,gpg13_7.8,gdpr_IV_35.7.d,gdpr_IV_32.2,hipaa_164.312.b,nist_800_53_AU.14,nist_800_53_AC.7,nist_800_53_SI.4,tsc_CC6.1,tsc_CC6.8,tsc_CC7.2,tsc_CC7.3, + + + 31509 + + CMS (WordPress or Joomla) brute force attempt. + + T1110 + + pci_dss_6.5,pci_dss_11.4,pci_dss_6.5.10,pci_dss_10.2.4,pci_dss_10.2.5,gdpr_IV_35.7.d,gdpr_IV_32.2,hipaa_164.312.b,nist_800_53_SA.11,nist_800_53_SI.4,nist_800_53_AU.14,nist_800_53_AC.7,tsc_CC6.6,tsc_CC7.1,tsc_CC8.1,tsc_CC6.1,tsc_CC6.8,tsc_CC7.2,tsc_CC7.3, + + + 40700 + status=1/FAILURE + Systemd: Service exited due to a failure. + gpg13_4.3,gdpr_IV_35.7.d, + + + + + syscheck + wp-config\.php$ + Cloudron: wp-config.php changed - #823-pattern persistence vector (backdoor was injected here). Verify the change is sanctioned. + T1505.003 + + + syscheck + uploads/.*\.php$ + Cloudron: PHP file in uploads dir - #823-pattern webshell staging (41 webshells found there 2026-09-06). + T1505.003 + + + 31100, 31101, 31103, 31104, 31106, 31108, 31109, 31110, 31115, 31120, 31121, 31122, 31123 + uploads/.*\.php + Cloudron: web request to PHP inside uploads - possible webshell access (#823 pattern). + T1505.003 + +