CFM Web Detector: Challenge Engine — a Major Step Toward Hosting-Grade HTTP Mitigation

by

CFM (Configurable Firewall Manager) started as a modern nftables-first firewall manager designed for high-security hosting and infrastructure operators.

Over time, it evolved into a complete security platform: dynamic firewalling, log-driven detection, autoblocking, system hardening, notifications, DNS/GeoIP enrichment, and API integration.

After introducing the Unified Web Detector (real-time vhost analytics and suspicious scoring), the next obvious question was:

“How do we *act* on HTTP abuse in a way that’s fast, safe, and works in real hosting stacks?”

This update answers that with three big additions:

✅ A built-in Challenge Server (HTTP + HTTPS)

✅ Challenge abuse protection (bots that attack the challenge itself)

✅ OpenResty in-path mode (challenge without per-IP DNAT rules)

This post explains what’s new, how it works, and how to deploy it.

☑️ In Short — What’s New?

🔥 1) The Challenge Engine

Web Detector can now *route suspicious clients through a browser challenge* instead of hard-blocking immediately.

This makes a huge difference for:

  • false-positive protection (shared hosting is noisy)
  • “stubborn” scanners that keep changing IPs
  • mixed traffic patterns (humans + bots)
  • edge cases where rate/ratio looks bot-like but isn’t

🧱 2) Hard-Block Signals (Deterministic)

Alongside challenge, CFM now supports explicit hard-block thresholds (for when challenge is pointless).

Examples:

  • extreme `404` storms
  • `403` storms
  • suspicious User-Agents (“python-requests”, “spider”, etc.)
  • 40x combos + high unique-path probing (classic scanner behavior)
  • malpath hits from known exploit sweeps

🧩 3) OpenResty In-Path Mode (No DNAT)

This is the real architectural jump.

Instead of managing per-IP nftables redirect/DNAT rules for challenge, CFM can push decisions to OpenResty through a Unix socket.

OpenResty then:

  • allows legitimate traffic
  • challenges suspicious browsers *directly*
  • blocks what must be blocked

No per-IP nft redirect churn. No “DNAT per attacker”. Just in-path enforcement.

🔥 The Challenge Server (HTTP + HTTPS)

CFM now includes an embedded challenge server with dedicated listeners:

CHALLENGE_HTTP_LISTEN  = 127.0.0.1:9098
CHALLENGE_HTTPS_LISTEN = 127.0.0.1:9099
CHALLENGE_ACCESS_LOG   = /var/log/cfm/challenge.access.log

CHALLENGE_LOG     = 1
CHALLENGE_NOTIFY  = 0

CHALLENGE_COOLDOWN     = 15m
CHALLENGE_COOKIE_LIFE  = 15m
OPENRESTY_OK_IP_TTL    = 15m

Key idea:

  • **Challenge is not a block**
  • It’s a “slow down + prove you’re a browser” gate

You get the best of both worlds:

  • bots suffer
  • humans pass and continue normally

🧨 Protecting the Challenge Itself (Challenge Abuse)

One of the first things you notice when you deploy a challenge endpoint:

bots start attacking the challenge server too.

So CFM now includes a dedicated protection layer that tracks repeated “bad” challenge outcomes and blocks abusers:

CHALLENGE_ABUSE_ENABLED    = 1
CHALLENGE_ABUSE_WINDOW     = 10s
CHALLENGE_ABUSE_BAD_N      = 15
CHALLENGE_ABUSE_BLOCK_TTL  = 1h
CHALLENGE_ABUSE_COOLDOWN   = 30m

This prevents:

  • PoW spammers
  • cookie bypass loops
  • scripted hammering
  • challenge endpoint brute behavior

And it keeps the challenge server *stable under pressure*.

🧱 Hard-Block Triggers (When “Challenge” Is a Waste)

Challenge is ideal for “maybe-bot / maybe-human” cases.

But scanners do very deterministic things. For those, CFM now supports explicit thresholds:

IP404_COUNT = 220
IP403_COUNT = 120

AGENT_LIST  = "python-requests,spider"
AGENT_COUNT = 25

IP40X_COMBO        = 180
IP40X_UNIQUE_PATHS = 20
IGNORE40X_PREFIXES = "/.well-known/,/robots.txt,/favicon.ico,/sitemap"

BLOCK          = 1h
BLOCK_COOLDOWN = 30m

This is especially important for hosting stacks where you see:

  • WordPress plugin exploit sweeps
  • random /wp-admin/ /xmlrpc.php / .env scanning
  • mass URL bruteforce
  • “inventory” scans across thousands of sites

Hard-block is fast, cheap, and accurate in these cases.

🧪 MalPath Detection: Stop Exploit Sweeps Early

CFM can now track “known bad path” sweeps from a file list:

MALPATH_COUNT = 20
MALPATH_FILE  = /etc/cfm/webdetector_malpaths.txt

This is perfect for:

  • recurring CVE spray paths
  • common webshell patterns
  • known botnet probe URLs

…and because it’s file-driven, you can update it instantly without code changes.

🏷 Per-Vhost Challenge Controls (Manual + Automatic)

Shared hosting has one painful truth:

some vhosts need extra protection sometimes, and others must *never* be challenged.

CFM now supports both:

Manual “panic mode” per-vhost

CHALLENGE_VHOST = victim.com, *.victim.com, www.nixpal.com

Ignore list for manual mode

CHALLENGE_VHOST_IGNORE = api.mybank.gr

Absolute bypass (wins over all triggers)

CHALLENGE_HOST_BYPASS = api.mybank.gr, health.victim.com, *.internal.victim.com

That last one is extremely important for:

  • API endpoints
  • health checks
  • internal services
  • monitoring probes
  • upstream proxies you don’t control

⚡ Auto “Under-Attack Mode” (Based on Long-Window Score)

The Web Detector already computes suspicious vhost scores.

Now you can turn that into an automatic “under attack” switch with hysteresis and holddown:

CHALLENGE_SUSPICIOUS_VHOST = 1
CHALLENGE_SUSPICIOUS_VHOST_SCORE_ON  = 0.70
CHALLENGE_SUSPICIOUS_VHOST_SCORE_OFF = 0.65
CHALLENGE_SUSPICIOUS_VHOST_MIN_UNIQIP = 80
CHALLENGE_SUSPICIOUS_VHOST_HOLDDOWN = 10m

So instead of manually reacting at 3am:

  • score rises → vhost enters challenge mode
  • score drops below OFF threshold → returns to normal
  • holddown prevents flapping

This is hosting-grade behavior.

🧩 OpenResty In-Path Mode (No Per-IP DNAT)

Traditional model:

  • CFM detects abusive IPs
  • nftables redirects those IPs to challenge server

That works. But under heavy churn, “per attacker DNAT” becomes operationally annoying.

OpenResty in-path mode replaces that with a cleaner enforcement model:

OPENRESTY_MODE  = 1
OPENRESTY_SOCK  = /var/run/cfm/cfm_nginx.sock
OPENRESTY_TOKEN = cfm

Socket API endpoints (served by CFM, consumed by OpenResty)

  • `GET /nginx/decision?ip=1.2.3.4&host=example.com`

→ {"ip_action":"allow|challenge|block","vhost_action":"allow|challenge"}

    • `POST /nginx/ip {“ip”:”1.2.3.4″,”action”:”challenge|block”,”ttl_sec”:600}`
  • `POST /nginx/ip/clear {“ip”:”1.2.3.4″}`
  • `POST /nginx/vhost {“host”:”example.com”,”action”:”challenge”,”ttl_sec”:600}`
  • `POST /nginx/vhost/clear {“host”:”example.com”}`
  • `GET /nginx/status`

All requests require:

X-CFM-Token: <OPENRESTY_TOKEN>

High-level flow

Client → OpenResty (in-path)
          |
          | query decision socket
          v
     CFM decision engine
          |
     allow / challenge / block

Result:

  • Zero per-IP DNAT churn
  • OpenResty can challenge directly
  • CFM stays the “brain”
  • OpenResty becomes the “enforcer”

🧠 Still ML-Ready (and now even more actionable)

You already had the ML-ready suspicious scoring pipeline.

Now you have something equally important:

a safe enforcement pipeline (challenge + under-attack + bypasses)

So the next phase (HBOS / eHBOS / Isolation Forest) can be introduced in observe-only first, then gradually influence decisions.

That’s the correct operational way to introduce ML in security:

deterministic first, ML second, and always with guardrails.

🏁 Final Notes

With these additions, CFM’s Web Detector is no longer “analytics only”.

It becomes a full mitigation layer:

✔ behavior-driven detection

✔ deterministic block triggers

✔ safe browser challenges

✔ challenge-abuse protection

✔ per-vhost control and bypass

✔ automatic under-attack vhost mode

✔ OpenResty in-path enforcement (no per-IP DNAT)

If you run WordPress / WooCommerce / WHM/cPanel/DirectAdmin style stacks, this is exactly the missing piece between “traffic visibility” and “traffic control”.

Post Views: 132