sysadmin horror stories
Sysadmin Horror Stories — Vol. 1 Ubuntu 24.04 & RAID1 EFI: A Comedy of Errors Platform: AMD AM5 / B650 Disks: 2× Samsung 960GB SATA SSD Mood: Progressively deteriorating A completely true account of what happens when you try to do something perfectly reasonable — installing Ubuntu Server 24.04 on a proper software RAID1 … Read more
Every piece of security infrastructure needs a real-world test. Unit tests are fine. Staging environments are fine. But nothing validates your upload scanning pipeline quite like an actual threat actor uploading a PHP webshell to your server while you’re mid-development. Allow us to introduce our tester: ~XBumbbleB33~. The Setup For context: CFM is our homegrown … Read more
CFM (Configurable Firewall Manager) started as a modern nftables-first firewall manager designed for high-security hosting and infrastructure operators. Over time, it evolved into a complete security platform: dynamic firewalling, log-driven detection, autoblocking, system hardening, notifications, DNS/GeoIP enrichment, and API integration. After introducing the Unified Web Detector (real-time vhost analytics and suspicious scoring), the next obvious … Read more
A Production-Grade, ML-Ready Traffic Analytics & Abuse-Detection Engine for Nginx, Apache & LiteSpeed CFM (Configurable Firewall Manager) started as a modern nftables-first firewall manager designed for high-security hosting and infrastructure operators.Over time, it evolved into a complete security platform: dynamic firewalling, live log-driven detection, autoblocking, system hardening, notifications, DNS/GeoIP enrichment, and API integration. Today, CFM … Read more
🧠 Building a Hybrid Anomaly Detection Engine for Network Flows 1️⃣ Background flowenricher already enriched NetFlow/IPFIX data with ASN, GeoIP, DNS, etc., and had an Isolation Forest (iForest)–based anomaly detector. We wanted to make the anomaly detection more stable, explainable, and sensitive to different attack patterns without constant retuning — so we added two complementary … Read more
Teaching FlowEnricher to Spot Weirdos: Isolation Forest Joins the Party tl;dr: We added unsupervised anomaly detection to FlowEnricher using an Isolation Forest microservice. It scores per-IP behavior in real time and helps catch stealthy port scans and low-and-slow DoS bursts that signatures miss. Yeap, Machine Learning in netflows. Why Isolation Forest? Rule engines are great … Read more
When CSF (ConfigServer Firewall) announced it was closing, it left a big hole in the Linux hosting world.CSF had been the de-facto standard for years — a reliable mix of iptables, Perl scripts, and clever wrappers that kept countless servers secure. But for those of us who’ve been running modern systems, one question was already … Read more
It all started, as it often does, with a simple hosting setup on dedicated servers, hosting customers on the usual suspects: Hetzner, OVH, and the like. As things grew, the natural evolution was to dive deeper into infrastructure — first a few racks in a datacenter (Nova), then IPs and circuits from a provider, and … Read more
server { listen 80; server_name kibana; error_log /var/log/nginx/kibana.error.log; access_log /var/log/nginx/kibana.access.log; location / { rewrite ^/(.*) /$1 break; proxy_ignore_client_abort on; proxy_pass http://localhost:5601; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; } }
No need to rescue / reboot or anything. yum install cloud-utils-growpart growpart /dev/sda 1 resize2fs /dev/sda1 resize2fs 1.46.5 (30-Dec-2021) Filesystem at /dev/sda1 is mounted on /; on-line resizing required old_desc_blocks = 16, new_desc_blocks = 28 The filesystem on /dev/sda1 is now 58719995 (4k) blocks long.