Locking unmanaged tenants to their IP: anti-spoofing for dedicated servers and Proxmox VMs (MikroTik, Cisco Nexus, Proxmox)

by

If you rent out unmanaged dedicated servers or VMs on a shared subnet, you give root to people you don’t control. Sooner or later one of them will add a second IP “just to test”, clone a config with someone else’s address, or install something that starts answering DHCP or sending IPv6 Router Advertisements. On a shared L2 segment, any of those can break other customers.

This post is the setup we run for exactly that: one tenant, one MAC, one IP (plus its IPv6), enforced at the edge and backed up at the router. It covers three cases — the router (MikroTik), bare metal behind a switch (Cisco Nexus), and VMs on Proxmox VE — and every snippet below was checked against the live boxes before publishing.

All addresses are documentation ranges: 203.0.113.0/24 is the shared customer subnet, 203.0.113.1 the gateway, 2001:db8:1:1::/64 the IPv6 on-link prefix.

The threat model

  • Extra / arbitrary IPs — tenant adds 203.0.113.90 next to its own .39, or takes a free address that belongs to the next customer.
  • MAC changes — which silently defeats any rule that trusts the MAC.
  • ARP / NDP spoofing — claiming the gateway’s or a neighbour’s address.
  • Rogue DHCP server — hands out leases to the whole segment.
  • Rogue IPv6 RA — the nastiest one: every host with SLAAC on the segment gets a new default gateway.

The key insight: the router only sees what gets routed. ARP poisoning between two customers, rogue DHCP, rogue RA, stealing a free IP on the same /24 — the router never sees most of it. Enforcement has to live at the edge (switch port or hypervisor tap). The router is the second line.

Layer Bare metal Proxmox VM
Edge: lock the MAC Switch port-security PVE macfilter
Edge: lock IP / ARP / NDP — (port-security is L2 only) PVE ipfilter
Edge: block rogue DHCP / RA DHCP snooping (not covered here) PVE rules + radv: 0
Router: lock IP given the MAC MikroTik raw rule — required MikroTik raw rule — backstop

1. MikroTik: one raw rule per tenant

/ip firewall raw
add action=drop chain=prerouting place-before=0 in-interface=bridgeMain \
    src-mac-address=AA:BB:CC:00:00:39 src-address=!203.0.113.39 \
    comment="tenant-39 lock"

“Anything coming from this MAC that doesn’t carry this source IP gets dropped.” Why raw/prerouting and not filter:

  • One rule instead of two. In filter you need one rule in input (traffic to the router) and one in forward (traffic through it). Raw prerouting runs before that routing decision, so one rule covers both. Our first version had exactly those two filter rules.
  • Before conntrack and fasttrack. The spoofed packet is dropped before it creates a connection entry, and fasttrack can’t skip it.
  • Rule ordering can’t bite you. Our old filter drops sat below broad accepts such as accept chain=forward src-address=10.0.0.0/8. A tenant spoofing a source from that range would hit the accept first and never reach the drop. In raw, place-before=0 puts the lock at the top.

Optionally pin the ARP entry as well, so nobody else can claim the tenant’s IP towards the router:

/ip arp add address=203.0.113.39 interface=bridgeMain mac-address=AA:BB:CC:00:00:39

What the static ARP entry does not do: with the bridge at the default arp=enabled, the router still learns any other IP the tenant announces. Closing that for the whole subnet means arp=reply-only on the bridge plus add-arp=yes on the DHCP server and static entries for every static host. That’s a real migration with real outage risk, so inventory every host before you try it.

The limitation you must understand

This rule keys on the MAC. If the tenant can change its MAC, the rule no longer matches and does nothing. On its own, the router rule is trivially bypassable. It only becomes a real lock when something at the edge pins the MAC — a switch port or the hypervisor.

2. Bare metal: Cisco Nexus port-security

For a dedicated server on its own access port, pin the one MAC that may use the port:

feature port-security

interface Ethernet1/11
  description TENANT-39-eno1
  switchport access vlan 10
  switchport port-security
  switchport port-security mac-address AABB.CC00.0039
  switchport port-security violation restrict
  • violation restrict drops frames from any other MAC and counts them, without err-disabling the port. shutdown is stricter but means a site visit or a manual no shut every time someone experiments.
  • Port-security is L2 only. It knows nothing about IP addresses. A tenant keeping its MAC can still add a second IP. That’s why the MikroTik rule in section 1 is mandatory for bare metal: port pins the MAC, router pins the IP to that MAC. Neither one alone is enough.
  • Don’t do this on a trunk to a hypervisor. That port carries the MACs of every VM on the host; port-security there means either a huge maximum (useless) or violations that take down the wrong customers whenever you create or migrate a VM. VMs get locked at the tap instead — section 3.

Going further: DHCP snooping, Dynamic ARP Inspection and IP Source Guard would give you IP enforcement on the switch itself, and isolated private VLANs would stop tenant-to-tenant L2 traffic entirely. We haven’t deployed those in production yet, so they’re out of scope for a “tested” guide.

3. Proxmox VE: lock the VM at its tap

For a VM, the equivalent of “its own switch port” is its tap interface, and the Proxmox firewall can filter MAC and IP and ARP/NDP there — more than port-security gives you on metal. Tested on PVE 9.1 with the default iptables/ebtables backend.

Step 0 — audit before touching anything

pve-firewall status
pvecm status                       # are you in a cluster?
ls -l /etc/pve/firewall/           # leftover .fw files?
grep -H 'firewall=1' /etc/pve/qemu-server/*.conf /etc/pve/lxc/*.conf
pve-firewall compile > /root/fw-before.txt

Most guests created in the GUI already have firewall=1 on their NIC. That’s harmless while the firewall is off — and exactly what wakes up when you turn it on. Look for stale <vmid>.fw files too: we found an old one with policy_in: DROP that would have started blocking a VM on the other node.

Step 1 — enable the datacenter firewall, neutrally

/etc/pve/firewall/cluster.fw is cluster-wide: turning it on for one node turns it on for all of them. Keep the hosts themselves unfiltered first, then flip the switch with ACCEPT policies:

# for EVERY node:
printf '[OPTIONS]\nenable: 0\n' > /etc/pve/nodes/<node>/host.fw

# then:
printf '[OPTIONS]\nenable: 1\npolicy_in: ACCEPT\npolicy_out: ACCEPT\n' > /etc/pve/firewall/cluster.fw

pve-firewall compile > /root/fw-after.txt
diff /root/fw-before.txt /root/fw-after.txt

In the diff you should only see helper chains (PVEFW-Drop, PVEFW-smurfs, …) and a bridge FORWARD that accepts IPv4/IPv6. No per-VM chains — guests without a .fw file have their own firewall disabled by default, so this step is a no-op for them. If you see a DROP aimed at a specific VM, stop.

Know your emergency brake before you start. It’s local and works even if cluster quorum is lost and /etc/pve is read-only:

pve-firewall stop

Step 2 — find what the VM really uses

No guest agent needed. Ask the network what the VM’s MAC is claiming. On the MikroTik:

/ip arp print where mac-address="BC:24:11:00:00:81"
/ipv6 neighbor print where mac-address="BC:24:11:00:00:81"

Bonus: if a second IPv4 already shows up here, the tenant has already added one. Look closely at the IPv6 result — including the link-local addresses. See the gotchas below.

Step 3 — the per-VM file

# /etc/pve/firewall/<vmid>.fw
[OPTIONS]
enable: 1
policy_in: ACCEPT
policy_out: ACCEPT
macfilter: 1
ipfilter: 1
dhcp: 1
ndp: 1
radv: 0

[RULES]
OUT DROP -p udp -dport 68 -sport 67 -log nolog

[IPSET ipfilter-net0]
203.0.113.81
2001:db8:1:1:be24:11ff:fe00:81
  • policy_*: ACCEPT — we are not firewalling the customer’s services, only preventing spoofing. That’s what you want for unmanaged.
  • macfilter: 1 — frames with any other source MAC are dropped (it’s the default, but be explicit).
  • ipfilter: 1 + ipfilter-net0 — only these source IPs may leave net0. It also filters ARP (--arp-ip-src), so no ARP spoofing.
  • dhcp: 1 — required if the VM gets its address from DHCP; otherwise the DHCPDISCOVER (source 0.0.0.0) is dropped by ipfilter.
  • radv: 0 — the VM may not send Router Advertisements. This single line protects the IPv6 default gateway of every host on the segment.
  • The OUT DROP rule blocks a rogue DHCP server inside the VM.

There is no “save now, activate later”. The pve-firewall daemon notices the file and applies it within about ten seconds. Write it only when you’re ready to verify. Never enable ipfilter with an empty or incomplete ipset — you cut the VM off.

The NIC needs firewall=1 (net0: virtio=…,bridge=vmbr0,firewall=1). If it’s already there, you get zero downtime. If you have to add it on a running VM (qm set <vmid> -net0 …,firewall=1), Proxmox inserts the fwbr bridge in front of the tap live; in our case the VM didn’t even drop a ping.

Step 4 — verify what actually got installed

ebtables-save  | grep -i tap<vmid>i0
iptables-save  | grep -i tap<vmid>i0-OUT
ip6tables-save | grep -i tap<vmid>i0-OUT
for s in $(ipset list -n | grep PVEFW); do echo "== $s"; ipset list $s | sed -n '/Members/,$p'; done
ip6tables -L tap<vmid>i0-OUT -vn     # watch the DROP counters

What we saw on the live host (trimmed):

# ebtables — MAC and ARP lock
-A tapNi0-OUT -s ! bc:24:11:0:0:81 -j DROP
-A tapNi0-OUT-ARP -p ARP --arp-ip-src 203.0.113.81 -j RETURN
-A tapNi0-OUT-ARP -j DROP

# iptables — IPv4 source lock + rogue DHCP
-A tapNi0-OUT -m mac ! --mac-source bc:24:11:00:00:81 -j DROP
-A tapNi0-OUT -m set ! --match-set PVEFW-N-ipfilter-net0-v4 src -j DROP
-A tapNi0-OUT -p udp -m udp --sport 67 --dport 68 -j DROP

# ip6tables — rogue RA dead
-A tapNi0-OUT -p ipv6-icmp -m icmp6 --icmpv6-type 134 -j DROP

# IPv6 ipset
2001:db8:1:1:be24:11ff:fe00:81
fe80::/10 nomatch
fe80::be24:11ff:fe00:81

Note the trick in the IPv6 set: fe80::/10 nomatch excludes all link-local addresses, then the VM’s own link-local is allowed back. The VM can’t impersonate another host’s link-local, so no NDP spoofing. But Proxmox only adds the EUI-64 link-local derived from the MAC — see the gotchas.

Then test from the VM’s console (not SSH, you’ll lock yourself out):

ip addr add 203.0.113.90/24 dev eth0
ping -c3 -I 203.0.113.90 203.0.113.1        # must FAIL
ip addr del 203.0.113.90/24 dev eth0
ping -c3 203.0.113.1                        # must work

“But what about eth0:1?” — an alias is just another address on the same NIC, same MAC, same tap. It hits the same ipset rule and its ARP never leaves the host.

Finally, add the MikroTik raw rule from section 1 for the VM’s MAC. It adds no new check, but it’s a second, independent trust domain: it still catches spoofed IPs if the Proxmox layer is ever lost — as long as the MAC stays pinned.

Gotchas we actually hit

IPv6 breaks silently

Our first ipset only had the IPv4 address. IPv4 kept working, so everything looked fine — but the VM’s global SLAAC address had just been cut off. Always check /ipv6 neighbor before writing the ipset. Also: our hypervisor had no IPv6 of its own, so ping6 from it said “Network is unreachable”, which proves nothing about the VM. Test IPv6 from the router.

EUI-64 today, privacy extensions tomorrow

An EUI-64 address (…be24:11ff:fe…) is derived from the MAC and stable. But on the next VM we locked (a fresh Debian 13 install), the router’s neighbour table showed both the EUI-64 and a random-looking 2001:db8:1:1:550f:902a:… — RFC 4941 / RFC 7217 privacy and stable-privacy addresses, the default on many modern distros and on Windows. Temporary addresses rotate, and a fixed ipset will cut them. Your options:

  • Put the whole shared /64 in the ipset (2001:db8:1:1::/64): never breaks and keeps the VM inside your prefix, but it can still spoof a neighbour’s IPv6. This is what we use for such VMs today.
  • Give every tenant its own /64: strict and robust. With a /48 you have 65,536 of them; the cost is router work, not address space. This is where we’re heading.

The link-local that isn’t EUI-64

The same stable-privacy setting also changes the link-local address (fe80::2c57:c702:… instead of fe80::be24:11ff:fe…). Proxmox auto-allows only the EUI-64 one, and fe80::/10 nomatch drops everything else. Right after enabling the filter we saw the IPv6 DROP counter tick up. Link-local is what the VM uses to talk NDP to the gateway, so once the neighbour cache expires, IPv6 would quietly die. The fix is to add that exact link-local to the ipset:

[IPSET ipfilter-net0]
203.0.113.67
2001:db8:1:1::/64
fe80::2c57:c702:2b10:8c77

Don’t add fe80::/10 — that would reopen link-local spoofing. A stable-privacy link-local is stable for that interface, so a single entry is enough (unless the customer reinstalls).

A router rule alone is not a lock

While re-checking everything for this post, we found a newer VM protected only by the MikroTik raw rule: its NIC had firewall=0 and no .fw file. It looked locked in the router config, but as explained in section 1, a MAC-keyed rule is bypassed by changing the MAC. Every tenant needs the edge layer.

Destroying a VM deletes its firewall file

When a VM is destroyed, Proxmox removes /etc/pve/firewall/<vmid>.fw with it — but the matching MikroTik raw rule and static ARP entry stay behind. Add “remove router lock” to your decommission checklist, or a new customer who inherits that IP will get a surprise. Conversely, the protection is not automatic for new VMs: it’s a provisioning step, so put it in your provisioning script.

Smaller ones

  • DAD probes: ARP probes and Neighbor Solicitations with an unspecified source (0.0.0.0 / ::) are dropped by ipfilter. Harmless in practice; if a customer reports odd DHCP or “tentative” IPv6 behaviour, you know where to look.
  • Static IPs are fine: ipfilter checks the source on the wire and doesn’t care whether the address came from DHCP or /etc/network/interfaces. When renumbering, add the new IP first, switch inside the VM, then remove the old one.
  • Things that will break with macfilter: keepalived/VRRP virtual MACs, nested hypervisors, Docker/LXC macvlan inside the VM, and VMs that act as routers or bridges. Exempt them deliberately.
  • Silent failure: the lock disappears if someone restores a VM from backup without its .fw, flips firewall=0, or moves it to a node where the cluster firewall is off. Monitor it.

Checklist

  Bare metal Proxmox VM
Pin MAC at the edge port-security on the access port firewall=1 on NIC + macfilter: 1
Pin IP (v4 + v6) at the edge — ipfilter: 1 + ipset with every address it really uses, including a non-EUI-64 link-local
Block rogue DHCP / RA (DHCP snooping — future) OUT DROP 67→68 + radv: 0
Router lock raw rule — mandatory raw rule — backstop
Static ARP optional optional
Decommission remove port-security MAC + raw rule remove raw rule + ARP (the .fw goes with the VM)

None of this replaces proper tenant isolation (per-customer VLANs or isolated PVLANs). But on an existing shared /24, it closes the practical abuse cases in an evening, with no downtime for anyone else.