What is it about

The following short howto describes the setup of a xen installation with the following properties:

• the host domU is isolated from the domUs
• the domUs share a bridged network interface
• a firewall in dom1 manages the traffic from domUs to the internet

What is the problem?

Maybe you would like to accomplish the previously described setup by using dom0 as the routing firewall?

It could look like the following:
(dia source: xen-network-not-working.dia)

I tried hard to get this setup running, but a problem of iptables with the xen network bridge rendered this approach impossible.

What went wrong?

• the traffic originating from the bridge xenbr0 should get forwared to eth in dom0

• the problem: the masquerading iptables rules failed to change the source address of the packets - thus the packets had an invalid return path

Symptoms

• pings from domUs reach the external router, but the source address is 10.0.1.x instead of 192.168.1.2 (as it should be)

Root cause

• the packets are treated as IN=xenbr0 (which is ok) and OUT=xenbr0 (this should be eth0 instead)

• despite this wrong attributes, the packets still get sent via eth0 (as they should)

• even more mysterious: after adding a masqerading rule that would also match packets marked as OUT=xenbr0, the packets do not even reach eth0 anymore

Other reports

In the shorewall howtos you can find the following:

I know of no case where a user has successfully used NAT (including Masquerade) in a bridged Xen Dom0. So if you want to create a masquerading firewall/gateway using Xen, you need to do so in a DomU (see how I did it) or you must configure Xen to use routing  or NAT rather than the default bridging.

Explanation

See: http://www.woitasen.com.ar/2011/09/confusion-using-iptables-nat-and-bridge/

• Problem: packets don’t cross nat table twice.

• Solution 1: echo 0 > /proc/sys/net/bridge/bridge-nf-call-iptables

• Solution 2: iptables -t raw -I PREROUTING -i BRIDGE -s x.x.x.x -j NOTRACK

How it can be done

The following picture describes a setup that avoids the previously described problem, since there are no bridges configured in dom1.

(dia source: xen-network-two-bridges.dia)

The configuration details (xen and shorewall configuration files) will follow soon ...

Links

• xen network overview: http://wiki.xensource.com/xenwiki/XenNetworking

• shorewall firewall setup for xen: http://www.shorewall.net/Xen.html

• using multiple bridges for separate domU networks: http://www.debian-administration.org/articles/470

• bridged firewall in domU: http://www.shorewall.net/XenMyWay.html (hard to reproduce)

• routed firewall in dom0: http://www.shorewall.net/XenMyWay-Routed.html (hard to reproduce)