kvm and bridged networking. How to disable nat on the physical server

[Steve Holdoway]

I've got a bit of a problem with a setup I built for a local charity,
where I set them up with a mail server in a virtual host. Seemed to be a
good security measure at the time.

Setup - all phys/virt servers are all ubuntu 10.04 LTS.

bridging setup on physical server /etc/network/interfaces:

auto br0
iface br0 inet static
address 192.168.x.x
netmask 255.255.255.0
network 192.168.x.0
broadcast 192.168.x.255
gateway 192.168.x.y
bridge_ports eth0
bridge_stp off
bridge_fd 0
bridge_maxwait 0

As it turns out, the bridging is performing NAT on the traffic
travelling between the physical and virtual servers, which effectively
screws up the mailer configuration - making all traffic look local, and
it's ended up as an open relay. Extensive googling and reading of
manuals has not told me how ( or even if ) this Nating can be disabled.

( Mail traffic is port forwarded directly from the firewall dd-wrt ).

Can anyone here shed any light on the matter/inprove on my google foo??

Cheers,

Steve

[Hadley Rich]

Post by Steve Holdoway
I've got a bit of a problem with a setup I built for a local charity,
where I set them up with a mail server in a virtual host. Seemed to be a
good security measure at the time.

[snip]

Post by Steve Holdoway
As it turns out, the bridging is performing NAT on the traffic
travelling between the physical and virtual servers, which effectively
screws up the mailer configuration - making all traffic look local, and
it's ended up as an open relay. Extensive googling and reading of
manuals has not told me how ( or even if ) this Nating can be disabled.

I'm confused, that shouldn't perform NAT at all. I use similar setups
myself.

Does this section look famlilar from /etc/libvirt/qemu/$HOST.xml;

<interface type='bridge'>
<mac address='01:23:45:ab:cd:ef'/>
<source bridge='br0'/>
</interface>

[Steve Holdoway]

Post by Hadley Rich

Post by Steve Holdoway
I've got a bit of a problem with a setup I built for a local charity,
where I set them up with a mail server in a virtual host. Seemed to be a
good security measure at the time.

[snip]

Post by Steve Holdoway
As it turns out, the bridging is performing NAT on the traffic
travelling between the physical and virtual servers, which effectively
screws up the mailer configuration - making all traffic look local, and
it's ended up as an open relay. Extensive googling and reading of
manuals has not told me how ( or even if ) this Nating can be disabled.

I'm confused, that shouldn't perform NAT at all. I use similar setups
myself.

So am I! My mail server is a standard that I haven't changed for
probably years ( except to upgrade ) - sendmail + milter-greylist +
spamhaus/zen RBL + clamav-milter.

But it's operating as an open relay as it sees all of the incoming
traffic as coming from the ip address of the physical server the vm's
are built upon. This address is neither the default gateway, nor is used
to specifically route anything from.

Post by Hadley Rich
Does this section look famlilar from /etc/libvirt/qemu/$HOST.xml;
<interface type='bridge'>
<mac address='01:23:45:ab:cd:ef'/>
<source bridge='br0'/>
</interface>

<interface type='bridge'>
<mac address='01:23:45:ab:cd:ef'/>
<source bridge='br0'/>
<model type='virtio'/>
</interface>


Cheers, Steve

[Steve Holdoway]

Post by Steve Holdoway

Post by Hadley Rich

Post by Steve Holdoway
I've got a bit of a problem with a setup I built for a local charity,
where I set them up with a mail server in a virtual host. Seemed to be a
good security measure at the time.

[snip]

Post by Steve Holdoway
As it turns out, the bridging is performing NAT on the traffic
travelling between the physical and virtual servers, which effectively
screws up the mailer configuration - making all traffic look local, and
it's ended up as an open relay. Extensive googling and reading of
manuals has not told me how ( or even if ) this Nating can be disabled.

I'm confused, that shouldn't perform NAT at all. I use similar setups
myself.

So am I! My mail server is a standard that I haven't changed for
probably years ( except to upgrade ) - sendmail + milter-greylist +
spamhaus/zen RBL + clamav-milter.
But it's operating as an open relay as it sees all of the incoming
traffic as coming from the ip address of the physical server the vm's
are built upon. This address is neither the default gateway, nor is used
to specifically route anything from.

Post by Hadley Rich
Does this section look famlilar from /etc/libvirt/qemu/$HOST.xml;
<interface type='bridge'>
<mac address='01:23:45:ab:cd:ef'/>
<source bridge='br0'/>
</interface>

<interface type='bridge'>
<mac address='01:23:45:ab:cd:ef'/>
<source bridge='br0'/>
<model type='virtio'/>
</interface>
Cheers, Steve
_______________________________________________
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug

I think I may have got to the bottom of this. There was a temporary
masquerade set up on network bridge on the physical server while we are
setting stuff up over openvpn, which seems to have also affected traffic
to the VMs as well.

Is this the case? Having read up on this, it does look like prerouting
stuff is performed before bridging decisions are made...

Steve

[Hadley Rich]

Post by Steve Holdoway
I think I may have got to the bottom of this. There was a temporary
masquerade set up on network bridge on the physical server while we
are setting stuff up over openvpn, which seems to have also affected
traffic to the VMs as well.

Aha yes that sounds more likely. I'm not entirely sure where the
prerouting stuff is performed but I can say that a normal bridge
shouldn't do that!

Post by Steve Holdoway
Is this the case? Having read up on this, it does look like prerouting
stuff is performed before bridging decisions are made...

[Dagan McGregor]

Post by Hadley Rich

Post by Steve Holdoway
I think I may have got to the bottom of this. There was a temporary
masquerade set up on network bridge on the physical server while we
are setting stuff up over openvpn, which seems to have also affected
traffic to the VMs as well.

Aha yes that sounds more likely. I'm not entirely sure where the
prerouting stuff is performed but I can say that a normal bridge
shouldn't do that!

In my experience playing with KVM and libvirt on Fedora boxes,
KVM/libvirt manages it's own iptables entries into the firewall,
external from the iptables configuration file.

I think if you compare iptables -L output against what your iptables
config file says, you'll notice extra entries added by KVM/libvirt.

There are open bugs in the RH Bugzilla to get this changed, so they
are properly managable by config entries.

YMMV.

Dagan

_______________________________________________
NZLUG mailing list ***@linux.net.nz
http://www.linux.net.nz/cgi-bin/mailman/listinfo/nzlug