Setting up docker containers with nat

Question

I am setting up two docker containers

     container1                 container2
     |        |                     |
    eth0     eth1                   |
     |        |                    eth1
   docker0   docker1<----------------
     |            
     |
   internet

docker0 and docker1 are the bridges.

I have ip forwarding to 1 in both host and in containers. I have setup

iptables -I POSTROUTING -t nat -o eth0 -j MASQUERADE in container 1

Still i am not able to ping anything from container 2 to internet. I can see that packets are being received at eth1 of container 1.

OS: ubuntu 13.10
docker version: 0.11.1, build fb99f99

Am i missing some configuration?

Steps to reproduce:

SERV=$(docker run --privileged=true -i -d -t -v ~/Projects/code/myproject/build:/build:ro debian:7.4 /bin/bash)
CLI=$(docker run --privileged=true -i -d -t -v ~/Projects/code/myproject/build:/build:ro debian:7.4 /bin/bash)
sudo pipework br1 $SERV 10.1.0.1/8
sudo pipework br1 $CLI 10.1.0.3/8 

In $SERV: iptables -I POSTROUTING -t nat -o eth0 -j MASQUERADE

In $CLI Disable the interface eth0. Set default route to eth1 interface.

Now ping is happening to 10.1.0.1 from $CLI but not to the internet.

Answer 1

Hm, it should work as you described. Maybe the default route is not configured correctly. This is what I did:

SERV=$(docker run -i --privileged -d -t debian:7.4 /bin/bash)
CLI=$(docker run --privileged -i -d -t debian:7.4 /bin/bash)

docker exec -ti $CLI ping google.de  # Internet up

docker exec -ti $CLI ip link set eth0 down
docker exec -ti $CLI ping google.de  # Internet down

pipework br1 $SERV 10.1.0.1/8
pipework br1 $CLI 10.1.0.2/8

docker exec -ti $SERV apt-get install -y iptables
docker exec -ti $SERV iptables -I POSTROUTING -t nat -o eth0 -j MASQUERADE
docker exec -ti $CLI ip route add default via 10.1.0.1 dev eth1
docker exec -ti $CLI ping google.de  # Internet up

docker exec -ti $CLI apt-get install -y traceroute
docker exec -ti $CLI traceroute google.de

Answer 2

The only way iptables is changed is when executed from Docker host on a containers run with

--privileged

Here is a script:

iptables along with a couple of tools are installed during the image build (Dcokerfile) inetutils-traceroute iputils-tracepath iptables

Here I use "phusion-dockerbase", you can use whatever image you want:

#!/bin/bash

### ==> Install & configure iptable during build
#RUN sudo apt-get install -y inetutils-traceroute iputils-tracepath iptables
# Build the image 
#sudo docker build -t mybimage -f phusion-dockerbase .

### container1
C1=$(docker run --privileged -i -d -t mybimage /bin/bash)
sudo docker exec -ti $C1 iptables -I POSTROUTING -t nat -o eth0 -j MASQUERADE
sleep 2
sudo pipework br6 -i eth1 $C1 192.168.66.1/24

### container2
lxterminal -e "sudo docker run -ti --name c2name mybimage /bin/bash"
sleep 2
C2="$(sudo docker ps | grep c2name | awk '{ print $1; }')"
sudo pipework br6 -i eth1 $C2 192.168.66.2/[email protected]

---

Result:

./lab.sh

From the Container1 (I use lxterminal to open it in a new window):

Note that, as soon as you stop container1, corresponding pipework and iptable modification are lost and even when restart the stopped container, you will need to reissue the commands:

pipework br6 -i eth1 52b95d6052f7 192.168.66.1/24

docker exec 52b95d6052f7 iptables -I POSTROUTING -t nat -o eth0 -j MASQUERADE

for container1 to act again like nat box.

Even commiting the running container1 to a new image and run a new container from it, doesn't help.