WireGuard Containers for Overlapping Networks

When using WireGuard to connect from the same server to different private client networks that use the same network address space, you can use policy routing to prevent address collisions. That approach was detailed in the original WireGuard With Overlapping Client Networks article.

But another approach is to use a different network namespace for each client network. This is especially convenient if you’re already using Docker containers on the server — you can simply add an additional container for each client network you want to connect. Each container can encapsulate the WireGuard configuration and firewall rules needed to connect to a particular client network.

For example, say we have a server (Host C) in our network running two Docker containers: the first container runs an HTTP server application that our customers need to access through WireGuard; and the second container runs a DB client application that needs to access databases at each customer site through WireGuard.

Two of our customers, Customer A and Customer B, want to use the same exact WireGuard configuration — including using an IP address 10.0.0.1 for their own end of the WireGuard connection, and an IP address 10.0.0.2 for our end:

Overlapping Client Networks — Figure 1. Two overlapping WireGuard networks

We can make this work by running each WireGuard connection in its own separate container, and attaching all the containers (both WireGuard and application containers) to the same bridge network (wg-network). Within each WireGuard container, we can set up a few firewall NAT (Network Address Translation) rules to translate the conflicting WireGuard IP addresses to our own private address space that does not conflict.

With our NAT rules in place, within our own application containers, we can use a private address of 10.100.100.1 to identify Customer A’s server (Host A), and a private address of 10.100.100.2 to identify Customer B’s server (Host B). Customer A and B will know nothing of this, as connections to and from their own servers will use the addresses they expect (10.0.0.1 for their servers and 10.0.0.2 for ours):

Detail of Container Network — Figure 2. Bridge network on our server

In our HTTP server container, its access logs will look like the following, with requests from Customer A showing up with an IP address of 10.100.100.1, and requests from Customer B showing up with an IP address of 10.100.100.2:

10.100.100.1 - - [03/Jan/2024:02:48:24 +0000] "GET / HTTP/1.1" 200 3 "-" "curl/7.81.0"
10.100.100.2 - - [03/Jan/2024:02:48:25 +0000] "GET / HTTP/1.1" 200 3 "-" "curl/7.81.0"

And in our DB client container, we can use an host IP address of 10.100.100.1 to connect to the DB at the Customer A site:

$ mysql -u customer_a_user -h 10.100.100.1 customer_a_db

While for Customer B, we can use a host IP address of 10.100.100.2:

$ mysql -u customer_b_user -h 10.100.100.2 customer_b_db

Connection to Host A

On Customer A’s private server, Host A, they want to use the following WireGuard config file:

# local settings for Host A
[Interface]
PrivateKey = <some private key they keep secret>
Address = 10.0.0.1

# remote settings for Host C
[Peer]
PublicKey = <some public key we provide>
Endpoint = <some endpoint address and port we provide>
AllowedIPs = 10.0.0.2
PersistentKeepalive = 25

And for our side of the connection, on Host C, Customer A wants us to use the following WireGuard config file:

# local settings for Host C
[Interface]
PrivateKey = <some private key we choose and keep secret>
Address = 10.0.0.2
ListenPort = <some port we choose>

# remote settings for Host A
[Peer]
PublicKey = /TOE4TKtAqVsePRVR+5AA43HkAK5DSntkOCO7nYq5xU=
AllowedIPs = 10.0.0.1

Ports

We need to provide each customer with a unique UDP port by which to connect to their particular WireGuard container on our Host C. We’ll set up Host C with a public IP address (203.0.113.2), and open up our network firewall to allow public access to its UDP port 51821. So for Customer A, we’ll tell them to use the following as the Endpoint setting to Host C in their WireGuard config:

Endpoint = 203.0.113.2:51821

And we’ll set the ListenPort in our WireGuard config to 51821:

ListenPort = 51821

Keys

We’ll also generate a new private key to use for Customer A, and put it in the PrivateKey setting of our WireGuard config:

$ wg genkey | tee customer-a.key
0F11111111111111111111111111111111111111110=

Then we’ll use the private key to calculate the corresponding public key to provide to the customer:

$ wg pubkey < customer-a.key
hN+4fjPihcKbEDFQhqrMmW8oPqqaT4CUQYtyg3FUx3k=

Tip

On most systems, the wg command is part of the wireguard-tools package. If you don’t have this package installed, you can run it instead by spinning up a temporary procustodibus/wireguard container, and using the wg command in the temporary container. For example, the following two commands will generate a new private key, and then print the corresponding public key:

$ sudo docker run --rm procustodibus/wireguard wg genkey > customer-a.key
$ sudo docker run -i --rm procustodibus/wireguard wg pubkey < customer-a.key
hN+4fjPihcKbEDFQhqrMmW8oPqqaT4CUQYtyg3FUx3k=

NAT Rules

Finally, we’ll add some firewall NAT rules to our WireGuard config. We’ll use one iptables rule to map incoming traffic from the customer on TCP port 8080 to the container that runs our HTTP server (listening on TCP port 80 at 10.100.20.21 of our bridge network):

PreUp = iptables -t nat -A PREROUTING -i wg0 -p tcp --dport 8080 -j DNAT --to-destination 10.100.20.21:80

We’ll use a second iptables rule to map outgoing traffic from the container that runs our DB client app to use the IP address for the customer’s end of the WireGuard connection (10.0.0.1):

PreUp = iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 3306 -j DNAT --to-destination 10.0.0.1

And we’ll use a third iptables rule to map the source address of each forwarded connection to the IP address of the network interface out of which it is forwarded (10.100.100.1 for connections forwarded out to the bridge network; 10.0.0.2 for connections forwarded out to the WireGuard network):

PreUp = iptables -t nat -A POSTROUTING -j MASQUERADE

Config

Putting it all together, this is the complete WireGuard config file we’ll use for Customer A’s WireGuard container:

# /srv/containers/wg-network/customer-a/wg0.conf

# local settings for Host C
[Interface]
PrivateKey = 0F11111111111111111111111111111111111111110=
Address = 10.0.0.2
ListenPort = 51821

# port forward incoming traffic to our HTTP server
PreUp = iptables -t nat -A PREROUTING -i wg0 -p tcp --dport 8080 -j DNAT --to-destination 10.100.20.21:80
# port forward outgoing traffic to their DB server
PreUp = iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 3306 -j DNAT --to-destination 10.0.0.1
# masquerade all forwarded traffic
PreUp = iptables -t nat -A POSTROUTING -j MASQUERADE

# remote settings for Host A
[Peer]
PublicKey = /TOE4TKtAqVsePRVR+5AA43HkAK5DSntkOCO7nYq5xU=
AllowedIPs = 10.0.0.1

Connection to Host B

On Customer B’s private server, Host B, they want to use the following WireGuard config file:

# local settings for Host B
[Interface]
PrivateKey = <some private key they keep secret>
Address = 10.0.0.2

# remote settings for Host C
[Peer]
PublicKey = <some public key we provide>
Endpoint = <some endpoint address and port we provide>
AllowedIPs = 10.0.0.2
PersistentKeepalive = 25

And for our side of the connection, on Host C, Customer B wants us to use the following WireGuard config file:

# local settings for Host C
[Interface]
PrivateKey = <some private key we choose and keep secret>
Address = 10.0.0.2
ListenPort = <some port we choose>

# remote settings for Host A
[Peer]
PublicKey = fE/wdxzl0klVp/IR8UcaoGUMjqaWi3jAd7KzHKFS6Ds=
AllowedIPs = 10.0.0.1

(Notice that these two files are identical to what Customer A gave us, with the exception that Customer B has their own unique WireGuard key pair.)

Ports

We need to provide a unique UDP port for Customer B; we’ll choose 51822, and open up our network firewall to allow it through to our Host C. We’ll tell Customer B to use the following Endpoint setting in their WireGuard config:

Endpoint = 203.0.113.2:51822

And we’ll set the ListenPort in our WireGuard config to 51822:

ListenPort = 51822

Keys

We’ll also generate a new private key to use for Customer B:

$ wg genkey | tee customer-b.key
2G22222222222222222222222222222222222222220=

And provide Customer B with the corresponding public key:

$ wg pubkey < customer-b.key
777Ntbnv/AA9Fvd53yVR6Fsrd02CNCM2ySXgzXbbSUI=

NAT Rules

We’ll add the exact same firewall NAT rules to our config for Customer B as we used used for Customer A:

PreUp = iptables -t nat -A PREROUTING -i wg0 -p tcp --dport 8080 -j DNAT --to-destination 10.100.20.21:80
PreUp = iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 3306 -j DNAT --to-destination 10.0.0.1
PreUp = iptables -t nat -A POSTROUTING -j MASQUERADE

Config

Putting it all together, this is the complete WireGuard config file we’ll use for Customer B’s WireGuard container:

# /srv/containers/wg-network/customer-b/wg0.conf

# local settings for Host C
[Interface]
PrivateKey = 2G22222222222222222222222222222222222222220=
Address = 10.0.0.2
ListenPort = 51822

# port forward incoming traffic to our HTTP server
PreUp = iptables -t nat -A PREROUTING -i wg0 -p tcp --dport 8080 -j DNAT --to-destination 10.100.20.21:80
# port forward outgoing traffic to their DB server
PreUp = iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 3306 -j DNAT --to-destination 10.0.0.1
# masquerade all forwarded traffic
PreUp = iptables -t nat -A POSTROUTING -j MASQUERADE

# remote settings for Host A
[Peer]
PublicKey = fE/wdxzl0klVp/IR8UcaoGUMjqaWi3jAd7KzHKFS6Ds=
AllowedIPs = 10.0.0.1

(Notice that this is almost identical to our WireGuard config for Customer A, differing only in the PrivateKey, ListenPort, and PublicKey settings.)

Running via Docker

Create Network

To run these containers via docker run commands, first we need to set up a bridge network. This will allow our WireGuard containers to connect to our app containers, and our app containers to connect to our WireGuard containers. We’ll name our network wg-network, and use the 10.100.0.0/16 network address space for it:

$ sudo docker network create \
    --subnet 10.100.0.0/16 \
    wg-network
7c3873589a8b3bd055c6ad06224e6f736caabd3b4410d7625723292b28f4e31b

Run App Containers

Then we need to attach our application containers to our new bridge network, using 10.100.20.21 for our HTTP server app and 10.100.20.22 for our DB client app. If our application containers are not yet running, we can attach them to our bridge network when we start them up:

$ sudo docker run \
    --name http-server-app \
    --network wg-network \
    --ip 10.100.20.21 \
    my-custom-http-server-app-image
$ sudo docker run \
    --name db-client-app \
    --network wg-network \
    --ip 10.100.20.22 \
    my-custom-db-client-app-image

Or we can attach them to our bridge network after they’re already up and running:

$ sudo docker network connect \
    --ip 10.100.20.21 \
    wg-network \
    http-server-app
$ sudo docker network connect \
    --ip 10.100.20.22 \
    wg-network \
    db-client-app

Run WireGuard Containers

Next, we need to start up our WireGuard containers and attach them to the bridge network. With our WireGuard config for Customer A saved at /srv/containers/wg-network/customer-a/wg0.conf, we can start it up with an IP address of 10.100.100.1 on our bridge network by running the following command:

$ sudo docker run \
    --cap-add NET_ADMIN \
    --name customer-a \
    --network wg-network \
    --ip 10.100.100.1 \
    --publish 51821:51821/udp \
    --volume /srv/containers/wg-network/customer-a:/etc/wireguard \
    procustodibus/wireguard

 * /proc is already mounted
rm: can't remove '/run/lock': Resource busy
rm: can't remove '/run/secrets': Resource busy
 * /run/lock: correcting mode
 * /run/lock: correcting owner
   OpenRC 0.52.1 is starting up Linux 6.6.8-200.fc39.x86_64 (x86_64) [DOCKER]

 * Caching service dependencies ... [ ok ]
 * Starting WireGuard interface wg0 ...Warning: `/etc/wireguard/wg0.conf' is world accessible
[#] iptables -t nat -A PREROUTING -i wg0 -p tcp --dport 8080 -j DNAT --to-destination 10.100.20.21:80
[#] iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 3306 -j DNAT --to-destination 10.0.0.1
[#] iptables -t nat -A POSTROUTING -j MASQUERADE
[#] ip link add wg0 type wireguard
[#] wg setconf wg0 /dev/fd/63
[#] ip -4 address add 10.0.0.2 dev wg0
[#] ip link set mtu 1420 up dev wg0
[#] ip -4 route add 10.0.0.1/32 dev wg0
 [ ok ]

And we can start up the WireGuard container for Customer B with the following command:

$ sudo docker run \
    --cap-add NET_ADMIN \
    --name customer-b \
    --network wg-network \
    --ip 10.100.100.2 \
    --publish 51822:51822/udp \
    --volume /srv/containers/wg-network/customer-b:/etc/wireguard \
    procustodibus/wireguard

 * /proc is already mounted
rm: can't remove '/run/lock': Resource busy
rm: can't remove '/run/secrets': Resource busy
 * /run/lock: correcting mode
 * /run/lock: correcting owner
   OpenRC 0.52.1 is starting up Linux 6.6.8-200.fc39.x86_64 (x86_64) [DOCKER]

 * Caching service dependencies ... [ ok ]
 * Starting WireGuard interface wg0 ...Warning: `/etc/wireguard/wg0.conf' is world accessible
[#] iptables -t nat -A PREROUTING -i wg0 -p tcp --dport 8080 -j DNAT --to-destination 10.100.20.21:80
[#] iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 3306 -j DNAT --to-destination 10.0.0.1
[#] iptables -t nat -A POSTROUTING -j MASQUERADE
[#] ip link add wg0 type wireguard
[#] wg setconf wg0 /dev/fd/63
[#] ip -4 address add 10.0.0.2 dev wg0
[#] ip link set mtu 1420 up dev wg0
[#] ip -4 route add 10.0.0.1/32 dev wg0
 [ ok ]

Running via Docker Compose

Alternatively, we could create a bridge network and run all these containers together with one big docker-compose.yml file, like the following:

# /srv/containers/wg-network/docker-compose.yml
version: '3'

networks:

  wg-network:
    ipam:
      config:
      - subnet: 10.100.0.0/16

services:

  http-server-app:
    image: my-custom-http-server-app-image
    networks:
      wg-network:
        ipv4_address: 10.100.20.21

  db-client-app:
    image: my-custom-db-client-app-image
    networks:
      wg-network:
        ipv4_address: 10.100.20.22

  customer-a:
    image: procustodibus/wireguard
    cap_add:
    - NET_ADMIN
    networks:
      wg-network:
        ipv4_address: 10.100.100.1
    ports:
    - 51821:51821/udp
    volumes:
    - ./customer-a:/etc/wireguard

  customer-b:
    image: procustodibus/wireguard
    cap_add:
    - NET_ADMIN
    networks:
      wg-network:
        ipv4_address: 10.100.100.2
    ports:
    - 51822:51822/udp
    volumes:
    - ./customer-b:/etc/wireguard

See the Troubleshooting section of the Building, Using, and Monitoring WireGuard Containers article if you need to debug errors starting or running your WireGuard containers with Docker.

Running via Podman

Prerequisites

We can also run these containers under rootless Podman. Before we do this, however, we’ll need to load the WireGuard kernel module as root, along with several iptables and nftables modules. On a system running systemd, the best way to do this is create a new file in the /etc/modules-load.d/ directory, with the following content:

# /etc/modules-load.d/wireguard.conf
# WireGuard module
wireguard
# iptables/nftables modules for basic DNAT/SNAT and masquerading
nft_chain_nat
nft_compat
xt_nat

Then run the following command to load the modules immediately (they will be loaded automatically on subsequent system boots):

$ sudo systemctl restart systemd-modules-load

See the Kernel Module Loading section of the WireGuard in Podman Rootless Containers for more details.

Also, if Host C is running a host-based firewall (like firewalld), we’ll need to update its firewall as root to allow new inbound connections to our WireGuard containers’ listen ports. For example, if Host C was running firewalld, and using its public zone for its public network interface (eg eth0), we’d run the following commands to open up its firewall for our WireGuard listen ports:

$ sudo firewall-cmd --zone=public --add-port=51821/udp --add-port=51822/udp

See the Firewall Modifications section of the WireGuard in Podman Rootless Containers for more details.

Create Network

Now to run these containers via podman run commands, we need to set up a bridge network. This will allow our WireGuard containers to connect to our app containers, and our app containers to connect to our WireGuard containers. We’ll name our network wg-network, and use the 10.100.0.0/16 network address space for it:

$ podman network create \
    --subnet 10.100.0.0/16 \
    wg-network
wg-network

(The Podman commands for this and the next step are practically identical to the rootfull Docker commands — we can simply substitute podman for sudo docker.)

Run App Containers

$ podman run \
    --name http-server-app \
    --network wg-network \
    --ip 10.100.20.21 \
    my-custom-http-server-app-image
$ podman run \
    --name db-client-app \
    --network wg-network \
    --ip 10.100.20.22 \
    my-custom-db-client-app-image

Or we can attach them to our bridge network after they’re already up and running:

$ podman network connect \
    --ip 10.100.20.21 \
    wg-network \
    http-server-app
$ podman network connect \
    --ip 10.100.20.22 \
    wg-network \
    db-client-app

Run WireGuard Containers

$ podman run \
    --cap-add NET_ADMIN \
    --name customer-a \
    --network wg-network \
    --ip 10.100.100.1 \
    --publish 51821:51821/udp \
    --sysctl net.ipv4.conf.all.forwarding=1 \
    --volume /srv/containers/wg-network/customer-a:/etc/wireguard:Z \
    docker.io/procustodibus/wireguard

 * /proc is already mounted
rm: can't remove '/run/lock': Resource busy
rm: can't remove '/run/secrets': Resource busy
 * /run/lock: correcting mode
 * /run/lock: correcting owner
   OpenRC 0.52.1 is starting up Linux 6.6.8-200.fc39.x86_64 (x86_64) [DOCKER]

 * Caching service dependencies ... [ ok ]
 * Starting WireGuard interface wg0 ...Warning: `/etc/wireguard/wg0.conf' is world accessible
[#] iptables -t nat -A PREROUTING -i wg0 -p tcp --dport 8080 -j DNAT --to-destination 10.100.22.21:80
[#] iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 3306 -j DNAT --to-destination 10.0.0.1
[#] iptables -t nat -A POSTROUTING -j MASQUERADE
[#] ip link add wg0 type wireguard
[#] wg setconf wg0 /dev/fd/63
[#] ip -4 address add 10.0.0.2 dev wg0
[#] ip link set mtu 1420 up dev wg0
[#] ip -4 route add 10.0.0.1/32 dev wg0
 [ ok ]

And we can start up the WireGuard container for Customer B with the following command:

$ podman run \
    --cap-add NET_ADMIN \
    --name customer-b \
    --network wg-network \
    --ip 10.100.100.2 \
    --publish 51822:51822/udp \
    --sysctl net.ipv4.conf.all.forwarding=1 \
    --volume /srv/containers/wg-network/customer-b:/etc/wireguard:Z \
    docker.io/procustodibus/wireguard

 * /proc is already mounted
rm: can't remove '/run/lock': Resource busy
rm: can't remove '/run/secrets': Resource busy
 * /run/lock: correcting mode
 * /run/lock: correcting owner
   OpenRC 0.52.1 is starting up Linux 6.6.8-200.fc39.x86_64 (x86_64) [DOCKER]

 * Caching service dependencies ... [ ok ]
 * Starting WireGuard interface wg0 ...Warning: `/etc/wireguard/wg0.conf' is world accessible
[#] iptables -t nat -A PREROUTING -i wg0 -p tcp --dport 8080 -j DNAT --to-destination 10.100.20.21:80
[#] iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 3306 -j DNAT --to-destination 10.0.0.1
[#] iptables -t nat -A POSTROUTING -j MASQUERADE
[#] ip link add wg0 type wireguard
[#] wg setconf wg0 /dev/fd/63
[#] ip -4 address add 10.0.0.2 dev wg0
[#] ip link set mtu 1420 up dev wg0
[#] ip -4 route add 10.0.0.1/32 dev wg0
 [ ok ]

The commands to run our WireGuard containers with Podman require a few slightly-different arguments than Docker; see the WireGuard in Podman Rootless Containers for a full explanation. Also see the Troubleshooting section of that article for tips on debugging errors when running WireGuard containers under Podman.

1/3/2024

by Justin Ludwig