Poor Man’s BeyondCorp with WireGuard

A true BeyondCorp architecture — the archetypal zero-trust network architecture — is notoriously difficult to implement. It requires re-arranging the way access is managed for all your internal applications, and is particularly difficult to integrate with non-web apps. This article will show you how you can use WireGuard and nftables to set up a simplified version of BeyondCorp — one that is highly practical for small organizations to build and use.

First, we’ll examine conceptually how BeyondCorp is meant to work, and how a simplified version with just WireGuard and nftables compares to it (including what’s missing from the full BeyondCorp vision). Then we’ll walk through the concrete steps needed to implement this simplified version.

Concept

BeyondCorp Architecture

Stripped of all its complicated implementation details, the core BeyondCorp architecture looks like the following, where a user device on any public network is able to connect to a private internal web application through a BeyondCorp access proxy:

BeyondCorp Architecture
Figure 1. BeyondCorp architecture

The access proxy terminates the TLS connection from the device. It uses the user and device credentials included as part of the underlying HTTP request to identify the user and device, and verifies with the BeyondCorp access control engine that the user and device is authorized to access the app. If so, it proxies the HTTP request to the back-end web-app server for the server to process and respond.

There’s a lot of special sauce that goes into the BeyondCorp access proxy and access control engine, as well as the user and device identification process. That special sauce, while powerful, makes a true BeyondCorp system difficult to implement.

Simplified Architecture

A dumbed-down version of this with WireGuard and nftables would look like the following:

Simplified Architecture
Figure 2. Poor man’s BeyondCorp

With this simplified architecture, HTTP requests from the device are tunneled through WireGuard to the WireGuard hub, and from the WireGuard hub to the app server. Devices are authenticated via their WireGuard key, and nftables access-control rules on the WireGuard hub reject requests from devices that have not been authorized for the app.

This gets you 80% of the benefits of BeyondCorp, but with 20% of the difficulty. Like BeyondCorp, it enables a user device anywhere in the world to connect securely to a private internal app. Unlike BeyondCorp, however, it authenticates and authorizes only devices, not users — it relies on the internal application itself for user authentication and authorization.

But a big advantage of this simplified version is that it can also be used with non-web applications — secure shell (SSH), remote desktop (RDP, VNC, etc), file shares (SMB, NFS, etc), mail (SMTP, IMAP, etc), voice over IP (SIP, RTP, etc), printing (LDP, IPP, etc), and so on — just as easily as it can be used with web apps. And you can still plug in whatever SSO (Single Sign-On) solution you might use with BeyondCorp into each individual application.

The original BeyondCorp paper identifies the following as the major steps for setting up a BeyondCorp architecture:

Here’s how each step compares to our simplified WireGuard and nftables implementation:

Securely Identifying the Device

In this guide, we’ll set up WireGuard on all user devices (as well as back-end applications). Our WireGuard hub will use the public-key pair we generate on each device to securely identify the device.

While this is pretty good, it’s not as strong as the BeyondCorp vision, which calls for identifying devices via a key stored in the TPM (Trusted Platform Module) of the device (so the key can’t be stolen and used on an different device); plus using other device information (such as the device location or patch level) to derive a graduated level of trust for the device.

Securely Identifying the User

This is the main piece missing from our simplified BeyondCorp implementation — unlike a true BeyondCorp architecture, it does not attempt to identify, authenticate, and authorize the user who’s using the device. (But note that you could add user authentication via WireGuard MFA with Pro Custodibus.)

Removing Trust From the Network

This guide will show you how to set up a host-based firewall with nftables on each application server (and user device) to block access to each host except through the WireGuard hub. While this is perfectly adequate, in a real implementation of this simplified BeyondCorp architecture (like with a full BeyondCorp architecture) you would usually also use network-level firewalls and segmentation.

Externalizing Applications and Workflows

Instead of implementing a protocol-aware proxy like a true BeyondCorp architecture, in this guide we’ll simply send all device-to-application traffic, internal and external, through a central WireGuard hub. WireGuard can handle any IP-based protocol (HTTP, SSH, RDP, SMB, etc), without having to be customized for the semantics of the protocol itself.

And unlike with a true BeyondCorp access proxy, where you’d use DNS hostnames for applications that resolve to the proxy’s own IP address, with this simplified BeyondCorp implementation, we’ll use the WireGuard IP address of each application server to access its hosted application. However, if you still want to use DNS names for your services, you can simply set up a private DNS server that resolves hostnames to WireGuard IP addresses (like to point mail.corp.example.com to the WireGuard IP address for the Mail Server in our example, 10.0.0.3).

Implementing Inventory-Based Access Control

In the grand BeyondCorp vision, this is where the magic happens: a specialized access control engine sorts through a multitude of user and device information, and evaluates a sophisticated set of access control policies to determine if a particular user is authorized to access a particular application from a particular device.

With our poor man’s BeyondCorp, we’ll just use some simple nftables rules on the WireGuard hub to define and enforce our inventory-based access control. WireGuard’s cryptokey routing system enables us to securely map devices to IP addresses; and with nftables it’s easy to set up rules to grant access through our WireGuard hub from each authorized device IP address to each application IP address (and deny all other access).

Implementation

Now let’s walk through a complete example implementation. For this example, we’ll use the following simple network with three applications (a Mail Server, a Web Server, and a VNC Server) and three user devices (Alice’s Workstation, Bob’s Workstation, and Cindy’s Laptop), some located at the main office site, and some scattered across the country:

Example Network
Figure 3. Network allowed access

Alice’s Workstation, located at the main office, needs to be able to connect to TCP ports 22 (SSH, for secure shell access), 25 (SMTP, to send email), and 143 (IMAP, to check her email) on the Mail Server; TCP ports 22 (SSH), 80 (the main web app), and 8080 (a secondary “admin” web app) on the Web Server; and TCP port 5900 on the VNC Server (to access some desktop application running on the server).

Bob’s Workstation, located at a remote office, needs to be able to connect to TCP ports 25 (SMTP) and 143 (IMAP) on the Mail Server; TCP ports 80 (main HTTP app) and 8080 (“admin” HTTP app) on the Web Server; and TCP port 5900 on the VNC Server.

Cindy’s Laptop, located at her home office or on the road, needs to be able to connect to TCP ports 25 (SMTP) and 143 (IMAP) on the Mail Server; and TCP port 80 on the Web Server.

In tabular form, our ACLs (Access Control List) would look like this:

Table 1. Network ACLs
Destination Host Destination Port Source Host Access

Mail Server

TCP 22 (SSH)

Alice’s Workstation

Allow

Mail Server

TCP 25 (SMTP)

Alice’s Workstation

Allow

Mail Server

TCP 25 (SMTP)

Bob’s Workstation

Allow

Mail Server

TCP 25 (SMTP)

Cindy’s Laptop

Allow

Mail Server

TCP 143 (IMAP)

Alice’s Workstation

Allow

Mail Server

TCP 143 (IMAP)

Bob’s Workstation

Allow

Mail Server

TCP 143 (IMAP)

Cindy’s Laptop

Allow

Web Server

TCP 22 (SSH)

Alice’s Workstation

Allow

Web Server

TCP 80 (HTTP)

Alice’s Workstation

Allow

Web Server

TCP 80 (HTTP)

Bob’s Workstation

Allow

Web Server

TCP 80 (HTTP)

Cindy’s Laptop

Allow

Web Server

TCP 8080

Alice’s Workstation

Allow

Web Server

TCP 8080

Bob’s Workstation

Allow

VNC Server

TCP 5900

Alice’s Workstation

Allow

VNC Server

TCP 5900

Bob’s Workstation

Allow

Everything else

Everything else

Everything else

Deny

In between all these devices and applications we’ll insert a central WireGuard Hub. Each user device and application server will have its own connection to this hub, and each will have its own unique WireGuard IP address:

WireGuard Hub-and-Spoke Network
Figure 4. WireGuard network

We’ll put our WireGuard Hub in the main office network, to which the Internet router at the office will forward UDP port 51821. All the other hosts in the WireGuard network will connect to it, instead of directly to each other: The remote WireGuard hosts will connect to it through the main office’s Internet address of 198.51.100.10, and the local WireGuard hosts will connect to it through its LAN (Local Area Network) address of 192.168.1.101.

Here’s a table of each host’s public endpoint IP address and port, its endpoint IP address and port within the main office LAN, and its internal WireGuard IP address:

Table 2. Hub and spoke addresses
Host Internet Address LAN Address WireGuard Address

WireGuard Hub

198.51.100.10:51821

192.168.1.101:51821

10.0.0.1

Alice’s Workstation

N/A

dynamic

10.0.0.2

Mail Server

N/A

192.168.1.63:51823

10.0.0.3

Web Server

N/A

192.168.1.44:51824

10.0.0.4

VNC Server

203.0.113.50:51825

N/A

10.0.0.5

Bob’s Workstation

dynamic

N/A

10.0.0.6

Cindy’s Laptop

dynamic

N/A

10.0.0.7

The WireGuard configuration of the hub and each spoke will be vanilla, just the same as described in the WireGuard Hub and Spoke Configuration Guide (see that guide for details about each WireGuard setting used here). The nftables firewalls on the user devices and application servers will be similarly basic, mimicking the base spoke configs from the hub-and-spoke section of the WireGuard Nftables Configuration Guide. You can easily replace nftables on any spoke with any other firewall technology that’s available for the particular device or server.

The most interesting part of this implementation is the Hub Nftables Config section (covered last), which actually implements our access control rules. These are the major sections of the implementation:

Preparation

First, provision a Linux server for your WireGuard Hub, and install WireGuard on it. For our little example WireGuard network, you wouldn’t need a beefy server — something as small as a Raspberry Pi would be fine (but for a larger network, you’ll probably need a more capable server). Make sure the server’s WireGuard port is exposed to the Internet; in our example, this UDP port is 51821.

Generate a WireGuard public-key pair on the server, as described by the Generate WireGuard Keys section of the hub-and-spoke guide. Copy the public key to your server inventory. You’ll need it, plus the server’s public IP address and WireGuard port, to configure WireGuard on all the other spokes.

Decide what range of private IP addresses you want to use for your WireGuard network (see RFC 6890 for options). If all your devices and applications support IPv6, you can use IPv6 ranges; otherwise you’ll need to stick with IPv4. In this guide, we’ll mix all our applications and devices together in the 10.0.0.0/24 block — but you may want to group your various WireGuard peers into an orderly hierarchy of IP address blocks, with devices and applications (and other infrastructure) grouped into the roles for which they will be authorized (like say the devices used by the IT team in the 172.31.1.0/24 block, devices of the R&D team in the 172.31.2.0/24 block, devices of the Sales team in the 172.31.3.0/24 block, etc).

In your device/server inventory, assign a unique WireGuard IP address in the appropriate range to each device, application server, or other host that will be connected to this WireGuard network.

Device WireGuard Config

On each user device, install WireGuard, and generate a WireGuard key pair for it. This key pair is what’s used for Securely Identifying the Device in our simplified BeyondCorp architecture. Copy the public key to your device inventory.

Then set up a WireGuard config on each device similar to the Configure WireGuard for Endpoint A section of the hub-and-spoke guide. This is what we’d use for Bob’s Workstation:

# /etc/wireguard/wg0.conf

# local settings for Bob's Workstation
[Interface]
PrivateKey = EFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFA=
Address = 10.0.0.6

# remote settings for WireGuard Hub
[Peer]
PublicKey = /TOE4TKtAqVsePRVR+5AA43HkAK5DSntkOCO7nYq5xU=
Endpoint = 198.51.100.10:51821
AllowedIPs = 10.0.0.0/24

In the [Interface] section of the config, put the private key you generated for the device in the PrivateKey field, and the WireGuard address you assigned to it in the Address field.

In the [Peer] section of the config, put the public key you generated for the WireGuard Hub in the PublicKey field, the public IP address and UDP port of the WireGuard Hub in the Endpoint field, and all the IP address ranges included in your WireGuard network in the AllowedIPs field.

Tip

On devices or other servers that can access the WireGuard Hub through an internal network, instead of the Internet, like Alice’s Workstation, you can point the Endpoint setting to the hub’s internal IP address (192.168.1.101 in this example) instead of to its external one (198.51.100.10).

For devices that may sometimes be able to access the WireGuard Hub on an internal network and sometimes through the Internet — like if Cindy sometimes uses her laptop at home and sometimes brings it into the office — you may want to set up two separate WireGuard configs on the device: one with the hub’s internal IP address (eg /etc/wireguard/corp-internal.conf), and one with its external IP (eg /etc/wireguard/corp-external.conf). Both configs should share all the same settings, except for Endpoint.

Device Nftables Config

On each user device, install nftables (or some other host-based firewall). Configure the firewall to deny all new inbound connections (while allowing established connections). With nftables, we’d use a simplified version of the base configuration from the nftables guide:

#!/usr/sbin/nft -f
flush ruleset

table inet filter {
    chain input {
        type filter hook input priority 0; policy drop;

        # accept all loopback packets
        iif "lo" accept
        # accept all icmp/icmpv6 packets
        ip protocol icmp accept
        ip6 nexthdr ipv6-icmp accept
        # accept all packets that are part of an already-established connection
        ct state established,related accept

        # reject with polite "port unreachable" icmp response
        reject
    }

    chain forward {
        type filter hook forward priority 0; policy drop;
        reject with icmpx type host-unreachable
    }
}

For simplicity in this guide we’re going to skip the suggested drop-bad-packets and drop-bad-ct-states tables from the base configuration (feel free to include them here, although they aren’t all that useful for end-user devices). Also, since our user devices will always be the initiator of new connections, and never the recipient, we don’t need the base configuration’s rules to accept new WireGuard or SSH connections.

App WireGuard Config

On each application server, install WireGuard. Then generate a WireGuard key pair, and copy the public key to your server inventory.

Set up the WireGuard config on each server similar to the Configure WireGuard for Endpoint B section of the hub-and-spoke guide. This is what we’d use for the VNC Server:

# /etc/wireguard/wg0.conf

# local settings for VNC Server
[Interface]
PrivateKey = EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE=
Address = 10.0.0.5
ListenPort = 51825

# remote settings for WireGuard Hub
[Peer]
PublicKey = /TOE4TKtAqVsePRVR+5AA43HkAK5DSntkOCO7nYq5xU=
AllowedIPs = 10.0.0.0/24

In the [Interface] section of the config, put the private key you generated for the server in the PrivateKey field, the WireGuard address you assigned to the server in the Address field, and the public UDP port you want WireGuard to use on the server in the ListenPort field.

Note

The WireGuard Hub must be able to initiate new connections to the application server through the UDP port specified by the ListenPort setting, so make sure you adjust any network-level firewall settings you have in place to allow for this.

In the [Peer] section of the config, put the public key you generated for the WireGuard Hub in the PublicKey field, and all the IP address ranges included in your WireGuard network in the AllowedIPs field.

App Nftables Config

On each application server, install nftables (or some other host-based firewall). Configure the firewall to deny all new inbound connections except:

  1. Connections to the server’s public WireGuard port.

  2. Connections through WireGuard to ports on which application listens (and optionally SSH, for server administration).

Tip

In addition to, or in place of, a host-based firewall on the server itself, you may want to put the server on an isolated network segment, and use a network-level firewall to deny all new inbound connections to the server except to the server’s public WireGuard port.

With nftables, we’d start with the base configuration from the nftables guide, and customize it for the server’s WireGuard port and application ports. For the VNC Server, we’d use this (where the VNC daemon is listening on TCP port 5900):

#!/usr/sbin/nft -f
flush ruleset

define pub_iface = "eth0"
define wg_iface = "wg0"
define wg_port = 51825
define app_tcp_port = { 5900 }
define app_udp_port = { discard }

table inet filter {
    chain input {
        type filter hook input priority 0; policy drop;

        # accept all loopback packets
        iif "lo" accept
        # accept all icmp/icmpv6 packets
        ip protocol icmp accept
        ip6 nexthdr ipv6-icmp accept
        # accept all packets that are part of an already-established connection
        ct state established,related accept

        # accept all WireGuard packets received on a public interface
        iif $pub_iface udp dport $wg_port accept
        # accept packets on application ports received through WireGuard
        iifname $wg_iface tcp dport $app_tcp_port accept
        iifname $wg_iface udp dport $app_udp_port accept

        # reject with polite "port unreachable" icmp response
        reject
    }

    chain forward {
        type filter hook forward priority 0; policy drop;
        reject with icmpx type host-unreachable
    }
}

For simplicity in this guide we’re going to skip the suggested drop-bad-packets and drop-bad-ct-states tables from the base configuration, but you may want to include them, especially for servers that are exposed to the Internet without any other network-level filtering in place. Also, with our BeyondCorp architecture, we’re going to send all traffic, including SSH, through the WireGuard Hub, so we won’t include the rule from the base configuration that accepts new SSH connections on the public interface.

Instead, with this configuration, we’re going to accept new SSH and other application-specific connections only through the WireGuard interface, via these two rules:

        iifname $wg_iface tcp dport $app_tcp_port accept
        iifname $wg_iface udp dport $app_udp_port accept

For convenience, we’ve defined the specific ports the application accepts at the top of the config file, with these two variables:

define app_tcp_port = { 5900 }
define app_udp_port = { discard }

For the VNC Server, we expose only one port, TCP port 5900. On other servers, we’ll expose multiple ports — like on the Mail Server, we’ll expose TCP ports 22 (SSH), 25 (SMTP), and 143 (IMAP):

define app_tcp_port = { ssh, smtp, imap }
define app_udp_port = { discard }

In this way we can use the same core configuration on each app server, and just plug in the custom server-specific values at the top of the config.

Tip

Nftables doesn’t allow variables to be defined with empty values, so we’ve used the discard port, 9, to indicate “no ports”. You may also want to simply omit or comment out the rule that uses the $app_udp_port variable when you don’t have any application UDP ports to expose (and omit or comment out the rule that uses the $app_tcp_port variable when you don’t have any application TCP ports to expose).

Hub WireGuard Config

On the WireGuard Hub, set up its WireGuard config similar to the Configure WireGuard on Host C section of the hub-and-spoke guide. Make sure you add a [Peer] entry for each user device and application server (or other host) that you want to be part of the WireGuard network. With our example WireGuard network, it would look like this:

# /etc/wireguard/wg0.conf

# local settings for WireGuard Hub
[Interface]
PrivateKey = AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEE=
Address = 10.0.0.1
ListenPort = 51821

# packet forwarding
PreUp = sysctl -w net.ipv4.conf.all.forwarding=1

# remote settings for Alice's Workstation
[Peer]
PublicKey = fE/wdxzl0klVp/IR8UcaoGUMjqaWi3jAd7KzHKFS6Ds=
AllowedIPs = 10.0.0.2/32

# remote settings for Mail Server
[Peer]
PublicKey = jUd41n3XYa3yXBzyBvWqlLhYgRef5RiBD7jwo70U+Rw=
Endpoint = 192.168.1.63:51823
AllowedIPs = 10.0.0.3/32

# remote settings for Web Server
[Peer]
PublicKey = MMsBeTT9v/7XNWB8a/jSMn9O8olPVNduUwvUPJ6eB14=
Endpoint = 192.168.1.44:51824
AllowedIPs = 10.0.0.4/32

# remote settings for VNC Server
[Peer]
PublicKey = kAYKIv6gpBDqueaVNOsY8ddKmIC2dnnXJQ0iKzWxfBI=
Endpoint = 203.0.113.50:51825
AllowedIPs = 10.0.0.5/32

# remote settings for Bob's Workstation
[Peer]
PublicKey = af24MfZ5LzUedF5WlpK2+O8602g2fmiKO8dYdv8dUyI=
AllowedIPs = 10.0.0.6/32

# remote settings for Cindy's Laptop
[Peer]
PublicKey = QHy/1+olsMKePzg/044wWUunKs/IfWzy4Ub/iJbzJ00=
AllowedIPs = 10.0.0.7/32

In the [Interface] section of the config, put the private key you generated for the WireGuard hub in the PrivateKey field, the WireGuard address you assigned to it in the Address field, and the public UDP port WireGuard will use on the hub in the ListenPort field.

To ensure that the hub has packet forwarding turned on, in this example we’ve also used a PreUp command in the [Interface] section to turn it on when the WireGuard interface is started up. If you’ve turned on packet forwarding elsewhere (like via /etc/sysctl.conf or a /etc/sysctl.d/ config file), you don’t need to do it here. If you’re using IPv6 addresses, make sure you set the IPv6 version of this property (net.ipv6.conf.all.forwarding) instead.

Configure one user device or application server in each [Peer] section, putting the device or server’s WireGuard public key in the PublicKey field, and WireGuard IP address in the AllowedIPs field.

Since our user devices will always be the initiator of WireGuard connections, we don’t need to include an Endpoint setting for them in their [Peer] sections. Our application servers will usually be the recipient of WireGuard connections, however, and not the initiator, so we do need to configure an Endpoint setting for each. Put the public IP address of the application server and its WireGuard port in the Endpoint setting for each app server.

Tip

On servers that the WireGuard Hub can access through an internal network, instead of the Internet, like the Mail Server in this example, you can point the server’s Endpoint address to its internal IP address (eg 192.168.1.63 for the Mail Server) instead of an external one. Note that you can never use a server’s own WireGuard IP address for its Endpoint setting.

Hub Nftables Config

WireGuard’s strict cryptokey routing system ensures that when the hub, with the above WireGuard config for example, sees a packet from 10.0.0.2 come in on its wg0 interface, it knows the packet’s source must have had access to the private key generated on Alice’s Workstation. This allows us to trust that the packet legitimately came from Alice’s Workstation (to the extent we trust that the private key never left her workstation).

With this guarantee in place, we only need a few basic nftables rules to accomplish Implementing Inventory-Based Access Control in our simplified BeyondCorp architecture. We’ll start with the hub configuration (“Host C”) from the nftables guide, and customize it with the Network ACLs we laid out above for our example network:

#!/usr/sbin/nft -f
flush ruleset

define pub_iface = "eth0"
define wg_iface = "wg0"
define wg_port = 51821

# host inventory
define alice_workstation = 10.0.0.2
define mail_server = 10.0.0.3
define web_server = 10.0.0.4
define vnc_server = 10.0.0.5
define bob_workstation = 10.0.0.6
define cindy_laptop = 10.0.0.7

table inet filter {
    chain input {
        type filter hook input priority 0; policy drop;

        # accept all loopback packets
        iif "lo" accept
        # accept all icmp/icmpv6 packets
        ip protocol icmp accept
        ip6 nexthdr ipv6-icmp accept
        # accept all packets that are part of an already-established connection
        ct state established,related accept

        # accept all WireGuard packets received on a public interface
        iif $pub_iface udp dport $wg_port accept
        # accept SSH packets from Alice's Workstation received via WireGuard
        iif $wg_iface tcp dport ssh ip saddr $alice_workstation accept

        # reject with polite "port unreachable" icmp response
        reject
    }

    chain wg-forward {
        # forward all icmp/icmpv6 packets
        ip protocol icmp accept
        ip6 nexthdr ipv6-icmp accept
        # forward all packets that are part of an already-established connection
        ct state established,related accept

        # access control rules
        ip daddr                . tcp dport . ip saddr {
          $mail_server          . ssh       . $alice_workstation,
          $mail_server          . smtp      . $alice_workstation,
          $mail_server          . smtp      . $bob_workstation,
          $mail_server          . smtp      . $cindy_laptop,
          $mail_server          . imap      . $alice_workstation,
          $mail_server          . imap      . $bob_workstation,
          $mail_server          . imap      . $cindy_laptop,
          $web_server           . ssh       . $alice_workstation,
          $web_server           . http      . $alice_workstation,
          $web_server           . http      . $bob_workstation,
          $web_server           . http      . $cindy_laptop,
          $web_server           . 8080      . $alice_workstation,
          $web_server           . 8080      . $bob_workstation,
          $vnc_server           . 5900      . $alice_workstation,
          $vnc_server           . 5900      . $bob_workstation
        } accept

        # reject with polite "administratively prohibited" icmp response
        reject with icmpx type admin-prohibited
    }

    chain forward {
        type filter hook forward priority 0; policy drop;

        # filter all packets transiting WireGuard network via wg-forward chain
        iifname $wg_iface oifname $wg_iface goto wg-forward

        # reject with polite "host unreachable" icmp response
        reject with icmpx type host-unreachable
    }
}

The “boilerplate” parts of this configuration are:

  1. The input chain, which filters connections inbound to the server itself, and allows new connections only through WireGuard;

  2. The forward chain, which filters forwarded connections, and allows the forwarding of packets only in from the WireGuard interface and back out to the WireGuard interface itself, and delegates further filtering of those packets to the custom wg-forward chain; and

  3. The starting and ending stanzas of the wg-forward chain, which allows packets from established connections to be forwarded, and rejects the forwarding of everything else not explicitly allowed by our access-control rules.

The “interesting” parts of the configuration are the inventory definitions at the top of the config:

# host inventory
define alice_workstation = 10.0.0.2
define mail_server = 10.0.0.3
define web_server = 10.0.0.4
define vnc_server = 10.0.0.5
define bob_workstation = 10.0.0.6
define cindy_laptop = 10.0.0.7

And the access-control rules that use those definitions in the middle of the wg-forward chain:

        # access control rules
        ip daddr                . tcp dport . ip saddr {
          $mail_server          . ssh       . $alice_workstation,
          $mail_server          . smtp      . $alice_workstation,
          $mail_server          . smtp      . $bob_workstation,
          $mail_server          . smtp      . $cindy_laptop,
          $mail_server          . imap      . $alice_workstation,
          $mail_server          . imap      . $bob_workstation,
          $mail_server          . imap      . $cindy_laptop,
          $web_server           . ssh       . $alice_workstation,
          $web_server           . http      . $alice_workstation,
          $web_server           . http      . $bob_workstation,
          $web_server           . http      . $cindy_laptop,
          $web_server           . 8080      . $alice_workstation,
          $web_server           . 8080      . $bob_workstation,
          $vnc_server           . 5900      . $alice_workstation,
          $vnc_server           . 5900      . $bob_workstation
        } accept

The host definitions at the top of the config make the access-control rules easy to read — and notice how the rules line up neatly with the Network ACLs table laid out at the beginning of the Implementation section.

Without this nftables configuration, any user device on our WireGuard network would be able to access any application service; but with it in place, only authorized devices can access the applications for which they’ve been authorized.

Tip

With nftables, you can use the service names defined in /etc/services in place of port numbers. We’ve done this above, with ssh in place of 22, smtp in place of 25, etc.

In this example we don’t have any applications which use UDP; but if you do, you’ll need to add a second access-control rule section specifically for UDP ports. For example, if you added a NTP server with UDP port 123 exposed to your WireGuard network, you’d need to add the following below the TCP access-control rules to grant access to it from specific devices:

        ip daddr                . udp dport . ip saddr {
          $ntp_server           . ntp       . $alice_workstation,
          $ntp_server           . ntp       . $boby_workstation,
          $ntp_server           . ntp       . $cindy_laptop
        } accept

And note that for simplicity in this guide we’ve skipped the suggested drop-bad-packets and drop-bad-ct-states tables from the base configuration of the nftables guide — but they’re exactly the sort of hardening that you normally should include on critical pieces of your infrastructure, like this hub.

Try It Out

Start up WireGuard and nftables on your WireGuard Hub. On Linux with systemd, run the following commands:

$ sudo systemctl start wg-quick@wg0.service
$ sudo systemctl start nftables.service

To ensure that both start up on system boot, also run the following commands:

$ sudo systemctl enable wg-quick@wg0.service
$ sudo systemctl enable nftables.service

Do the same for all connected user devices and app servers.

Then try out the connections. From Alice’s Workstation, SSH into the WireGuard Hub:

$ ssh 10.0.0.1

SSH into the Mail Server:

$ ssh 10.0.0.3

Access the Web Server:

$ curl 10.0.0.4

And so on. You should be able to access each application from each device that’s been granted access to the app through the hub’s access-control rules.

Variations

For applications where you can’t install WireGuard on the application server itself (like say because it’s on a managed server in a cloud environment), you can as an alternative put the application server on a private subnet, and set up a WireGuard server in the same subnet (the same local “site”). Use a network-level firewall to block inbound connections to the subnet except to the WireGuard port of the WireGuard server.

Then configure the spoke WireGuard server in the private subnet to forward connections to the application server, using one of the two following approaches:

Port Forwarding

With port forwarding, you don’t have to make any WireGuard changes to any hosts — just apply the same App WireGuard Config to the WireGuard Spoke that you would otherwise apply to the app server itself.

WireGuard Port Forwarding
Figure 5. WireGuard spoke with port forwarding

You do need a different nftables configuration, however — one similar to the “Host β” configuration from the Point to Site With Port Forwarding section of the nftables guide (but with a few different details). For the VNC Server, we’d use this nftables config on its WireGuard spoke (to forward TCP port 5900 connections from the WireGuard spoke to the VNC Server at IP address 192.168.200.52 on the private subnet):

#!/usr/sbin/nft -f
flush ruleset

define pub_iface = "eth0"
define wg_iface = "wg0"
define wg_port = 51825
define app_address = 192.168.200.52
define app_tcp_port = { 5900 }
define app_udp_port = { discard }

table inet filter {
    chain input {
        type filter hook input priority 0; policy drop;

        # accept all loopback packets
        iif "lo" accept
        # accept all icmp/icmpv6 packets
        ip protocol icmp accept
        ip6 nexthdr ipv6-icmp accept
        # accept all packets that are part of an already-established connection
        ct state established,related accept

        # accept all WireGuard packets received on a public interface
        iif $pub_iface udp dport $wg_port accept
        # accept SSH packets received via WireGuard
        iif $wg_iface tcp dport ssh accept

        # reject with polite "port unreachable" icmp response
        reject
    }

    chain site-forward {
        # forward (rewritten) packets to app
        ip daddr $app_address tcp dport $app_tcp_port accept
        ip daddr $app_address udp dport $app_udp_port accept

        # reject with polite "administratively prohibited" icmp response
        reject with icmpx type admin-prohibited
    }

    chain forward {
        type filter hook forward priority 0; policy drop;

        # forward all packets that are part of an already-established connection
        ct state established,related accept
        # filter all packets forwarded through WireGuard via site-forward chain
        iifname $wg_iface oif $pub_iface goto site-forward

        # reject with polite "host unreachable" icmp response
        reject with icmpx type host-unreachable
    }
}
table inet nat {
    chain prerouting {
        type nat hook prerouting priority -100; policy accept;
        # rewrite destination address for app ports to use app IP address
        tcp dport $app_tcp_port dnat ip to $app_address
        udp dport $app_udp_port dnat ip to $app_address
    }
    chain postrouting {
        type nat hook postrouting priority 100; policy accept;
        # masquerade all packets from WireGuard to local site
        iifname $wg_iface oif $pub_iface masquerade
    }
}

Unlike our standard App Nftables Config, here we don’t accept connections to the application’s ports in the input chain (we accept only SSH connections to the WireGuard Spoke itself, for remote administration). Instead, we add the following to the forward chain, to send incoming traffic from WireGuard to the site-forward chain for filtering:

        # forward all packets that are part of an already-established connection
        ct state established,related accept
        # filter all packets forwarded through WireGuard via site-forward chain
        iifname $wg_iface oif $pub_iface goto site-forward

And we set up this custom site-forward chain to accept connections to the application’s ports (much like we accept them in the input chain of the standard App Nftables Config):

    chain site-forward {
        # forward (rewritten) packets to app
        ip daddr $app_address tcp dport $app_tcp_port accept
        ip daddr $app_address udp dport $app_udp_port accept

        # reject with polite "administratively prohibited" icmp response
        reject with icmpx type admin-prohibited
    }

If we fully trusted the WireGuard Hub, we could have alternatively just omitted the custom site-forward chain, and set up our forward chain like this:

    chain forward {
        type filter hook forward priority 0; policy drop;
        ct state established,related accept
        iifname $wg_iface accept
        reject with icmpx type host-unreachable
    }

The advantage of adding this extra filtering through our site-forward chain is that even if the WireGuard Hub is compromised (or misconfigured), only minimal access to the application server is exposed through the WireGuard Spoke.

The most important part of this nftables config, however, is actually the last part — the nat table with its prerouting and postrouting chains:

table inet nat {
    chain prerouting {
        type nat hook prerouting priority -100; policy accept;
        # rewrite destination address for app ports to use app IP address
        tcp dport $app_tcp_port dnat ip to $app_address
        udp dport $app_udp_port dnat ip to $app_address
    }
    chain postrouting {
        type nat hook postrouting priority 100; policy accept;
        # masquerade all packets from WireGuard to local site
        iifname $wg_iface oif $pub_iface masquerade
    }
}

This is what does the actual port forwarding, with the prerouting chain using nftables DNAT (Destination Network Address Translation) rules to forward our configured ports (TCP port 5900 in this example) to the application server. The postrouting chain uses SNAT (Source Network Address Translation) to “masquerade” forwarded packets from the WireGuard network as if their source was from the WireGuard Spoke itself (so that the VNC Server will send replies back through the WireGuard Spoke).

Tip

If you want to use SSH to administer both the WireGuard Spoke and the app server in its subnet, you have the problem of deciding where the spoke’s TCP port 22 should go. With the above configuration, when Alice runs ssh 10.0.0.5 from her workstation, she’ll SSH into the WireGuard Spoke. However, if you added ssh to the $app_tcp_port definition, she would instead SSH into the VNC Server.

The simplest way to resolve this is to add another DNAT rule to the bottom of the prerouting chain that redirects a different TCP port (like 222) to the spoke’s SSH server — for example:

        tcp dport 222 redirect to ssh

With this rule in place, when Alice runs ssh 10.0.0.5 from her workstation, she’ll SSH into the VNC Server; but when she runs ssh -p 222 10.0.0.5, she’ll SSH into the WireGuard Spoke itself. Note that this assumes you also grant Alice’s Workstation access to both ports (22 and 222) in your ntfables access-control rules on the WireGuard Hub:

          $vnc_server           . ssh       . $alice_workstation,
          $vnc_server           . 222       . $alice_workstation,

One last configuration change we need to make on the WireGuard Spoke is to enable packet forwarding. The easiest way to do that is to add it as a PreUp command to the [Interface] section of the WireGuard config on the spoke:

PreUp = sysctl -w net.ipv4.conf.all.forwarding=1

With that in place, for the VNC Server in our example scenario, Alice will be able to access the VNC Server on her workstation using the WireGuard IP address of the WireGuard Spoke:

$ vncviewer 10.0.0.5:5900

The WireGuard Spoke will transparently forward the VNC connection to the actual VNC Server, running at 192.168.200.52 on the private subnet it shares with the spoke.

Site Gateway

As an alternative to using port forwarding, you could instead set up the WireGuard Spoke as a gateway from the app server on its private subnet (its local “site”) to the WireGuard network. With this approach, however, you do need to make some changes to your WireGuard configuration — both on the hub, and potentially on every other spoke as well.

WireGuard Site Gateway
Figure 6. WireGuard spoke with site gateway

With the site gateway approach, the app server’s IP address on the private subnet needs to be exposed to the rest of the WireGuard network, as part of the AllowedIPs configuration setting of each member in the network (although you can omit this IP address from the configuration of user devices and other app servers that don’t actually need to access the app server — but, of course, special-casing this on some devices and not others can quickly become a maintenance burden). If you need to use this approach for several servers on several different subnets, you may want to group all such subnets into a larger range of IP addresses (eg 192.168.192.0/20) that you can safely add as a single block to the AllowedIPs setting on all devices (as long as that range doesn’t collide with any other private network ranges that the devices may need to access).

Even with the additional subnet IP address (or addresses) exposed, the central WireGuard Hub will still control access from each individual user device to each individual app server — including the special-cased app server (or servers) — using the same nftables access-control rules as before. The only difference is that the access control rules will use the subnet IP address instead of WireGuard IP address for each special-cased app server.

For a concrete example, we’ll add a dedicated WireGuard server as a spoke in the VNC Server’s private subnet, making the following changes to our standard BeyondCorp configuration:

Site Gateway WireGuard Config

The configuration of the WireGuard Spoke serving as the site gateway for the VNC Server should be the same as with Port Forwarding. It should use our standard App WireGuard Config; and it should turn on packet forwarding, like with this PreUp line added to the standard config:

# /etc/wireguard/wg0.conf

# local settings for VNC Server
[Interface]
PrivateKey = EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE=
Address = 10.0.0.5
ListenPort = 51825

# packet forwarding
PreUp = sysctl -w net.ipv4.conf.all.forwarding=1

# remote settings for WireGuard Hub
[Peer]
PublicKey = /TOE4TKtAqVsePRVR+5AA43HkAK5DSntkOCO7nYq5xU=
AllowedIPs = 10.0.0.0/24

Site Gateway Nftables Config

The nftables config for the spoke should also be the same as the same as with Port Forwarding — except to omit the nat table which would otherwise do port forwarding:

#!/usr/sbin/nft -f
flush ruleset

define pub_iface = "eth0"
define wg_iface = "wg0"
define wg_port = 51825
define app_address = 192.168.200.52
define app_tcp_port = { 5900 }
define app_udp_port = { discard }

table inet filter {
    chain input {
        type filter hook input priority 0; policy drop;

        # accept all loopback packets
        iif "lo" accept
        # accept all icmp/icmpv6 packets
        ip protocol icmp accept
        ip6 nexthdr ipv6-icmp accept
        # accept all packets that are part of an already-established connection
        ct state established,related accept

        # accept all WireGuard packets received on a public interface
        iif $pub_iface udp dport $wg_port accept
        # accept SSH packets received via WireGuard
        iif $wg_iface tcp dport ssh accept

        # reject with polite "port unreachable" icmp response
        reject
    }

    chain site-forward {
        # forward packets to app
        ip daddr $app_address tcp dport $app_tcp_port accept
        ip daddr $app_address udp dport $app_udp_port accept

        # reject with polite "administratively prohibited" icmp response
        reject with icmpx type admin-prohibited
    }

    chain forward {
        type filter hook forward priority 0; policy drop;

        # forward all packets that are part of an already-established connection
        ct state established,related accept
        # filter all packets forwarded through WireGuard via site-forward chain
        iifname $wg_iface oif $pub_iface goto site-forward

        # reject with polite "host unreachable" icmp response
        reject with icmpx type host-unreachable
    }
}

App Routing for Site Gateway

On the VNC Server itself (or on the router for its private subnet), you need to manually configure the routes for the WireGuard network. If the VNC Server (or router) is running Linux, you can do it with the following command, where 10.0.0.0/24 is the IP range used by the WireGuard network, 192.168.200.105 is the IP address of the WireGuard Spoke on the private subnet, and eth0 is the VNC Server’s (or router’s) network interface connected to the subnet:

$ sudo ip route add 10.0.0.0/24 via 192.168.200.105 dev eth0

Add a separate route for each separate IP range used by the WireGuard network.

Warning

Routes added through the ip route command will be lost on reboot, so you should also add those routes to a startup script or some other appropriate network config file.

Hub WireGuard Config for Site Gateway

In the WireGuard configuration on the WireGuard Hub, adjust the AllowedIPs setting for the VNC Server to include both the WireGuard IP address of the WireGuard Spoke (10.0.0.5), and the IP address of the VNC Server on its private subnet (192.168.200.52):

# remote settings for VNC Server
[Peer]
PublicKey = kAYKIv6gpBDqueaVNOsY8ddKmIC2dnnXJQ0iKzWxfBI=
Endpoint = 203.0.113.50:51825
AllowedIPs = 10.0.0.5/32, 192.168.200.52/32

Hub Nftables Config for Site Gateway

In the nftables configuration on the WireGuard Hub, change the $vnc_server variable definition to use the IP address of the VNC Server on its private subnet (192.168.200.52), and add a new variable definition for the WireGuard Spoke as $vnc_server_gateway (using its WireGuard IP address, 10.0.0.5):

# host inventory
define alice_workstation = 10.0.0.2
define mail_server = 10.0.0.3
define web_server = 10.0.0.4
define vnc_server = 192.168.200.52
define vnc_server_gateway = 10.0.0.5
define bob_workstation = 10.0.0.6
define cindy_laptop = 10.0.0.7

Then add a new access-control rule to allow SSH access to the WireGuard Spoke server from Alice’s Workstation, for administration:

        # access control rules
        ip daddr                . tcp dport . ip saddr {
          $mail_server          . ssh       . $alice_workstation,
          $mail_server          . smtp      . $alice_workstation,
          $mail_server          . smtp      . $bob_workstation,
          $mail_server          . smtp      . $cindy_laptop,
          $mail_server          . imap      . $alice_workstation,
          $mail_server          . imap      . $bob_workstation,
          $mail_server          . imap      . $cindy_laptop,
          $web_server           . ssh       . $alice_workstation,
          $web_server           . http      . $alice_workstation,
          $web_server           . http      . $bob_workstation,
          $web_server           . http      . $cindy_laptop,
          $web_server           . 8080      . $alice_workstation,
          $web_server           . 8080      . $bob_workstation,
          $vnc_server_gateway   . ssh       . $alice_workstation,
          $vnc_server           . 5900      . $alice_workstation,
          $vnc_server           . 5900      . $bob_workstation
        } accept

Device WireGuard Config for Site Gateway

Finally, in the WireGuard configuration for each user device (or at least all user devices that need to access the VNC Server), update their AllowedIPs setting to include the IP address of the VNC Server on its private subnet (192.168.200.52). This is what we’d configure on Bob’s Workstation:

# /etc/wireguard/wg0.conf

# local settings for Bob's Workstation
[Interface]
PrivateKey = EFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFA=
Address = 10.0.0.6

# remote settings for WireGuard Hub
[Peer]
PublicKey = /TOE4TKtAqVsePRVR+5AA43HkAK5DSntkOCO7nYq5xU=
Endpoint = 198.51.100.10:51821
AllowedIPs = 10.0.0.0/24, 192.168.200.52/32

Now Bob will be able to access the VNC Server from his workstation using VNC Server’s private IP address:

$ vncviewer 192.168.200.52:5900