Critical Pro Custodibus Agent Update for DNS

On December 15, Pro Custodibus agent versions between 1.7.0 and 1.10.1 will stop working, due to an upcoming incompatibility between its default DNS resolver (Quad9, aka 9.9.9.9) and the way it resolves DNS by default (using DNS-over-HTTPS, aka DoH, with HTTP 1.1). Versions before 1.7.0 are not affected.

After December 15, you can check if you are affected by running the agent CLI with the --test option; if so, this is what you’ll see:

$ sudo /opt/venvs/procustodibus-agent/bin/procustodibus-agent --test
... 1 wireguard interfaces found ...
!!! cannot resolve pro custodibus api ipv4 address: dns error resolving api.custodib.us: 400 Client Error: Bad Request for url: https://9.9.9.9:443/dns-query?dns=FeIBAAABAAAAAAAAA2FwaQhjdXN0b2RpYgJ1cwAAAQAB !!!
!!! unhealthy pro custodibus api: server unavailable (dns error resolving api.custodib.us: 400 Client Error: Bad Request for url: https://9.9.9.9:443/dns-query?dns=1poBAAABAAAAAAAAA2FwaQhjdXN0b2RpYgJ1cwAAAQAB) !!!
!!! cannot access host record on api (dns error resolving api.custodib.us: 400 Client Error: Bad Request for url: https://9.9.9.9:443/dns-query?dns=E9sBAAABAAAAAAAAA2FwaQhjdXN0b2RpYgJ1cwAAAQAB) !!!
Issues encountered; see https://docs.procustodibus.com/guide/agents/troubleshoot/ to fix

We recommend you either:

  1. Upgrade the Agent to version 1.10.2

  2. or Customize Your DNS Settings to use a different resolver

Upgrade the Agent

Version 1.10.2 of the agent fixes this problem, defaulting to using the host’s own system resolver for DNS.

Linux

To upgrade the agent on Linux, download the latest version from the Linux Downloads page.

Copy the downloaded tarball to the host, extract it, and run the install script:

$ tar xf procustodibus-agent-latest.tar.gz
$ cd procustodibus-agent-*/
$ sudo ./install.sh --install
running as root
install 1.10.2
agent configuration found at /etc/wireguard/procustodibus.conf
...
WARNING agent 1.10.1 package outdated (1.10.2 available)
upgrade agent from /root/procustodibus-agent-1.10.2? ([y]es, [q]uit): y
OK will upgrade agent
...
WARNING daemon requires restart
restart daemon? ([y]es, [q]uit): y
OK will restart daemon
restarted daemon
install SUCCESS

Enter y at all the prompts (or run the script with the --force option to automatically select y without prompting).

Windows

To upgrade the agent on Windows, download the latest version from the Windows Downloads page.

Then simply run the downloaded MSI package as the Administrator user on the host:

> procustodibus_agent-latest-win64.msi

Docker

To upgrade the agent with Docker, pull the latest agent image to the container’s host with the following command:

$ sudo docker pull procustodibus/agent
latest: Pulling from procustodibus/agent
2d35ebdb57d9: Already exists
...
0b669aef92ab: Pull complete
Digest: sha256:59defb473cad1e990f6830fdf35e92f55aa7e071c9a31632015bb4e3324c428d
Status: Downloaded newer image for procustodibus/agent:latest
docker.io/procustodibus/agent:latest

List the available versions and tags for the agent image with the following command:

$ sudo docker image ls procustodibus/agent
IMAGE                        ID             DISK USAGE   CONTENT SIZE   EXTRA
procustodibus/agent:1.10.0   9fd98605ccd9        318MB             0B
procustodibus/agent:1.10.1   280089289823        318MB             0B
procustodibus/agent:1.10.2   2ce743f4cf26        318MB             0B
procustodibus/agent:latest   2ce743f4cf26        318MB             0B

If you specified a particular tag or image ID for the agent image in your docker run command or docker-compose.yml config file, update it to use a tag or the ID of the latest image. Then recreate the container.

If you started the container with docker run (and the container’s name is wireguard), run the following command to stop and delete the container:

$ sudo docker stop wireguard && sudo docker rm wireguard

Then re-run the docker run command you used originally to start the container.

If you started the container with docker compose, run the following command from the directory containing the docker-compose.yml file (if the service’s name is wireguard) to recreate the container:

$ sudo docker compose up -d wireguard

Customize Your DNS Settings

Alternatively, you can avoid this DNS issue entirely by customizing the Pro Custodibus agent’s DNS settings. Open up its config file at /etc/wireguard/procustodibus.conf (or C:\Program Files\Pro Custodibus Agent\cnf\procustodibus.conf on Windows), and make one of the following changes; then restart the agent.

Use Host System Resolver

To use the host operating system’s own DNS resolver, add a Dns = off setting to the agent config file, then restart the agent:

# /etc/wireguard/procustodibus.conf
[Procustodibus]
# My Host Agent
Agent = ABC123def45
# My Host
Host = DEF456ghi78

Dns = off
Note
 You should choose one of the other options if you are using WireGuard for the default route (aka 0.0.0.0/0 or ::/0) on the host, to ensure that the agent’s own DNS queries do not use the WireGuard tunnel.

Use Cloudflare DoH Resolver

Alternately, to use Cloudflare’s DNS resolver, add the following settings to the agent config file, then restart the agent:

# /etc/wireguard/procustodibus.conf
[Procustodibus]
# My Host Agent
Agent = ABC123def45
# My Host
Host = DEF456ghi78

Dns = 1.1.1.1, 1.0.0.1, 2606:4700:4700::1111, 2606:4700:4700::1001
Doh = cloudflare-dns.com

Use Google DoH Resolver

Alternately, to use Google’s DNS resolver, add the following settings to the agent config file, then restart the agent:

# /etc/wireguard/procustodibus.conf
[Procustodibus]
# My Host Agent
Agent = ABC123def45
# My Host
Host = DEF456ghi78

Dns = 8.8.8.8, 8.8.4.4, 2001:4860:4860::8888, 2001:4860:4860::8844
Doh = dns.google

Use Custom Resolver

Alternately, to use your own custom DNS resolver, add the following setting to the agent config file, replacing 10.0.0.1 with the DNS resolver’s actual IP address:

# /etc/wireguard/procustodibus.conf
[Procustodibus]
# My Host Agent
Agent = ABC123def45
# My Host
Host = DEF456ghi78

Dns = 10.0.0.1

Then restart the agent.

Restart

After making a change to the procustodibus.conf file, restart the Pro Custodibus agent.

Linux With Systemd

To restart the agent on a Linux host using systemd, run the following command:

$ sudo systemctl restart procustodibus-agent

Linux With OpenRC

To restart the agent on a Linux host using OpenRC, run the following command:

$ doas rc-service procustodibus-agent restart

Windows

To restart the agent on Windows, run the following commands as an Administrator user:

> sc stop ProCustodibusAgentService
> sc start ProCustodibusAgentService

Docker Run

If you started the container with docker run (and the container’s name is wireguard), run the following command to restart the container:

$ sudo docker restart wireguard

Docker Compose

If you started the container with docker compose, run the following command from the directory containing the docker-compose.yml file (if the service’s name is wireguard) to restart the container:

$ sudo docker compose restart wireguard
by Justin Ludwig
  • Pro Custodibus
    •