Running an OpenBSD Tor proxy in Qubes

Install OpenBSD as HVM Template

Follow the normal process for creating a template that is not cloned from an existing template. In the GUI select the options to Launch Qube settings after creation and Do not clone. In the qube Settings make sure that virt_mode is set to HVM, Kernel is (provided by qube), deselect Include in memory balancing, and set Initial memory to 400. Boot the new template from an OpenBSD iso, and run a standard install on to the first disk. Alternatively, there is a pre-built OpenBSD template available from 3isec

Install and configure tor and pf

I have already covered Running a Tor proxy on OpenBSD so I will summarise here.

pkg_add tor

I also install curl (for testing) and nyx to monitor circuits. pkg_add curl nyx

SOCKSPort "127.0.0.1:9050 IsolateClientAddr IsolateSocksAuth" # Default: Bind to localhost:9050 for local connections.
TransPort "127.0.0.1:9040 IsolateClientAddr" #
TransProxyType pf-divert
VirtualAddrNetwork "172.16.0.0/12"

DNSPort "127.0.0.1:53 IsolateClientAddr IsolateSocksAuth"
AutoMapHostsOnResolve 1

SOCKSPolicy accept 127.0.0.1
SOCKSPolicy accept 10.0.0.0/8

ext_if    = "xnf0"
internal  = "10.0.0.0/8"
SOCKS_IP  = "127.0.0.1"
TRANS_IP  = "127.0.0.1"
DNS_IP    = "127.0.0.1"
SOCKSPort = 9050
TRANSPort = 9040
DNSPort         = 53

block all
set skip on lo0

pass out on $ext_if proto tcp from any user _tor

pass in quick on $ext_if proto tcp \
  from any to any port $SOCKSPort \
    rdr-to $SOCKS_IP port $SOCKSPort

pass in on $ext_if proto udp \
  from $internal to port $DNSPort \
  rdr-to $DNS_IP port $DNSPort

pass in on $ext_if proto tcp \
  from $internal to any \
    divert-to $TRANS_IP port $TRANSPort

nameserver 127.0.0.1

pfctl -f /etc/pf.conf

rcctl enable tor
rcctl start tor

curl -x socks5h://127.0.0.1:9050 -s https://check.torproject.org |head -n20|grep Tor

Configure the net qube

Create a new qube - here called bsd_firewall to act as net qube for the OpenBSD Tor qube and clients. You could clone sys-firewall. Make sure you set the qube preferences to include provides_network, and that the upstream qubes do not include one providing Tor services. (You can no longer run Tor over Tor.)

In the bsd_firewall qube, add the following to /rw/config/rc.local:

# SET THE IP OF THE BSD_TOR QUBE HERE
TOR_IP="10.137.0.105"
# SET THE IP OF THE BSD_TOR QUBE HERE

mkdir  /etc/iproute2/
echo "100 to_proxy" >> /etc/iproute2/rt_tables
ip route add default via $TOR_IP table to_proxy

Replace 10.137.0.105 with the IP address of the OpenBSD qube running tor.

What this does is create a new routing table, called to_proxy, when the bsd_firewall starts up. We will use this to override normal Qubes networking routing, so that all traffic from client qubes gets directed to the OpenBSD Tor qube.

You could set nftables to pass all traffic between connected qubes, but lets do something a little more interesting. It's possible to run a script when there is a change in the status of an attached interface: the file is created in /rw/config/network-hooks.d/ : you may need to create that directory. Executable files placed here are run when Qubes interfaces are configured, as documented here

Here is a sample script: change TOR_IP to that of the OpenBSD qube:

#!/bin/bash

# SET THE IP OF THE BSD_TOR QUBE HERE
TOR_IP="10.137.0.105"
# SET THE IP OF THE BSD_TOR QUBE HERE

TOR_VIF=$(ip route |awk -v var="$TOR_IP" '$1 == var { print $3 }')

command="$1"
vif="$2"
vif_type="$3"
ip="$4"

echo "$command  $vif  $vif_type  $ip " >> /rw/config/log
echo "=================== " >> /rw/config/log

table_qubes=$(mktemp)

set_nft() {
  nft list table ip qubes > $table_qubes
  sed -i -e '1i \#\!/usr/sbin/nft -f\nflush table ip qubes\nflush set ip qubes allowed\n' \
         -e "s/10.139.1.[12]/$1/2 " \
         -e "/drop forward:/d " \
         -e "/eth0/d " \
         -e "/chain antispoof/a iifname $2 accept" \
         -e "/chain custom-forward/a iifname $2 oifname eth0 ct state new,established,related counter accept" \
         -e "/chain custom-forward/a iifname eth0 oifname $2 ct state established counter accept" \
         -e "/chain custom-forward/a oifname $2 ip saddr $3 ct state new,established counter accept" \
         -e "/chain custom-forward/a iifname $2 ip daddr $3 ct state established counter accept " \
         -e "/chain custom-forward/a iifname $4 ip daddr $1 ct state established,new counter accept " \
      -e '/chain custom-forward {/,/^[[:space:]]*}/{
        /^[[:space:]]*}/i\
    counter log prefix "drop forward: " drop
          }' \
      $table_qubes

  nft -f $table_qubes
}

clear_nft () {
  echo $1
  echo $2
  rm /tmp/tmp*
  nft list table ip qubes > $table_qubes
  sed -i -E -e '1i \#\!/usr/sbin/nft -f\nflush table ip qubes\nflush set ip qubes allowed\n' \
    -e "/$2/{ /[{}]/!d; s/\"[^\"]*\" \. $2[, ]*//; s/$2,? *//; }" \
    -e "/$3/d" \
      $table_qubes
  nft -f $table_qubes
}


if [ "$ip" == $TOR_IP ]; then
  case "$command" in
    online)
      ip route add default via $TOR_IP table to_proxy
      ;;
    offline)
      ;;
  esac
else
  case "$command" in
    online)
      ip route add default via $TOR_IP table to_proxy
      ip rule add from $ip lookup to_proxy
      set_nft "$TOR_IP" "$TOR_VIF" "$ip" "$vif"
      ;;
    offline)
      ip rule del from $ip lookup to_proxy
      clear_nft "$TOR_VIF" "$ip" "$vif"
      ;;
  esac
fi

What this does is create a default entry to the OpenBSD qube in the routing table, and configures routing on the firewall to ensure that clients use the to_proxy table. The script uses sed to create specific nftables rules in the qubes custom-forward chain, allowing traffic between the attached qube and the OpenBSD qube, and dropping any other traffic. If a client were to attempt to circumvent the default routing in some way, this will ensure this will fail.

The result is that qubes which have netvm set to the bsd_firewall will have all traffic routed through Tor, either by specifically setting SOCKS proxy to the IP of the OpenBSD Tor qube, or through the transparent proxy. Both provide circuit isolation between client qubes.

Testing

From a client on the same network, you can test DNS, SOCKS and the transparent proxy, using a simple script:

#!/bin/sh
  curl -x socks5h://X.X.X.X:9050 https://check.torproject.org |grep Congratulations
  curl https://check.torproject.org |grep Congratulations

Replace X.X.X.X with the IP address of the Tor proxy.

If successful you will see "Congratulations. This browser is configured to use Tor." for both tests. Notice that you do not need to specify a proxy address for the second curl test: the pf rules take care of that.