Binding containers to a specific host VLAN – Docker

I have a host which is connected to multiple VLANs that have certain routing rules at my router, they have different properties.

I’ve seen other suggestions about running docker in a VM for each VLAN but that seems ugly and messy too.

For example

/etc/network/interfaces:

# Is routed out a VPN on my router
auto bond0.3
iface bond0.3 inet dhcp
iface bond0.3 inet6 auto

# Is null routed at my router (LAN only)
auto bond0.4
iface bond0.4 inet dhcp
  dns-nameserver 192.168.3.1
iface bond0.4 inet6 auto

# Routed directly to my ISP for containers only
auto bond0.7
iface bond0.7 inet dhcp
iface bond0.7 inet6 auto

I used a macvlan interface on a docker interface called "external_7":

networks:
  external_v7:
    name: external_v7
    driver: macvlan
    driver_opts:
      parent: bond0.7
    enable_ipv6: false
    ipam:
      config:
      - subnet: 192.168.7.0/24
        gateway: 192.168.7.1
        ip_range: 192.168.7.128/26

Next up I have a Bittorrent client behind a VPN, the main reason for having separate VPN tunnel container and not using the one on my router is because when downloading something it chokes the tunnel, and I can’t use it for anything else, so I have a dedicated VPN tunnel for Bittorrent:

services:
  vpn_bittorrent:
    extends:
      file: ../docker-vpn/docker-compose-vpn.yml
      service: openvpn-client
    container_name: vpn_bittorrent
    networks:
      external_v7:
        ipv4_address: 192.168.7.128
    volumes:
      - /mnt/data/container_data/vpn:/data/vpn
    ports:
      - 192.168.7.128:26129:26129
      - 192.168.7.128:26129:26129/udp

Which takes extension from:

services:
  openvpn-client:
    image: ghcr.io/wfg/openvpn-client
    cap_add:
      - NET_ADMIN
    devices:
      - /dev/net/tun
    restart: unless-stopped

Finally I have qbittorrent running:

  qbittorrent:
    image: ghcr.io/linuxserver/qbittorrent
    container_name: qbittorrent
    network_mode: service:vpn_bittorrent
    environment:
      - PUID=1004
      - PGID=1003
      - WEBUI_PORT=8080
    volumes:
      - /mnt/data/container_data/qbittorrent:/config
      - /mnt/data/shared/incoming:/mnt/shared/incoming
    restart: unless-stopped

Now of course I like to have a reverse proxy in front of this, because I like to use standard ports like :443 or whatever.

  web:
    image: nginx:stable-alpine
    container_name: web
    hostname: web.internal
    networks:
      external_v4:
        ipv4_address: 192.168.4.133
    volumes:
      - /mnt/data/container_data/web/nginx/auth-basic.conf:/etc/nginx/auth-basic.conf:ro
      - /mnt/data/container_data/web/nginx/htpasswd:/etc/nginx/htpasswd:ro
      - /mnt/data/container_data/web/nginx/http.d:/etc/nginx/http.d:ro
      - /mnt/data/container_data/web/nginx/nginx.conf:/etc/nginx/nginx.conf:ro
      - /mnt/data/container_data/web/nginx/proxy-control.conf:/etc/nginx/proxy-control.conf:ro
      - /mnt/data/container_data/web/nginx/ssl.conf:/etc/nginx/ssl.conf:ro
      - /mnt/data/container_data/web/nginx/ssl:/etc/nginx/ssl:ro
      - /mnt/data/container_data/web/nginx/html:/etc/nginx/html:ro
    command: [nginx-debug, '-g', 'daemon off;']
    ports:
      - 8080:8080
      - 9090:9090
      - 8989:8989
      - 7878:7878
    restart: unless-stopped

Each service must require a config:

server {
    listen 443;
    listen [::]:443;
    include /etc/nginx/ssl/qbittorrent.web.internal/qbittorrent.web.internal.conf;

    server_name qbittorrent.web.internal;


    location / {
        proxy_pass http://192.168.7.128:8080/;
        include    /etc/nginx/proxy-control.conf;
        include    /etc/nginx/auth-basic.conf;
    }
}

Now there’s a couple of problems:

1. If for some reason the VPN container vpn_bittorrent restarts the nginx proxy web will not be able to see the bittorrent container anymore until it too is restarted, the result is a "502 Bad Gateway" error. I suspect this has something to do with the connection between qbittorrent and vpn_bittorrent ie the network_mode: service:vpn_bittorrent.

2. I want to replace my reverse proxy with something a little more flexible like https://traefik.io/, I don’t want to have to configure nginx configs manually to match a certain IP address eg 192.168.7.128

3. I think moving away from macvlan addresses is probably a good thing, so I created a bridge on my host

auto br7
iface br7 inet static
  address 192.168.7.252
  netmask 255.255.255.0
  gateway 192.168.7.1
  bridge_ports bond0.7
  bridge_stp 0

I was somehow hoping to bind my container to that interface using a user-defined bridge and then not have to worry about VLANs at all on the docker side of things.

It seems the only way to make docker use a specific interface for egress is either macvlan or routing rules on the host:

• https://medium.com/@havloujian.joachim/advanced-docker-networking-outgoing-ip-921fc3090b09
• https://stewartadam.io/blog/2019/04/04/routing-packets-specific-docker-container-through-specific-outgoing-interface

If there’s a simpler way I’ve overlooked please tell me.