Meshtastic Bridging using Docker Compose

Starting with 2.6.4, Meshtastic introduced the idea of dumping mesh packets onto the local area network (LAN) using a UDP multicast. This essentially opened the door to allow for quick and easy bridging of modem presets over a network. What that means is that you could have nodes on LongFast, MediumFast, ShortSlow, etc. all able to talk to each other quickly and easily using these packets.

What this also means is that we can now run multiple instances of meshtasticd on a single machine and have them all communicate with each other over the local docker network.

Examples

Here's a quick example of a docker-compose.yml file that includes 2 instances of meshtasticd running side-by-side.

services:
  meshtasticd_node1:
    image: meshtastic/meshtasticd:beta-debian
    container_name: meshtasticd_node1
    devices:
      - "/dev/spidev0.0"
      - "/dev/gpiochip0"
    cap_add:
      - SYS_RAWIO
    ports:
      - 4403:4403
    volumes:
      - ./meshtasticd_node1/:/var/lib/meshtasticd
      - ./meshtasticd_node1.yml:/etc/meshtasticd/config.yaml:ro
    restart: unless-stopped

  meshtasticd_node2:
    image: meshtastic/meshtasticd:beta-debian
    container_name: meshtasticd_node2
    devices:
      - "/dev/bus/usb/001:/dev/bus/usb/001"
    group_add:
      - "plugdev"
    ports:
      - 4404:4403
    volumes:
      - ./node2/:/var/lib/meshtasticd
      - ./node2.yml:/etc/meshtasticd/config.yaml:ro
    restart: unless-stopped

Each of these docker containers utilizes a different way of communicating with a radio. The first container (ShortSlow) uses SPI to connect to a radio hat via GPIO headers. The second container (LongFast) passes in a device for a USB radio. Each has their own dedicated config.yml file that's specific to their radio chipset and pinouts.

Finding Your USB Device

To find the path of your USB radio, you'll need to run a few commands in your terminal. All of the following commands work on a Raspberry Pi 3B+ and Raspbery Pi 4. I can't promise they'll work on a Mac or PC though, so your mileage may vary.

  1. Plug in your USB radio to your Raspberry Pi

  2. run lsusb and look for a device matching your manufacturer. I've used a MeshToad in the past and typically look for a manufacturer that looks like this:

Bus 001 Device 007: ID 1a86:5512 QinHeng Electronics CH341 in EPP/MEM/I2C mode, EPP/I2C adapter

  1. Write down the bus number and device number from the lsusb command. From the example above, it would be 001 for the bus and 007 for the device.

  2. Run ls /dev/bus/usb/{USB Bus Number} to make sure that your device number shows up here.

  3. Modify your docker-compose.yml and pass in the full path of your USB device into the container. You can pass in the entire USB bus directory, but I do recommend passing in the direct path if you can help it. ESPECIALLY if you're using 2 USB radios at the same time.

  4. Start your docker containers and test for connectivity.

Finding your SPI Device

SPI is a bit easier than USB devices. Here's what you have to do:

  1. Make sure SPI is enabled by running sudo raspi-config and going to Interface Options and enabling SPI.

  2. Power down your Pi and install the radio hat.

  3. Boot the Raspberry Pi back up and check that you have a directory for the SPI device by running ls /dev/spidev* It should be spidev0.0 or spidev0.1. You can also run modprobe spidev to see if it's showing up properly. For what it's worth, 99.999% of the time for me it's spidev0.0.

  4. Pass this device path into the docker container. You may need to enable additional permissions as well. See the example above.

  5. Start your container and test.

Configuring 2 Instances of Meshtasticd

The real tricky business with having 2 copies of meshtasticd running at the same time is that there are overlapping usage of the default port 4403. If you see in the LongFast node, I manually mapped port 4404:4403 to get around this. The problem with this is that as of right now, only the Android app allows you to specify the port number when connecting to a TCP interface. This may change in the future though.

If you don't have this option, what you can do is spin up one container at port 4403, configure it, spin it down, then spin up the second container on the same port, and configure that second instance. Then once both nodes are configured, remap the ports to not overlap and then spin them up in tandem. A bit of a pain, I know, but it works.

In Practice

I've utilized the configuration above to test out a single node that bridges LongFast to ShortSlow and other modem presets. The main purpose of this is to help migrate the local mesh to a spread factor that supports more traffic while also allowing new users using the default of LongFast to still be heard. We're seeing channel usage upwards of 35% at some of our infrastructure nodes, which starts to enter the "degraded network" territory.

There are a few nodes in the Metro Atlanta area that utilize this bridge, most of them bridging Longfast with either MediumFast or ShortSlow.

A Word of Warning

One thing I have noticed in my own testing with bridged networks is that you really need to ensure that you can transmit from your node to other nodes successfully on the first try. What I've noticed is that the UDP packets bridging networks can stop the primary transmitter from re-trying a message transmit over LoRa if it has not received a receipt yet. The UPD ack from the bridged node essentially silences any follow-up retransmissions.

For instance, I had my primary node up on my roof with a 1-watt output. I had that connected to a MediumFast node in my basement for testing via UDP. I would try to transmit from the roof node and it was not escaping the house. However, when I powered-down the MediumFast basement node, it was able to retry transmissions in the case of collisions.

The best way around this would be to have your bridged UDP network at a serious height advantage and continue to evaluate the current modem preset you're using to match the demands of the network.