P1 | Trying to create an open community mesh network on a shoestring budget

Introduction

Hey! In this series of blog posts, I am going to try and document my journey of attempting to setup an open community mesh network. What does this mean? It means that anybody can join and add to the mesh, (hence the “open”), and can expand the network for others, or just connect to it. This is not intending to be “free WiFi”, although in the future with internet gateways it very well could become that, but almost like a decentralized internet alternative. Read on to find out more about the network architecture.

Network Architecture

This network will utilize multiple technologies and is heavily based on MassMesh. This is a mesh network in Massachusetts, and they actually distribute home internet off of it. I don’t think the network that I’m trying to build will ever be competitive against the offerings from traditional ISPs in my area, but who knows! For multipoint to multipoint (mesh) between radios, they use 802.11s, but with it’s built in forwarding turned off. This means that when the neighboring devices get connected to each other, it is as if they are both plugged into an Ethernet switch. They know of the existence of the other device, but only have the MAC address of it, no IP addressing yet. Why? This is because they use a separate 3rd layer protocol that runs on top of the physical mesh, called Yggdrasil. I don’t have the time to explain the intricacies of this very cool decentralized IPv6 network, so please click the link (or if you are paranoid do a google search for “Yggdrasil mesh”) to learn more. Essentially, each device gets its own IPv6 address based on public key cryptography, it doesn’t change even if the device is reconnected to the network from another location, and peers can be connected over the “clearnet” (IPv4 and IPv6) as well as through local peering (multicast). This last part is important for our network, because if we bridge a radio that is part of the 802.11s mesh to a spare port (with multicast turned on), anything connected to that port will be able to “see” other clients on a neighboring node.

Initial Setup

After hypothesizing this idea in the MassMesh Matrix Server to ensure my basic (and crude) understanding of it, I began to search for hardware to do this on. But soon after, a member offered to ship me 10 AeroHive AP330 Access Points, which I gladly accepted! I paid for shipping, and sure enough, a few days later, 10 WAPs showed up at my doorstep! He said he got 25 of them for $2 each, and wanted the extras to get some use. Getting WAPs from a community member was also quite convenient as documentation had already been written, and I could flash OpenWRT to them with relative ease.

OpenWRT Flash

This part was pretty simple, just tedious. Reading the previously mentioned documentation, I was able to flash all 10 AP330s with OpenWRT. I had to order a $9 USB console cable online, but everything went smoothly. I did have to repeat the flashing process on each AP, however.

Yes, that is a 2008 Macbook Pro running Ubuntu. Still usable enough to flash 10 WAPs!

Mesh Setup

For simplicity, (and because I am not proficient with vim) I decided to configure the 802.11s mesh using the OpenWRT LuCI web interface. I started out by deleting the default wireless networks, and then created a new 802.11s network. I set it to use channel 11, defined a new “Network” (I called it “mesh”), and most importantly, unchecked “Forward mesh peer traffic”. For testing purposes, I left it unsecured, but in the future, I will probably add some kind of security for encryption. Then, in interfaces, I deleted the WAN interface. This is because the new “mesh” interface is effectively the “WAN” network. Make sure OpenWRT is up to date and all packages have been installed before this step, because this will break internet access. Configuring this mesh interface, I set the protocol to “unmanaged”, bring up on boot, force link and use default gateway. I’ll be honest, I don’t have a clue what most of these settings do, but hopefully in the future I will have a better configuration ready. In “Physical Settings”, I selected Bridge ports, and made sure “eth0” and “wlan0” were added. This will allow any computer connected to port eth0 on the AP to “see” other nodes over the 802.11s network. I also selected “bring up empty bridge”, “enable IGMP snooping”, and “enable multicast querier” (once again, not sure if these are required). In “Firewall Settings”, I created a new “mesh” firewall zone and assigned it. In the firewall settings for this zone, I put “accept” under Input, Output, and Forward.

Now for the important part, go into the “Devices” tab of the Interfaces page. For eth0, ensure that “Multicast routing” is set to “always”. It is crucial that this is done on the wlan0 interface as well, otherwise yggdrasil will not be able to form a local peer. This is what prevented my setup from working initially, so don’t make the same mistake I made.

I did not mess with any of the LAN settings. If I need to configure something, all I need to do is connect my laptop to the eth1 port of the AP330 and load up the LuCI interface.

Now, repeat this on another “node”. After looking at the status in OpenWRT, you should see an Associated Station, or another node on the same 802.11s network.

Assuming your configuration is correct, yggdrasil should work! I connected my macbook pro to the eth0 port on one of the WAPs, and connected an old laptop to a node in the room over. Then, typing “sudo yggdrasilctl getPeers” showed me that a local peer was found on both devices! Using ping6, I was able to ping each device, going over the 802.11s mesh network. Cool!

Conclusion

Now that we have a basic, 2 node network setup, I will conclude this first entry. Later, I will add more nodes to the setup, and setup services behind each of the nodes (because why not??). Yes, this is a sloppy post, but I hope it is interesting (or possibly helpful *gasp*) for some people out there. Please feel free to comment below or contact me on matrix here: @randomodbuild:matrix.org

Thanks for stopping by!