Skip to content

Instantly share code, notes, and snippets.

@stiltr

stiltr/!Overview.md

Last active February 4, 2022 03:35
Show Gist options
  • Save stiltr/b392f14b18a121f4c3510cf355927631 to your computer and use it in GitHub Desktop.
Save stiltr/b392f14b18a121f4c3510cf355927631 to your computer and use it in GitHub Desktop.
Download ZIP
EspressoBin v5 Router
Raw
!Overview.md

EspressoBin v5 Router

I use my EspressoBin as a router and it does a great job of it. It handles DHCP, DNS and firewalling for my network across multiple VLANs. Initially I tried using Arch Linux and Armbian, which had support for the board at the time, but I had various issues with those and eventually whipped up my own Devuan image. It has been quite stable for me across several releases (ascii->beowulf->chimaera). I do however, still use the kernel package from Armbian's repo.

My plan is to leave some notes here on some various things I've done with the EspressoBin since setting it up as my router here at work.

Raw
IPv6.md

IPv6 - WIP

After having my router running well for a while, I eventually realized that my ISP supported IPv6 and would, upon request, grant me a /56 to play with. (Protip: just because you haven't explicitly enabled IPv6 doesn't mean you aren't getting assigned a globally routable address and may or may not be hanging out in the breeze without a firewall. It's not nearly the concern it would be with IPv4, but it's still worth checking out.) I was eventually able to get this working and assign each of my VLANs a /64. Here's how I did it.

More to come...

Raw
RTC.md

Setting up an RTC module on the EspressoBin v5.

Disclaimer: I'm writing this up from memory with the help of bash history, so it's possible I might have skipped a step here or there. If you find something that doesn't quite make sense, let me know and I'll try and fix it.

After some minimal research and a quick check on linux kernel compatibility I grabbed the DS3231 breakout board from Adafruit. It seemed a bit expensive, but it was available on Amazon with fast and free shipping, so I pulled the trigger.

Once it arrived, I grabbed some wire and my soldering iron and made up a harness to get it connected to the EspressoBin. According to the documentation, power was available on pins 2 & 4, and I2C is on 20 & 21. Easy enough!

So now that it was hooked up, could I see it from linux?

stiltr@router:~$ sudo i2cdetect
-bash: i2cdetect: command not found

Ok, dumb, install i2c-tools: sudo apt install i2c-tools

stiltr@router:~$ sudo i2cdetect 
Error: No i2c-bus specified!
Usage: i2cdetect [-y] [-a] [-q|-r] I2CBUS [FIRST LAST]
       i2cdetect -F I2CBUS
       i2cdetect -l
  I2CBUS is an integer or an I2C bus name
  If provided, FIRST and LAST limit the probing range.

Oh, right, better check which buses the kernel sees.

stiltr@router:~$ ls /dev/ | grep i2c
stiltr@router:~$

That's a problem.

So it turns out that the kernel I'm using from Armbian doesn't enable I2C. Now I could build my own kernel, but that would likely take a little trial and error to get it just right. Plus, I'd have to manage it on my own whereas now, I just pull the latest kernel from their repos. So for simplicity's sake I elected to just compile kernel modules and load them that way. Granted, I'll still have to recompile the modules when I update the kernel, but that's not a huge deal.

I hopped over to my LXD container that's setup to cross-compile for arm64 and used git to pull the source for the 5.10.y branch since I'm running a 5.10.60 kernel.

stiltr@router:~$ uname -a
Linux router 5.10.60-mvebu64 #21.08.1 SMP PREEMPT Wed Aug 25 19:19:23 UTC 2021 aarch64 GNU/Linux

(LXD)root@buildbot-1: ~/espressobin-kernel # git clone --depth 1 --branch linux-5.10.y --single-branch git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git

Now at this point I mucked around with the config for a bit, including grabbing the config from Armbian.

wget https://raw.githubusercontent.com/armbian/build/master/config/kernel/linux-mvebu64-current.config

Eventually I remembered that the config for the exact kernel I'm running is sitting in /boot/config-5.10.60-mvebu64. That proved to be a better starting point. I copied that over and renamed it to .config. I ran a make menuconfig and everything looked ok.

After some research I found that I2C_PXA is the module we need to enable I2C. I hunted for it, but I couldn't find it, even though all it's dependencies seemed to be satisfied. I think it has something to do with ARCH_MVEBU not being recognized properly by menuconfig.

Symbol: ARCH_MVEBU [=ARCH_MVEBU]
Type  : unknown

In the end I just opened up the config with a text editor and enabled I2C_PXA as a module: CONFIG_I2C_PXA=m

For the RTC, the DS3231 is covered by the DS1307 kernel module. After a quick search I found and enabled that with menuconfig.

Now, the version of the kernel in git was a bit newer than the one I'm targetting, so I just changed the Makefile to show the same version as I needed. Technically I should probably have gone back to an earlier commit, but I didn't figure there were likely to be any changes that would impact the I2C module. I also added an extra version so that it would match the version string of my kernel. (More on that later.)

diff --git a/Makefile b/Makefile
index a113a2954..40dec5eb6 100644
--- a/Makefile
+++ b/Makefile
@@ -1,8 +1,8 @@
 # SPDX-License-Identifier: GPL-2.0
 VERSION = 5
 PATCHLEVEL = 10
-SUBLEVEL = 92
-EXTRAVERSION =
+SUBLEVEL = 60
+EXTRAVERSION =mvebu64
 NAME = Dare mighty things
 
 # *DOCUMENTATION*

So now we're ready to build the modules. There are ways to build individual kernel modules, but I was lazy and just had it build all of them.

export ARCH=arm64
export CROSS_COMPILE=aarch64-linux-gnu-
make modules -j20

Now that the modules are built, let's copy them over and see if they'll load.

stiltr@router:~$ sudo insmod rtc-ds1307.ko
insmod: ERROR: could not insert module rtc-ds1307.ko: Invalid module format
stiltr@router:~$ dmesg | grep ds1307
rtc_ds1307: version magic '5.10.60mvebu64+ SMP preempt mod_unload aarch64' should be '5.10.60-mvebu64 SMP preempt mod_unload aarch64'

Ok, so clearly I messed up on the version string. Luckily we can cheat, since the strings are the same length. Opening the module up in a hex editor will let you change the string to match. Luckily, there's an even easier way with the help of bbe.

stiltr@router:~$ bbe -e 's/5.10.60mvebu64+/5.10.60-mvebu64/' rtc-ds1307.ko > rtc-ds1307.ko.2; mv rtc-ds1307.ko{.2,};

Now, lets try again.

stiltr@router:~$ sudo insmod rtc-ds1307.ko
stiltr@router:~$ lsmod | grep rtc
rtc_ds1307             32768  0

Great! It worked and didn't crash anything. After the version string fix, I2C_PXA loaded as well.

Unfortunately, there were still no i2c devices under /dev.

After a little searching I found that the I2C controllers were disabled in the device tree.

i2c@11000 {
        compatible = "marvell,armada-3700-i2c";
        reg = <0x11000 0x24>;
        #address-cells = <0x1>;
        #size-cells = <0x0>;
        clocks = <0x2 0xa>;
        interrupts = <0x0 0x1 0x4>;
         mrvl,i2c-fast-mode;
        status = "disabled";
};

i2c@11080 {
        compatible = "marvell,armada-3700-i2c";
        reg = <0x11080 0x24>;
        #address-cells = <0x1>;
        #size-cells = <0x0>;
        clocks = <0x2 0x9>;
        interrupts = <0x0 0x2 0x4>;
        mrvl,i2c-fast-mode;
        status = "disabled";
};

I'd previously modified my device tree to enable the watchdog timer, so this was an easy enough fix. Ideally you'd start with the dts/dti files in the kernel source directory and rebuild it from there, but I went the easy route and just decompiled the binary with dtc. Something like this:

dtc -I dtb -O dts -o armada-3720-espressobin.dts armada-3720-espressobin.dtb

Then I just changed the status = "disabled" to status = "okay" for both of the i2c entries and recompiled.

dtc -O dtb -o armada-3720-espressobin.new.dtb -b 0 armada-3720-espressobin.dts

Copy this back over to the espressobin and backup and replace the existing dtb.

Now at this point, I went ahead and setup the modules to load automatically by adding them to the /etc/modules file and placing them in the proper modules folder. I also had to run depmod so it would know they were there.

stiltr@router:~$ echo -e '\n\n#Enable I2C\ni2c-pxa\n\n#Enable RTC\nrtc-ds1307' | sudo tee -a /etc/modules
#Enable I2C
i2c-pxa

#Enable RTC
rtc-ds1307
stiltr@router:~$ sudo cp i2c-pxa.ko /lib/modules/5.10.60-mvebu64/kernel/drivers/i2c/busses/
stiltr@router:~$ sudo cp rtc-ds1307.ko /lib/modules/5.10.60-mvebu64/kernel/drivers/rtc/
stiltr@router:~$ sudo depmod

Now it was time to reboot. I hooked up the serial console just in case, but it booted up without issue and now I could see the i2c devices!

stiltr@router:~$ ls /dev/ | grep i2c
i2c-0
i2c-1

So now let's see if we can see the RTC...

stiltr@router:~$ sudo i2cdetect 0
WARNING! This program can confuse your I2C bus, cause data loss and worse!
I will probe file /dev/i2c-0.
I will probe address range 0x08-0x77.
Continue? [Y/n] 
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:                         -- -- -- -- -- -- -- -- 
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
60: -- -- -- -- -- -- -- -- 68 -- -- -- -- -- -- -- 
70: -- -- -- -- -- -- -- --                         
stiltr@router:~$ sudo i2cdetect 1
WARNING! This program can confuse your I2C bus, cause data loss and worse!
I will probe file /dev/i2c-1.
I will probe address range 0x08-0x77.
Continue? [Y/n] 
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:                         
-- -- -- -- ^C

Sure enough! It shows up at address 68, just like you'd expect. I tried to scan i2c-1, but it seemed to hang and I already had what I wanted, so I killed it.

Now, I wasn't quite sure if the I2C_PXA module was actually the right one, since there was a CONFIG_I2C_MV64XXX=y already enabled in the kernel config that seemed more like what I would expect. A quick test showed that it was indeed the correct one.

stiltr@router:~$ sudo rmmod i2c_pxa 
stiltr@router:~$ ls /dev/ | grep i2c
stiltr@router:~$ sudo modprobe i2c_pxa
stiltr@router:~$ ls /dev/ | grep i2c
i2c-0
i2c-1
stiltr@router:~$

Ok, so let's see if we can talk to the RTC now.

stiltr@router:~$ echo ds1307 0x68 | sudo tee /sys/class/i2c-adapter/i2c-0/new_device
ds1307 0x68
stiltr@router:~$ dmesg | tail | grep -v inet-fw
[  606.329089] rtc-ds1307 0-0068: registered as rtc0
[  606.334496] rtc-ds1307 0-0068: setting system clock to 2000-01-01T03:03:31 UTC (946695811)
[  606.344920] i2c i2c-0: new_device: Instantiated device ds1307 at 0x68
stiltr@router:~$ date
Fri 31 Dec 1999 07:03:49 PM PST

Awesome! It worked! Not quite as awesome, the time was now roughly twenty years off and I was a handful of hours away from seeing how my router would handle Y2K...

stiltr@router:~$ sudo /etc/init.d/ntp stop
Stopping NTP server: ntpd.
stiltr@router:~$ sudo ntpd -gq
<SNIP>
 2 Feb 19:02:18 ntpd[3643]: ntpd: time set +697161421.036497 s
ntpd: time set +697161421.036497s
stiltr@router:~$ date
Wed 02 Feb 2022 07:02:20 PM PST
stiltr@router:~$ sudo service ntp start
Starting NTP server: ntpd.
stiltr@router:~$ date
Wed 02 Feb 2022 07:02:39 PM PST

Now that the date is fixed, let's set the RTC to the correct time.

stiltr@router:~$ sudo hwclock -w
stiltr@router:~$ date
Wed 02 Feb 2022 07:13:05 PM PST
stiltr@router:~$ sudo hwclock -r
2022-02-02 19:13:08.949115-08:00

Given that there was a few seconds in between those commands, it looks like everything is in good shape.

Now, adding the RTC via /sys/class/i2c-adapter/i2c-0/new_device is the old way of doing things and ideally this should be handled by the device tree. So let's modify that real quick. First I had to figure out which entry in the device tree mapped to which device in /dev/. I figured it was probably in order, but I wanted to be sure. Eventually I found that you could look at the clocks to see which was which.

stiltr@router:~$ cat /sys/class/i2c-adapter/i2c-0/device/of_node/clocks | xxd
00000000: 0000 0002 0000 000a                      ........
stiltr@router:~$ cat /sys/class/i2c-adapter/i2c-1/device/of_node/clocks | xxd
00000000: 0000 0002 0000 0009

Looking back at the dts file, we see that i2c@11000 has clocks = <0x2 0xa>;, while i2c@11080 has clocks = <0x2 0x9>;. So i2c@11000 is what we want. So here's our final version of the i2c@11000 node with an RTC module entry added based on the kernel documentation at Documentation/devicetree/bindings/rtc/rtc-ds1307.txt

i2c@11000 {
        compatible = "marvell,armada-3700-i2c";
        reg = <0x11000 0x24>;
        #address-cells = <0x1>;
        #size-cells = <0x0>;
        clocks = <0x2 0xa>;
        interrupts = <0x0 0x1 0x4>;
        mrvl,i2c-fast-mode;
        status = "okay";

        ds3231: rtc@68 {
                compatible = "maxim,ds3231";
                reg = <0x68>;
        };

};

Recompile with dtc, copy it back to the router and reboot!

stiltr@router:~$ dmesg | grep rtc
[    8.840141] rtc-ds1307 0-0068: SET TIME!
[    8.846287] rtc-ds1307 0-0068: registered as rtc0
[    8.860352] rtc-ds1307 0-0068: setting system clock to 2022-02-03T03:46:37 UTC (1643859997)

Awesome! Accurate time much earlier in the boot process than when depending on NTP. This fixed some issues with DHCP as well as making the logs for dnsmasq and others accurate. It was a bit of a pain to get here, but I wish I'd done it sooner.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment