How to set up a Mastodon dev instance on an M1 Macbook Air

MAC_DEV.md

How to run a Mastodon dev instance on an M1 Macbook Air

I think I now hoave a minimal set of steps required to get Mastodon dev running on an M1 Macbook Air. I may have missed some, because I got there through a bunch of trial and error, which may have installed extra dependencies. If this doesn't work, please let me know.

These instructions are based around https://docs.joinmastodon.org/dev/setup/#vagrant and https://github.com/ppggff/vagrant-qemu .

• Start with a working Homebrew install.
• brew install vagrant
• vagrant plugin install vagrant-hostsupdater
• brew install qemu
• vagrant plugin install vagrant-qemu
• Enable SMB, following https://developer.hashicorp.com/vagrant/docs/synced-folders/smb#macos-host
• git clone https://github.com/mastodon/mastodon.git
• Replace the supplied Vagrantfile file with mine.
• Edit the file, replacing 192.168.1.150 with your machine's IP address.
• vagrant up

That should pretty much be it, and if all's well you can access your Mastodon dev instance on mastodon.local:3000.

If it's not working... look carefully at the logs. vagrant up --debug is your friend. Vagrant has a very lax approach to error-checking, but you can probably get your instance into a working state by doing a vagrant ssh into the VM, and running through the scripted steps one-by-one, correcting errors as you go. Good luck.

Issues I faced

An alternative to "How to do this right" is "How I did this wrong", which may both motivate my guide above, and help people debug some of the failure modes.

This is an abridged version, cutting out a fair number of dead-ends and silly mistakes I don't wish to own up to.

My main machine is an M1 Macbook Air. I thought, as such a stereotypical dev machine, it would be well-supported, but I think I was wrong!

My starting point was the Vagrant section of https://docs.joinmastodon.org/dev/setup/ . If I'm building a thing with a bunch of services, including database, I really want to put it inside some kind of VM, because I hate the idea of stateful weirdness floating around on my machine. I want it all in a little box I can throw away, leaving my physical machine in a sensible, clean state.

Of the virtualisation options, Vagrant seemed... well, designed for the task. The right tool for building temporary dev environments. How hard can it be? Off I go. Did I mention I've never used it before?

So, Vagrant depends on something to run the VMs. How about VirtualBox? Sounds great, apart from the fact it doesn't work on Apple M1 ARM processors (aka aarch64). Well, it can, but then I tried actually installing it and it kept crashing.

Ok. Change of plan. How about something that actually runs on M1? I tried qemu, using https://github.com/ppggff/vagrant-qemu . This rather glosses over my actual experience, which involved eventually realising that trying to run x86 images on aarch64 virtualisation isn't going to end well. For a while I thought I should be trying to run an x86 virtualised environment via emulation, in order to match vanilla Mastodon as much as possible, but... nah, making my virtual env ARM was fine.

I also got a bit of confusion as to whether I should be trying to access qemu through qemu or libvirt, but in the end I seem to have settled on qemu. I also got to learn way more than I wanted to about how to start up qemu VMs.

I think it was around this point I had a VM that booted, and could be accessed via vagrant ssh, but just didn't work. The ppggff/centos-7-aarch64-2009-4K box doesn't do apt, which the Mastodon Vagrantfile assumes, but I switched to perk/debian-11-genericcloud-arm64, which seemed to do the job.

Vagrant was very unhappy with the Vagrantfile trying to use NFS. It pushed me towards SMB. Of course, SMB didn't work either, and for multiple reasons. First, you need to make it work on the host side. Then you need to make it work on the guest side.

It took me far too long to work out that when it was unhappy with SMB on the guest side, this was not a problem with SMB, it was a problem with the guest accessing the host network. It then took me even longer to realise that actually host network access was OK, once I'd worked out how to configure it, it's just "ping" doesn't work. I was somewhat distracted by the warning that config.vm.network doesn't work on QEMU. I think this may be (partial) lies.

If only I'd read the instructions more carefully!

I now have a box that boots and doesn't complain about much and even has a /vagrant directory mounted... but it doesn't work. Random messy errors all over the place. Have I mentioned hashicorp/vagrant#8301 ? Vagrant provisioning scripts don't care about failing, so the set-up was broken, and it didn't care or want to warn me. Awesome.

The thing that caused me the most hassle was yarn. Why it was a problem, I didn't know, because it's just a test suite thing. EXCEPT IT'S NOT. It's a package manager thing with the same name as a test suite thing. Guess which one I got installed? Yes! The test suite I didn't want, also known as cmdtest.

What's going on here? Well, the Mastodon Vagrant provisioning script calls apt-add-repository, which doesn't exist on the box I'm using, so it silently decides to install the wrong Yarn, and everything's a mess. Other stuff broke, and I have no idea if it's because of this, or something else.

Anyway, long story short, after much manual wrangling inside vagrant ssh, the thing seems to start up. A further few iterations, and I think I have it working in a scripted manner.

Vagrantfile

# -*- mode: ruby -*-
# vi: set ft=ruby :

ENV["PORT"] ||= "3000"

$provision = <<SCRIPT

cd /vagrant # This is where the host folder/repo is mounted

# Add the yarn repo + yarn repo keys
sudo apt-get install gnupg -y
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list

# Add repo for NodeJS
curl -sL https://deb.nodesource.com/setup_14.x | sudo bash -

# Add firewall rule to redirect 80 to PORT and save
sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port #{ENV["PORT"]}
echo iptables-persistent iptables-persistent/autosave_v4 boolean true | sudo debconf-set-selections
echo iptables-persistent iptables-persistent/autosave_v6 boolean true | sudo debconf-set-selections
sudo apt-get install iptables-persistent -y

# Add packages to build and run Mastodon
sudo apt-get install \
  git-core \
  g++ \
  libpq-dev \
  libxml2-dev \
  libxslt1-dev \
  imagemagick \
  nodejs \
  redis-server \
  redis-tools \
  postgresql \
  postgresql-contrib \
  yarn \
  libicu-dev \
  libidn11-dev \
  libreadline-dev \
  libpam0g-dev \
  -y

# Install rvm
read RUBY_VERSION < .ruby-version

curl -sSL https://rvm.io/mpapis.asc | gpg --import
curl -sSL https://rvm.io/pkuczynski.asc | gpg --import

curl -sSL https://raw.githubusercontent.com/rvm/rvm/stable/binscripts/rvm-installer | bash -s stable --ruby=$RUBY_VERSION
source /home/vagrant/.rvm/scripts/rvm

# Install Ruby
rvm reinstall ruby-$RUBY_VERSION --disable-binary

# Configure database
sudo -u postgres createuser -U postgres vagrant -s
sudo -u postgres createdb -U postgres mastodon_development

# Install gems and node modules
gem install bundler foreman
bundle install
yarn install

# Build Mastodon
export RAILS_ENV=development
export $(cat ".env.vagrant" | xargs)
bundle exec rails db:setup

# Configure automatic loading of environment variable
echo 'export RAILS_ENV=development' >> ~/.bash_profile
echo 'export $(cat "/vagrant/.env.vagrant" | xargs)' >> ~/.bash_profile

SCRIPT

$start = <<SCRIPT

echo 'To start server'
echo '  $ vagrant ssh -c "cd /vagrant && foreman start"'

SCRIPT

VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

  config.vm.box = "perk/debian-11-genericcloud-arm64"

  config.vm.provider :virtualbox do |vb|
    vb.name = "mastodon"
    vb.customize ["modifyvm", :id, "--memory", "2048"]
    # Increase the number of CPUs. Uncomment and adjust to
    # increase performance
    # vb.customize ["modifyvm", :id, "--cpus", "3"]

    # Disable VirtualBox DNS proxy to skip long-delay IPv6 resolutions.
    # https://github.com/mitchellh/vagrant/issues/1172
    vb.customize ["modifyvm", :id, "--natdnsproxy1", "off"]
    vb.customize ["modifyvm", :id, "--natdnshostresolver1", "off"]

    # Use "virtio" network interfaces for better performance.
    vb.customize ["modifyvm", :id, "--nictype1", "virtio"]
    vb.customize ["modifyvm", :id, "--nictype2", "virtio"]

  end

  # This uses the vagrant-hostsupdater plugin, and lets you
  # access the development site at http://mastodon.local.
  # If you change it, also change it in .env.vagrant before provisioning
  # the vagrant server to update the development build.
  #
  # To install:
  #   $ vagrant plugin install vagrant-hostsupdater
  config.vm.hostname = "mastodon.local"

  if defined?(VagrantPlugins::HostsUpdater)
    config.vm.network :private_network, ip: "192.168.42.42", nictype: "virtio"
    config.hostsupdater.remove_on_suspend = false
  end

  # Set to your machine's IP address. See https://github.com/ppggff/vagrant-qemu#1-failed-to-create-shared-folder for details.
  config.vm.synced_folder ".", "/vagrant", type: "smb", smb_host: "192.168.1.150"

  # Otherwise, you can access the site at http://localhost:3000 and http://localhost:4000 , http://localhost:8080
  config.vm.network :forwarded_port, guest: 3000, host: 3000
  config.vm.network :forwarded_port, guest: 4000, host: 4000
  config.vm.network :forwarded_port, guest: 8080, host: 8080

  # Full provisioning script, only runs on first 'vagrant up' or with 'vagrant provision'
  config.vm.provision :shell, inline: $provision, privileged: false

  # Start up script, runs on every 'vagrant up'
  config.vm.provision :shell, inline: $start, run: 'always', privileged: false

end