Services that are (relatively) easy to set up Link to heading
Some services are easy to host in a FreeBSD jail because they don’t require too many additional steps to get them running. On this page, I am only going to provide links or references on how to install them in a jail plus any additional steps that I have used. These services are generally provided as pre-compiled packages (or available to build as ports) where the dependency-hell has been sorted out for you. Most of the time, the author(s) of the services/packages has provided a guide on what needs to be done to get the service up and running.
Postgresql Link to heading
I mention postgresql first because it is typically my database of choice. To get postgresql to run in a jail, you will need to allow access to something called ‘System V shared memory’. I don’t know what that means, but Postgresql won’t run if you don’t allow it.
There are two ways to allow a jail access to shared memory: the ‘proper` way and the outdated, insecure, deprecated way.
The proper way Link to heading
This is the proper way to allow your jail to access shared memory that is recommended by the FreeBSD jail man page.
Add the following line to your jail.conf file:
sysvshm = new;
Or you can simply run:
host # bastille config JAIL set sysvshm=new
The old way Link to heading
Don’t use them.
There are two versions of the old way.
The first was to set the following on the host which applied this to all jails:
security.jail.sysvipc_allowed=1
Alternatively, you set this on a per jail-basis in the jail.conf file:
allow.sysvipc=1
Don’t do this.
I mention the old instructions here only so that you recognise them. There are lots of pages that still present these outdated instructions, probably because this was the original way of letting the jail access shared memory. The FreeBSD jail man page explicitly states that this method is deprecated in favour of sysvshm, sysvmsg, and sysvsem. These methods are deprecated because they allow the jail to see all of the shared memory on the host which introduces unnecessary security compromises that defeat ths purpose of jails. For more information, have a read of skyforge.at, but in short: “Don’t use allow.sysvipc=1, instead set sysvmsg, sysvsem and sysvshm to “new”, if you need to use sysvipc inside a jail.” Despite this, I came across pages written in November 2023 still suggesting these methods. (I won’t include any links because I don’t want to give the impression that I am shaming anyone).
Mastodon Link to heading
Hosting a Mastodon instance in a jail is now easier than ever. Since late-2022 Muhammad Moinur Rahman has been maintaining a package and installation instructions for version 4 of Mastodon which makes it easy to install. To get mastodon up and running in a jail, you can follow the installation instructions by Muhammad and the notes by Stefano Marinelli for setting up the jail. Specifically, you will need to add lines to the jail configuration file (mentioned above) to allow postgresql to run.
Running a single-user instance Link to heading
Once the application is up and running, you will need to do a few things to allow your mastodon instance to find material on other servers. Julia Evans and Stefan Bohacek have put together some good tips on running a single-user instance. Personally, I find that relays (e.g. FediBuzz) and FediFetcher allow me to find content that I want to see/read.
Setting up FediFetcher Link to heading
The instructions on the FediFetcher github page will let you get FediFetcher up and running very easily and quickly. You will then notice that replies to posts will start to get populated into your timeline.
Here is what I did to have FediFetcher run every 15 minutes via a cron job:
Step one, create a config.json file:
{
"access-token": <REDACTED>,
"server": "mastodon.brendans-bits.com",
"home-timeline-length": 200,
"max-followings": 80,
"max-followers": 80,
"from-notifications": 1,
"http-timeout": 60
}
Step two, set up a cron job:
jail $ crontab -l
*/15 * * * * /bin/sh /usr/local/www/mastodon/FediFetcher/artifacts/FediFetcher.sh
Step three, create a script file:
#!/bin/sh
/usr/local/bin/python3.9 \
/usr/local/www/mastodon/FediFetcher/find_posts.py \
--config=/usr/local/www/mastodon/FediFetcher/artifacts/config.json \
--state-dir=/usr/local/www/mastodon/FediFetcher/artifacts \
| logger -p local1.info
(I cloned FediFetcher into my mastodon folder, but it can go anywhere)
The last line of this script is optional. It is only there to log the output of the FediFetcher tools so that you can see what is going on.
Alternatively, you can simply redirect the output to a file and then skip step four:
#!/bin/sh
/usr/local/bin/python3.9 \
/usr/local/www/mastodon/FediFetcher/find_posts.py \
--config=/usr/local/www/mastodon/FediFetcher/artifacts/config.json \
--state-dir=/usr/local/www/mastodon/FediFetcher/artifacts \
>> /usr/local/www/mastodon/FediFetcher/log/`date +\%Y\%m\%d\%H\%M\%S`-cron.log 2>&1
Step four, set up logging: (only if you are using the logger utility)
# Create the syslog.d folder
jail # mkdir -p /usr/local/etc/syslog.d/
# Create fedifetcher.conf
jail # vim /usr/local/etc/syslog.d/fedifetcher.conf
# Put a line like this into the .conf file
local1.info /var/log/fedifetcher.log
# Restart the syslogd service
jail # service syslogd restart
Step 5: run FediFetcher for the first time:
jail $ sh FediFetcher/artifacts/FediFetcher.sh
Step 6: tail the log file to make sure that everything is working:
jail # tail -f /var/log/fedifetcher.log
Dec 17 20:54:05 mastodon-app mastodon[31336]: 2023-12-17 20:53:16.212789 AEDT: Added context url https://mastodon.coffee/@xinit/111590987410678014
Dec 17 20:54:05 mastodon-app mastodon[31336]: 2023-12-17 20:53:19.980956 AEDT: Added context url https://hachyderm.io/@Luisa_Donato/111586055948592693
Dec 17 20:54:05 mastodon-app mastodon[31336]: 2023-12-17 20:53:19.980999 AEDT: Added 2 new context toots (with 0 failures)
Dec 17 20:54:05 mastodon-app mastodon[31336]: 2023-12-17 20:53:25.762127 AEDT: Added context url https://tech.lgbt/@LilahTovMoon/111585658427553813
Dec 17 20:54:05 mastodon-app mastodon[31336]: 2023-12-17 20:53:26.702738 AEDT: Got context for toot https://tech.lgbt/@LilahTovMoon/111585658427553813
Dec 17 20:54:05 mastodon-app mastodon[31336]: 2023-12-17 20:53:26.702797 AEDT: Found 10 known context toots
Dec 17 20:54:05 mastodon-app mastodon[31336]: 2023-12-17 20:53:33.001886 AEDT: Added context url https://techhub.social/@LMNOChris/111586237435899385
Dec 17 20:54:05 mastodon-app mastodon[31336]: 2023-12-17 20:53:38.815461 AEDT: Added context url https://better.boston/@timmc/111586957828569287
Dec 17 20:54:05 mastodon-app mastodon[31336]: 2023-12-17 20:53:40.174580 AEDT: Added context url https://infosec.exchange/@paul_ipv6/111592834860299499
Dec 17 20:54:05 mastodon-app mastodon[31336]: 2023-12-17 20:53:41.222196 AEDT: Added context url https://dmv.community/@jcrabapple/111585731806305998
Dec 17 20:54:05 mastodon-app mastodon[31336]: 2023-12-17 20:53:44.205304 AEDT: Added context url https://evil.social/notes/9nc9s2nemqa47o29
Dec 17 20:54:05 mastodon-app mastodon[31336]: 2023-12-17 20:53:45.006112 AEDT: Added context url https://dmv.community/@jcrabapple/111585771523890999
Dec 17 20:54:05 mastodon-app mastodon[31336]: 2023-12-17 20:53:49.205586 AEDT: Added context url https://tech.lgbt/@normjess/111586686368322374
Dec 17 20:54:05 mastodon-app mastodon[31336]: 2023-12-17 20:53:54.211339 AEDT: Added context url https://fedi.ikeran.org/notes/657c907a95e3b8ae6f9666bb
Dec 17 20:54:05 mastodon-app mastodon[31336]: 2023-12-17 20:53:56.801458 AEDT: Added context url https://mas.to/@BackFromTheDud/111592744386346963
Dec 17 20:54:05 mastodon-app mastodon[31336]: 2023-12-17 20:54:02.759203 AEDT: Added context url https://hachyderm.io/@stonebear/111586309387202851
<AND SO ON....>
The first run will take a very long time, so go out for a coffee.
Separating out the cache directory Link to heading
I found that the mastodon directory quickly grows in size, especially if you have relays sending data to your instance.
Examining the folder more closely revealed that almost all of the growth was in the mastodon/public/system folder.
jail # du -sh /usr/local/www/mastodon/* | sort -h
<trimmed output>
24M /usr/local/www/mastodon/app
76M /usr/local/www/mastodon/tmp
160M /usr/local/www/mastodon/vendor
825M /usr/local/www/mastodon/node_modules
34G /usr/local/www/mastodon/public
This folder holds all of the images, videos, avatars, etc that mastodon collects in case I want to view it. One way to deal with this is to use an aggressive cleaning policy - basically get mastodon to delete everything that is more than a set nuber fo days old. But this still leaves files that are not strictly necessary to run mastodon and make the backups much larger.
If I have a separate public/system folder and I have to restore the mastodon jail from a backup, I will lose the stored media. Annoying: not at all, I’m the only one who uses this instance. End of the world: certainly not, I can get mastodon to retrieve anything that is missing.
I created a separate ZFS dataset outside the mastodon jail which would hold all of this stuff. Mounting this ZFS dataset into the jail was very easy. All I had to do was add the following line to the jail fstab file:
/storage/mastodon /usr/local/bastille/jails/mastodon-app/root/usr/local/www/mastodon/public/system nullfs rw 0 0
This lets me back up my mastodon jail (~4G) without pulling in 34G of stuff that mastodon could retrieve if needed.
Giving mastodon and fedifetcher a separate DNS resolver Link to heading
Fedifetcher makes lots of DNS requests when it runs. When I first installed Fedifetcher, all of these requests were going via my Adguard server. By default, Adguard limits the number of requests per second, and Fedifetcher was exceeding this limit.
The workaround was to give the mastodon jail its own DNS server: unbound.
By default, FreeBSD comes with unbound_local, which can cache DNS records in the jail. This is the easiest to set up. All you have to do is enable the unbound_local service:
# sysrc unbound_local=yes
And then start it:
# service unbound_local start
Starting unbound_local will overwrite your /etc/resolv.conf file and use itself as the DNS cache.
But let’s go a step further…
Calomel has put together a good write up on how to install an authoritative DNS server.
I did a few things differently. When installing files needed for unbound I changed the directory from /var/unbound/... to /usr/local/etc/unbound/.
# /usr/local/etc/unbound/unbound.conf
server:
interface: 127.0.0.1
port: 53
cache-min-ttl: 0
cache-max-ttl: 86400
root-hints: "/usr/local/etc/unbound/root.hints"
hide-identity: yes
hide-version: yes
prefetch: yes
local-zone: "brendans-bits.com." static
local-data: "mastodon.brendans-bits.com. IN A 10.10.10.21"
python:
dynlib:
remote-control:
control-enable: yes
control-interface: 127.0.0.1
control-port: 8953
control-use-cert: "no"
#/etc/resolv.conf
nameserver 127.0.0.1
This one trick will reduce your Adguard traffic by 90%!