Admin stuff

Moving media files to self-hosted Minio S3 storage

In this guide, we will create a self-hosted MINIO S3 bucket called "mastodata" at the address "https://media.your-server-social:9000" and then proxy the bucket address as "" for use by Mastodon.

(When copying over instructions, watch out for the extra blank space in https: // prefixes.)

Minio server

Configure machine & install certificates

  1. sudo apt update && sudo apt upgrade -y
  2. Install & configure fail2ban
  3. Open firewall ports: 80 (http), 443 (https), 9000 (S3 API), 45135 (Admin console, you can pick a different port in your minio configuration)
  4. Ensure DNS entry for is there
  5. sudo apt install certbot
  6. sudo certbot certonly --standalone -d --staple-ocsp -m --agree-tos

See also:

Install Minio server with HTTPS

Create user & path
  1. adduser --disabled-login minio-user
  2. mkdir -p /storage/location/for/minio
  3. chown minio-user /storage/location/for/minio

Link certificates
  1. su - minio-user
  2. mkdir -p .minio/certs
  3. ln -s /etc/letsencrypt/live/ /home/minio-user/.minio/certs/public.crt
  4. ln -s /etc/letsencrypt/live/ /home/minio-user/.minio/certs/private.key
  5. exit

Install Minio, but don't start the service yet

Allow MinIO to use port 443:
sudo setcap 'cap_net_bind_service=+ep' /usr/local/bin/minio

There are 2 configuration files:

  • Systemd service: /usr/lib/systemd/system/minio.service
    • Follow instructions from the installation link above

  • Server config: /etc/default/minio
    • Set server URL with https: MINIO_SERVER_URL=""
    • Set fixed port for console to have it permanent in the firewall: MINIO_OPTS="--console-address :45135"

Test with:
  • sudo systemctl restart minio
  • systemctl status minio

The status output should contain 2 https links:

  • S3-API: https: //
  • Console: https: //

If it's working, enable the service:
sudo systemctl enable minio

See also Generate Let’s Encrypt certificate using Certbot for MinIO

Configure MinIO bucket

Visit the MinIO console at and log in with the MINIO_ROOT_USER from the server config file at /etc/default/minio.

  1. Go to Settings
  2. In Region, enter a region
  3. In API, double-check that "Cors Allow Origin" is set to "*"
  4. Create a bucket "mastodata"
  5. Configure access using the instructions from
  6. Visit the "mastodon" user, then Service Accounts. Create an access key there - this will be used by the Mastodon server to authenticate.
  7. Visit the "mastodata" bucket and add Anonymous access for "readonly"
  8. Double-check the bucket's custom policy to ensure that it won't allow the listing of objects

Mastodon server

Copy the media files

Before you start copying, consider deactivating all relays so that you won't accumulate too many new media files while you work.

Clean up the Mastodon cached media:

  1. su - mastodon
  2. live/bin/tootctl media remove --days=2
  3. live/bin/tootctl preview_cards remove --days=2
  4. live/bin/tootctl media remove-orphans

While that's running, install the media copy tool.

Set an alias to access the S3 bucket with the copy tool:
./mc alias set media https: //media.yourserver-social:9000 accesskey secretkey

Once the file cleanup has finished, you can do your first round of copying:

./mc cp -r --md5 /home/mastodon/live/public/system/site_uploads media/mastodata
./mc cp -r --md5 /home/mastodon/live/public/system/custom_emojis media/mastodata
./mc cp -r --md5 /home/mastodon/live/public/system/media_attachments media/mastodata
./mc cp -r --md5 /home/mastodon/live/public/system/accounts media/mastodata
./mc cp -r --md5 /home/mastodon/live/public/system/cache media/mastodata

Double-check the bucket file structure in the MinIO admin console to ensure that the paths are correct (e.g. no duplicated accounts/accounts)

Configure Mastodon

Create a proxy in nginx using the instructions from

Matching the example links above, this is what you replace in the nginx configuration:

  • server_name;
  • set $s3_backend 'https: //media.your-server-social:9000/mastodata';
  • proxy_set_header Host;
  • ssl_certificate /etc/letsencrypt/live/; # managed by Certbot
  • ssl_certificate_key /etc/letsencrypt/live/; # managed by Certbot

Edit /etc/nginx/sites-available/mastodon (create a backup first, just in case!) and change the contents of the ~ ^/system/ location:

location ~ ^/system/ {
rewrite ^/system(.*) https: //$1 permanent;

Edit /home/mastodon/live/.env.production and add the S3 configuration:


Restart the services:
sudo systemctl restart nginx
sudo systemctl restart mastodon-*

Now test your site. Right-click on the images and check they are being served from
  • Header on the "about page"
  • Custom Emojis
  • Profile headers & avatars
  • Preview cards
  • Existing media upload
  • New media upload
  • Media federate correctly to other servers

Visit the admin panel for any warnings about the S3 security configuration.

If media addresses are correct and you can see them in a new tab, but they are blank on your server, check the Content-Security-Policy headers in the ~ ^/system/ and @proxy locations in /etc/nginx/sites-available/mastodon.

Copy any new media files

Once everything is confirmed to be working, you are ready to copy any new files since you last copied them.

We use the mirror option to save time. You can optionally define a time interval too to skip older files:

./mc mirror --overwrite --newer-than 1d2h30m -a --md5 /home/mastodon/live/public/system/ media/mastodon/

To empty out the server, we can also use the move command, but make sure to create a backup of your server's media files, in case anything should go wrong (you can skip the cache). This command does not come with an --md5 option.

./mc mv -r /home/mastodon/live/public/system/site_uploads media/mastodata
./mc mv -r /home/mastodon/live/public/system/custom_emojis media/mastodata
./mc mv -r /home/mastodon/live/public/system/media_attachments media/mastodata
./mc mv -r /home/mastodon/live/public/system/accounts media/mastodata
./mc mv -r /home/mastodon/live/public/system/cache media/mastodata

This will still leave you with a tree of empty directories, so:
cd /home/mastodon/live/public/system
rm -rf accounts/
rm -rf cache/
rm -rf custom_emojis/
rm -rf media_attachments/
rm -rf site_uploads/

You might also have accumulated some orphans during the process, so:
su - mastodon
live/bin/tootctl media remove-orphans

If you should be missing a remote status image here and there, you can restore them with the tootctl media refresh command.

And don't forget to reactivate your relays!

Automated backups

For setting up automated alerts and backups, follow the procedure in the Maintenance article.

I skip the cache to save disk space.

In the long run, it's probably best to set up a Multi-Drive MinIO.

rsync commands for settings & raw data:
  • rsync -av --delete-after -e ssh admin@media.server:/root/ /path/to/backup/media/root --exclude=".*"
  • rsync -av --delete-after -e ssh admin@media.server:/home/ /path/to/backup/media/home
  • rsync -av --delete-after -e ssh admin@media.server:/usr/lib/systemd/ /path/to/backup/media/usr/lib/systemd
  • rsync -av --delete-after -e ssh admin@media.server:/etc/default/ /path/to/backup/media/etc/default
  • rsync -av --delete-after -e ssh admin@media.server:/var/spool/cron/crontabs /path/to/backup/media/
  • rsync -av --delete-after -e ssh admin@media.server:/storage/location/for/minio/ /storage/location/for/minio --exclude=minio/mastodata/cache --exclude=minio/.minio.sys/tmp

You can also use mc (see installation instructions above). Configure once with:
./mc alias set media-server https: // mastodata password

Then run regularly:
  • /path/to/mc mirror --overwrite -a --md5 media-server/mastodata/accounts/ /path/to/backup/media/mastodata/accounts/
  • /path/to/mc mirror --overwrite -a --md5 media-server/mastodata/custom_emojis/ /path/to/backup/media/mastodata/custom_emojis/
  • /path/to/mc mirror --overwrite -a --md5 media-server/mastodata/site_uploads/ /path/to/backup/media/mastodata/site_uploads/
  • /path/to/mc mirror --overwrite -a --md5 media-server/mastodata/media_attachments/ /path/to/backup/media/mastodata/media_attachments/

My rsnapshot commands look like this:

  • backup_exec /path/to/mc mirror --overwrite --remove -a --md5 media-server/mastodata/accounts/ /path/to/backup/media/mastodata/accounts/
  • backup_exec /path/to/mc mirror --overwrite --remove -a --md5 media-server/mastodata/custom_emojis/ /path/to/backup/media/mastodata/custom_emojis/
  • backup_exec /path/to/mc mirror --overwrite --remove -a --md5 media-server/mastodata/site_uploads/ /path/to/backup/media/mastodata/site_uploads/
  • backup_exec /path/to/mc mirror --overwrite --remove -a --md5 media-server/mastodata/media_attachments/ /path/to/backup/media/mastodata/media_attachments/
  • backup /path/to/backup/media/mastodata media/

Last edited: 27 August 2023 12:23:33