Music syncing with Nextcloud

April 02, 2020 in digital music by Dan Gravell

What’s difficult doesn’t get done. I’m often reminded of that when I realise I forgot to sync my phone with my latest music. Such is the plight of the self hoster.

So something I’ve been looking into recently is automated synchronisation of music libraries. Ideally I want a way that my phone, an SD card, and other locations get their data synchronised automatically. Here’s one way of doing this - using Nextcloud.

Nextcloud is self hosted server software that works a bit like Google Drive or Microsoft OneDrive. The big difference is that self hosted bit.

You can upload or create files on the service, and edit them with various web applications. You can store your documents, presentations, photos, videos and more. And, of course, you can store your music.

“Self hosted” means you own the infrastructure on which Nextcloud runs. That doesn’t just mean you have to install it on a server or computer in your house; you could equally use a rented server in something like Amazon EC2. In addition, you can purchase devices with Nextcloud already installed so it’s plug-and-play, or you can rent Nextcloud ready-hosted by a provider.

Nextcloud is free to use (support costs though), so I decided to give Nextcloud a go - here’s my walkthrough of setting up music synchronisation.

Installation

Aiming to get an easy proof of concept up and running, I decided to use the Docker installation method. There are lots of other ways to install, though.

Docker is easy and convenient because it is a pre-packaged virtual machine image which you can just run and everything is already setup in terms of software installation; all you have to do is configure it.

$ sudo apt install docker.io
$ usermod -a -G docker [your username]

We’re now able to install Nextcloud. I issued a docker pull command, which downloads the Docker image from Docker Hub (an online repository of Docker images - and yeah, there are Docker images for bliss too)

$ docker pull nextcloud
Using default tag: latest
latest: Pulling from library/nextcloud
68ced04f60ab: Downloading  5.024MB/27.09MB
1d2a5d8fa585: Download complete 
5d59ec4ae241: Downloading  7.515MB/76.65MB
d42331ef4d44: Download complete 
408b7b7ee112: Downloading  3.677MB/18.68MB
570cd47896d5: Waiting 
2419413b2a16: Waiting
... 

So that’s the image downloaded, and ready to run.

When Nextcloud runs it has a Web interface, like bliss. We need a port on which the Web interface can run.

$ sudo lsof -i -P -n | grep LISTEN
rpcbind    1103                root    8u  IPv4  19126      0t0  TCP *:111 (LISTEN)
rpcbind    1103                root   11u  IPv6  19129      0t0  TCP *:111 (LISTEN)
rsync      1120                root    4u  IPv4  26710      0t0  TCP *:873 (LISTEN)
rsync      1120                root    5u  IPv6  26711      0t0  TCP *:873 (LISTEN)
...
...
java       1499               music    4u  IPv4  33856      0t0  TCP 192.168.0.3:8000 (LISTEN)
java       1499               music   10u  IPv6  32067      0t0  TCP 127.0.1.1:3222 (LISTEN)
java       1499               music  120u  IPv6  32445      0t0  TCP *:3220 (LISTEN) **<-- bliss!**
...
java       1764             tomcat8   41u  IPv6  33941      0t0  TCP *:8080 (LISTEN)
...

In my case the default port 8080 was already taken, so I’ll arbitrarily choose 8081.

The next thing to do is make our music accessible to Nextcloud. The easiest way to do this is map existing storage on the host computer to the Nextcloud container.

I create a folder which we’ll be able to use for Nextcloud data:

sudo mkdir /mnt/vm/docker/nextcloud.data

(There’s nothing special about /mnt/vm/docker, it’s just where I store Docker related files on this host).

I copy music files into a well-known location inside nextcloud.data:

$ sudo -u www-data cp -r /mnt/media/music.nextcloud nextcloud.data/admin/files

Copying to admin/files means the files are stored for the admin user.

Finally we need to setup permissions otherwise we will get an error:

Can’t create or write into the data directory /var/www/html/data

That’s because our new data folder doesn’t have the correct permissions. The source directory /mnt/vm/docker/nextcloud.data must be readable and writeable by the www-data user in the Nextcloud container. The easiest way to do this is to find out the user and group ID, then chown the source directory.

This means we need to start the Docker container, just to retrieve the user ID:

$ docker run -d nextcloud
f366db973377237b6190c0ae3cc3984bfc34ceac28bfffed89fccaacb67f6b6e

The long string returned from the command is the container ID. It can be used to stop, restart and run commands in the container. So I can find out the ID of the www-data user by doing:

$ docker exec -u www-data f366db973377237b6190c0ae3cc3984bfc34ceac28bfffed89fccaacb67f6b6e id
uid=33(www-data) gid=33(www-data) groups=33(www-data)

(i.e. this simply runs id on the Docker container, as the www-data user)

Now I can set ownership of the music files:

$ sudo chown -R 33:33 /mnt/vm/docker/nextcloud.data

And stop the temporarily started container:

$ docker stop f366db973377237b6190c0ae3cc3984bfc34ceac28bfffed89fccaacb67f6b6e

Finally we can try re-starting Nextcloud with:

• The “mapping” of the port from port 8081 on the host computer to port 80 on the Docker container.
• The volume mapping from our music to Nextcloud’s data directory.

$ docker run -d -p 8081:80 -v /mnt/vm/docker/nextcloud.data:/var/www/html/data nextcloud
a51ca66f06c0eddcade46f71d99e363b66df1dd2de1e93903731b1e0b1f121c2

(Note the container ID has changed).

Nextcloud should now be running with its web pages served from port 8081. My home server’s name is bordeaux, so I browse to http://bordeaux:8080:

Thus begins the Installation Wizard. The first step is to create an administrator username/password. I enter my choices and accept the default Data folder and Configure the database options under Storage & Database.

There is a strong warning to not use the default SQLite database for file syncing. However, for this proof of concept I’ll just go with SQLite with the intention to replace it with Postgresql if it all works out (using Docker it is, after all, pretty easy to start over and get running quickly).

I’m only interested in file synchronisation so I deselect the option to Install recommended apps, but this is where you can install other cool stuff like photo sharing, document editing etc. Everything to create your own personal cloud!

Once I click Finish setup there’s a bit of a wait, and then the app begins to initialise.

Now for the music

Our music files, mapped previously, are now visible:

We can now do some cool stuff, like share links to the files, review version history of the files and more.

At this point I found the apps weren’t loading properly in Firefox. I found a bunch of Content-Security-Policy (CSP) issues and I switched to Chrome, which worked better.

$ docker exec -u www-data a51ca66f06c0eddcade46f71d99e363b66df1dd2de1e93903731b1e0b1f121c2 php occ files:scan --all
Starting scan for user 1 out of 1 (admin)
+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 46      | 339   | 00:00:35     |
+---------+-------+--------------+

Hit play!

There are a couple of music playing apps for Nextcloud - Music and Audio player. The latter also has an ID3 editor (aside: that gives me an idea…).

I tried Music, though, because it has slightly better reviews. It scanned the music files and was able to play my files (once the scanning had completed).

But that’s an aside… what I was really interested in was file syncing.

File syncing

My original goal was to find a way of easily syncing (meaning: copying to local storage) music from the Nextcloud server to my ‘phone.

The primary way Nextcloud offers this is via the WebDAV protocol. There are official Nextcloud apps available for mobile but, being me, I decided I wanted a generic client. The fact that Nextcloud supports an open protocol like WebDAV makes that possible.

I found FolderSync did exactly what I wanted.

FolderSync home screen

The home screen provides a menu into other parts of FolderSync - but initially you should click CREATE NEW SYNC

FolderSync add account wizard

Choose a WebDAV account, then fill in the details to your Nextcloud server.

We point WebDAV to our Nextcloud server and to a well known location: /remote.php/webdav/

I found I got a Method not allowed message when trying to point FolderSync at the IP address. I found I needed to add the complete WebDAV path.

We can then test the connection - click TEST. This lets us know if Nextcloud is setup to allow the synchronisation.

If Nextcloud refuses the WebDAV connection.

I realised I needed to add the local IP address as a `trusted_domain` in Nextcloud

The fix to this is to add the IP address to the trusted_domains list.

$ docker exec -u www-data a51ca66f06c0eddcade46f71d99e363b66df1dd2de1e93903731b1e0b1f121c2 php occ config:system:get trusted_domains
bordeaux:8081
$ docker exec -u www-data a51ca66f06c0eddcade46f71d99e363b66df1dd2de1e93903731b1e0b1f121c2 php occ config:system:set trusted_domains 1 --value=192.168.0.3
System config value trusted_domains => 1 set to string 192.168.0.3
$ docker exec -u www-data a51ca66f06c0eddcade46f71d99e363b66df1dd2de1e93903731b1e0b1f121c2 php occ config:system:get trusted_domains
bordeaux:8081
192.168.0.3

FolderSync select folders to synchronise

For Remote folder, choose the folder on the Nextcloud server, relative to the Admin's files. Because we copied music.nextcloud into that location with its children, that's the folder to choose.

For Local folder, choose the local folder to synchronise to. In this case it's a folder on the phone's storage.

Click Next and on the next step ignore the schedule.

FolderSync synchronisation status

Now click Folderpairs and then Sync

Click back to Sync Status to show the synchronisation status.

For me, file synchronisation appears to be pretty slow, but it was over wifi, after all.

Finally, our music completed its synchronisation:

What’s the verdict?

This is a solution that works, for sure. Once configured, it’s great to have scheduled synchronisations configured so it’s essentially automated.

The whole setup process is way too complicated for most people - it’s understandable how many people would prefer a hosted service already setup.

It would be great if there was a way of filtering or transforming the music files somehow, during the synchronisation. For example, I only want MP3s or Oggs to save space. Furthermore, I don’t want my entire collection, just new music and some old favourites.

I can also see this working well, in reverse, for use cases where content is created on the phone, and then backed up. For example, photos taken on the phone.

If you have the technical chops, give Nextcloud a try!

More like this:

• This month in digital music libraries - November 2024
• This month in digital music libraries - October 2024
• This month in digital music libraries - September 2024
• This month in digital music libraries - August 2024