Organizing My Music

Organizing Failure

As I go about my daily business, which usually involves crouching over a 15 inch screen for 12 hours plus a day, I certainly like to play music in the background. I find it helps me to focus and tune out any distractions other than the ones right in front of me. I've had people in the same office as me ask me if I "heard that ruckus today" with me not having any clue what they are talking about, which is exactly how I want it unless that ruckus is a fire. If you don't integrate music into your daily work routine I highly recommend it.

Recently, I had gotten a subscription to Spotify and was using their "Discover" service to find new music I hadn't heard of before. Spotify is really great for searching, finding, and playing the music you know you want to listen to; however, in my experience it's been particularly bad at finding new music for me. My first recommended artist was Tune-yards, and that is a band I still really like today. However, subsequent recommendations were more...just blah. I somehow got into a groove of pedestrian fuzzy garage pop that I couldn't get out of. Every new band that was recommended to me somehow had that benign formula where it could be played in most lobbies and not offend anyone. Let's not ever talk about Spotify radio, as I think it isn't enough of anything to warrant a proper discussion. 

I just wasn't getting the "fresh" feelings, like when I discovered artists from a bunch of music I and friends had pirated, from Spotify. I and a few friends had pirated a lot of music in the neighborhoods of 150GBs or so. We would just horde the music, and I personally would shuffle through my entire collection. A lot of the time, I would just let it play and skip through the opera and other annoying tracks, but occasionally, I would stop what I was doing and really look into the artist/album/track that I was hearing. I might then put that artist on repeat or start downloading more from them. After all of my Spotify trials and tribulations, I thought it would be a great idea to start going through my catalogue of sounds again with fresh ears.

A Spinning Wheel of Death

The only problem was that when I went to start up the music player I liked for Mac OS X, it just spun. I had to force quit it every time I tried to play music. I quickly realized that I needed to figure out a new program to play my music. I had a few caveats.

I really liked to change the track from the menu bar so that I didn't have to go back to the music app to change music. Since I use full-screen mode on my Mac, I would end up spending 10 or more seconds locating the next track button and then going back to what I was doing. Having that ability right at the top of the screen was important for me to not interrupt my flow when working.

I also really like to shuffle through my entire music library and maybe focus in on an artist or album if it really strikes me. Sure, I get some phases where I have to skip through 10 tracks of opera, serial music, and spoken word tracks, but picking out music for a playlist usually takes up too much cognitive energy for me to care about that. I find that by shuffling through my music it's the only way I'll faithfully try tracks from artists I'm unfamiliar with. If I have to pick them myself, I just stick to what I know.

With those two caveats in mind, I searched the Apple store high and low for my golden music playing solution.

Skip Tunes....Skip That

I tried out a few apps and then settled on Skip Tunes, an app that lets you connect to Spotify and other services and skip through tracks using a menu bar drop-down. Initially, I was decently happy, but I had to start up Spotify and then start up Skip Tunes and the only benefit I was getting was faster skipping through tracks, not anything to do with rediscovering my own music collection. So, I had to search harder.

Somehow, I ended up finding some app called Bowtie that lead me to an open source player called Tomahawk. Tomahawk's claim to fame is that it only handles metadata. Since Tomahawk doesn't ascribe to any one source for music and only needs metadata about each song, you can theoretically hook up any source you want. To hook up sources, you have to create a "resolver" that sends Tomahawk the correct metadata in the correct format. There are currently as many resolvers as I would think anyone would need, but if you want to add one, you can certainly do so since it's open source. The best part is that each resolver is separate from the main development work on Tomahawk so no politics or opinions would ever get in the way of someone connecting to Tomahawk with a new service.

You can also let other people connect to your Tomahawk instance and share playlists as well as streaming media to them. I haven't played music with remote streaming, but I can say that sharing playlists works. What's interesting is that you can share a playlist where you have all the music on your computer and your friend doesn't. Let's say your friend is connected to Spotify though. They can then listen to the same music and Tomahawk will automatically look through any connected services to find the right music to play. What's really interesting with how Tomahawk finds music is that you can have an album with 9/10 tracks locally on your computer and Tomahawk will fill in the missing track with a service like Spotify. I have already listened to an album I had played over and over again only to find out I was missing a couple of tracks I never knew where on the album. Tomahawk was filling in those track from Spotify without me even noticing. Xang!

Beets Rules the World

So, I had a metadata player for music, but you might wonder how that relates to the music collection I wanted to rediscover. To be loaded onto Tomahawk, your music must have the proper ID3 tags and there needs to be a resolver to send the metadata information to Tomahawk. As I looked through the available resolvers for Tomahawk, I saw some familiar faces (Spotify, Soundcloud, Grooveshark) but I also saw a host of other music hosting options, one of which was called Beets.

I happen to love beet juice, but other than drinking the stuff I had never heard of any piece of software with the same name. Beets claims to be a music geek's organizer and has a boat-load of options for organizing and categorizing your music. You can grab lyrics to songs, look up a piece of unknown music's acoustic fingerprint, and do a whole host of other things that I didn't care to look into. I chose simply to organize my music by artist and grab the album cover art for each album I had.

The process is pretty straightforward, so I won't go over what the guides tell you, but I will tell you some things to do/not do. 

  • Always copy files instead of moving/renaming them. Beets can do some strange shit, and I always feel more comfortable copying the files rather than messing up my entire collection because I didn't fully understand a command. 
  • Really think about where you want your music to end up path-wise. I initially copied my music to a folder that I later thought was a bad location. You'd think that you just have to change the library root path in the config options and that would take care of it, but Beets must keep an absolute path for each file. I had to use the "beet move..." command to clear up my mess and it took several hours. 
  • Group albums with multiple CDs properly before trying "beet import..." I never could get this one right and always had multiple versions of the same album for 2CD+ sets. I'll have to correct these as I go, but if you don't know what I mean here, pick an album that has multiple CDs and experiment on importing it before you try your whole collection. Through trial and error, you should find something that works. 
  • DON'T TRY "BEET UPDATE" without the -p flag. The -p flag just shows you what will happen, but once again Beets can do some weird shit to various artists albums. In the end, it might make more sense to split these albums up, but I am paranoid and didn't want to do what the update command was suggesting. One the move command, you might get some "error, can't find..." on certain files. the update command will clean these up, but you can also just fix each file not found error as it comes up without having to succumb to everything the update command wants to do. 
  • Keep a copy of your .blb library database file before doing anything major. I have screwed up my collection after painstakingly importing it for hours and only had to restore this .blb file from a copy to get everything back to the way it was. 
  • If you have a large collection, try importing by letter. I split up my collection this way and it was more manageable. I figured if I was going to organize my collection, I better do it right so it took me 15 - 20 hours just sitting there and saying "yeah, that is the album" or "no, this is the proper album for this directory". In the end, I was glad I split it up into 26 chunks. 
  • If you have a lot of classical music, consider separating those albums out and spending extra time on importing them. I'd say less than 10% of the time that Beets guessed the right classical album tags for my music. Considering there are a lot of compilations of the same classical music, this isn't surprising, but it is and will be frustrating for you.
  • If you don't know, just keep the tags you had. I kept a lot of music, most all of my classical music, as is. It can be handy to look up the Musicbrainz id from their website rather than trying to search through the beets terminal interface. 

With all of those caveats, importing music through Beets is fairly straightforward. To get Beets on your Unix system, just open a terminal and type:

$ pip install beets

That will install Beets using the Python package manager. If you don't have "pip" installed, then go figure that out...After you are cool with Beets, you will need to make some changes to the configuration file. I'll show you mine just so you can see...

directory: /Volumes/MUSIC-STUFF/beets


    copy: yes

    write: yes

    resume: ask

    quiet_fallback: skip

    timid: no

    log: beetslog.txt

ignore: .AppleDouble ._* *~ .DS_Store

art_filename: albumart

plugins: web fetchart

pluginpath: ~/beets/myplugins

threaded: yes

color: yes

library: ~/beets/musiclibrary.blb


    strong_rec_thresh: 0.08

The important parts here are:

  • directory - this is where Beets will import all of the music you tell it to. Make sure this is where you will eventually want your music before importing as not being sure about this can waste hours of your time. You can't just move the files yourself, you have to use "beet move" to change the directory after you have imported music. 
  • import - I won't explain these, but they all impact the import command as you would expect. I like to copy the files and write to the ID3 tags. 
  • plugins - these are where you specify which plugins you want to use. I wanted to capture album art and use the web server. The web server is essential for playing music from Beets in Tomahawk. You must have the server running for Tomahawk to see all your music. 
  • threaded - Setting this to yes just makes it faster by using multiple processing threads. I never had this off, but I imagine you never want to leave it off unless you have a really old computer. 
  • library - This was set by default, and I just kept it that way. My music is technically on another hard drive, but I figured keeping my library file on my computer was okay since the space it takes up is not that much. 
  • match - This by default is set to 0.04 I think, which means that a Musicbrainz match has to be 96% or greater for Beets to not ask you if the album is what it thinks it is. I relaxed that to 92% and above since the matches were generally right at that percentage. This change sped up my import time a lot. 

Once you have your configuration options set, the fun can begin. All you then need to do is run the following command to import your music.

$ beet import path/to/music

It took me a little while to figure out the difference between directory and the  "path/to/music" parameter. "path/to/music" is where your music currently is located and the directory is where Beets will place imported music. If you have the import options set a little differently, then your import might not work the same and copy the files, but for me, "beet import path/to/music" is equivalent to "cp -R path/to/music $beets_library_path" and will recursively copy everything over with all the other stuff Beets is doing. I suggest making a test with multiple CD albums and maybe some other odd quirks (classical music, compilations) to test your import before you try your whole collection. 

Once you are done importing, you need to run

$ nohup beet web &

in your terminal to start the beets server and then add the resolver (plugin) in Tomahawk. It should connect to the default port and show your music collection under "cloud collections". The "&" after "beet import" just means that the process will run in the background and not take up space in your terminal window. The "nohup" before the beet command means that the process won't get hangup if you close the terminal window. You can type

$ ps

then to see that the process is in fact running. My process was using the command "/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.fra". Putting the process in the background is just nicer than having to commandeer a terminal window to run the process, and I have successfully kept it going even after disconnecting my external drive and letting my laptop go to sleep. If I were more advanced, I would make a script to run at start-up that enabled this process, but whatevs...

Final Thoughts

I am pretty impressed with the combination of Tomahawk and Beets. I feel like my music is much more organized and I always like using open source software. I think Tomahawk's player, particularly the menu bar, can be improved and maybe someday I will get into helping out with that source code. However for now, I use this combination everyday and am happy to rediscover music I had forgotten about and discover music I hadn't listened to before. Supposedly, there is a flat tree view coming to Tomahawk that will allow you to shuffle through an entire cloud collection, and I am very much looking forward to using that feature in the future.