Today, RecoverMy.Video posted a shutdown notice. Unfortunately, it looks like because Youtube changed their developer policy to prohibit monetization from people using its API, it cannot economically operate anymore.

This is very sad, but I'm not here to talk about why it's sad, or why Google is making a terrible mistake. Other people can do that better, on other platforms. Instead, since I was using this to monitor my own playlists, and I'm going to share what I'm using to replace it.

In short, RecoverMy.Video was a web-based tool that periodically downloaded all of your Youtube playlists. Whenever a video is deleted off of your playlist due to Youtube removing the video, it sends you an email notification with the saved details of the video title and link, so that you can be aware it went missing and can find a replacement video to put back on the playlist.

Otherwise, if a video is deleted by Youtube, you don't know immediately, and later when you check the playlist on Youtube, all you see is [Deleted Video]. That's not useful for finding a replacement video.

Here's what I did to replace the functionality of RecoverMy.Video in a self-hosted way.

urlwatch monitors more than just URLs

I'm already using urlwatch for many things. As the name indicates, it's a command-line tool that watches URLs. It sends you an email notification when the content of a certain URL change.

However, it's very versatile. It can both do more than watch URLs and do more than send email notifications.

Instead of watching URLs, urlwatch can watch the output of arbitrary shell commands. This small feature makes this an incredibly a useful tool for monitoring... anything, really, since many things can be scripted. For instance, I have urlwatch watching the output of a shell script that ssh's into all of my EC2 instances and checks the disk space, outputting a warning if it's running out of free space:^[1]

#!/bin/bash

[[ "$(ssh mail.redacted.example.com findmnt -fnrDoUSE% / | grep -Pho "\d+")" -ge 90 ]] \
    && echo Mail server size warning
[[ "$(ssh home.redacted.example.com findmnt -fnrDoUSE% / | grep -Pho "\d+")" -ge 90 ]] \
    && echo Home NAS size warning

exit 0

Poor man's EC2 instance monitoring, with five minutes of setup!

I also have another one running tail /path/to/nginx/errors.log so I know instantly if someone gets a 500 error out of one of my microservices, or is trying to brute force a .htaccess, or something is down (nginx reverse proxy could not connect to backend service).

Also, in addition to sending email notifications, it can also send local unix mail, Pushover or Pushbullet notifications, messages on various chat platform (Discord webhooks, Telegram, Slack, Matrix...) and more.

Basically, this tool is great for quickly and hackily connecting anything you want to monitor (which you can express as a shell script or command) with the human-friendly best ways to get your attention (for me, this is a Discord ping and an email). It is by no means a robust industry-standard monitoring system, but it serves its purpose well, and for a technically-inclined user, is a great time-saver for everyday automation.

youtube-dl downloads more than just videos

If you use a tool which downloads youtube videos regularly, you're almost certainly using either youtube-dl or something which uses youtube-dl internally. youtube-dl is an excellent and well-supported open source command line Youtube downloader.

It uses the web client of Youtube, rather than any official Google API, which means you don't have to generate an authorization key or OAUTH your Google account or anything to pull publicly-available data. The main drawback of a scraper-based interface is that it might change chaotically and randomly, but youtube-dl is very good about keeping up with updates, and even has an auto-self-updater you can run by doing youtube-dl -U.

In addition to downloading Youtube videos, there's a huge list of other video platforms it supports, including Facebook videos, Instagram, and more. The list literally has over a thousand different platforms.^[2]

What we care about for the purpose of replacing RecoverMy.Video, however, is its ability to download and parse entire Youtube playlists.

Putting it all together

Now, here's a youtube-dl command that will download the playlist, listing without downloading the videos on the playlist (hence the --flat-playlist flag), and output in JSON (hence the -j flag) the data.

youtube-dl -j --flat-playlist "https://www.youtube.com/playlist?list=PLFsQleAWXsj_4yDeebiIADdH5FMayBiJo"

Once you confirm this works, you can simply do urlwatch --edit to edit your local urlwatch file^[4] to add a section that runs this command every time urlwatch runs:

---
name: 'important video update'
command: 'youtube-dl -j --flat-playlist "https://www.youtube.com/playlist?list=PLFsQleAWXsj_4yDeebiIADdH5FMayBiJo"'
---

And then you'll get a notification whenever the playlist changes, which includes a diff of the changes.^[5] You can test it by adding a video to the playlist, and then running urlwatch or waiting for cron to run it.

Sidenote on private playlists

This doesn't work for private playlists. But, just making them unlisted should be fine. Nobody's going to guess your playlist URL, and anyone with enough access to your computer to read your urlwatch config can also copy out the saved Chrome cookies and log in to your Youtube account to access your secret playlist anyways. It's just a roundabout unlisted playlist, from an attack surface perspective.

---

1. The exit 0 is because otherwise urlwatch assume the script failed, depending on the exit code of the last command, which becomes the exit code of the shell script. ↩︎

2. Also, did you know, you can use youtube-dl to download arbitrary HLS streams! For instance, if you're viewing a completely legal "watch online free" video on a sketchy domain, and you want to download it instead of enduring their ads, 90% of the time you can just pop open the console (disabling breakpoints if they have the trivial anti-console while (1) { debugger; } running), reload the page, hit Play, identify the .m3u8 file in the network log, copy the URL, and then youtube-dl "https://sketchyvideosite.example.com/5f4dcc3b5aa765d61d8327deb882cf99.m3u8" -o out.mp4 and it just works!^[3] ↩︎

3. Yes, you can also use ffmpeg for this, but I've never managed to memorize to the point of intuition how ffmpeg flags work, let alone the ones needed to allow HTTPS, allow crypto, etc. Perhaps I'm just ignorant, but, in my ignorant opinion, youtube-dl is much easier to just use. (Also, yes, youtube-dl internally uses ffmpeg, so at the end of the day it's the same thing). ↩︎

4. Of course, if you just installed urlwatch, you'll need to follow the instructions to add urlwatch to your crontab so it runs regularly, and also set up reporters (which are places where notifications are delivered). ↩︎

5. Unfortunately I can't supply a screenshot of the notification, because I can't modify the important videos playlist (yes, I can confirm that I'm not that person), and all the playlists I'm actually using this to watch link me to my irl identity. But eh, my Discord client is CSS-modified to the point that you wouldn't recognize it as Discord anyways, so it wouldn't exactly help illustrate anything. ↩︎