Indexing Video “Playlists” in Splunk

In my last blog post entry, I talked about indexing radio stations’ playlists and described my reference implementation. This brings up a question whether the same approach can be used for indexing playlists for videos, not just songs. The answer is yes. One thing to keep in mind is that most people don’t spend time wondering what was the last video played on a certain web site or cable channel so that they can purchase it. In other words, discovering new videos on TV channels is not as popular an activity as discovering new songs on the radio. Nevertheless, it is a popular activity on the web. To try this out, I created two reference implementations that you can find on Splunkbase to try out yourself.

Youtube

When one thinks about discovering new videos, Youtube may be the first site that comes to mind as it is the most popular video sharing site out there. Youtube provides API’s to get information about a particular video and also a ready to use RSS feed to grab the lists of the most popular and recent videos of the day. I used the RSS feed to create a relatively simple Splunk app that indexes the most popular and recent videos that are currently on Youtube. I also put a basic simple XML dashboard in the app as a starting point to view the data.

Youtube Dashboard

(Please note as a disclaimer, this is real time data, so I have no control over its contents, which may contain videos that you may not find agreeable). If you click on any of the titles, you can get to the raw data. The interesting part here is that now you can use a supplied workflow action initiated from the Splunk flashtimeline view to see the video yourself from the link field.

Youtube Workflow Action

Now, you can sit back and use Splunk as your catalog for the most popular and recent Youtube videos. In essence, this is a playlist. You can download the app yourself and use your own index for it, if you want to control its retention policies.

Vimeo

I discovered that Vimeo has some very nice high quality user submitted content that they label as HD videos. Similar to the Youtube approach, I used their RSS feed to index their latest feeds every 8 hours. Unlike the Youtube approach, instead of indexing the latest popular and recent videos indiscriminately, with my Vimeo app, I used a text file to list a set of channels and had the app only index the content that is recently played in these supplied channels. When you download the app from Splunkbase, be sure to change the channels.txt file to reflect the names of the channels that you would like to index. Here’s a simple XML dashboard for the Vimeo app.

Vimeo Dashboard

Since the list of some of the content may be repeated every 8 hours, I used dedup in all the panels to get unique results. Just like in the Youtube approach, if you click on one of the titles or categories and go to the raw data in the flashtimeview, you can use a supplied workflow action to view the actual video. In this case, use the url field to initiate the workflow action.

Usage

The purpose of these two apps was purely for entertainment and proof point exercises. I had shown them at the last two Splunk User Conferences in the “Splunking Outside the Box” sessions just to have a little fun. You can use Splunk as a gateway for your enjoyment beyond doing the usual high value operational intelligence with it. There may be some business use for these apps as you can use them to send yourself Splunk alerts whenever some key words (such as the name of your company) are in the title or description of the videos so that you can take proactive steps for market monitoring or data leakage investigations. For now, they are meant to provide some levity alongside your more serious apps.