Data Claus is Coming to Town!

What happens when one of our London-based Splunkterns, Rupert Truman combines his new Splunk knowledge with some festive spirit? He produces the following guide on the holiday season! Too good not to share....

Christmas season is rapidly approaching it’s bauble-filled climax, and consequently there’s plenty of questions to answer: What am I going to buy? What am I going to do? Is anyone else feeling faint, or is it just me? I decided to see if I could use data to answer these questions. This is the story of how I Splunk’d Christmas.

The Presents

Traditionally I find it hard picking the right gift for my sister - so this year I decided to harness the Twitter hivemind. To determine what to get her, I ingested into Splunk this dataset of 50,000+ tweets from Christmas 2016.

At first I searched the phrase "all I want for Christmas is…”, in the hope of finding  the gifts people were anticipating most. But this backfired when, perhaps unsurprisingly, for 95% of events, the next word was “you”, “youuuu” or even “youuuuuuuuu”. Moving away from my ongoing conflict with Mariah Carey, I opted to try a different tactic...

Remembering recent remarks my sister made about updating her wardrobe, I refined my search to clothes. This is typically very risky territory to venture into, but filled with confidence in the data, (and making a mental note to keep the receipt of my eventual purchase), I pushed on.

First, I searched tweets related to ‘sweatshirt’, ‘jacket’ or ‘shoes’. This gave a count for each item, and from there I searched tweets containing some variety of smiling emoji, indicating a positive (hopefully not sarcastic), reaction. Correlating the two gave a number of mentions for each item, and the percentage which contained a positive emoji. While shoes were the most tweeted item, sweatshirts were the safest bet because:

1. 80% of all positive tweets were for sweatshirts opposed to shoes and jackets, and
2. of all tweets mentioning ‘sweatshirt’, 5.48% featured a positive emoji - the highest percentage!

The Dinner

Next up, the centerpiece of the day, Christmas dinner. Deciding on the appropriate amount to spend on Christmas dinner can be difficult. With various dishes to cook and multiple relatives visiting, the cost can rapidly soar!. Wanting to estimate how much I would spend, I decided to predict the average Christmas dinner cost for 2017, using this historic data.

After ingesting the data into a table showing the average historical prices for each food item, and the annual meal average (calculated using addcoltotals), I needed to convert to timechart compatible format:

Using “transpose”, I converted the table into two columns - year and total, showing the average yearly totals, inflation adjusted, from 2006 onwards. Using this, I created a time chart of the average costs, and added “predict” to predict 2017’s average.

And there you have it, Splunk expects an average 2017 Christmas dinner spend of £116.44, add to that at your own discretion.

The Film

Feeling well fed and slightly sleepy, families everywhere will trudge from the dinner table to the TV. But which film to watch? Well, stealing shamelessly from Matt Davies’ Valentine’s Day blog, I ingested this dataset of 5,000+ movies to help me decide.

I narrowed the pool down to the 50 featuring a ‘Christmas’ keyword and sorted by vote_average. This was a start, but some interesting selections remained - while “Spotlight” is a fantastic film, it doesn’t make an ideal family watch.

Next, I limited to films from the family or comedy genre and, conceding that younger family members may not be entirely captivated by 1940’s 130 minute classic “It’s a Wonderful Life”, only included films under 2 hours long from 1970 onwards. This left me with a top five and a winner, which is...A Christmas Story!

So there we are, Christmas Splunk’d and my present, dinner and film sorted. Now, what about Hanukkah?

Merry Christmas and thanks for reading!