Predicting and Preventing Crime with Machine Learning - Part1

Let’s imagine we could use technology to make the world a better place, with less violence and crime. Recently, my colleague Greg wrote about Machine Learning for Social Good. In it he shared interesting thoughts on the potential of AI and ML powered use cases, but also their challenges in the light of bias and privacy. Predicting and preventing crime is certainly related to this. There are different ways how AI and ML can help with that. 

In this two part blog series, I’d like to reference two interesting projects that I came across in the Splunk community. The first is about a new app on splunkbase by the German consulting firm ADVISORI FTC. They demonstrate how Police operations can be improved using Splunk’s Machine Learning Toolkit. The second project we’ll touch on in part 2. 

Automate the boring stuff – how Splunk offers ML-based Decision Support

Video by ADVISORY FTC

If you have successfully integrated Splunk into your system landscape (for example as a SIEM as shown in the video above), then you’re ready to start thinking about the next steps, such as, can I automate time-consuming, manual and repetitive tasks and execute them at machine speed? These tasks can include incident classification, KPI forecasting, failure prediction or fraud detection. The underlying issue remains the same: you need to derive meaning from data first. But then, what if you could automate your decision-making to free up some time?

Let’s dive into a real-world example: imagine you got promoted to Chief of Operations of the Chicago Police Department.

“Welcome to your new job, Chief!“ you tell yourself as you look outside your office window. You notice the long line of police cars parked along the street. You ask yourself, “Where should I send my officers today?” You open the dashboard of the operation planning tool. And there it is. Chicago. Neatly sectioned into squares, and yet, blotches of red overshadow the city. Those are all crimes. And they all happened just yesterday...

“Great! How do I get rid of those now?” you ask yourself.

As you scroll through the past months of data you realize why your predecessor resigned. There is so much data. This will take hours to look through. How could you possibly come up with a feasible plan within the next thirty minutes? Overwhelmed and frustrated, you open the operational planning screen and make the call to send additional officers to yesterday’s worst affected areas – aware of the wasted potential of a more thorough analysis.

It’s lunch time and you meet your system admin Jeff. You tell him about your hopeless endeavor, but to your surprise he doesn’t seem phased. Jeff explains that your operational planning tool is based on Splunk Enterprise and, as he states with a grin, it has the Machine Learning Toolkit installed. Machine Learning... it rings a bell but I have no idea. “Don’t worry, it’s easy,” Jeff explains. You decide to meet him after lunch to rethink your planning tool.

After lunch you enter Jeff’s office and find him already in front of the planning tool. “Look!” he says, “I thought about your problem and prepared a new dashboard for your planning tool app. The first thing we need are some features that we can feed into our model”. You suggest using data from the past days. “Great idea. Let’s also use some moving averages over a longer timescale! This should add another perspective.” Jeff opens a new Splunk Search and within a few moments he created the new feature. “Done. Now we can create our model! We should start with something simple. Let’s go for a linear regression.” With a sigh Jeff saves the query and closes the Machine Learning Toolkit Experiment Page. “And that’s it!” Jeff exclaims. It looks almost like your old tool, but all you need to do is push a button to create a forecast. “Tell me tomorrow how many bad guys you’ve caught with it!”

Hey Jeff, you made my day!

It’s Wednesday morning. Same desk in the same office on the same street. Yesterday was the first day you tested your brand new machine learning powered operational planning. You open the application to check the reported crimes. As you compare the results your interest and hope are peaked. It’s already better than before, and all just by using such a simple model! 

At lunch you meet Jeff to tell him about your success. Asking him if the model could be further improved by means of a more complex version, like a neural network, Jeff responds: “Great idea! And the newer versions of Splunk should be able to support it. I recently came across the Deep Learning Toolkit for neural networks. I’ll check it out and see if we can do better!”

The Chicago Crime Predictor that you and Jeff built is not just a tutorial on the Splunk Machine Learning Toolkit, but can serve as a starting point for your organization. Check it out to see what Jeff had in mind to make the idea work.

Video by ADVISORY FTC

Feel free to explore the Predictive Crime Showcase app on splunkbase if you are interested in learning more. Stay tuned for the next blog post related to predicting and preventing crime!

I'd like to thank the whole team at ADVISORI FTC and congratulate Mike, Eric and Sebastian on their app!

Happy Splunking, 

Philipp

P.s. Read Predicting and Preventing Crime with Machine Learning - Part 2 here.