A First-Timer's RSA Conference Experience: Part 2

This is the second of a two-part blog series about a data scientist's first experience at RSA Conference from guest blogger Lauren Deason, lead data scientist at PUNCH Cyber Analytics.

Read part one here.

Caught Between Swag and Buzzwords

Throughout the week, radiating out of several rows of exhibit booths, one could find a near-endless list of companies signaling their value to potential customers by flaunting it in the form of open bars and obscene quantities of swag.

As a naturally competitive person, I felt compelled to get my (Splunk's) money's worth by maximizing my swag haul. But even I had to throw in the (logo emblazoned) towel after a couple hours of booth-hopping. There's only so many t-shirts one can layer on top of each other to avoid bag checking on the plane ride home.

Luckily, I had one of my Splunk benefactors, Ryan, showing me around the expo so I was able to benefit from his knowledge about which companies were doing what, and who I might be interested in talking to. You could tell he was an experienced conference veteran by his complete disinterest in acquiring swag. ("Ryan, look! This one has lightsabers!") Through his connections, I visited the booths of several companies I hadn't previously heard of and I was able to talk with people actually doing the work they were passionate about. There was nary an exhibit that didn't advertise the use of machine learning or AI to futuristically catch bad guys, and I enjoyed asking technical questions of those touting these technologies to get to the bottom of what they were actually doing (spoiler alert: usually not much). 

A prevalent theme throughout the conference was a hyper awareness amongst all of the buzzwords that are ridiculously overused to market security solutions of little to no substance ("Check out our AI! Now with 50 percent more blockchain!¹"). It seems to have become a requirement in this field for any speaker to make reference to this phenomenon so that the audience knows that "Yes, I too am aware of this overhype, but you and I? We're the smart ones who actually know how this stuff works, wink, wink."

Except we can't ALL be part of the 'in the know' minority—with tens of thousands of conference attendees all self-professed to be wise to the overhype, who exactly are all these other naive members of the uninformed majority? The reality, I believe, is that most people in the cybersecurity field are both aware of the overhype, but largely unable to distinguish truly useful technology from snake oil (except, maybe when it is explicitly labeled as such).

Because they lack the tools to accurately assess these technologies, I believe many people either give up or convince themselves that they understand more than they do, and quietly carry around their membership cards as part of the informed minority.

What is truly needed, in my opinion, is an improvement in education—at a low level of technical detail—for those in the security field about what questions they should ask of vendors in order to assess technical merits of analytic tools. For example, if a vendor tells you their product “uses AI to detect bad actors,” you should be able to find out exactly what specific question is answered by the referenced algorithm and what the required inputs are to arrive at that answer. (If you’re at a fine dining establishment and inquire about the specialties of the house, it’s not terribly meaningful to be told that they’re made out of ‘food’ using ‘cooking’, right?) Additionally, measurable benchmarks agreed upon in the industry that can be used by third parties to evaluate security solutions would take some of this responsibility off the customer by allowing them to trust such an industry standard. 

While I think there is a long way to go in terms of educating non-technical consumers of security products about the abilities and weaknesses of various machine learning applications, I did find it encouraging to hear the 'base-rate fallacy' mentioned at multiple venues during my RSA experience, indicating that this shortcoming of several machine learning applications to the world of cybersecurity is becoming more widely understood and accepted (namely, that when the event you are trying to detect is extremely rare relative to benign events, even seemingly low false-positive rates can render the output of a perfectly accurate detection analytic useless in practice.) 

Hooray for Diversity in Cybersecurity

Another encouraging thing I noticed at the conference was that—while clearly still heavily male dominated (best bathroom line ratio I have ever encountered!)—everyone that I spoke with about the grant I received from Splunk to attend the conference was impressed by the initiative, and (maybe?) will think further about what they can do within their own companies to promote a diverse workforce.

Honestly, when offered this opportunity I initially felt a bit sheepish about taking it; I consider myself sufficiently smart and capable to make my way professionally without needing artificial advantages to supplement my lady brain capabilities, thank you very much. But the truth is that without this effort by Splunk, there would have been one less female attendee at RSA, and I would have missed out on insight gained into the business side of the industry I work in.

I'm sure that the causal factors leading to the massive imbalance in gender and race representation in this particular field are many and complex, but initiatives such as this can only help to re-balance things. Far from just providing assistance to the individual recipients of such grants, you never know when someone who wouldn't have otherwise been part of such an event may take away newfound skills or inspiration to take on a leadership role and inspire others from an underrepresented community, simply by being a visible part of the industry.

Overall, my experience at RSA was very valuable—both as an eye opener in terms of how the business side of the field works, as well as in terms of content and connections that will directly benefit my ability to produce high-quality work. I met a number of talented individuals working in either security, data science, or the intersection of the two at various events surrounding the conference, and my conversations with them sparked new ideas that I am eager to explore further as I continue to research machine learning applications for cybersecurity.

¹Credit: John Lankau, PUNCH Cyber