Ask an Expert: How Splunk is Addressing New Technology Challenges

Organizations of all kinds have been forced to pivot dramatically over the last few months. What was deemed a priority in January may or may not be on the list anymore. Even while managing in a crisis, it’s vital for companies to look ahead and make careful decisions about the new normal we will need to embrace. That was one of the key messages we discussed in the first of Splunk’s “Ask An Expert” series of virtual events, designed to bring together leading technologists to discuss key challenges businesses, governments and organizations are facing during the Covid-19 pandemic.

It was my distinct pleasure to host the first event where we brought together James Hodge, Chief Technical Advisor and Interim CIO Steve McMahon to discuss how we at Splunk are responding to new challenges and what an accelerating rate of change means in an already disrupted digital landscape. 

Removing Human Middleware 

For Splunk, McMahon noted, digital transformation is built around automation. Like so many other organizations, Splunk mostly uses cloud-based software. But having a variety of tools can lead to integration challenges which people typically mask with workarounds or point solutions — by becoming “human middleware.” For example, many teams still manage workflows in spreadsheets, making changes and re-uploading the document to a shared server. That kind of workflow is the opposite of scalable.

First of all, McMahon said, the organization in general and the IT department in particular need to understand that customization is bad. Splunk’s business processes aren’t special, he added. We operate just like every other software company and we need to act accordingly. The way we make our business exceptional is by being as efficient as possible with our processes and system integrations. We achieve that efficiency not with customization but with automation, removing humans where appropriate so that our people can work on higher-value tasks.

It’s easy to justify the cost of automation, McMahon said. Take for example a scenario where an organization is paying 12 people to take data from the CRM and create tickets for provisioning new cloud instances. There's nothing about that process that requires creativity or judgment. It's a better use of resources to invest in creating an API to automate that process and free up those 12 people to do something with a much higher value-add.

McMahon said the businesses exist to make and sell goods and services, and the job of the IT department is to make those processes scalable and repeatable. If you've built applications and systems in such a way that you don't need legions of people to run them, you've done it right. “And that's really the net of our digital transformation,” he concluded.

Shifting to a Remote Workforce

For Splunk, the shift to becoming a remote workforce was focused on ensuring our data center had the capacity to handle a tremendous increase in VPN connections. We use many cloud-based applications, but we have around 14 mission-critical applications in our data center that require VPN access. Splunk went from roughly 100 to 150 concurrent users on VPN to nearly 3,000 overnight. 

Fortunately, McMahon said, Splunk had a sales conference in February where the APAC-based team could not travel and the impact of COVID-19 on China became apparent. The Networking IT team gathered information from around the world and quickly made the decision to increase VPN capacity by 5X before many were aware of the issue.

“Thank goodness we did,” he added, “because it's meant that we've been 100 percent up since everybody started working from home.”

If you’d like to hear more, including a discussion of the challenges of maintaining compliance with a distributed workforce, the complete recording is now available, along with our second event, "Ask an Expert About Remote Work Insights."