Security Advisories for Splunk 9.0

Splunk is committed to using inclusive and unbiased language. This blog post might contain terminology that we no longer use. For more information on our updated terminology and our stance on biased language, please visit our blog post. We appreciate your understanding as we work towards making our community more inclusive for everyone.

Customer security and trust are our top priorities. On June 14, 2022 Splunk published eight Security Advisories regarding vulnerabilities related to Splunk Enterprise and Splunk Cloud Platform. We’ve received customer feedback about the vulnerabilities and our process, following the release of the advisories, which we appreciate and are addressing as part of our commitment to continuously improving Splunk's security posture.

We’re committed to reporting new vulnerabilities consistent with our Security Advisory Policy and expediting maintenance releases for supported versions to address critical-risk, high-impact vulnerabilities outlined in our security program here. 

Different advisories may be applicable to your Splunk environment depending on the Splunk deployment type you are using, such as Splunk Cloud Platform (across regions, cloud providers, and compliance environments) and Customer Managed Platform (CMP).

The advisories and their links are listed below:

• SVD-2022-0601 - Splunk Enterprise disabled TLS validation using the CA certificate stores in Python 3 libraries by default
• SVD-2022-0602 - Splunk Enterprise lacked TLS certificate validation for Splunk-to-Splunk communication by default
• SVD-2022-0603 - Splunk Enterprise lacked TLS hostname certificate validation
• SVD-2022-0604 - Risky commands warnings in Splunk Enterprise dashboards
• SVD-2022-0605 - Universal Forwarder management services allow remote login by default
• SVD-2022-0606 - Splunk Enterprise and Universal Forwarder CLI connections lacked TLS certificate validation 
• SVD-2022-0607 - Splunk Enterprise deployment servers allow unauthenticated forwarder bundle downloads
• SVD-2022-0608 - Splunk Enterprise deployment servers allow client publishing of forwarder bundles

To remediate all the vulnerabilities listed in the advisories, we recommend customers upgrade to 9.0. We understand that not all of our customers will be able to upgrade to the latest release immediately. To reduce the severity of these vulnerabilities during the process of upgrade, we have published partial mitigations as additional security controls to help limit security exposure. We will continue to update our guidance on our Splunk advisories page as applicable. 

Our Rationale and Process

We issued a major release instead of backporting all the security vulnerabilities to alert customers to material changes to product behavior and avert potential issues with customer production deployments. The intent was to be consistent with our major/minor patch release policy. Below are some of the specific reasons why we didn’t backport initially by vulnerability, and why we feel it’s not practical to backport other Splunk 9.0 security fixes.

• For the single critical vulnerability SVD-2022-0608, a backport is currently in development for all supported versions of Splunk Enterprise (currently 8.1.x and 8.2.x). Additionally, SVD-2022-0608 cannot be mitigated without turning off the deployment server. After additional analysis, we believe this backport has the least risk of introducing a regression. We’re in the process of releasing a backport for the versions under support (8.1.x / 8.2.x). We had initially ruled out a backport due to the risk of introducing regressions to Splunk instances that co-locate other services with the Deployment Server (such as License Manager, Deployer, etc.). We’ve performed additional work to confirm that a 9.0 Deployment Server is compatible with older versions of Universal Forwarders, License Managers, etc. Customers who have a Deployment Server co-located should review the compatibility matrix in the Upgrading Splunk Enterprise document to ensure the backported fix will work in your environment.
• SVD-2022-0607 (Deployment servers allow unauthenticated bundle access) requires both the Deployment Server (DS) and Universal Forwarders (UF) be version-consistent and doesn't allow for a mixture of UF versions.
• Our testing demonstrates that backporting SVD-2022-0601 (Splunk Enterprise disabled TLS validation using the CA certificate stores in Python 3 libraries by default) would break existing customer applications that use private certificates.
• Similarly, SVD-2022-0605 (UF management services allows remote login by default), would have impacted the Splunk CLI and could have broken customer integrations with tools such as Chef, Ansible or other scripted automation. 
• SVD-2022-0603 (Splunk Enterprise lacked TLS hostname certificate validation) requires customers to have properly configured x509 certificates across all Splunk nodes, including valid SAN and CN values. These changes, intended to make Splunk more secure, require thoughtful deployment planning beyond the expectations of a patch release.

We recommend opening Support cases for environment-specific assistance and issue tracking and we will update ideas.splunk.com as we make progress on a backport for SVD-2022-0608. 

Risk Mitigation Resources

To stay up-to-date on any actions required (e.g.patching) and to mitigate risks, please leverage the resources below:

• “Upgrading Splunk Enterprise” Lantern page. This page contains information on best practices for updating Splunk Enterprise, frequently asked questions, partial mitigations and more.
• A Tech Talk presented by members of Splunk Product Management, Engineering and Professional Services explains the security advisories and discusses Splunk Enterprise 9.0’s newest security features.

Next steps

We remain committed to helping customers identify and remediate security issues quickly.

For “Critical” or “High” vulnerabilities we plan to provide advisories and any available patches as close to real-time as possible. For vulnerabilities considered “Moderate” or “Low Risk”, we’re planning quarterly releases of any available patches so that Splunk administrators can plan for patches and upgrades on a predictable schedule.  Please continue to watch the Splunk advisories page for the latest advisories or use the RSS feed with your favorite aggregator.

Thank you to our community for your feedback. We will be more responsive and will communicate as clearly as possible going forward. 

----------------------------------------------------
Thanks!
Garth Fort