Splunk Cloud "Automated Private App Validation" (APAV) - General Availability

The ecosystem of apps and add-ons is a big part of what makes Splunk valuable and useful to so many of our customers. Part of that value is grounded in the portability of apps between our self-managed and full-managed offerings (aka "on-prem" and "Cloud"). However, one area that we've consistently received feedback on is private apps. Specifically, the ability to easily deploy and maintain them in Splunk Cloud Platform in a fully-self-service fashion. Historically, all private apps with runtime code (custom search commands, alert actions, modular inputs, rest handlers, front-end code in the form of javascript, etc.) needed to go through the same manual inspection process (what we often refer to as "vetting") as SplunkBase hosted apps. This process was a supplement to the automated App Inspect process, and was an additional layer of review that was meant to ensure apps are secure and safe for other apps.

While app vetting is a useful and valuable feature of Splunkbase, for private apps, it often resulted in a suboptimal experience. Around 12 months ago (December 2020) we started a project to eliminate manual vetting requirements for private apps, and we're excited to announce that we are now starting the rollout of Automated Private App Validation to the Splunk Cloud Platform fleet.

What Is Happening Exactly?

With the release of 8.2.2112 to Splunk Cloud Platform, we are excited to announce the general availability of "Automated Private App Validation" (APAV). We are rolling out this capability to the fleet for both our Classic and Victoria experiences. This new capability will remove the requirement for manual review for all of your private apps, meaning you can now fully self service apps in Splunk Cloud Platform, no more support tickets, no more maintenance windows for installs and upgrades, no need to wait for your app to go through the manual review queue. There is also an update to the UI to remind you that private apps are still subject to our general terms, and are not supported directly by Splunk.

How Do I Know It's Working (Or Not…)

As mentioned earlier, APAV is tightly integrated with our App Inspect service, and as a famous web slinger's uncle once said, "with great power comes great responsibility". With the rollout of APAV, this does mean abiding by some new, and long standing App Inspect checks. We've linked to our standard developer guidance in the references section, but I strongly encourage you to start integrating app inspect into your existing Splunk app development process today, if you have not already done so. 

To take advantage of APAV in production, you now have two options, the first is to use the existing UI-based process that you're already familiar with. If that's your process today, the good news is that all of the underlying changes have already been integrated as of the 8.2.2112 release, and so you can simply continue to upload your apps as you do, and the only difference will be no more manual checks. The second option is for those of you that have been working to integrate Splunk Cloud Platform into their CI/CD process, APAV is an important ingredient, along with our Admin Config Service (ACS). If you're not familiar with ACS, the TL;DR is that it's the "control plane" for your Splunk Cloud Stack(s), and provides a clean and modern set of APIs for a number of admin related operations. ACS does require that your app pass all required App Inspect checks, but will be taking advantage of the new "private_app" tag.

Below are some handy SPL snippets to determine if APAV is enabled, with example results. The first example is to determine if the feature flag is enabled or not. For the second example, the "included_tags" tag should exclusively be 'private_app'. If you're still receiving "manual_check" values greater than zero in your app inspect reports, this likely means the APAV feature flag has not been rolled out to your stack yet.

Victoria Experience

Feature Flag SPL:

| rest /servicesNS/-/100-whisper/configs/conf-server/applicationsManagement | stats values(splunk_server) as SH by  private_app_vetting_global

Example Result:
 

Note: that the value for "private_app_vetting_global" should be set to 1, when the feature flag is enabled, if this search does not return any results you likely are not on the proper version of Splunk Cloud

App Inspect Flag SPL:

index=_internal  comp="splunk.appinspect.client" included_tags 

Example Result:

2021-12-17 00:08:12,389 +0000, level=INFO, pid=24528, tid=139933258651456, comp=splunk.appinspect.client, loc=client:103, Submit app to AppInspect with app: /tmp/tmp753ifzctcustomer_app.tar.gz data: {'included_tags': 'private_app', 'excluded_tags': ''}

Note: this example assumes that you have uploaded at least one private app to your stack

Classic Experience

Feature Flag SPL:

| rest /servicesNS/-/dmc/configs/conf-dmc_agent | stats values(splunk_server) as SH by  private_app_vetting_global

Example Result:

Note: that the value for "private_app_vetting_global" should be set to 1, when the feature flag is enabled, if this search does not return any results you likely are not on the proper version of Splunk Cloud

App Inspect Flag SPL:

index=_internal included_tags sourcetype=dmc_agent

Example Result:

2021-12-17 23:35:45,988 INFO (CommandControllerWorker-0) AppInspectValidateCommand: Params used for appinspect _validate method: {'app_package': '/opt/splunk/etc/repository/_unvetted/E382A089-FEED-4804-8FBD-26C328DB3831_bmo_csoc_security.tar.gz', 'app_package_name': 'E382A089-FEED-4804-8FBD-26C328DB3831_customer_app.tar.gz', 'included_tags': ['private_app'], 'stack_id': 'joking-jaguar-hj8'}

For those customers who participated in our limited availability release, thank you for your feedback and engagement!
For those of you looking to learn more, below are some helpful links to get you acquainted with the details.

Happy Splunking!
Kyle Champlin

References

• Splunk Cloud Ready Guidelines For Apps
• App Inspect API Reference
• App Inspect Swagger
  
• App Inspect Tag Reference
• ACS App Management Blog
• ACS API Spec
• ACS API Docs
• ACS Availability
   

----------------------------------------------------
Thanks!
Kyle Champlin