Storing encrypted credentials

Updated Oct. 22, 2021: Please note that this blog post is deprecated. For an updated blog post on secret storage, see Store Encrypted Secrets in a Splunk App.

Splunk 4.2 was released today and your new resolution:

Build the greatest Splunk app that gathers data from all different source, some that are public and others that require credentials, index them in Splunk and then do some cool things with it.

This blog post will only be concerned with one small, but important aspect of your great app: how to securely store user credentials yet be able to safely access them in clear text when needed. I will split up the post into four sections: get credentials from the user,  access them from your script, where are the credentials stored and security implications.

Get and securely store user credentials

The best time to get user credentials for you app would be during app setup.  If your scripted input needs credentials to work properly you should ship it as disabled and then during app setup ask the user to supply the credentials and activate the input.  So, as you probably already know, Splunk provides a mechanism for asking the user to provide the necessary information to setup an app –  this utility in Splunk lingo is known as setup.xml, for more info see Configure a setup screen. In your setup screen you would want the user to provide you with a new set of credentials so that section of your setup.xml should look like:

    <block title="Add new credentials" endpoint="storage/passwords" entity="_new">
            <input field="name">
                    <label>Username</label>
                    <type>text</type>
            </input>
            <input field="password">
                    <label>Password</label>
                    <type>password</type>
            </input>
            <!-- feel free to comment realm out if your app is not using it -->
            <input field="realm">
                    <label>Realm</label>
                    <type>text</type>
            </input>
    </block>

The above block will gather: username, password and (optionally) realm and will create a new REST entity under the storage/passwords endpoint in your app. For those who don’t speak REST that basically means the credentials will be stored by Splunk somewhere, read further down to see exactly where they’re written.

Access credentials from your script

This section, for simplicity, assumes that your scripted input is written in python. To access and use the credentials requires that you:

(1) tell Splunk to pass the script an auth token that the script can use to access Splunk’s REST endpoints – in this case we just want to access the passwords endpoint.

inputs.conf
--------------------------------
[script://./bin/YourInputScript.py]
......
passAuth   = splunk-system-user

(2) access and use the credentials that the user provided

YourInputScript.py
--------------------------------
import splunk.entity as entity
....
# access the credentials in /servicesNS/nobody/<YourApp>/storage/passwords
def getCredentials(sessionKey):
   myapp = 'name-of-your-app-here'
   try:
      # list all credentials
      entities = entity.getEntities(['admin', 'passwords'], namespace=myapp, 
                                    owner='nobody', sessionKey=sessionKey) 
   except Exception, e:
      raise Exception("Could not get %s credentials from splunk. Error: %s" 
                      % (myapp, str(e)))

   # return first set of credentials
   for i, c in entities.items(): 
        return c['username'], c['clear_password']

   raise Exception("No credentials have been found")  

def main():
        # read session key sent from splunkd
        sessionKey = sys.stdin.readline().strip()

        if len(sessionKey) == 0:
           sys.stderr.write("Did not receive a session key from splunkd. " + 
                            "Please enable passAuth in inputs.conf for this " +
                            "script\n")
           exit(2)

        # now get twitter credentials - might exit if no creds are available 
        username, password = getCredentials(sessionKey)

        # use the credentials to access the data source 
        ......

… and that should be it! To recap we asked the user for credentials, stored them encrypted somewhere and used them in your script. If you don’t care where the credentials are stored and how secure they are feel free to skip the rest of this post and go finish your great app!!

Where are the credentials stored?

The encrypted credentials are stored in: $SPLUNK_HOME/etc/apps//local/app.conf in stanzas that look like this:

[credential:<realm>:<username>]
password = $1$<encrypted-password>

Please read $SPLUNK_HOME/etc/system/README/app.conf.spec for more info on the format.

Security implications

The passwords are encrypted and stored – note that we don’t store password hashes because we need to be able to decrypt it and provide you with the clear text password for the script to use. Since the encrypted data and the encryption key are stored on the same machine, cryptographically this is equivalent to obfuscation. While some people might argue that this is very weak encryption, it is the best we can do with storing the encrypted data and encryption key on the same machine and it is definitely better than clear text passwords To improve on the security we also:

– restrict access on storage/passwords endpoint to only system/admin users (users that have admin_all_objects capability)
– audit all accesses on the endpoint (run the following search to see the relevant audit entries: index=_audit action=*password)

Have fun building your app !!

----------------------------------------------------
Thanks!
Ledion Bitincka