Splunking Microsoft Cloud Data: Part 1

So, you've got Office365, Azure, EOP, EOL and a shopping trolley full of Microsoft Cloud Services. What now?

This is Part 1, in a series of step-by-step guides for accessing, configuring and retrieving all the valuable intel from Microsoft Cloud Services. Microsoft are finally coming to the party and opening up more and more API endpoints for their cloud services. We'll start with some of the common sources that are easily configurable using the Splunk Add-on for Microsoft Cloud Services, and in later posts we'll cover some other sources including mail logs, EOP reports, threat intel and billing data.

Part 1: Office365 Management Logs and Azure Audit Logs

Some things required along the way:

_{Splunk Add-on for Microsoft Cloud Services}

• Redirect URL
• Certificate (Auto-generated or Self-signed)

_{Azure AD Application}

• Application ID (Client ID)
• API Key
• Tenant ID
• Subscription ID

1)    Install the Splunk Add-on for Microsoft Cloud Services

https://splunkbase.splunk.com/app/3110/

2)    Inside the Add-on, open the Configuration tab, then click Add Account

3)    Copy the Redirect URL. – We’ll need this shortly!

If your data collection node does not have a Splunk Web UI, use

http://<host:port>/enUS/app/Splunk_TA_microsoft-cloudservices/redirect

Create an Azure Active Directory Application

An Azure Active Directory application is required to allow Splunk to read information from Azure. The application provides permissions and API access to data for your subscription.

Log in to your Azure Account through the Azure portal.

https://portal.azure.com/

4)    Select Azure Active Directory.

5)    Select App Registrations

6)    Select New Application Registration

7)    Enter a Name, Select Web app / API and enter a Sign-on URL. Select Create

Note: Sign-on URL does NOT have to be an active URL. The App registration requires the field populated with a value.

8)    Select the application you just created

9)    Select Reply URLs, insert your redirect URL from step 3. Select Save

10) Copy Application ID (Client ID) – We’ll need this shortly!

11) Select Keys, Enter Description and Duration. Select Save. Once saved, Copy key value. – We’ll need this shortly too!

12) In Azure Active Directory, Select Properties. Copy Directory ID (Tenant ID) – We’ll need this shortly!

Assign Application to Subscription Role

13) Select Subscriptions from the side menu

14) Select your Subscription, Select Access Control (IAM), Select Add, Select Reader Role, search for Application Name, Select Application

Add permissions to your Active Directory Application

The application you created needs permissions to read information the Add-on is configured to retrieve. This includes reading activity reports, activity data and service health information for the specified subscription.

15)  Select your application from Azure Active Directory > App Registrations

16) Select Required Permissions, then select Add

17) Select an API, Select Windows Azure Service Management API. Press Select

18) Select Permissions, Select Access Azure Service Management as organisation users under Delegated Permissions. Press Select

19) Select an API, Select Office 365 Management APIs. Press Select

20) Select Permissions, For BOTH Application and Delegated Permissions, Select

·      Read activity reports for your organisation

·      Read activity data for your organisation

·      Read service health information for your organisation

21) Ensure permissions have been saved correctly

Connect accounts using Splunk Web

With Azure configured, we can now add an account that will be used by the inputs.

22) Open a new browser, or a private browsing session. Existing sessions or cached data could cause O365 sign-on issues.

23) In the add-on, then Select Configuration > O365 Account > Add Account.

24) Enter Friendly Name, Client ID (Step 11), API Key (Step 12). Select Add

25) A Microsoft Sign-in page should appear. Enter your Azure Credentials.

26) If the Azure configuration has been done correctly, you should see this page. Select Accept

If successful, the account will be added to the configuration

Add Azure App Account

27) In the add-on, then Select Configuration > O365 Account > Add Account.

28) Enter Friendly Name, Client ID (Step 11), API Key (Step 12), Tenant ID (Step 13), Select Add

If successful, the account will be added to the configuration

Configure Certificate

The certificate is used to enable service-to-service calls, which allow you to avoid reconfiguring authorization tokens if they expire. You can either use an auto-generated certificate or a self-signed certificate.

29) In the Add-on, Select Configuration, Certificate, Select Generate under Option 2

30) Copy the credentials to your clipboard. We’ll need this shortly!

31) Navigate to your Registered App in Azure. Azure AD > App Registration > App. Select Manifest

32) Locate the keyCredentials stanza. Paste certificate BETWEEN square brackets.

33) Delete duplicate code( “keyCredentials”: [

34) Delete duplicate “]” from the end

Should look similar to this:

35) If configured correctly, the O365 Troubleshooting page will show the following:

If errors are detected or the status shows as invalid, restart Splunk.

Configure the Inputs

Now that we’ve created our AD app and have working credentials, we can configure the inputs.

36) Select Inputs > Create New Input > Azure Audit

37) Enter Name, specify Account, Subscription ID and Index. If required, modify the start time and interval. Select Add.

38) Repeat Step 37 and select Office 365 Management APIs. Enter Name, specify Account and Index. If required, modify the data source intervals. Select Add.

39) Once inputs are configured, data should be populated in the mscs:azure:audit and ms:o365:management sourcetypes.

Now we've got the data coming in, check out some examples of what we can use it for:

Office365 Service Status:

sourcetype="ms:o365:management" StatusDisplayName=* | stats first(FeatureStatus{}.FeatureServiceStatusDisplayName) AS Status by WorkloadDisplayName | rename WorkloadDisplayName AS Workload

OneDrive Operations:

sourcetype="ms:o365:management" Workload=OneDrive | stats count by UserId, Operation

File Activity Investigation:

sourcetype="ms:o365:management" Workload=* | dedup file_name, user, file_path, ClientIP | table _time file_name user EventSource file_path ClientIP action | rename user AS User EventSource AS App file_path AS "File Path" ClientIP AS "Client IP" action AS Action

Stay tuned for Part 2!