Splunk App for Active Directory 1.1.

4
Deploy and Use the Splunk App for Active Directory
Generated: 5/08/2013 2:32 am

Copyright 2013 Splunk, Inc. All Rights Reserved

Table of Contents
Introduction..........................................................................................................1 About the Splunk App for Windows Server Active Directory.....................1 How this app fits into the Splunk picture....................................................2 How to get support and find more information about Splunk....................2 Before you install.................................................................................................4 Platform and hardware requirements........................................................4 What data the Splunk App for Active Directory collects............................7 Other deployment considerations.............................................................7 Deploy the Splunk App for Active Directory.....................................................9 What a Splunk App for Active Directory deployment looks like .................9 How to deploy the Splunk App for Active Directory .................................12 Enable auditing and local PowerShell script execution on Active Directory servers......................................................................................17 Configure and deploy the technology add-ons ........................................23 Deploy TAs and configurations with a deployment server......................29 Configure the SA-ldapsearch supporting add-on....................................31 Install the app onto the central Splunk instance ......................................34 Upgrade the Splunk App for Active Directory .................................................37 Upgrade the Splunk App for Active Directory ..........................................37 Use the Splunk App for Active Directory .........................................................41 Log in and get started.............................................................................41 Configuration...........................................................................................41 Dashboard reference overview...............................................................42 Dashboard reference: Operations...........................................................43 Dashboard reference: Security...............................................................50 Dashboard reference: Change Management..........................................55 Troubleshoot the Splunk App for Active Directory........................................58 Troubleshoot the Splunk App for Active Directory..................................58 Release notes.....................................................................................................63 Release notes.........................................................................................63

Introduction
About the Splunk App for Windows Server Active Directory
Caution: The Splunk App for Active Directory does not currently work with Splunk universal forwarder versions 5.0 and later. If you run the Splunk App for Active Directory, do not upgrade any of the universal forwarders in that deployment. For additional details, see the release notes. The Splunk App for Windows Server Active Directory (hereafter known as the Splunk App for Active Directory) provides deep insight into your Windows Server Active Directory deployment. You can monitor the health of your forest, assess and dispatch security threats, and much more. Use the Splunk App for Active Directory to: Get a detailed topology report on all aspects of your AD forest, including all domains, sites, domain controllers (complete with operations master roles) and AD objects. Monitor AD Directory Services performance, including replication throughput, search performance, and any anomalous events that might signal upcoming problems. Explore various security aspects in your AD forest, including failed and anomalous logons and account utilization Track changes to various AD objects such as users, groups, computers and group policy objects.

How does it work?

The Splunk App for Active Directory runs on top of a Splunk deployment and gathers extensive Active Directory metrics, including but not limited to: AD replication and health statistics LDAP search statistics Performance monitor statistics Security, Directory Service and Domain Name System (DNS) server event logs

The app presents this data to you with reports and dashboards to give you full visibility into your Active Directory deployment.

How do I get it?

Download the Splunk App for Active Directory from Splunkbase.

How do I upgrade from a previous version?

If you are already running the Splunk App for Active Directory and want to upgrade, be sure to read "Upgrade the Splunk App for Active Directory" for important information and specific upgrading instructions. For information on what's been fixed from the previous version, as well as any known issues in this version, review the release notes.

How this app fits into the Splunk picture

The Splunk App for Active Directory is one of a variety of apps and add-ons available within the Splunk ecosystem. All Splunk apps and add-ons run on top of a core Splunk installation. You need to install Splunk first, and then install the app and/or add-on components of the Splunk App for Active Directory. For details about apps and add-ons, refer to "What are apps and add-ons?" in the core Splunk product documentation. To download Splunk, visit the download page on splunk.com. To get more apps and add-ons, visit Splunkbase.

How to get support and find more information about Splunk

If you need customer support for the Splunk App for Active Directory, log a case via the Splunk Support Portal.

Find more information about Splunk

There are a variety of options for finding more information about Splunk: The core Splunk documentation
2

 Splunk Answers The #splunk IRC channel on EFNET

Before you install

Platform and hardware requirements
This topic discusses the system and hardware requirements for running the Splunk App for Active Directory. Important: Installing and configuring the Splunk App for Active Directory is a complex, intense procedure. It is not for beginners. It requires an in-depth knowledge of Windows and Active Directory, as well as at least a general understanding of how to deploy Splunk in a distributed environment. We recommend that you contact Splunk's Professional Services for assistance in deploying the app. The Splunk App for Active Directory consists of several components: The main Splunk App for Active Directory component installs directly onto a full Splunk instance, otherwise known as the "central" Splunk instance. It does not install onto a universal forwarder or a light forwarder, because Splunk Web is required to use the app's dashboards and reports. The Splunk App for Active Directory includes four technology add-ons (TAs) which collect data from your Active Directory domain controllers and DNS servers. You deploy these TAs based on the version of Windows and the role that the server performs. More information on how to configure and deploy the TAs is available at "What a Splunk App for Active Directory deployment looks like" in this manual. The SA-ldapsearch supporting add-on installs onto both the central Splunk instance and the universal forwarders that collect data for the Splunk App for Active Directory.

Hardware requirements
The Splunk App for Active Directory has hardware requirements similar to core Splunk. Depending on the size of your Active Directory environment, the Splunk App for Active Directory might require multiple servers to handle indexing and searching of AD data. We do not recommend installing the Splunk App for Active Directory in a virtual environment.
4

For additional information on hardware requirements, review "System requirements" in the core Splunk documentation.

Operating system requirements

The main Splunk App for Active Directory component and the SA-ldapsearch supporting add-on install onto a Splunk instance running on any of the following operating systems: Linux Windows XP Windows Vista Windows 7 Windows Server 2003 (with SP2 or later) or Server 2003 R2 (with SP1 or later) Windows Server 2008 (with SP2 or later) or Server 2008 R2 (with SP1 or later) Windows Server 2012 The app is not supported on the following Windows versions: Windows 95 Windows 98 Windows Me Windows NT Workstation/Server 3.1 Windows NT Workstation/Server 3.5 Windows NT Workstation/Server 4.0 Windows 2000 Workstation/Server Windows Server 2008 Core You can also install the Splunk App for Active Directory on a non-Windows Splunk instance to display Active Directory information. The technology add-ons included with the Splunk App for Active Directory install into a universal forwarder. You install universal forwarders on the domain controllers and DNS servers in your AD environment. The TAs (as well as the SA-ldapsearch SA) support the following versions of Windows Server: Windows Server 2003 (with SP2 or later and PowerShell 2.0 or later) Windows Server 2003 R2 (with SP1 or later and PowerShell 2.0 or later) Windows Server 2008 (with SP2 or later and PowerShell 2.0 or later)
5

 Windows Server 2008 R2 (with SP1 or later and PowerShell 2.0 or later) Windows Server 2008 R2 Core (with SP1 or later and PowerShell 2.0 or later) Windows Server 2012 Important: The TAs do not work on computers that run Windows Server 2008 Core because that version of Windows does not support PowerShell. For additional details about supported versions of Windows for Splunk, refer to "System requirements" in the core Splunk product documentation.

What other items are required?

In addition to the operating systems and versions of Splunk listed above, the Splunk App for Active Directory also requires the following: PowerShell version 2.0 or later (PowerShell v1.0 does not work on this platform.) The Splunk support for Active Directory (SA-ldapsearch) supporting add-on version 1.1.4 or later (which requires Java Standard Edition Runtime Environment version 1.7 or later.) The Splunk Technology Add-ons for Active Directory (which are included in the Splunk App for Active Directory installation package). Sideview Utils version 1.3.2 or later. The Splunk Technology Add-on for Windows version 4.5 or later.

What versions of Splunk are supported?

All instances of full Splunk in a Splunk App for Active Directory deployment must run version 4.3.1 or later. All universal forwarders in a Splunk App for Active Directory deployment must run version 4.2.5 or later. Be sure to download the correct version for your platform. In particular, ensure that you're running the 64-bit version of Splunk on 64-bit platforms.

What data the Splunk App for Active Directory collects

This topic describes what data the Splunk App for Active Directory collects. Windows event logs: Security, Application, System, Distributed File System Replication (DFSR), NT File Replication Services (FRS), DNS Server Active Directory schema changes (through Splunk's Active Directory monitoring input) Active Directory forest-wide health, information and replication statistics (through PowerShell scripts) Domain controller health and performance metrics (through performance monitoring inputs for memory, CPU, disk, network, and NTDS operations and connectivity performance counters) DNS server health and information (through PowerShell scripts) General Windows network information (through the Splunk Technology Add-on for Windows)

Other deployment considerations

This topic explores other tactics to consider when deploying the Splunk App for Active Directory and provides answers to frequently asked questions.

Frequently asked questions

Installation 1. Can I collect data remotely from my domain controllers? No. You must install a universal forwarder on your domain controllers so that PowerShell scripts can run and collect data. 2. But what about domain controller performance? Won't a universal forwarder utilize resources? Yes, but universal forwarders are designed to utilize as few resources as possible. Data Collection 1. Can I collect the Windows event logs via a third-party method like Syslog-NG or Snare? No. The Splunk App for Active Directory expects events
7

generated by Splunk's event log inputs and the Splunk TA for Windows. 2. Can I collect data without using a universal forwarder? No. You need the universal forwarder in order to run the included PowerShell scripts.

Deploy the Splunk App for Active Directory

What a Splunk App for Active Directory deployment looks like
This topic explains the overall architecture of a Splunk App for Active Directory deployment. For instructions on how to deploy the app, read "How to deploy the Splunk App for Active Directory."

Overview
The Splunk App for Active Directory is a complex installation that contains many components. They are described in detail below. Installing the app requires in-depth knowledge of Windows systems and at least a basic knowledge of how to deploy Splunk in a distributed environment. We strongly suggest that you read the following core Splunk documentation topics before beginning a Splunk App for Active Directory installation: "Distributed Splunk overview" in the Distributed Deployment Manual. "Hardware capacity planning for a distributed Splunk deployment" in the Distributed Deployment Manual. "Introducing the universal forwarder" in the Distributed Deployment Manual, as well as the individual forwarder installation topics in the same chapter. If your Active Directory environment is large or complex, you might want to engage a member of Professional Services for assistance in planning your Splunk App for Active Directory deployment.

Components of a Splunk App for Active Directory deployment

At a minimum, the Splunk App for Active Directory consists of the following components: A "central" Splunk instance that consists of one or more Splunk servers. The central Splunk instance indexes incoming Active Directory data for the app and allows you to view, search, and report on that data by navigating the app's user interface.
9

 Universal forwarders that install onto the DNS servers and domain controllers in your Active Directory environment. These universal forwarders use the Splunk App for Active Directory's technology add-ons to collect AD data, then forward that data to the central Splunk instance. About the universal forwarders During the setup process, you install a universal forwarder onto the domain controllers and DNS servers in your AD environment. The forwarders then collect data from the servers using technology add-ons and send that data to the central Splunk instance for display and searching. You must install a universal forwarder on each domain controller and DNS server in your AD environment. About the Splunk for Active Directory supporting and technology add-ons The Splunk App for Active Directory comes with four technology add-ons (TAs). These TAs install into universal forwarders on the DNS servers and domain controllers in your AD environment. Each TA is a folder that contains objects that the Splunk App for Active Directory uses to collect data from a DNS server or domain controller. The TAs are specific to the Splunk App for Active Directory. The name of each TA corresponds to the version of Windows that runs on the DNS server or domain controller. The Splunk App for Active Directory installation package contains the TAs, in Splunk_for_Active_Directory\appserver\addons. You install the appropriate TAs for the Windows version and AD role into the universal forwarders on each AD server as part of the deployment process. The SA-ldapsearch supporting add-on is available for download from Splunkbase. You download this add-on and install it onto the central Splunk instance. The following table describes the add-ons and where you install them in the course of deploying the Splunk App for Active Directory: Name
SA-ldapsearch TA-DomainController-NT5

Description
Performs LDAP searches on specified AD forests and domains.

10

Collects AD, event log, and performance metrics on Windows Server 2003 or Server 2003 R2 domain controllers. TA-DomainController-NT6 Collects AD, event log, and performance metrics on Windows Server 2008, Server 2008 R2, or Server 2012 domain controllers. Collects DNS event and debug logs from Windows Server 2003 or Server 2003 R2 DNS servers. Collects DNS event and debug logs from Windows Server 2008, Server 2008 R2, or Server 2012 DNS servers.

TA-DNSServer-NT5 TA-DNSServer-NT6

About the central Splunk App for Active Directory instance The "central" Splunk instance receives AD data from the domain controllers and DNS servers in your AD environment. It can be a single Splunk server that both indexes and presents the data in the app, or it can be a distributed deployment with multiple indexers and search heads to handle increased data and search load. Its size depends on the size and scope of your Active Directory environment. A larger environment requires a distributed deployment because of the amount of data that the AD servers generate.

Example deployment
This diagram depicts a typical Splunk App for Active Directory deployment.

11

How to deploy the Splunk App for Active Directory

This topic details the deployment procedure for the Splunk App for Active Directory.

Overview
There are three main steps to installing the Splunk App for Active Directory: First, you prepare your Active Directory environment so that it properly generates and formats the data for the app. Then, you configure the Splunk App for Active Directory on your central Splunk instance to receive and search the incoming Active Directory data. Finally, you install and configure universal forwarders on your domain controllers and DNS servers so that they send AD data to the central Splunk instance.

Prepare your Active Directory environment

Before you can deploy the Splunk App for Active Directory, you must prepare your AD environment to generate the required data for the app. Important: You must have administrator-level privileges to complete the following steps. If you do not have these credentials, then find someone in your organization who does, as you cannot finish the procedure without this access. To prepare your AD environment for the Splunk App for Active Directory: 1. Verify that all of the domain controllers and DNS servers in your environment have the latest service packs and hot fixes installed. If your AD computer runs this version of Windows: then confirm that it has (at a minimum):
* All service packs Windows Server 2003

Windows Server 2003 R2

* The Windows Management Framework Core Package (KB 968930) * PowerShell v2.0 installed and enabled * The Administrative Templates for Microsoft PowerShell
12

* All service packs Windows Server 2008 R2 Core

* PowerShell v2.0 installed and enabled (Learn how to enable PowerShell)

Windows Server 2008 * All service packs Windows Server 2008 R2 Windows Server 2012 Important: The Splunk App for Active Directory does not support computers that run Windows Server 2008 Core because that version of Windows does not support PowerShell. You must upgrade or reinstall those systems with a version of Windows that the app supports. Review the platform and hardware requirements for additional information.

2. Confirm that PowerShell v2.0 or later is installed. Versions of PowerShell earlier than v2.0 are not compatible with the Splunk App for Active Directory. 3. Set your AD environment's forest and domain functional levels to "Windows Server 2003" or higher. For additional information on forest and domain functional levels, review "What are Active Directory functional levels?" (http://technet.microsoft.com/en-us/library/cc787290%28v=ws.10%29.aspx) on MS TechNet. 4. Enable Security event log auditing and local PowerShell script execution on every domain controller in your AD environment. Caution: When you enable Security event log auditing on your domain controllers, the DCs generate a large number of events. These events significantly impact indexing volume and might cause indexing license violations. You might also see decreased performance on your domain controllers. Read this topic carefully to understand what events the Splunk App for Active Directory must collect to function properly and which events you can choose not to include. 5. If you want detailed DNS server statistics, enable debug logging on your DNS servers by following the instructions at "Select and enable debug logging options on the DNS server" (http://technet.microsoft.com/en-us/library/cc759581%28v=ws.10%29.aspx) on MS TechNet. Caution: When you enable debug logging on your DNS servers, you must consider the following caveats:

13

 If you enable DNS server debug logging, individual DNS server performance will decrease significantly. Debug logging generates significant amounts of data that might exhaust disk space on your DNS servers, which can potentially cause downtime. You must watch and rotate your DNS server logs to prevent disk capacity issues from occurring. Debug logging also greatly increases the overall amount of data indexed by the Splunk App for Active Directory. Ensure that you have a Splunk license that can accommodate the additional indexing volume.

Install and configure the central Splunk instance

Once you have configured your AD environment to send the appropriate data, you must now configure the central Splunk instance to receive, index, and present that data. If your central Splunk instance is: then install the following items onto indexer(s):
* SA-ldapsearch

and install the following items onto search head(s):

a single indexer

* Sideview Utils v1.3.2 or later * The Splunk Technology nothing Add-on for Windows * The Splunk App for Active Directory
* SA-ldapsearch * SA-ldapsearch

* Sideview Utils v1.3.2 or * Sideview Utils v1.3.2 or later a distributed environment with later multiple indexers and search * The Splunk Technology * The Splunk Technology heads Add-on for Windows Add-on for Windows * The Splunk App for * The Splunk App for Active Directory Active Directory 1. Install a full copy of Splunk or designate an existing installation as your central Splunk instance. Important: We strongly recommend a distributed Splunk deployment for the central Splunk instance in a Splunk App for Active Directory installation. Review the Distributed Deployment Manual for information on distributed deployments. 2. Configure Splunk to be a receiving indexer by telling it to listen on a port for incoming AD data.
14

3. Download the SA-ldapsearch supporting add-on. 4. Install and configure SA-ldapsearch on the central Splunk instance. 5. Download and install Sideview Utils 1.3.2 or later on the central Splunk instance. Note: If your central Splunk instance is distributed, then you must install Sideview Utils onto both the search heads and indexers in the instance. 6. Download and install the Splunk Technology Add-on for Windows. Note: If your central Splunk instance runs on Unix or Linux, you might receive a compatibility warning if you install the Splunk TA for Windows through Manager. You can safely ignore this warning. The Splunk App for Active Directory requires several modules that the Splunk TA for Windows provides, and cannot run without the TA installed. 7. Install and configure the Splunk App for Active Directory onto your central Splunk instance. Note: If your central Splunk instance is distributed, then you must install the app onto both the search heads and indexers in the instance. 8. Restart all instances in your Splunk App for Active Directory deployment to ensure that installation and configuration changes take effect. Restart your central Splunk instance first. If your central Splunk instance is distributed, restart both the search heads and indexers. Then, restart all universal forwarders in the deployment.

Install and configure universal forwarders on AD servers

Once you have configured your central Splunk instance to receive incoming AD data, you must now install universal forwarders to send that data from domain controllers and DNS servers in your AD environment. To forward AD data from your AD servers to the central Splunk App for Active Directory instance: 1. Download the Splunk App for Active Directory installation package and unpack it to a known, accessible location.
15

2. Download the Splunk Technology Add-on for Windows and unpack it to a known, accessible location. Caution: The Splunk App for Active Directory is not compatible with the Splunk App for Windows. You must only install the Splunk Technology Add-on for Windows. 3. Download and install a Splunk universal forwarder onto each of the domain controllers and DNS servers in your environment. Important: Install only one universal forwarder on each domain controller or DNS server. When asked for the user to install Splunk as, choose the "Local System" user. When asked for the receiving indexer (where the forwarder should send data), enter the host name or IP address and port of a receiving indexer on your central Splunk instance. Do not enable any of the inputs during the installation. 4. Prepare the Splunk App for Active Directory technology add-ons for the AD servers in your environment. Note: The TAs for the Splunk App for Active Directory reside in Splunk_for_ActiveDirectory\appserver\addons in the Splunk App for Active Directory installation package. 5. If you use a Splunk deployment server to deploy the app, copy the configured TAs into %SPLUNK_HOME%\etc\deployment-apps on your deployment server. Note: We strongly recommend that you use a deployment server to distribute apps, add-ons and configuration files for the Splunk App for Active Directory. 6. If you use a Splunk deployment server to deploy the app, configure serverclass.conf on your deployment server to distribute the add-ons across the AD servers in your environment. 7. Install or deploy the appropriate TAs onto each universal forwarder, according to the table shown below: If the AD computer is: and it runs this version of Windows:
16

then install or deploy these TA(s):

Windows Server 2003 or Server 2003 R2 a domain controller Windows Server 2008, Server 2008 R2, Server 2008 R2 Core, or Server 2012 Windows Server 2003 or Server 2003 R2 a DNS server Windows Server 2008, Server 2008 R2, Server 2008 R2 Core, or Server 2012 Windows Server 2003 or Server 2003 R2

Splunk_TA_Windows TA-DomainController-NT5 Splunk_TA_Windows TA-DomainController-NT6 Splunk_TA_Windows TA-DNSServer-NT5 Splunk_TA_Windows TA-DNSServer-NT6 Splunk_TA_Windows TA-DomainController-NT5 TA-DNSServer-NT5 Splunk_TA_Windows TA-DomainController-NT6 TA-DNSServer-NT6

a domain controller and a DNS server

Windows Server 2008, Server 2008 R2, or Server 2008 R2 Core

Note: If you do not have a deployment server, or do not want to use one to deploy the TA(s), then you must manually copy them to %SPLUNK_HOME%\etc\apps on each Active Directory domain controller or DNS server.

If your Splunk deployment is large or complex, you might want to engage a member of Splunk's Professional Services team to assist you in deploying the Splunk App for Active Directory into your environment.

Enable auditing and local PowerShell script execution on Active Directory servers
The Splunk App for Active Directory requires that you enable certain features in your Active Directory (AD) environment in order for the app to function optimally. This topic discusses how to enable auditing of AD events and execution of local PowerShell scripts.

Auditing overview
By default, Active Directory does not automatically audit certain security events. You must enable auditing of these events so that your domain controllers log them into the Security event log channel.

17

You do this by creating a Group Policy object (GPO) and deploying that GPO to all domain controllers (DCs) in your AD environment. Once you activate the GPO, your DCs will log these security events into the Security event log. After you deploy universal forwarders with the appropriate technology add-ons onto your DCs, the forwarders collect the logs and forward them to the central Splunk App for Active Directory instance. Note: This topic shows you how to create individual Group Policy objects (GPOs) for both sets of settings. If you wish, you can combine both the PowerShell and audit settings into a single GPO. For ease of administration, you should create and deploy these GPOs separately from other GPOs.

Important information on security event auditing and indexing volume

When you enable auditing of the Security Event Log on your domain controllers, the DCs generate a lot of data. These events significantly increase indexing volume and might cause indexing license violations. You might also see decreased performance on your domain controllers based on how much additional data the servers generate. If you are concerned about the impact that enabling security event auditing might have on your indexing volume, you can tweak policy settings to generate only the data that is important to you. Refer to the table below to learn about which policy settings generate which event types, and how the Splunk App for Active Directory uses those events to populate its dashboards, reports and lookups. If you choose to disable certain policy settings in an effort to curb indexing volume, you directly affect how much data gets sent to the Splunk App for Active Directory. The table below lists what data you lose if you do not enable a particular policy setting. This is not an all-inclusive list - some lookups are correlated across various policy settings, as multiple events often derive a single knowledge object. Failure to enable all of the policy settings might cause the Splunk App for Active Directory to display incomplete or incorrect knowledge objects in its dashboards and reports. Policy setting:
Audit Account Logon Events Yes

Required?

What the Splunk App for Active Directory uses it for:

Administrator Audit dashboards

Security->Logon dashboards Security->Reports->New (Computer or Domain)

18

Accounts Session ID-to-User (tSessions) lookup Computer-to-IP Address (tHostinfo) lookup

Administrator Audit dashboards Audit Account Management No

Change Management dashboards

Administrator Audit dashboards

Audit Logon Events

No

Logon and access information

Administrator Audit dashboards

Audit Object Access

No

Information on who changed a GPO and when

Security->Reports->Group Policy Reports

Audit Policy Change

No

GPO Change Management dashboard

Directory Services replication events

Audit System Events

No

Advanced Audit Policy settings

You might alternatively want to use the Advanced Audit Policy (AAP) configuration settings to control which events your domain controllers send to the Splunk App for Active Directory. While we do support this method, it is outside the scope of this document to list all available AAP configuration options. This is because of the number of available AAP configuration options and the fact that those options change with different Windows versions - for example, the options for the Windows Server 2003 family differ from those in the Windows Server 2008 family. Windows Vista and earlier versions of Windows do not support AAP. If you need more granularity in the types of audit events you want generated, you can review eventtypes.conf (located in the Splunk App for Active Directory installation at %SPLUNK_HOME%\etc\apps\Splunk_for_ActiveDirectory\default) for the event codes that the app looks for. With that information, you can create a GPO that enables AAP and generates audit events for only those specific event codes. Note: When you enable AAP, Windows disables configurations for standard Audit Policy.

19

Enable auditing
To enable auditing of security events in your AD domain or forest: On Windows Server 2003 and Server 2003 R2 1. Create a new Active Directory GPO: a. Click Start > Administrative Tools > Active Directory Sites and Services. b. In the left pane, under "Sites", locate the forest for which you want to set group policy. c. Right-click the site, then select Properties. d. In the window that appears, click the Group Policy tab. e. Click New. f. Enter a unique name for your new GPO that you will remember. 2. Open the GPO for editing by clicking the Edit... button in the Group Policy properties window. 3. In the GPO Editor, select Computer Configuration > Windows Settings > Security Settings > Local Policy > Audit Policy. 4. Enable both Success and Failure auditing of the following policy settings: Audit account logon events Audit account management Audit directory service access Audit logon events Audit object access Audit policy change Audit privilege use Audit system events 5. Close the Group Policy Object Editor window to save your changes. 6. Deploy the GPO:

20

a. Open Active Directory Users and Computers. Click Start > Administrative Tools > Active Directory Users and Computers. b. In the left pane of the window that appears, right-click Domain controllers then click Properties. c. Click the Group Policy tab. d. Click the Add... button. e. In the dialog that appears the All tab. f. Select the GPO you created in Step 1, then click OK. g. Move your GPO up or down in the priority list to your liking. h. Close the window to save changes. On Windows Server 2008 and Server 2008 R2 1. Create a new GPO: a. Click Start > Administrative Tools > Group Policy Management. b. In the left pane, under "Group Policy Management," expand the forest and domain for which you want to set group policy. c. Right-click Group Policy objects and select New. d. In the dialog window that opens, enter a unique name for your new GPO that you will remember in the Name field, and select None for the Source Starter GPO field. 2. Open the GPO for editing by right-clicking the newly created GPO In the Group Policy Objects window and selecting Edit. 3. In the GPO editor, select Computer Configuration > Policies > Windows Settings > Security Settings > Local Policy > Audit Policy. 4. Enable both Success and Failure auditing of the following policy settings: Audit account logon events Audit account management
21

 Audit directory service access Audit logon events Audit object access Audit policy change Audit privilege use Audit system events 5. Close the Group Policy Object Editor window to save your changes. 6. Deploy the GPO: a. In Group Policy Management, in the left pane of the window, right-click on the Domain Controllers item and click Link an existing GPO..." b. In the window that appears, select the GPO you created in Step 1. c. Click OK. The GPMC will refresh to show that your GPO is now linked to the Domain Controllers organizational unit.

Enable local PowerShell script execution

The Splunk App for Active Directory Technology Add-ons contain PowerShell scripts that must run on the domain controllers and DNS servers in your AD environment. You must configure your domain controllers to allow local execution of PowerShell scripts so that they can run. To enable local execution of PowerShell scripts on your domain controllers: 1. If required, download PowerShell (http://support.microsoft.com/kb/968929) from Microsoft's Support site and install it. Note: All versions of Windows Server 2003 R2, Windows Server 2008 SP2 (except Core), and Windows Server 2008 R2 have PowerShell installed by default. 2. If required, download the Administrative Templates for Microsoft PowerShell (http://www.microsoft.com/en-us/download/details.aspx?id=25119) from Microsoft and install them. Note: All versions of Windows Server 2003 R2, Windows Server 2008 (except Core) and Windows 2008 R2 have the required templates for PowerShell installed.

22

3. Create a new Active Directory GPO: 4. Open the GPO for editing. 5. In the GPO editor, select Computer Configuration > Policies > Administrative Templates > Windows Components > Windows PowerShell. 6. Right-click "Turn on script execution", then select "Edit". 7. In the window that appears, click the "Enabled" radio button. 8. In the "Execution Policy" drop-down, select Allow local scripts and remote signed scripts. 9. Click "OK" to accept the changes. 10. Close the Group Policy Object editor to save your changes. 11. Deploy the GPO.

Configure and deploy the technology add-ons

This topic discusses what you must do to deploy and configure the technology add-ons (TAs) on your Active Directory domain controllers and DNS servers.

Overview
The Splunk App for Active Directory uses several indexes to store its data for later use. The universal forwarders in a Splunk App for Active Directory deployment tag the incoming data with the correct index, which the app then uses in its dashboards, reports, and lookups. By default, the Splunk App for Active Directory is configured to store data in the following indexes: Data type
Security, System and Application event logs* Active Directory replication and DNS Server event logs Performance monitoring / metrics PowerShell logs for Active Directory health (ldapsearch) / metrics

Index
main winevents perfmon msad

23

* The Splunk Technology add-on for Windows collects these logs. It uses the main index by default. If you want the Splunk App for Active Directory to use different indexes (for example, if you are using an existing Splunk instance as the central Splunk instance, or are upgrading from a previous version that used different indexes), then follow the instructions in this topic to configure the technology add-ons to use different indexes than what comes with the Splunk App for Active Directory out of the box. You can find the technology add-ons in the Splunk App for Active Directory installation package, at Splunk_for_ActiveDirectory\appserver\addons Important: If you want to deploy the Splunk App for Active Directory with default settings as shown in the table above, then do not proceed further in this topic. The work is already done for you, and you can proceed to the next step in the deployment process. You only need to edit the technology add-on configurations if you want the Splunk App for Active Directory to use different indexes than the ones shown above.

Configure TAs for domain controllers

You must configure the domain controller TAs on each domain controller in your AD environment. Once the TAs are properly configured, they collect data from the domain controllers and send it to the central Splunk App for Active Directory instance. This instance indexes the data and displays it when you run the app. The technology add-ons you configure depend on what version of Windows is installed on your domain controllers. Following is a table that shows which TAs should be installed on the domain controllers in your environment: If your domain controller runs:
Windows Server 2003 or Windows Server 2003 R2

then install or deploy these TA(s):

Splunk_TA_Windows TA-DomainController-NT5

24

Windows Server 2008, Server 2008 R2, Server 2008 R2 Core, or Server 2012

Splunk_TA_Windows TA-DomainController-NT6

To configure the TAs to send events to the appropriate indexes on your central Splunk instance: Edit the configuration files 1. In the Splunk for Active Directory app installation package, locate the proper TA for the version of domain controller that operates in your AD environment. Note: Use the table above to determine which TA(s) you should configure and deploy. 2. Copy the files located in TA_DomainController_NTx\default to TA_DomainController_NTx\local. Note: You might need to create the local directory if it does not exist. 3. In the TA_DomainController_NTx\local directory, open admon.conf for editing. 4. In the file, under the [nearestDc] stanza, add or change the index attribute to point to the correct index on for Active Directory monitoring data on the central Splunk instance. For example, if you configured the Splunk App for Active Directory to use the ad-monitor index, then configure the [nearestDc] stanza as follows:

[nearestDc] disabled = 0 monitorSubtree = 1 index=ad-monitor

5. Save the file and close it. 6. Open the perfmon.conf file in the same directory for editing. 7. In this file, edit the stanzas so that the index attribute for each stanza points to the correct index for performance metrics on the central Splunk instance. For example, if you configured the Splunk App for Active Directory to use the ad-perfmon index, then edit the perfmon.conf file as follows:

25

[PERFMON:Processor] object = Processor index=ad-perfmon counters = * instances = * interval = 10 disabled = 0 [PERFMON:Memory] index=ad-perfmon object = Memory counters = * interval = 10 disabled = 0 [PERFMON:Network_Interface] index=ad-perfmon object = Network Interface counters = * instances = * interval = 10 disabled = 0 [PERFMON:DFS_Replicated_Folders] index=ad-perfmon object = DFS Replicated Folders counters = * instances = * interval = 30 disabled = 0 [PERFMON:NTDS] index=ad-perfmon object = NTDS counters = * interval = 10 disabled = 0

8. Save the file and close it. 9. Finally, open the inputs.conf file for editing. 10. In this file, edit the stanzas so that the index attribute for each stanza points to the correct index for event log collection on the central Splunk instance For example, if you configured the Splunk App for Active Directory to use the ad-eventlogs index, then edit the inputs.conf file as follows:

26

### ### Windows Event Logs ### ### Application, System and Security logs are handled ### by Splunk_TA_windows and should be compatible with ### what we need ### # # Application and Services Logs - DFS Replication # [WinEventLog:DFS Replication] disabled=0 index=ad-eventlogs sourcetype="WinEventLog:DFS Replication" queue=parsingQueue # # Application and Services Logs - Directory Service # [WinEventLog:Directory Service] disabled=0 index=ad-eventlogs sourcetype="WinEventLog:Directory Service" queue=parsingQueue # # Application and Services Logs - File Replication Service # [WinEventLog:File Replication Service] disabled=0 index=ad-eventlogs sourcetype="WinEventLog:File Replication Service" queue=parsingQueue # # Application and Services Logs - Key Management Service # [WinEventLog:Key Management Service] disabled=0 index=ad-eventlogs sourcetype="WinEventLog:Key Management Service" queue=parsingQueue # # Collect Replication Information # [script://.\bin\runpowershell.cmd ad-repl-stat.ps1] index=ad-monitor source=Powershell sourcetype=MSAD:NT6:Replication

27

interval=300 disabled=false # # Collect Health and Topology Information # [script://.\bin\runpowershell.cmd ad-health.ps1] index=ad-monitor source=Powershell sourcetype=MSAD:NT6:Health interval=300 disabled=false # # Collect Site, Site Link and Subnet Information # [script://.\bin\runpowershell.cmd siteinfo.ps1] index=ad-monitor source=Powershell sourcetype=MSAD:NT6:SiteInfo interval=3600 disabled=false # # Perfmon Collection # [script://$SPLUNK_HOME\bin\scripts\splunk-perfmon.path] index=ad-perfmon interval=3600 disabled=false source=PerformanceMonitor queue=winparsing # # ADMon Collection # [script://$SPLUNK_HOME\bin\scripts\splunk-admon.path] index=ad-monitor interval=3600 disabled=false # # Subnet Affinity Log # [monitor://C:\Windows\debug\netlogon.log] index=ad-monitor sourcetype=MSAD:NT6:Netlogon disabled=false

11. Save the file and close it.

28

Configure TAs for DNS servers

For the DNS servers in your AD environment, configure and deploy the TA_DNSServer_NTx TAs in the same way that you configure the TAs for the domain controllers. If your DNS server runs:
Windows Server 2003 or Windows Server 2003 R2

then install or deploy these TA(s):

TA-DNSServer-NT5

Windows Server 2008, Server 2008 R2, Server 2008 R2 TA-DNSServer-NT6 Core, or Server 2012

Note: You do not need to install the Splunk for Windows TA on DNS servers.

Deploy the technology add-ons

To install the TAs, place them into the %SPLUNK_HOME%\etc\apps directory in the universal forwarder on each domain controller. If you have a deployment server in your Splunk deployment, you can use it to distribute the apps to your domain controllers by placing the TAs into the %SPLUNK_HOME%\etc\deployment-apps on the deployment server. Create a server class that differentiates domain controllers from member servers to ensure the TAs get deployed only to the appropriate computers. For more information on configuring deployment servers, read About deployment server" in the core Splunk documentation. For more information on how to create a server class on your deployment server to differentiate domain controllers from other servers, read Define server classes" in the core Splunk documentation.

Deploy TAs and configurations with a deployment server

This topic discusses what is required to deploy the Splunk App for Active Directory's technology add-ons from a deployment server.

Overview
We strongly recommend that you use a deployment server to distribute the technology add-ons (TAs) and configurations across the domain controllers and
29

DNS servers in your environment. Once configured, the deployment server makes configuration management much easier - you only have to make a change in one place, versus on each AD server. Important: If you do not have a deployment server, or you do not wish to use a deployment server to distribute add-ons and configuration files, then do not proceed further.

Deploy the Splunk App for Active Directory technology add-ons from a deployment server
Before you deploy the TAs onto the universal forwarders, configure them to point to the correct indexes on the central Splunk instance. Read "Configure and deploy the technology add-ons" for specific instructions. To distribute TAs and configurations across your AD servers: 1. Designate or install a full Splunk instance as a deployment server. Review "Plan a deployment" in the core Splunk documentation for instructions. 2. Create a serverclass.conf that controls the distribution of your Splunk App for Active Directory. Review "Define server classes" in the core Splunk documentation. We have provided an example serverclass.conf which you can edit to meet your needs. 3. Configure all universal forwarders to pull configurations from the deployment server. Review "Configure deployment clients" in the core Splunk documentation. 4. Restart Splunk on your deployment server. Example serverclass.conf Below is an example serverclass.conf that you can tailor to meet your specific needs. This file belongs in %SPLUNK_HOME%/etc/system/local on your deployment server.
30

[serverClass:windows] whitelist.0 = forest-* whitelist.1 = eng-* [serverClass:NT5_DC] whitelist.0 = forest-* [serverClass:NT5_DNS] whitelist.0 = forest-* [serverClass:NT6_DC] whitelist.0 = eng-* [serverClass:NT6_DNS] whitelist.0 = eng-* [serverClass:windows:app:Splunk_TA_windows] restartSplunkd = true [serverClass:NT5_DC:app:TA-DomainController-NT5] restartSplunkd = true [serverClass:NT5_DNS:app:TA-DNSServer-NT5] restartSplunkd = true [serverClass:NT6_DC:app:TA-DomainController-NT6] restartSplunkd = true [serverClass:NT6_DNS:app:TA-DNSServer-NT6] restartSplunkd = true

Configure the SA-ldapsearch supporting add-on

This topic discusses the steps needed to install the SA-ldapsearch supporting add-on (SA) into your Splunk App for Active Directory environment.

Configure the ldap.conf file in SA-ldapsearch

To install the SA-ldapsearch SA onto the central Splunk instance: 1. Place the SA-ldapsearch folder into %SPLUNK_HOME%\etc\apps on the central Splunk instance. Note: If your central Splunk instance is set up as a distributed environment, then you must install the app onto all servers that act as search heads. We strongly
31

suggest you use a deployment server to send apps and configurations to all of the search heads in a distributed central Splunk instance. 2. Make a copy of %SPLUNK_HOME%\etc\apps\SA-ldapsearch\default\ldap.conf and place it into %SPLUNK_HOME%\etc\apps\SA-ldapsearch\local. 3. Open the copied file in local for editing. 4. In this file, provide the host and credentials that should be used to search the Active Directory databases. For more information, see "Stanza types for ldap.conf" below. 5. Save the file and close it.

Stanza types for ldap.conf

The ldap.conf file has two types of stanza, all of which are required to monitor a single domain: Informational stanza The first stanza type - the informational stanza - provides information about a domain. The stanza name must be one of the following: The NetBIOS name of the domain (for example, SPL). The fully-qualified domain name (FQDN - for example, spl.com). This stanza has the attributes shown below. Attribute
server

Description
The host name or IP address of a domain controller on the domain that you wish to search. host1.spl.com

Example

port

The LDAP port that the SA-ldapsearch SA should connect to in order to authenticate into the server specified in the server attribute. This 389

attribute is optional, and if not present, will default to 389.

ssl

should attempt to connect to AD using Secure Sockets Layer (SSL). Set to true to connect with SSL and false to connect without SSL.
Whether or not SA-ldapsearch

true

32

Important: If you specify true for this attribute, then the AD server you specify must have a valid SSL certificate installed. For additional information, review "How to enable LDAP over SSL with a third-party certification authority" (http://support.microsoft.com/kb/321051) and "How to troubleshoot LDAP over SSL connection problems" (http://support.microsoft.com/kb/938703) on Microsoft's support site.
basedn The base Distinguished Name (DN) that the app should use when binding to the Directory Service dc=spl,dc=com to collect AD data

binddn

The user, in LDAP format, that the app should bind to Active Directory as for the purposes of collecting AD data. The user must be able to read all records in the directory, in every domain. We cn=Administrator,cn=Users,dc=spl,

do not recommend using the Administrator account.

The password for the user defined in the bindas password

attribute. The password can either be clear-text, or base-64 encoded. To specify a base-64 encoded password, place {64} before the password.
The name format for the domain which was not used in the stanza name. For example, if you used the FQDN for the domain as the stanza name, here you must specify the domain's NetBIOS name.

{64}fohhiuehgihgri

alternatedomain

SPL

Default stanza The second stanza type is the default stanza. Use this stanza when you want to specify the name of a forest-level global catalog (GC) server. Attribute
server port

Description
The host name or IP address of a global catalog (GC) server. Used for contextual AD lookups.

Example
dc1.spl.com

The LDAP port that the SA-ldapsearch SA should connect 389 to on the GC server specified in the server attribute.

33

This attribute is optional, and if not present, will default to 389. should attempt to connect to the GC server using Secure Sockets Layer (SSL). Set to true to connect with SSL and false to connect without SSL.
Whether or not SA-ldapsearch

Important: If you specify true for this attribute, then the GC server you specify must have a false ssl valid SSL certificate installed. For additional information, review "How to enable LDAP over SSL with a third-party certification authority" (http://support.microsoft.com/kb/321051) and "How to troubleshoot LDAP over SSL connection problems" (http://support.microsoft.com/kb/938703) on Microsoft's support site. Defaults to false. Example ldap.conf Following is an example ldap.conf. Important: Do not use this file as is. You must modify it to fit your specific use case.

[spl.com] server = host1;host2;host3 port = 389 ssl = false basedn = dc=spl,dc=com binddn = cn=Administrator,cn=Users,dc=spl,dc=com password = {64}fohhiuehgihgri alternatedomain = SPL [default] server = 172.19.0.2 port = 389 ssl = false

Install the app onto the central Splunk instance

After you have configured the various technology add-ons for the Splunk App for Active Directory, you must now install the app itself onto your central Splunk instance.
34

Once installed, users can log into the app and view data collected from your AD domain controllers and DNS servers.

Install the app onto your central Splunk instance

To install the Splunk App for Active Directory onto the central Splunk instance: 1. Place the Splunk_for_ActiveDirectory folder into %SPLUNK_HOME%\etc\apps onto the central Splunk instance. Note: If your central Splunk instance is set up as a distributed environment, then you must install the app onto all servers acting as search heads. We strongly suggest you use a deployment server to send apps and configurations to all of the search heads in the instance. 2. Make a copy of
%SPLUNK_HOME%\etc\apps\Splunk_for_ActiveDirectory\default\eventtypes.conf and place it into %SPLUNK_HOME%\etc\apps\Splunk_for_ActiveDirectory\local.

3. Open the copied file in local for editing. 4. In this file, ensure that the following stanzas have the correct index defined within each search attribute:

[admon] [wineventlog-security] [wineventlog-ds] [wineventlog-dns] [perfmon] [powershell] [ad-files]

Following is an example of the pertinent section of eventtypes.conf:

[admon] search = index=msad source=ActiveDirectory [wineventlog-security] search = index=main source=WinEventLog:Security [wineventlog-ds] search = index=winevents source="WinEventLog:Directory Service" [wineventlog-dns]

35

search = index=winevents sourcetype=WinEventLog:DNS-Server [perfmon] search = index=perfmon source="Perfmon:*" [powershell] search = index=msad source=Powershell [ad-files] search = index=msad

5. Remove any unchanged stanzas from the file. 6. Save the file and close it.

36

Upgrade the Splunk App for Active Directory

Upgrade the Splunk App for Active Directory
This topic discusses what you must do to upgrade your Splunk App for Active Directory environment from a previous version.

Overview
The upgrade process for the Splunk App for Active Directory is simple, particularly if you have a deployment server to distribute applications and configurations across your servers. There are some important steps you must take in order to upgrade successfully: Upgrading from version 1.0 to version 1.1.4? First, you must distribute the upgraded technology add-ons to all of the universal forwarders in your Splunk App for Active Directory environment. Next, you must upgrade the Splunk App for Active Directory itself and install the new SA-ldapsearch supporting add-on into your central Splunk instance. Finally, you must rebuild the lookup tables which the Splunk App for Active Directory uses to populate its dashboards, views and reports. This last step is very important and, if you do not perform it, will result in missing or incorrect information in your Splunk App for Active Directory deployment. Upgrading from version 1.1 to version 1.1.4? First, you must distribute the upgraded technology add-ons to all of the universal forwarders in your Splunk App for Active Directory environment. Then, you must rebuild the lookup tables which the Splunk App for Active Directory uses to populate its dashboards, views and reports. This last step is very important and, if you do not perform it, will result in missing or incorrect information in your Splunk App for Active Directory deployment.

37

Upgrade the Splunk App for Active Directory

To upgrade the Splunk App for Active Directory to the latest version, follow these steps: 1. Download the updated Splunk App for Active Directory package from Splunkbase and unpack it to an accessible location. 2. Download the new SA-ldapsearch supporting add-on and unpack it to an accessible location. Important: The SA-ldapsearch supporting add-on replaces the Perl LDAP commands that come with the Splunk App for Active Directory. 3. Upgrade the Splunk for Active Directory technology add-ons (TAs): a. Place the updated TAs into $SPLUNK_HOME/etc/deployment-apps on your deployment server. b. Edit the TAs, as described in "Configure and deploy the technology add-ons". c. Finally, deploy the TAs, as described in "Deploy TAs and configurations with a deployment server." Note: If you do not have a deployment server, or do not wish to use one to deploy the updated TAs, then you must manually copy the TAs to %SPLUNK_HOME%\etc\apps on each domain controller or DNS server, as described in "Configure and deploy the technology add-ons" and "How to deploy the Splunk App for Active Directory". You will also need to manually restart each universal forwarder in your Splunk App for Active Directory environment for the changes to take effect. 4. Edit the eventtypes.conf file within the main Splunk App for Active Directory package, as described in "Install the Splunk App for Active Directory. Note: You can also copy your existing eventtypes.conf on your central Splunk instance. 5. Edit the ldap.conf file within the SA-ldapsearch supporting add-on package, as described in "Configure the SA-ldapsearch supporting add-on". Note: You can use the activedirectory.conf from your existing central Splunk instance as the basis for the new ldap.conf.
38

6. Remove the existing Splunk App for Active Directory installation from all servers in your central Splunk instance by deleting the Splunk_for_ActiveDirectory folder within $SPLUNK_HOME/etc/apps. 7. Deploy the new Splunk_for_ActiveDirectory app by placing it into $SPLUNK_HOME/etc/apps on all servers in your central Splunk instance. 8. Deploy the new SA-ldapsearch supporting add-on by placing it into $SPLUNK_HOME/etc/apps on all search heads in your central Splunk instance. 9. Restart the central Splunk App for Active Directory instance: First, restart all search heads in the central Splunk instance. Then, restart all indexers in the instance.

Rebuild the Splunk App for Active Directory's lookup tables

Once you have updated the Splunk App for Active Directory, you must update the app's lookup tables so that it properly presents your data. Important: If you are upgrading from version 1.0 to version 1.1.4, be sure to read the upgrade instructions for version 1.1 for instructions on how to update additional lookup tables that are not shown in this topic. 1. Log into your central Splunk instance. 2. Once logged in, click the Splunk Home tab, then select Splunk App for Active Directory in the list. 3. On the Splunk App for Active Directory's Topology Report page, click the Search menu in the upper left corner. 4. On the Search page, set the time picker (the drop down menu on the right edge of the search bar), to "Last 30 days". 5. In the search bar, type in and execute the following search command to rebuild the Domain Selector lookup table: `domain-selector-search`|outputlookup DomainSelector.csv Note: Be sure to include the back-quotes (`) that surround the search command elements.
39

6. Next, type in and execute the following search command to rebuild the Host to Domain lookup table: `domain-list`|outputlookup DomainList.csv Note: Once each rebuild is complete, Splunk prints a message in the search window stating that the rebuild was successful.

40

Use the Splunk App for Active Directory

Log in and get started
This topic shows you how to log in to Splunk Web, access the Splunk App for Active Directory, and get started.

Log in to Splunk Web

To log into Splunk Web and access the Splunk App for Active Directory, navigate to:
http://<host>:8000

Use the host and port you chose during installation of Splunk. The default port is 8000. The first time you log in to Splunk, the default login details are: Username: admin Password: changeme Splunk recommends that you change the admin password to a secure password.

Access the Splunk App for Active Directory

Once you've logged in to Splunk Web, you'll see the Welcome page. Click the "Splunk Home" tab, which lists all the apps that are currently installed. You should see the Search and Getting Started apps, as well as the Splunk App for Active Directory. Then, to access the Splunk App for Active Directory, click on it in the list. To learn about the various dashboards available, review "Dashboard reference."

Configuration
The Splunk App for Active Directory does not have configurable elements within the application itself. You perform all configuration during the app's setup phase.
41

Review the following topics in this manual for additional information:

Overview
What a Splunk App for Active Directory deployment looks like

Installation
How to deploy the Splunk App for Active Directory Enable auditing and PowerShell on domain controllers Configure and deploy the technology add-ons Configure the SA-ldapsearch supporting add-on Deploy TAs and configurations with a deployment server Install the app onto the central Splunk instance

Dashboard reference overview

The Splunk App for Active Directory comes with several dashboards that give you in-depth access to the operation, health and security status of your Active Directory environment. This topic provides links to other topics that describe the dashboards in detail. The Splunk App for Active Directory has four menus which provide access to data collected by the app in various ways: Search: This menu allows you to use search commands to retrieve any data collected by the app, and presents events that match search terms that you enter, much like regular Splunk. Operations: The Operations menu offers information on your Active Directory's topology, domain, and DNS statuses, as well as reports on health, performance and replication status. Security: The Security menu provides insight into your Active Directory's security profile and operations, including activity on security principal objects as well as changes to AD itself. There are several audit reports which detail the current status of users, computers, groups, and group policy objects. Change Management: The Change Management menu provides details of the recent changes that administrators and valid delegates have made
42

to your Active Directory environment.

Dashboard reference: Operations

This topic discusses the various dashboards available under the Operations menu in the Splunk App for Active Directory.

Topology Report
When you first log into the Splunk App for Active Directory, it displays the Topology Report: a view of all of the AD forests, domains, and domain controllers known to the Splunk App for Active Directory at the present time. You can return to this dashboard at any time by selecting Operations > Topology Report from the menus below the Splunk App for Active Directory banner. The Topology Report dashboard is split into two halves, top and bottom. The top half of the dashboard is a selection panel which allows you to choose the forests, sites, domains, and domain controllers that are known to the Splunk App for Active Directory. You can select multiple objects at a time by holding down the Ctrl key and clicking on the desired entries. The bottom half of the dashboard displays additional information based on what you select on the top half. It displays detailed information on the domain controllers in the selected forest and domain, and includes the following statistics: The host name of the domain controller (DC). The AD site that the DC belongs to. The operating system and version of Windows the server runs. The AD Flexible Single Master Operation (FSMO) role(s) the server holds. Information on the Directory Service Agent (DSA) options available for the DC. Information on the status of the AD services that the machine runs. Information on whether or not the server has registered itself in DNS. Information on whether or not the machine's SYSVOL share is available on the network. In this dashboard, the operations master roles for each server are indicated by icons shown under the "Master Roles" column. Icon Role
43

Description

Schema Master

The Schema Master controls all updates to the Active Directory's schema, then replicates it to all other domain controllers in the forest. There can be only one Schema Master in an entire forest.

The Domain Naming Master controls the naming of all domains within Domain the forest. It is the only domain controller that can add or remove Naming Master domains from Active Directory. As such, only one Domain Naming Master can be present in a forest. The Relative ID Master domain controller maintains the relative ID (RID) resource pool and is responsible for allocating RIDs to other domain controllers within a domain when they are requested during the creation of security principle objects like users and groups. There can only be one RID Master in a domain. This domain controller emulates the Primary Domain Controller (PDC) role for a domain and handles time synchronization across the domain. It also handles various PDC duties (such as password changes, account lockouts and GPO manipulation) for domains which have both Windows Server 2000 and Server 2003 domain controllers present. Only one PDC emulator can be present in a domain. The Infrastructure Master handles updates to the security identifier (SID) and distinguished name (DN) of an object that is cross-referenced by another object in another domain. There can only be one Infrastructure Master in a domain.

Relative ID Master

PDC Emulator Master

Infrastructure Master

The DSA options are listed as icons under the "DSA Options" column: A globe indicates that the server is a Global Catalog (GC). A padlock indicates that the server is a Read-only Domain Controller (RODC). You can click on any domain controller in the list to get additional information about that domain controller. See Domain Controller status for more details. You can limit the number of domain controller objects displayed by selecting the Show n entries list box on the left. You can also search for a specific string (such as the name of a domain controller) by typing in the string in the Search: field on the right.

Domain Services
The Domain Services series of dashboards display information on the selected domains, sites, and domain controllers.

44

Domain Status The Domain Status dashboard gives you information on the selected domain, including: Which domain controllers in the domain hold AD operations masters roles Which site(s) the domain is a part of Which domain controllers control the domain You can choose which domain you want to view by choosing it in the Domain drop-down list in the Domain Status pane of the dashboard. You can click on one of the listed sites to get additional information about the site. See (Site status). You can click on one of the listed domain controllers to get additional information about that controller. See DC status. You can also adjust how much data you see by selecting the time range you desire in the time range picker. Site Status The Site Status dashboard gives you information about the sites in your Active Directory forest, including: Information on which domain controller holds the Inter-site Topology Generator AD operations master role. A list of the domains included in the site. A list of the domain controllers included in the site. A list of the IP network subnets configured for the site. The number and replication status of any site links between this and other AD sites. The targeted and actual weighting of Active Directory-related activity across all of the domain controllers for a particular domain. In the Site Status pane of this dashboard, you can select the site you want to view by choosing it in the Site drop-down list. This automatically updates the Domain drop down list next to it, which lets you view more information about the chosen domain. You can click on a domain in the Domains in Site list to get more information about that domain.
45

You can click on a domain controller in the Domain Controllers in Site list to get details about that domain controller. You can also adjust how much data you see by selecting the time range you desire in the time range picker. Domain Controller Status The Domain Controller Status dashboard gives you information on the domain controllers in your Active Directory environment, including: Information on Directory Services performance, with spark lines and average values over time for important DS related performance counters. Information on replication performance, also with spark lines and average values over time. Any anomalous events that you should be aware of. You can click on individual counters in both the Directory Services performance and Replication Performance sections of the dashboard to review specifics about the values returned by those objects. You can also adjust how much data is displayed by selecting the time range you desire in the time range picker.

DNS Services
The DNS Services series of dashboards displays information about the health, configuration, and performance of Active Directory DNS operations. As DNS is a vital component of Active Directory, problems displayed here might assist in the troubleshooting and analysis of Active Directory itself. DNS Services dashboards are accessible at any time by selecting Operations > DNS Svcs > DNS Status from the menus below the Splunk App for Active Directory banner. DNS Status The DNS Status dashboard displays an overview of current DNS operations and includes: A selectable list of known DNS servers in your AD environment. This includes server host name, the status of DNS on the server, the zones in which it participates, the OS version and service pack level, and a spark
46

line depicting the average amount of DNS queries per second. A selectable list of known DNS zones in the environment. This consists of the zone name, the servers that control the zone, the number of records in the zone and a breakdown of specific record types. A list of anomalous DNS related events that have recently occurred. You can select a server in the DNS Servers list to get more information about that server. See DNS Server status. You can select a zone in the DNS Zones list to get additional details about that zone. See DNS Zone Information. You can click on an anomalous event in the Anomalous events list to get specifics about that event. You can also adjust how much data is displayed by selecting the time range you desire in the time range picker at the top of the dashboard. When you click on the magnifying glass button above, you refresh the data shown in the dashboard. DNS Server Status The DNS Server Status dashboard is similar to the Domain Controller status dashboard described above. However, this dashboard contains information about DNS Query Performance and Recursion Performance instead of AD Directory Services and replication performance. You can click on a performance metric in either performance pane to get details about the selected metric. An Anomalous Events pane at the bottom of the dashboard lists events that warrant further investigation. You can also adjust how much data is displayed by selecting the time range you desire in the time range picker at the top of the dashboard. DNS Zone Information The DNS Zone Information dashboard contains details about a known Active Directory DNS zone, including: Important DNS zone configuration settings. A list of the DNS servers that control the zone. The status of replication of DNS servers that control the zone, and whether or not those servers are out of sync.

47

Note: You cannot change DNS settings in this dashboard. To change DNS settings, you must use the DNS configuration tool on the DNS server(s) that control the zone that you wish to change. You can get additional information about the DNS servers that control the zone by selecting the desired server in the DNS Servers list. See DNS Server status for additional information. You can choose which DNS Zone you want to display by selecting it in the DNS Zone: drop-down list at the top of the dashboard. You can also adjust how much data is displayed by selecting the time range you desire in the time range picker. DNS Performance The DNS Performance dashboard lets you view specific DNS performance metrics in chart form, based on the server and performance metrics you choose in the drop-down lists on the upper right portion of the dashboard. Each metric is overlaid with CPU performance information so that you can correlate anomalous readings with CPU usage in real time. You can adjust how much data is displayed by selecting the time range you desire in the time range picker on the upper left side of the dashboard. DNS Reports The DNS Reports collection allows you to generate reports on your DNS operations by running real-time searches against the collected DNS data. These reports include: DNS Failing Domains DNS Top Filing Domains DNS Top Hosts sending failing queries DNS Top Non-authoritative responses DNS Top Querying Hosts DNS Top Recursive Failure Domains DNS Top Requested Queries Note: In order to view these statistics, your DNS servers must have debug logging enabled. If this feature is not turned on, then these reports will be blank. Review "Deployment process" for instructions.
48

Reports
The Reports series of dashboards provide insight into major health and performance issues with your Active Directory environment. These dashboards provide one-step access to information on problems that are currently happening within your environment, allowing you to quickly analyze and take appropriate action. Health Issues The Health Issues dashboard displays active problems occurring with the domain controllers within your AD forest. It also displays anomalous events that you should be aware of, such as reboots, problems with Knowledge Consistency Checkers (KCCs) on domain controllers, and other unexpected circumstances. You can control how much information is displayed by selecting the time range you desire in the time range picker on the upper left side of the dashboard. Subnet Affinity Issues Occasionally, a server will appear from an IP address that is not associated with a site. The Subnet Affinity Issues dashboard provides a concise report for handling this case. When you see an IP address in this page, log on to your Forest Infrastructure Master and use the Active Directory Sites and Services tool to add the subnet and associate it with a Site. IP addresses that report more frequently are closer to the top of the list. You can control how much information is displayed by selecting the time range you desire in the time range picker on the upper left side of the dashboard. Replication issues The Active Directory Replication Health dashboard lets you review current AD replication agreements, and the status of those agreements. You can change the context in which you view the replication agreements by selecting the Naming Context drop-down on the upper right side of the dashboard. You can also adjust how much time is considered when constructing the reports by selecting the time range you desire in the time range picker on the upper left.

49

Performance The Performance dashboard lets you view all AD-related performance metrics across all domain controllers in your AD forest in a chart. To view a metric, select the desired domain controller from the Server drop-down list on the upper right of the dashboard. Then, select the performance Object and, finally, the desired Counter in the same fashion. The chart is displayed on the lower portion of the dashboard. You can also adjust how much data is displayed by selecting the time range you desire in the time range picker on the upper left portion of the window.

Dashboard reference: Security

The Security series of dashboards give you vision into the defense mechanisms of your Active Directory operations. They provide information on logon failures, attempts to controvert user security settings, and user utilization, as well as display audits and reports on all AD objects in your environment. Each of the Security dashboards is split into two sections. The top section of the dashboard is a selection panel which allows you to choose the forests, sites, domains, and domain controllers that are known to the Splunk App for Active Directory to narrow your search. You can select multiple objects at a time. The bottom portion of the dashboard displays additional information based on what you select on the top half. You can also control how much data is displayed by selecting the time range you desire in the time range picker on the upper left portion of the window. User Logon Failures The User Logon Failures dashboard provides insight into recent failed attempts by users to log into your domain. Specific statistics include: Failed logons over time. Failed interactive logons by IP address. Failed logons by reason (for example, expired password, locked account, or disabled account.) Failed interactive logons by username. Failed logons by logon type.
50

 Users failing to logon from multiple IPs (for example, an active attempt to break into the network.) Anomalous Logons Like the User Logon Failures dashboard, the Anomalous Logons dashboard contains information about questionable user activity on your network. It also shows the more sinister attempts to access restricted network resources. Specific statistics displayed here include: Users logging on from more than one AD site Users logging on from more than one workstation Attempts to log on to disabled or expired accounts User Utilization The User Utilization dashboard displays statistics on: The number of logons over time. The top number of successful logons, by user. The number of locked accounts. The top number of authenticating workstations.

Audit
The Audit series of dashboards allow you to take stock of changes that have happened to your Active Directory environment over time. The audits you can perform are: Administrator audit Computer audit User audit Group audit Group Policy Audit Organizational Unit (OU) Audit In all audit dashboards, you can control how much data is displayed by selecting the time range you desire in the time range picker on the upper left portion of each dashboard.

51

Administrator Audit The Administrator Audit dashboard displays information about recent activity by administrators in your AD environment. The dashboard displays the following specifics: Administrator logons. Attempts by administrators to unlock accounts. Other administrative changes to user accounts. Administrative changes to computer accounts. Administrative changes to groups. Administrative changes to Group Policy and Group Policy objects. Additions, changes or deletions of computer accounts. In the upper portion of the dashboard, you can choose the domain from which you want to display administrator audit data by selecting the Account Domain drop-down list. You can further narrow down your search by selecting an administrator from the Administrator drop-down list. Clicking on a chart in the Administrator Audit dashboard takes you to one of the five other dashboards shown below. Computer Audit The Computer Audit dashboard displays information about access to Active Directory from computer accounts, and includes statistics on: Active Directory record. Group Membership. Accounts that were locked out after attempting a logon from a specific workstation. Failed logons from specific computers. In the upper portion of the dashboard, you can choose the domain from which you want to display computer audit data by selecting the Account Domain drop-down list. You must do so in order to get information on computer account activity within the domain. You can further narrow down your search by typing in the name of a valid computer account in the Computer Account field.

52

User Audit The User Audit dashboard displays information about Active Directory user objects, and includes specifics on: Active Directory record. Group Membership. Accounts that were locked out after failing to logon properly. Failed logons by the selected workstation. In the upper portion of the dashboard, you can choose the domain from which you want to display user audit data by selecting the Account Domain drop-down list. You must do so in order to get information on user account activity within the domain. You can further narrow down your search by typing in the name of a valid user object in the User Account field. Group Audit The Group Audit dashboard displays information about Active Directory group objects, and includes statistics on: Active Directory record. A full Group Membership list. Recent changes to the group membership. In the upper portion of the dashboard, you can choose the domain from which you want to display user audit data by selecting the Account Domain drop-down list. You must do so in order to get information on group account activity within the domain. You can further narrow down your search by typing in the name of a valid group object in the Group Name field. Group Policy Audit The Group Policy Audit dashboard displays information about Active Directory Group Policy objects (GPOs), and includes statistics on: Which group policy objects are linked to which containers. Recent changes to group policy.

53

In the upper portion of the dashboard, you can choose the domain from which you want to display user audit data by selecting the Domain drop-down list. You can further narrow down your search by typing in a valid GPO in the Group Policy Name field. Organizational Unit (OU) Audit The OU Audit dashboard displays information about Active Directory Organizational Units and includes statistics on Active Directory record. In the upper portion of the dashboard, you can choose the domain from which you want to display user audit data by selecting the Domain drop-down list. You can further narrow down your search by typing in a valid OU in the Organizational Unit Name field.

Reports
The Reports series of dashboards displays detailed information about all aspects of your Active Directory environment. You can display and print the following reports: Computer Accounts: All Domain controllers only New Deleted Active Inactive Unused Disabled Trusted No Manager (The object does not have a delegate assigned to it.) Domain Accounts: All New Deleted Active Inactive Unused Disabled Accounts that don't expire
54

 Accounts where a password is not required Accounts where the password does not expire Accounts where the password is too old No manager Sensitive accounts Security Group Accounts: All New Deleted Changed type Empty Large Nested No Manager. Organizational Units: All New Deleted No Manager Those with a direct GPO link. Group Policy Objects: All New Deleted Disabled.

Dashboard reference: Change Management

The Change Management series of dashboards shows you what recent changes have been made to your Active Directory environment. They display changes that have been made by administrators or delegates with authority to make changes to the following objects: User objects Group objects Computer objects Group Policy objects

55

The top half of each Change Management dashboard is a selection panel which allows you to choose the forests, sites, domains, and domain controllers that are known to the Splunk App for Active Directory. You can select multiple objects at a time. The bottom section of the dashboard displays information based on the selection you make in the top section of the dashboard. On all dashboards, you can also control how much data is displayed by selecting the time range you desire in the time range picker on the upper left portion of the dashboard window. User Record Changes The User Record Changes dashboard shows information about changes to user objects in the AD environment, from both a security and a directory services perspective. You can narrow your search by typing in the name of a user in the Account User field in the upper portion of the dashboard. Group Changes The Group Changes dashboard shows information about changes to AD group objects, from the context of both changes to the group object itself and changes to the membership of the group. You can narrow your search by using one of the available drop-downs to limit results based on: Administrator (who made the changes) Member, Group, Group Class (Security or Distribution) Group Scope (Global, Local or Universal). Computer Changes The Computer Changes dashboard displays information about changes to AD computer objects. You can narrow your search by using one of the available drop downs to limit results based on Administrator (who made the changes) and Computer Name.

56

Group Policy Changes The Group Policy Changes dashboard displays information about changes to AD group policy objects (GPOs). You can narrow your search by using one of the available drop downs to limit results based on Administrator (who made the changes) and Group Policy Name.

57

Troubleshoot the Splunk App for Active Directory

Troubleshoot the Splunk App for Active Directory
This topic discusses how you can troubleshoot your Splunk for Active Directory deployment if you aren't seeing the data that you expect.

Is your central Splunk instance correctly configured?

The first thing to check when Splunk App for Active Directory data is incomplete or incorrect is to confirm that the central Splunk instance is properly configured and is receiving data. 1. Confirm that every indexer in the central Splunk instance is configured to receive data. Note: Review "Enable a receiver" in the core Splunk documentation for instructions. 2. If present, confirm that every search head in the central Splunk instance is configured properly. Note: Review "Install a dedicated search head" and "Configure distributed search" in the core Splunk documentation for specific instructions on configuring search heads and search peers. Review "How to deploy the Splunk App for Active Directory" for specific instructions on what specifically to install on the central Splunk instance's search heads. 3. Confirm that the Splunk App for Active Directory is installed properly. The Splunk App for Active Directory must reside on all indexers in the central Splunk instance. SA-ldapsearch must reside on all search heads and indexers in your central Splunk instance.

58

 Indexes that the Splunk App for Active Directory requires must be present on all indexers: msad: for AD health metrics winevents: for Directory Service, Replication Service, DNS server event logs perfmon: for performance metrics Review "About managing indexes" in the core Splunk documentation for instructions on how to confirm that the proper indexes exist. eventtypes.conf (in
%SPLUNK_HOME%\etc\apps\Splunk_for_ActiveDirectory\local)

must be

configured with the proper indexes for the defined event types. Check for typos in the configuration file. Review "Install the app onto the central Splunk instance" for specific installation instructions. ldap.conf (in %SPLUNK_HOME%\etc\apps\SA-ldapsearch\local) must be configured with the proper credentials to bind to Active Directory: Check for typos in the configuration file. Confirm that the credentials are valid, and that the account is not locked out, does not have an expired password, and is allowed full access to the AD schema which you are trying to monitor. Review "Configure the SA-ldapsearch supporting add-on" for specific installation instructions. Review the table below for specific credential troubleshooting instructions. Troubleshoot issues with ldapsearch When the Splunk App for Active Directory cannot complete a search using the SA-ldapsearch supporting add-on, it notifies you by displaying an error message in Manager's status bar (at the top of your browser window), as follows:
External search command 'ldapsearch' returned error code 1. ERROR: com.unboundid.ldap.sdk.LDAPException: 80090308: LdapErr: DSID-0C090334, comment: AcceptSecurityContext error, data 525, vece

The Splunk App for Active Directory also writes a message to $SPLUNK_HOME/var/log/splunk/SA-ldapsearch.log, similar to the following:
2012-08-10 14:58:34.108 -0700 pid=877 com.splunk.program.LDAPSearch:main#-1 ERROR Exception com.unboundid.ldap.sdk.LDAPException thrown: 80090308: LdapErr:

59

DSID-0C090334, comment: AcceptSecurityContext error, data 525, vece^@

If you see an error message similar to this when performing a search, use the following table to decode the data value and figure out how to resolve the error. Data value What it means What you should do

Confirm that the domain that you Either the domain was want to monitor exists and is not found or there was a configured properly, or that your syntax error in the search search string is properly formatted command. and syntactically correct. The username provided in ldap.conf is not valid.

525

and provide the correct user, then restart your central Splunk instance.
Edit ldap.conf

52E

The password provided in ldap.conf is not valid.

and provide the correct password, then restart your central Splunk instance.
Edit ldap.conf Remove the user's log on time restrictions from within Active Directory, then try again. Modify the local security policy of the server from which the specified user is trying to log in to Active Directory, then try again. Change the user's password or set the "Password never expires" bit from within Active Directory, then try again. Re-enable the user account from within Active Directory, then try again. Re-enable the user account from within Active Directory, then try again. Un-set the "User must reset password at next logon" bit for the user account from within Active Directory, then try again.

530

The user account provided is not allowed to log into Active Directory at this time. The user account provided is not allowed to log into Active Directory from the current server.

531

532

The user account provided has an expired password.

533

The user account provided is disabled. The user account provided has expired. The user account provided has the "User must reset password at next logon" bit set.

701

773

775

60

The user account provided is locked because an incorrect password has been entered too many times.

Re-enable the user account from within Active Directory and change the password to a known good one, then try again.

Are the universal forwarders on your instance installed and configured properly?
After you confirm that your central Splunk instance is set up correctly, make sure that the forwarders on the domain controllers and DNS servers are correctly configured and sending data. 1. Confirm that universal forwarders are installed on domain controllers and DNS servers as the Local System user. This allows direct access to the AD schema (on DCs) and DNS event logs (on DNS servers). Review "Deploy a Windows universal forwarder via the installer GUI" or "Deploy a Windows universal forwarder via the command line" for instructions on how to install a UF onto an domain controller or DNS server. If you use a deployment server to manage configurations, make sure that you specify the correct deployment server during UF setup. 2. Confirm that universal forwarders are properly configured to send data to the indexer(s) in the central Splunk instance. Review "Set up forwarding and receiving" for specific instructions on how to configure universal forwarders to send data to indexers. 3. Confirm that the appropriate Splunk App for Active Directory technology add-ons are deployed for the server role and version of Windows installed. Review "Configure and deploy the technology add-ons" for specific instructions. You might want to use a deployment server to more easily manage configurations across multiple forwarders. 4. Ensure that there are no network connectivity problems between the universal forwarders and the indexers in the central Splunk instance. The universal forwarders, indexers, and (if present) deployment servers in a Splunk App for Active Directory deployment must have TCP/IP connectivity between them.

61

 Make sure that firewalls - either installed on or between the universal forwarders, indexers, or deployment servers - are not blocking network traffic. Make sure that routers do not mistakenly filter traffic, particularly across WAN links.

Other common issues

Failed logon data is missing Confirm that your domain controllers are configured to audit failed logons. Review "Enable auditing and local PowerShell script execution on Active Directory servers". GPO change data is missing Confirm that auditing is enabled on your domain controllers. Also, confirm that Active Directory monitoring is enabled. AD topology data and domain information is missing Confirm that local PowerShell script execution is enabled on your domain controllers. The app displays numerous Java errors when it runs the LDAP search commands Confirm that Java SE (Standard Edition) runtime environment version 1.7 or greater is installed on all servers upon which you have installed the SA-ldapsearch supporting add-on.

Get the help you need

If the troubleshooting guidelines do not resolve your problem, then you have several options available to you: Splunk Answers. The Splunk Support Portal. Splunk's Professional Services.

62

Release notes
Release notes
This topic contains information on new features, known issues, and updates as we version the Splunk App and Technology Add-ons for Active Directory.

What's new
Here's what's new in the latest version of the Splunk App for Active Directory: Documentation! The Splunk App for Active Directory will have documentation maintained with every release of the app.

Current known issues

The Splunk App for Active Directory does not currently work with Splunk universal forwarder versions 5.0 and later. If you currently run the Splunk App for Active Directory, do not upgrade any of the universal forwarders in your deployment. If you have already upgraded your forwarders, then downgrade them to version 4.3.5 by uninstalling and reinstalling. You can use Splunk 5.0 and later on indexers and search heads in the deployment, however. Older versions of the universal forwarder might not correctly get some Windows events. To fix this issue, upgrade your forwarders to the latest available 4.3 version. Note: Currently, the Splunk App for Active Directory does not work with Splunk universal forwarder versions 5.0 or later. (SPL-51312)

63