Professional Documents
Culture Documents
2 SIEM
Advanced Administration Course
The following topics are to be covered in details: Regular Expressions, custom properties, building
Universal DSM Log Source eXtensions (LSX), and creation of rules.
Intended auditory: IBM Security QRadar SIEM Administrators. Each trainee must be given an ID to
successfully complete the lab training session.
1. Go to https://qradar.scnsoft.com
2. If successful, you should see QRadar console login page.
Agenda
• Day 1
1. Introduction to QRadar administration features and functionality
• User roles and security profiles
• Create and customize network and remote hierarchies
• Reference Sets management
• Configure Vulnerability Scanner
• Configure Routing Rules
• Configure Retention Periods
2. Security Events normalization
• Regular Expressions
• Common normalization fields
• Creating and managing custom properties
3. Building LSX (normalization part)
• LSX structure
• Obligatory fields
• Optional fields
• Patterns and matchers
• Values extraction
• Timestamp formatting
• LSX deployment
• Testing events normalization
Agenda
• Day 1
4. Building LSX (mapping part)
• QRadar event categories (High and Low Level)
• Proper category assignment best practice
• EventCRE vs Custom QID
• Creating new QIDs
• Mapping events in UI
• Testing events mapping
• Day 2
1. Building blocks (BB) overview and specifics. Enabling custom BB.
• Host definitions
• Network definitions
• False positives
• User Tuning / User Defined False Positives Tunings
2. Rules overview
• Custom Rules
o Event rule
o Flow rule
o Common rule
o Offense rule
Agenda
• Day 2
Anomaly Detection Rules
•
o Anomaly rule
o Behavioral rule
o Threshold rule
3. Creating rules
• Functions (rule tests) overview
• Using custom properties and reference sets in rules tests
• Rule responses
o Classic responses (SNMP, Syslog, E-Mail, IF-MAP)
o Specific responses (Reference Set, Reference Map, Trigger Scan)
o Including events into offense
o Indices
o Naming convention. Renaming offenses
4. Tuning rules
• Optimizing Custom Rules
• Creating an OR Condition within the CRE
• Cleaning the SIM Model
• Identifying Network Assets
Agenda
• Day 2
3. Fine tuning false positives
• From event properties
• By Routing Rules
• By rule thresholds update
• Discovering Servers
• Populating Building Blocks
• False Positive Rule Chains
• Cleaning-up fine tunings
4. Analyzing Offenses
5. QRadar Risk Manager
• QRM Overview
• QRM Deployment
• QRM Adapters
• QRM use cases
Day 1
Introduction to QRadar administration
features and functionality
• User roles and security profiles
All users that are allowed to access IBM Security QRadar SIEM must be configured first. The
following items are assigned for each user:
User role;
The role represents granted user privileges to access specific functionality and
information in QRadar. Default roles are Admin (full access) and All (limited access).
Additional user roles can be created to meet the requirements for user permissions.
The following basic operations are available for user roles management:
Create a user role;
Edit a user role;
Delete a user role.
Security profile;
The profile provides access to networks and log sources for QRadar users. Default
security profile is for administrative users and allows access to all networks and all log
sources. Additional security profiles can be created to meet the requirements for
accessing networks and log sources.
The following basic operations are available for security profiles management:
Create a security profile;
Edit a security profile;
Duplicate a security profile;
Delete a security profile.
Introduction to QRadar administration
features and functionality
Security profile management includes permission precedence, which allows QRadar to
display the relevant to permission information in Log Activity tab and Network Activity tab.
The following options are available:
No Restrictions;
Network Only;
Log Source Only;
Network AND Log Sources;
All conditions MUST be met.
Network OR Log Sources.
Any condition MUST be met.
The following basic operations are available for security profiles management:
Create a security profile;
Edit a security profile;
Duplicate a security profile;
Delete a security profile.
Introduction to QRadar administration
features and functionality
Task 1: Create 6 different users on the QRadar. Each user will be given “Admin” user role
and “Admin” security profile for the simplicity reasons. The password will be the same as
the username.
Trainee1
Trainee2
Trainee3
Trainee4
Trainee5
Trainee6
Introduction to QRadar administration
features and functionality
• Create and customize network and remote hierarchies
QRadar SIEM utilizes network hierarchy to understand the network traffic and provide the
ability to view the network activity across the entire network infrastructure. Network
hierarchy needs to be defined by a range of IP addresses representing geographical
location and/or business units (i.e. Europe Office, Marketing Dept.). The following best
practices are applied:
Group units with similar behavior;
Group units with similar traffic patterns;
Separate unique units;
Top the traffic consumers;
15 units per group average;
Group subnets (Classless Inter-Domain Routings, CIDRs) into single network group:
Group IP addresses
Accounting 10.1.1.0/25
Marketing 10.1.2.0/25
Supernets:
/1…/8 – Class A networks (16,777,214 - 2,147,483,392 hosts);
10.0.0.0/8
Subnets:
/25…/32 (except /31) – Classless networks (1 – 124 hosts).
10.1.1.1/32
Important: Ensure all internal address spaces, both routable and non-routable, are defined
within your network hierarchy. Failure to do so could result in QRadar generating an
excessive number of false positives.
Introduction to QRadar administration
features and functionality
Network hierarchy structure is being saved to the following files on the QRadar file system:
/store/configservices/staging/globalconfig/net.conf
/store/configservices/staging/globalconfig/netid.conf
Example: If your trainee ID is “Trainee 1”, you should create a reference set “Trainee1” and
put your trainee ID, name and surname (case insensitive) as 3 separate elements. Other
trainees should follow the suite.
Introduction to QRadar administration
features and functionality
• Configure Vulnerability Scanner
QRadar uses Vulnerability Integration Services (VIS) for building vulnerability assessment
profiles. These profiles use network activity data to determine vulnerabilities and threat
level on the business network assets. Vulnerability assessment integration is build upon
QRadar interaction with various vulnerability scanners. The following two types of
integration are supported:
Direct communication through specific API
Such integration allows vulnerability scanners management through QRadar GUI:
scan schedule, IP range set, vulnerability data transfer etc. This approach allows direct
interaction with particular vulnerability scanner, which can be used on a daily basis for
the most critical network assets.
Vulnerability data file import
Such integration allows import of already available vulnerability reports through file
import. This approach can be used on a monthly basis for large sets of vulnerability
data covering huge networks, which can be gathered in advance.
In order for QRadar to build vulnerability assessment profiles for the network assets, the
following steps must be performed on QRadar:
Add and configure suitable network scanner;
Configure scan schedules.
Introduction to QRadar administration
features and functionality
Task 4: Create Nessus Scanner instance corresponding to your Trainee ID to collect
scheduled results of the Nessus Vulnerability Scanner. Add suitable schedule to collect
the results by QRadar.
Example: If your Trainee ID is “Trainee 1”, you should create a Nessus scanner “Trainee1”
and configure Scheduled Result Import collection type with any other values of choice, i.e.
no specific hostname, etc. Other trainees should follow the suite.
Introduction to QRadar administration
features and functionality
• Configure Routing Rules
After receiving raw log data from log sources, QRadar can forward it to one or more
external systems, such as ticketing or alerting systems. Normalized event data can also be
forwarded to other QRadar systems. QRadar keeps all forwarded data unmodified. The
following items must be configured first in order for QRadar to forward the data:
Forwarding Destinations
These are external systems that QRadar will forward the data to. The following
options are available for the configurations:
Name;
Event Format (Raw or Normalized)
Destination Address;
Destination Port;
Protocol (Raw – TCP/UDP, Normalized – TCP only).
Routing rules
These are the rules that determine what log data is being forwarded and with what
routing options. The following types of rules are available:
Bulk event forwarding (through Admin tab);
Selective event forwarding (through Offenses->Rules tab).
Introduction to QRadar administration
features and functionality
The event forwarding is very convenient for the situations when there are specific
requirements for certain events handling and such events of matched criteria need to be
forwarded for the storage, immediate handling, and/or escalation. Sometimes, it is
necessary to filter out traffic based on a specific value of normalized fields to save storage
space.
The following routing options are available:
Forward;
Drop;
Bypass Correlation.
Introduction to QRadar administration
features and functionality
Task 5: Create a routing rule corresponding to your badge to drop events that contain
your name or surname in the “Username” normalized field.
Example: If you have a badge that says “Trainee 1”, you should create a routing rule called
“Trainee1 Drop” and put your ID trainee1, name or surname (all three) as a filter so that any
incoming event containing your name, surname, or ID as Username should be dropped by
this routing rule. Other trainees should follow the suite.
Introduction to QRadar administration
features and functionality
• Configure Retention Periods
The following list of retention periods is available for many QRadar configuration settings as
well as for the collected data from the log sources:
Automatic Updates
Backup Retention Period (days, 1-65535);
System Settings
Temporary Files Retention Period (6 hrs – 2 years);
Management Database Settings
Accumulator Retention Minute-By-Minute (1 day – 2 years);
Accumulator Retention Hourly (1 day – 2 years);
Accumulator Retention Daily (1 day – 2 years);
Payload Index Retention (1 day – 2 years);
Offense Retention Period (1 day – 2 years);
Attacker History Retention Period (1 day – 2 years);
Target History Retention Period (1 day – 2 years);
Ariel Database Settings
Search Results Retention Period (1 day – 3 month);
Introduction to QRadar administration
features and functionality
Ariel Database Settings
Search Results Retention Period (1 day – 3 month);
Asset Profile Settings
Asset Profile Retention Period (1 day – 2 years);
Events and Flows Settings
How long does the collected data is being kept? QRadar utilizes retention buckets to define
suitable retention periods for the collected events and flows that match custom filters in
order to satisfy different data storage period requirements. QRadar provides 11 retention
buckets: 10 unconfigured and 1 default. The precedence goes from top to bottom. If there
are no specific requirements on the different kind of data storage, the default bucket will
always be applied for all incoming events or flows as it has the lowest precedence, i.e.
always checked last. The following properties are available for the events and flows buckets
retention settings:
Name (convenient name for the bucket);
Keep data placed in this bucket for (1 day – 2 years);
Allow data in this bucket to be compressed (never – 2 weeks);
Delete data in this bucket (space vs retention period);
Current Filters (specific filters for custom only buckets).
Security Events normalization
• Regular Expressions
A regular expression (RE, regex, regexp) is a pattern describing a certain amount of text,
against which strings can be matched. Strings either match the pattern or they don't. The
shell/cmd wildcards and question marks (* and ?) might be considered as a very primitive
type of RE. Just imagine that *.* will match any filename with the “.” (dot) present, however,
?.? will only match a 3-character-long filename with the “.” present.
In simple words, regular expressions represent patterns with metacharacters.
The very first one to look at is . (dot), which represent any character. The other important
metacharacter is ? (question mark), which literally means optional. It comes in handy when
you want to match something that might have additional characters, but doesn’t
necessarily have to: cente?re? (American “center”, British “centre”).
The other two very import meta characters: ^ (carat) and the $ (dollar sign), which are the
start and end of a line respectively. Searching for the pattern ^test$ will find the word test,
but only if it’s on a line by itself.
There are also metacharacters that control how many things are being matched. These are
+ (plus) that matches one or more of the immediately preceding item; and * (asterisk) that
matches any number, including none, of the immediately preceding item.
Security Events normalization
GREEDY LAZY
.+test vs .+?test
Security Events normalization
Regular expressions make the use of character classes or sets, which are specified by [ ]
(square brackets). Simply put the required characters in the square brackets that will form a
class: [A-Za-z0-9].
This example defines a class of all English alphabet letters in both UPPER and lower cases as
well as digits from 0 to 9. The specified regex will match a single character out of this class.
Regular expressions can use () (parentheses) as placeholders for the specific regex match:
User\s([^\s]+)\slogged\sout
NOTE: ^ vs [^abc]
The placeholders or backreferences are used to return only a matched part of the string
instead of the whole string (Hint: Custom Properties)
Another useful metacharacter is | (pipe), which means OR. This metacharacter has to be
used with () (parentheses) identifying the scope: (T|t)est will match both Test and test
capturing T or t as a backreference #1. If we do not want to capture the backreference, the
regex has to be written as follows: (?:T|t)est
NOTE: a? vs (?:a)
Security Events normalization
Regular expressions use special metacharacters (not limited to, depending on regex flavor)
for word character \w, digit \d, space \s, tab \t etc.
There is additional repetition operator that allows to specify how many times a token can
be repeated. The syntax is {min,max}, where min is a positive integer number indicating the
minimum number of matches, and max is an integer equal to or greater than min indicating
the maximum number of matches. If the comma is present but max is omitted, the
maximum number of matches is infinite.
Examples:
\d{2,} will match two or more digits infinitely (10-~);
\d{2,4} will match digits between two and four (10-9999);
\d{2} will match two digits exactly (10-99).
Security Events normalization
• Common Regular Expressions
There is an infinite number of regular expressions that can be used to match a desired
string of text. The following are several common regular expressions:
This is only the beginning of regular expressions that cover very basics necessary for the
QRadar integration. Mastering regular expressions cannot be thought over night and
requires additional time for reading and practicing. Advanced reading material as well as
regex related software (Regex Buddy) can be found on the internet.
Security Events normalization
• Common normalization fields
QRadar includes a comprehensive list of available normalization fields for both Log Activity
and Network Activity. The following fields are considered to be common because they are
used as default fields while displaying the default views of Log Activity and Network Activity.
Common Fields for Log Activity
Example: If your Trainee ID “Trainee 1”, you should create a custom property called
“Trainee1: Affected User” to extract username “fake_user” out of the event. Username can
potentially have only letters, numbers, _ (underscore), and – (hyphen). Other trainees
should follow the suite.
Building LSX (normalization part)
• LSX structure
QRadar provides Universal Device Support Module (uDSM) Log Source eXtension (LSX)
framework that allows custom development and integration for unsupported log sources.
Additionally, LSX can provide parsing enhancements to already existing DSM to cover
additional reporting requirements. The LSX uses Java flavor regular expressions to provide
parsing logic for the data extraction and further normalization with QRadar. LSX consists of
the following items:
Preprocessor
Optional. A binary for collecting/preprocessing unsupported log source data to make it
available for QRadar standard protocols. The need for preprocessor is based on the
log source data availability.
LSX XML Parser
Mandatory. An XML file that contain QRadar parsing and matching rules for the data
normalization process. This is the core of any LSX and its development requires
advanced knowledge of regular expressions.
LSX Mapper
Optional. Ordinary, the mapping of successfully parsed data by LSX XML and assigning
suitable QRadar IDs (QID) is a manual process involving QRadar command prompt
and GUI for each event. Optionally, a shell script for creating QRadar QID mappings
for the data normalization process can be created to automate manual task.
Building LSX (normalization part)
• Obligatory fields
The development of any XML starts with LSX XML template. LSX XML parser consists of the
following obligatory fields:
EventName The action that the event represent. The value of this field plays vital
role in the data correlation process as QRadar at least needs to
know what kind of action was performed.
Building LSX (normalization part)
• Optional fields
The following LSX XML fields are optional and their presence depends on the
corresponding data availability within the event. Best practices suggest that all available
data should be parsed for better correlation and visibility.
Field Name Description
EventCategory specific category the event belongs to;
SourceIp IP address of the source;
SourcePort port of the source;
SourceIpPreNAT real IP address of the source;
SourceIpPostNAT mapped IP address of the source;
SourceMAC MAC identifier of the source;
SourcePortPreNAT real port of the source;
SourcePortPostNAT mapped port of the source;
DestinationIp IP address of the destination;
DestinationPort port of the destination;
DestinationIpPreNAT real IP address of the destination.
Building LSX (normalization part)
• Optional fields
c c c
Building LSX (normalization part)
• Testing events normalization
How to correctly specify the Log Source Identifier (LSI)? There are two approaches to solve
this:
Automatic
LSI is automatically identified as the IP address of the source host that send the
system. This approach is good for the Log Sources autodiscovery feature identifying
supported out-of-the-box Log Sources.
Manual
LSI can be manually set depending on the presence of the corresponding value
(hostname or IP address) in the incoming event. LSI for unsupported Log Sources
should always be specified manually.
Building LSX (normalization part)
• Testing events normalization
Further verification of event normalization would be to check if Event ID was correctly
parsed out of the incoming event as was specified in the LSX XML. This can be done by
checking the value of Event ID through the “Map Event” button in the Event Viewer.
Building LSX (normalization part)
Task 7: Create a Log Source corresponding to your Trainee ID. Look at the data first.
Example: If your Trainee ID is “Trainee 1”, you should create Log Source called
“Trainee1_LSX“. Log Source Identifier must be determined based on the data analysis.
Other trainees should follow the suite.
Building LSX (normalization part)
Task 8: Create a LSX XML corresponding to your Trainee ID. XML should only contain the
patterns and matchers to extract the relevant data out of the relevant log sample. Log
samples relevancy is determined through the corresponding Log Activity filter. After
completion, assign LSX XML to the appropriate Log Source.
Example: If your Trainee ID is “Trainee 1”, you should first create a filter called “Trainee1”
and filter out the events relevant to a specific Log Source. Then, create an XML file based
on the copy of LSX XML Template called “Trainee 1_LSX.xml“. After log samples analysis, only
the required patterns and matchers must be present, others deleted. After the
development is complete, LSX XML should be associated with the Log Source “Trainee1” by
creating suitable Log Source Extension. Other trainees should follow the suite.
Building LSX (mapping part)
• QRadar event categories (High and Low Level)
Prior LSX development finalization, identified Event IDs must be correctly mapped to the
corresponding Event Names. Although the event names might appear as understandable
values in the logs, e.g. DROP, DENY, and ACCEPT, QRadar has no understanding of what
these values actual represent.
From QRadar perspective, these value are strings of text that are not mapped to any known
values. It is necessary to map all unknown events to their equivalents in the QRadar ID
(QID) map. The value will then appear as expected and treated as normalized events.
All QIDs are unique within QRadar and belong to specific categories. Categories are fixed
and cannot be modified by user. These categories are classified as follows:
High-Level Category
Top level category of the QID identifying generic area that QID belongs to.
i.e. Authentication, Access, DoS, Recon, System, etc.
Low-Level Category
Bottom level category of the QID identifying specific section of a particular area that
QID belongs to. One High-Level Category can have many Low-Level Categories, which,
in turn, can have many QIDs.
i.e. Admin Login Successful, Admin Login Failure, Host Logout. As of current QRadar
release, there are 1180 Low-Level Categories.
Building LSX (mapping part)
• Proper category assignment best practice
Any successful Event ID mapping lies within understanding of what exactly the Event ID
represents, i.e. the action behind it. If the action is determined correctly, the appropriate
High-Level Category and then Low-Level Category is chosen for the event out of the list of
available categories. The Low-Level Category must be as close as possible to the meaning
of the action.
Example:
Analysis of the following event
<23>Oct 20 08:33:48 fakeware src=10.10.1.2 uid=root success=yes msg=user fake_user
was created on the system
yields the action of creating a user within the system. Therefore, the appropriate High-Level
and Low-Level Category would be as follows:
Options Description
-l Lists the low-level category
-c Creates a new QID map entry
-m Modifies an existing user-defined QID map entry
-i Imports QID map entries
-e Exports existing user-defined QID map entries
<filename> If you specify the -i or -e option, allows you to
-f
specify a file name to import or export QID map entries
If you specify the -i or -e option, allows you to specify a
-d delimiter for the import or export file. The default is a
comma
-h Display the help options
Building LSX (mapping part)
• Creating new QIDs
New QID are created with the following syntax:
qidmap_cli.sh -c --qname <name> --qdescription <description> --severity <severity> --
lowlevelcategoryid <ID>
where qname and qdescription can be anything of your choice;
severity - 1 to 10, the highest is the most severe;
lowlevelcategoryid – ID of the Low-Level Category (qidmap_cli.sh -l)
Example:
LSX XML matcher’s capture_group “User_Deleted_Success” yields the following CLI
command:
/opt/qradar/bin/qidmap_cli.sh -c --qname "Fakeware User Delete Success" --qdescription
"Fakeware Logging: User Successfully Deleted" --severity 3 --lowlevelcategoryid 3035
where 3035 is the ID for “Authentication-User Account Removed” High-Level, Low-Level
Category pair.
Building LSX (mapping part)
• Mapping events in UI
EventCRE vs new QID:
Building LSX (mapping part)
• Testing events mapping
After the mapping is successful, all new events that are identified by the corresponding
Event IDs will automatically be normalized and will participate in the correlation logic and
rules.
Building LSX (mapping part)
Task 9: Create corresponding EventCRE mapping for each event manually.
Example: All trainees should create suitable EventCRE mappings manually for all events
relevant to their Log Sources. Upon mapping completion, all events must have Event Name
and Low Level Category properly assigned.
Day 2
Building blocks (BB) overview and specifics.
Enabling custom BB.
• Host definitions
A Building Block (BB) is a reusable component that can be included in QRadar rules. BBs
can be created or edited to satisfy specific requirements. In simple words, BBs are testing
conditions for the rules, thus BBs are the rules themselves, but without a specific action or
rule respond defined. QRadar includes a comprehensive set of default predefined BBs to
cover various IT, networking, and computing areas, i.e.:
Definitions, Devices, Databases, Networks, Policies, Threats, etc.
The most common family of BBs used in rules creation is BB:HostDefinition family that
contain vast variety of predefined host assets that can be edited to reflect any
environment. Host definitions family includes many categories, i.e.:
Database Server, LDAP Servers, DMZ Servers, Mail Servers, Proxy Servers and many others.
Missing categories can be easily created and exported as new BBs.
BB:HostDefinition are created taking into the account IP addresses and ports of the
required assets.
After creating a new member of any BB family, its name has to be included into the
“System: Load Building Blocks” rule in order for the BB to work correctly.
Building blocks (BB) overview and specifics.
Enabling custom BB.
Task 10: Create BB:HostDefinition corresponding to your Trainee ID with the source or
destination IP address condition equal to the Source IP address from the relevant log.
Example: If your Trainee ID is “Trainee 1”, you should create a Building Block called
“BB:HostDefinition:Trainee1” and add a condition with the source or destination IP address
equal to 10.10.1.2. Other trainees should follow the suite.
Building blocks (BB) overview and specifics.
Enabling custom BB.
• Network definitions
Another frequently used in rules creation family of BBs is BB:NetworkDefinition family that
contain vast variety of predefined network assets that can be edited to reflect any network
environment. This BB can be regarded as a soft copy of the Network Hierarchy. Missing
networks can be easily created and exported as new BBs.
BB:NetworkDefinition are created taking into the account CIDRs of the required network
assets.
Best practices suggest that it is advisable to use the Network Hierarchy definitions rather
than Building Blocks for rules checking IP addresses for networks.
Building blocks (BB) overview and specifics.
Enabling custom BB.
• False positives
Like any other BBs, BB:FalsePositive family is used to match incoming events based on
certain conditions. However, the purpose of the false-positive identification through BBs is
to exclude matched events from contributing to the other rules that create offenses. This
approach is used to bulk-identify false-positive events with similar conditions, i.e. Low-Level
Category.
After creating a new false-positive BB, its name has to be included into the “FalsePositive:
False Positive Rules and Building Blocks” rule in order for the BB to work correctly.
Building blocks (BB) overview and specifics.
Enabling custom BB.
• User Tuning / User Defined False Positives Tunings
To reduce the number of offenses, the following BBs needs to be edited since they are
producing a lot of traffic:
BB:HostDefinition: VA Scanner Source IP
BB:HostDefinition: Network Management Servers
BB:HostDefinition: Virus Definition and Other Update Servers
BB:HostDefinition: Proxy Servers
BB:NetworkDefinition: NAT Address Range
BB:NetworkDefinition: TrustedNetwork
False Positive Tuning function can be used to tune out false positive events and flows from
creating offenses. The user must have appropriate permissions for creating customized
rules to tune false positives.
The following best practices tuning methodology is applicable:
Disable rules that produce numerous unwanted offenses.
Consider modifying rules to include local rather than remote network context.
When you edit a rule with the attach events for the next 300 seconds option enabled,
wait 300 seconds before closing the related offenses.
Building blocks (BB) overview and specifics.
Enabling custom BB.
• User Tuning / User Defined False Positives Tunings
Rules overview
• Custom Rules
A rule is a collection of tests that perform an action when certain conditions are met. Each
rule can be configured to capture and respond to a specific event, sequence of events, flow
sequence, or offense. The actions which can be triggered can include sending an email or
generating a syslog message. A rule can reference multiple building blocks by using the
tests found in the function sections of the test groups within the Rule Editor.
The following rules exist in QRadar:
Event Rule
This rule is only applicable on incoming events as it tests the conditions for event
properties.
Flow Rule
This rule is only applicable on incoming flows as it tests the conditions for flow
properties.
Common Rule
This rule is applicable for both events and flows as it tests the conditions for events
and flows simultaneously.
Offense Rule
This rule is applicable for offenses only as it tests the conditions for already created
offenses.
Rules overview
• Anomaly Detection Rules
QRadar also offers a functionality of testing the conditions on aggregated fields thus
creating anomaly rules. In order to enable the anomaly rules creation, the search must
contain a “Group By” field, be of Time series type, Capture Time Series data, and must also
be saved.
The following anomaly rules exist in QRadar:
Anomaly Rule
This rule performs tests on aggregated fields for anomalous network activity.
Behavioral Rule
This rule type performs tests on aggregated fields to alert on volume changes in
network activity that occurs in regular seasonal patterns.
Threshold Rule
This rule type performs tests on aggregated fields to alert on network activity that
exceeds a defined threshold.
Rules overview
• Anomaly Detection Rules
Rules overview
• Anomaly Detection Rules
Creating Rules
• Functions (rule tests) overview
QRadar provides a comprehensive set of already predefined rule tests that can be used
when constructing a rule. These tests are written in human language so that the meaning
of the test condition can be easily understood. The tests are grouped into the categories
for a better related to a particular category overview. These categories are:
All
Common Property
Date / Time
Event Property
Functions
Host Profile
IP / Ports
Log Source
Network Property
Creating Rules
• Functions (rule tests) overview
Creating Rules
• Using custom properties and reference sets in rules tests
The following rules are suitable for working with Custom Properties and Reference Sets
conditional testing:
when the event matches this search filter
when any of these event properties are contained in any of these reference set(s)
Introduction to QRadar administration
features and functionality
Task 11: Create rule corresponding to your Trainee ID that matches your ID, name or
surname from the custom property “Trainee X: Affected User” from the relevant log. No
rule response.
Example: If your Trainee ID is “Trainee 1”, you should create an Event Rule called
“Trainee1:Rule: Custom Property Match” and add a condition with the custom property
“Trainee1: Affected User” to match your ID, name or surname. No rule response should be
configured. Other trainees should follow the suite.
Introduction to QRadar administration
features and functionality
Task 12: Create rule corresponding to your Trainee ID that matches your trainee ID,
name or surname from the reference set “TraineeX” from the relevant log. No rule
response.
Example: If your Trainee ID is “Trainee 1”, you should create an Event Rule called
“Trainee1:Rule: Reference Set Match” and add a condition to match the value of custom
property “Trainee1: Custom Property” from the reference set “Trainee1”. No rule response
should be configured. Other trainees should follow the suite.
Creating Rules
• Rule Responses
During rule configuration wizard, QRadar provides the ability to configure the rule
response. The following responses can be set up:
Classic responses (SNMP, Syslog, E-Mail, IF-MAP)
This type of response generates outgoing SNMP trap, e-mail or IF-MAP.
To enable SNMP trap configuration, edit /opt/qradar/conf/offenseCRE.snmp.xml
To enable sending the data to IF-MAP server, system settings must be modified to
allow such kind of data transfer.
Specific responses (Reference Set, Reference Map, Trigger Scan)
This type of response updates reference set/map, or trigger the IP address scan
(destination, source, or both).
Including events into offense
This type of response allows to include the original event into generated offense.
Naming convention. Renaming offenses
The following naming convention is used when naming the rule:
Component.Number.Rule or Offense: Rule Name
i.e. OS.001.Offense: Password Share
The same name of the offense must be specified in the “Dispatch New Event”
Creating Rules
• Rule Responses
Tuning Rules
• Optimizing Custom Rules
When building custom rules, it is highly recommended that you optimize the order of the
testing. This ensures that the rules do not slow down the Common Rule Engine (CRE). The
tests in a rule are executed in the order in which they are displayed in the user interface.
The most memory intensive tests for the CRE are the payload and regular expression
searches. To ensure these tests run against a smaller subset of data and execute faster, it
is strongly recommend you first include one of the following tests:
when the event(s) were detected by one or more of these log source types
when the event QID is one of the following QIDs
when the source IP is one of the following IP addresses
when the destination IP is one of the following IP addresses
when the local IP is one of the following IP addresses
when the remote IP is one of the following IP addresses
when either the source or destination IP is one of the following IP addresses
when the event(s) were detected by one of more of these log sources
Tuning Rules
• Creating an OR Condition within the CRE
When adding more tests to a rule, each test can only be an AND or AND NOT conditional
test. To create an OR condition within the CRE put each separate set of conditions into a
building block and then create a new rule or building block that utilizes the following rule:
when an event matches any of the following rules
This will ensure both Building Blocks are loaded when the test is applied.
Tuning Rules
• Cleaning the SIM Model
When the tuning process is complete, it is recommended that you clean the SIM model.
This ensures that QRadar only displays recent offenses. This function ensures that offenses
are based on the most current rules, discovered servers, and network hierarchy. When you
clean the SIM model, all existing offenses are closed, but this does not affect existing events
and flows.
Tuning Rules
• Identifying Network Assets
The following network assets can be included in the BBs:
Example: If your Trainee ID is “Trainee 1”, you should create an Event Rule that will generate
an offense as a rule response, called “Trainee 1.Offense: Reference Set Match” and add a
condition with the reference set “Trainee1” to match your Trainee ID, name or surname out
of the custom property “Trainee1: Custom Property”. Other trainees should follow the suite.
Analyzing Offenses
• Offenses
QRadar Risk Manager
• QRM Overview
QRadar Risk Manager is an internal component of QRadar SIEM solution that proactively
helps in assessing the risks from vulnerabilities, correlating the network topology
information with data from QRadar SIEM, including assets configuration, events and flow
patterns. QRM also helps in detecting configuration errors in firewalls and IPS systems to
build a complete picture of a possible intrusion path.
QRadar Risk Manager
• QRM Overview
QRM provides the following key capabilities:
Policy monitoring to improve compliance
QRadar Risk Manager features an automated policy engine that simplifies the
assessment of a wide spectrum of information security and compliance policies.
Device configuration management to detect changes and profile future risks
QRadar Risk Manager provides automated collection, monitoring and auditing of
device configurations across an organization’s switches, routers, firewalls and
intrusion detection system (IDS)/intrusion prevention system (IPS) devices.
Modeling and simulation of attacks and network configuration changes
QRadar Risk Manager provides simulation capabilities that can check network
connectivity before and after a proposed network configuration change, such as
adding a firewall rule to one or more devices.
QRadar Risk Manager
• QRM Deployment
IBM Security QRadar Risk Manager deployment includes a IBM Security QRadar SIEM
Console and QRadar Risk Manager appliance as a managed host. QRM can be supplied
either as a hardware appliance or a software image to be installed on existing hardware
that meets certain requirements. During the installation, QRM requires the following
parameters in order to be properly initiated:
• Hostname - Type a fully qualified domain name as the system hostname.
• IP Address - Type the IP address of the system.
• Network Mask - Type the network mask address for the system.
• Gateway - Type the default gateway of the system.
• Primary DNS - Type the primary DNS server address.
• Secondary DNS - Optional. Type the secondary DNS server address.
• Public IP - Optional. Type the Public IP address of the server.
Once the initial installation and configuration are completed, QRM needs to be added as a
managed host to QRadar SIEM. This is achieved through the Deployment Editor in QRadar
Console Admin tab.
QRadar Risk Manager
• QRM Deployment: Adding as a Managed Host
QRadar Risk Manager
• QRM Adapters
QRM uses adapters to integrate the network devices into QRadar SIEM. With the help of
adapters, QRM collects and imports configuration information from the network devices
like firewalls, routers and switches.
The following adapters are supported:
• Check Point SecurePlatform Appliances
• Cisco Internet Operating System (IOS)
• Cisco Catalyst (CatOS)
• Cisco Security Appliances
• Juniper Networks ScreenOS
• Juniper Networks JUNOS
• Juniper Networks NSM
The adapters need to be installed on the QRM in order to provide the support for the
required devices.
QRadar Risk Manager
• QRM Adapters
The following methods of adding devices are available in QRadar Console through Admin
tab (Plugins->QRM):
• Add Device - Add one device.
• Discover Devices - Add multiple devices.
• Discover NSM - Add devices that are managed by a Juniper Networks NSM console.
• Discover CPSMS - Add devices that are managed by a Check Point Security Manager
Server (CPSMS).
For more details, please, consult the user documentation for QRM Adapters.
QRadar Risk Manager
• QRM use cases
The following use cases are identified with QRM key capabilities:
Configuration Audit
Collected by QRM configuration information for network devices, can be used for
audit compliance and to schedule configuration backups. The configuration
information for your devices is collected from device backups in Configuration Source
Management. Each time QRadar Risk Manager backs up your device list, it archives a
copy of your device configuration to provide a historical reference.
View network paths in the topology
The topology in QRadar Risk Manager displays a graphical representation of your
network devices. A topology path search can determine how your network devices are
communicating and the network path that they use to communicate. Path searching
allows QRadar Risk Manager to visibly display the path between a source and
destination, along with the ports, protocols, and rules.
Visualize the attack path of an offense
Attack path visualization ties offenses with topology searches. This visualization
allows security operators to view the offense detail and the path the offense took
through your network. The visual representation shows you the assets in your
network that are communicating to allow an offense to travel through the network.
QRadar Risk Manager
• QRM use cases
The following use cases are identified with QRM key capabilities:
Monitor policies
Use Policy Monitor to define tests that are based on the risk indicators, and then
restrict the test results to filter the query for specific results, violations, protocols,
or vulnerabilities.
Assess assets that have suspicious configurations and communications
Organizations use corporate security policies to define risks and the communications
that are allowed between assets and networks. To assist with compliance and
corporate policy breaches, organizations use Policy Monitor to assess and monitor
risks that might be unknown.
Simulate attacks on network assets
You can use a simulation to test your network for vulnerabilities from various sources.
Simulate the risk of network configuration changes
You can use a topology model to determine the effect of configuration changes on
your network using a simulation.
ScienceSoft’s Identity
WHO WE ARE: An international IT company, expert in the design, development
and delivery of custom software solutions, IT consulting and IT
outsourcing services.
• Locations in Finland and Belarus, customers in 25 countries
• 400 full-time staff; established in 1989
• Two major releases of IBM TCIM (2007-2008), three major releases of IBM
Tivoli Security Information and Event Manager (TSIEM) major releases
(2009-2011)
• More than 120 completed CISM, TCIM, and TSIEM Event Sources and
Compliance Management Modules projects, over 40 device rules and
core bug-fixing for TSOM
Let’s keep in touch