Intrusion Detection Systems
Intrusion Detection Systems
• Presently there is much interest in systems,
which can detect intrusions, IDS (Intrusion
Detection System).
• IDS are of very different character.
• Some focus on one machine and try to stop the
intruder from doing damage, such is LIDS for
Linux.
• Some can detect a worm attack from the way it
spreads from machine to machine, like GrIDS.
Tecniche di Sicurezza dei Sistemi
2
Intrusion Detection Systems
• Several are actually data mining, they determine
from logfiles if there is an intrusion based on
reasoning by an expert system, NSTAT is an
example.
• Many IDS implementations are listening
passively to some LAN segment, look at the
traffic and detect an intrusion. Snort IDS is a
popular freeware program of this Network IDStype.
• Other IDS solutions protect one machine by
access controls.
Tecniche di Sicurezza dei Sistemi
3
What is Intrusion Detection
• Intrusion detection systems (IDSs) are designed for
 detecting, blocking and reporting unauthorized
activity in computer networks.
• “The life expectancy of a default installation of
Linux Red Hat 6.2 server is estimated to be less
than 72 hours.”
• “The fastest compromise happened in 15 minutes
(including scanning, probing and attacking)”
• “Netbios scans affecting Windows computers were
executed with the average of 17 per day”
(source: Honeynet Project)
Tecniche di Sicurezza dei Sistemi
4
1. Motivation for Intrusion Detection
Unauthorized Use of Computer Systems Within Last 12 Months
(source CSI/FBI Study)
80
70
1996
60
1997
50
1998
Percentage of
40
Respondents
1999
2000
30
2001
20
2002
10
0
Y es
No
Tecniche di Sicurezza dei Sistemi
Don't Know
5
1. Motivation for Intrusion Detection
Most Common Attacks (source CSI/FBI)
In year 2002 most common attacks were:
•
•
•
•
Virus (78%)
Insider Abuse of Net Access (78%)
Laptop theft (55%)
Denial of Service and System Penetration (40%)
• Unauthorized Access by Insiders (38%)
(Red color shows the attack types, which IDS can decrease)
Tecniche di Sicurezza dei Sistemi
6
Definitions
• Intrusion
– A set of actions aimed to compromise the
security goals, namely
• Integrity, confidentiality, or availability, of a
computing and networking resource
• Intrusion detection
– The process of identifying and responding to
intrusion activities
Tecniche di Sicurezza dei Sistemi
7
Why Is Intrusion Detection
Necessary?
Prevent
Detect
React/
Survive
Security principles: layered mechanisms
Tecniche di Sicurezza dei Sistemi
8
Elements of Intrusion Detection
• Primary assumptions:
– System activities are observable
– Normal and intrusive activities have distinct
evidence
• Components of intrusion detection systems:
– From an algorithmic perspective:
• Features - capture intrusion evidences
• Models - piece evidences together
– From a system architecture perspective:
• Audit data processor, knowledge base, decision
engine, alarm generation and responses
Tecniche di Sicurezza dei Sistemi
9
Components of Intrusion
Detection System
system activities are
observable
Audit Records
Audit Data
Preprocessor
Activity Data
Detection
Models
Detection Engine
Alarms
Decision
Table
Decision Engine
normal and intrusive
activities have distinct
evidence
Action/Report
Tecniche di Sicurezza dei Sistemi
10
Different Types of IDSs
1. Application based;
2. Host based;
3. Network based.
Tecniche di Sicurezza dei Sistemi
11
Different Types of IDSs
Application IDS
– Watch application logs
– Watch user actions
– Stop attacks targeted against an application
•Advantages
Encrypted data can be read
•Problems
Positioned too high in the attack chain
(the attacks reach the application)
Tecniche di Sicurezza dei Sistemi
12
Different Types of IDSs
Host IDS
–
–
–
–
Watch kernel operations
Watch network interface
Stop illegal system operations
Drop attack packets at network driver
•Advantages
Encrypted data can be read
Each host contributes to the detection process
•Problems
Positioned too high in the attack chain (the
attacks reach the network driver)
Tecniche di Sicurezza dei Sistemi
13
Different Types of IDSs
Network IDS
– Watch network traffic
– Watch active services and servers
– Report and possibly stop network level attacks
•Advantages
Attacks can be stopped early enough (before they
reach the hosts or applications)
Attack information from different subnets can be
correlated
•Problems
Encrypted data cannot be read
Annoyances to normal traffic if for some reason
normal traffic is dropped
Tecniche di Sicurezza dei Sistemi
14
2. Different Types of IDSs
Application-, Host- and Network IDS –
Comparison
Technique
Data Rate
Placement
Cost ($)
Maintenance Effort
Encrypted Data
Switched Networks
Application-based
Application monitoring
Low
Application, userland process
Low to Moderate
Moderate
Supported
Not problematic
Host-based
Host system monitoring
Moderate
Kernel, system process
Moderate
Moderate to High
Supported
Not problematic
Tecniche di Sicurezza dei Sistemi
Network-based
Network segment monitoring
High
Network node
High
Low
Unsupported
Problematic
15
Simple Process Model for ID
Diagram
Parse data, filter data and execute
Detection Algorithms
Capture Data
Analyse Data
Respond
Iterate
For example applications log
network driver, or network cable
Drop packets, send alerts,
update routing tables,
kill processes etc.
Tecniche di Sicurezza dei Sistemi
16
IDS principle of detection
There are two basic methods used by ID Systems:
misuse detection and anomaly detection.
Misuse Detection
– Search attack signatures, which are patterns, byte code
or expressions belonging to a specific attack.
– often called signature-based detection
– A signature is created by analysing an attack method
– The patterns are stored inside the IDS
Example Rule:
Alert tcp !192.168.1.0/24 any -> 192.168.1.0/24 111
(Content: “|00 01 86 A5|”;msg:”External Mountd access”;)
Tecniche di Sicurezza dei Sistemi
17
Example of a NIDS, snort
• Enable NIDS mode of Snort
# ./snort -dev -l ./log -h 192.168.1.0/24 c snort.conf
• The above command means that let Snort work as NIDS
for the network 192.168.1.0/24 according to the rules
inside snort.conf file.
• Sample rule:
•
alert udp any any -> 192.168.1.0/24 5060
(content:"|01 6a 42 c8|"; msg: “SIP session signaling";)
• The rules are modular and it is easy to add new rules.
Typically the rules make alarms of all old security breaches
so that you cannot notice any new breaches.
Tecniche di Sicurezza dei Sistemi
18
IDS principle of detection
Anomaly Detection
“Distinguish abnormal from normal”
Threshold Detection
• X events in Y seconds triggers the alarm
Statistical Measures
• Current traffic profile matches the ”normal” profile
Rule-Based Methods
• Jack never logs in at 6 to 8 AM
• If Jack just sent email from Espoo office, he should
not send email from New York office at the same time
Tecniche di Sicurezza dei Sistemi
19
IDS principle of detection
Anomaly/Misuse Detection – Comparison
Method
Technique
Generalization
Specifity
Sensitivity
False Alarms
Adaptation
Misuse Detection
Detect Patterns of Interest
Problematic
Yes
High
Low
No
Anomaly Detection
Deviations from Learned Norms
Yes
No
Moderate
Moderate
Yes
Tecniche di Sicurezza dei Sistemi
20
IDS response principles
Responses
•Alerts and notifications: email, SMS, pager (important
issue: alert path must be bulletproof)
•Increase Surveillance: log more
•Throttling: slow down malicious traffic
•Blocking Access: drop data, update firewall/router
• Make Counterattack: Eye for an eye tactics
•Honey Pots and Padded Cells: route the hacker to a fake
system and let him play freely
Tecniche di Sicurezza dei Sistemi
21
IDS problems in the detection stage
Detection problems
•True positive, TP, is a malicious attack that is correctly
detected as malicious.
•True negative, TN, is a not an attack and is correctly
classified as benign.
•False positive, FP, is not an attack but has been classified as
an attack.
•False negative, FN, is an attack that has been incorrectly
classified as a benign.
Detection rate is obtained by testing the IDS against set of
intrusive scenarios
“…The false alarm rate is the limiting factor for the performance in an IDS”.
Tecniche di Sicurezza dei Sistemi
22
Advanced IDS Techniques
For Protection
•Stream Reassembly: follow connections and sessions
•Traffic Normalization: see that protocols are followed
• Bayesian Networks: Data mining and decision networks
•Graphical IDSs (for example GrIDS): use graphs to model attacks
•Feature equality heuristics: port stepping, packet gap recognition
•Genetic Programming, Human immune systems
• Tens of research systems exist
For Attacks
•
Evasion methods (fragmentation, mutation etc.)
• IDS trashing (DoS tools to like stick/snot to crash IDS capability
Tecniche di Sicurezza dei Sistemi
23
Evaluation of IDS
• Type I error: (false negative)
– Intrusive but not being detected
• Type II error: (false positive)
– Not intrusive but being detected as intrusive
• Evaluation:
– How to measure?
– ROC - Receiver Operating Characteristics
curve analysis - detection rate vs. False alarm
rate
– What else? Efficiency? “Cost?”
Tecniche di Sicurezza dei Sistemi
24
Example ROC Curve
% Detect
IDS
% False Alarm
• Ideal system should have 100% detection rate
with 0% false alarm
Tecniche di Sicurezza dei Sistemi
25
Next Generation IDSs
• Adaptive
– Detect new intrusions
• Scenario-based
– Correlate (multiple sources of) audit data and
attack information
• Cost-sensitive
– Model cost factors related to intrusion
detection
– Dynamically configure IDS components for
best protection/cost performance
Tecniche di Sicurezza dei Sistemi
26
Adaptive IDSs
ID
Modeling Engine
semiautomatic
anomaly data
IDS
anomaly
detection
ID models
(misuse detection)
ID models
ID models
IDS
IDS
Tecniche di Sicurezza dei Sistemi
27
Semi-automatic Generation of ID
Models
models
Learning
Data mining connection
/ session
records
raw audit data
packets/
events
(ASCII)
Tecniche di Sicurezza dei Sistemi
28
The Feature Construction Problem
dst … service … flag
dst … service … flag %S0
h1
h1
h1
http
http
http
S0
S0
S0
h1
h1
h1
http
http
http
S0
S0
S0
70
72
75
h2
http
S0
h2
http
S0
0
h4
http
S0
h4
http
S0
0
h2
ftp
S0
h2
ftp
S0
0
existing features
useless
syn flood
normal
construct features with
high information gain
How? Use temporal and statistical
patterns, e.g., “a lot of S0
connections to same service/host
within aTecniche
shortdi time
window”
Sicurezza
dei Sistemi
29
Feature Construction Example
• An example: “syn flood” patterns (dst_host
is reference attribute):
– (flag = S0, service = http), (flag = S0, service =
http)  (flag = S0, service = http) [0.6, 2s]
– add features:
• count the connections to the same dst_host in the
past 2 seconds, and among these connections,
• the percentage with the same service,
• the percentage with S0
Tecniche di Sicurezza dei Sistemi
30
An Adaptive IDS Architecture
Detection
Models
FW
Quick and
dirty
Real-time
IDS
Best-effort in
real-time
Dynamic
Cost-sensitive
Decision
Making
Backend
IDS
Thorough and slow
(scenario/trend)
Tecniche di Sicurezza dei Sistemi
31
Detecting Intruders
• Commercially the most used IDS systems are probably
misuse based Network ID Systems, but Host-level IDS is
also needed.
• As an example of a Host-level IDS let us look at LIDS for
Linux.
• The philosophy of LIDS is to have a three layer
protection:
– Firewall
– PortSentry
– LIDS
• The firewall limits access to only allowed ports. In a Webserver only the TCP port 80 is absolutely necessary.
• Disable ports which are not used, for instance by removing
the daemons or by modifying /etc/inetd.conf. Leave only
the basic activities needed.
Tecniche di Sicurezza dei Sistemi
32
Detecting Intruders
• PortSentry is put to some port, which is often
scanned but not used in the system.
• One should find suitable ports where to put
PortSentry by looking at ports which are scanned
often, like 143 or 111.
• Typically nowadays hackers do sweep scanning
looking at only one port in several machines.
• PortSentry monitors activity on specific TCP/UDP
ports. The PortSentry can take actions, like
denying further access to the port.
Tecniche di Sicurezza dei Sistemi
33
Detecting Intruders
• This is based on the assumption that the
hacker will first probe with a scanner the
machine for weaknesses.
• You install PortSentry in TCP-mode by
portsentry -tcp
• ports are in portsentry.conf -file.
Tecniche di Sicurezza dei Sistemi
34
Detecting Intruders
LIDS
• LIDS is an intrusion detection system
that resides in the Linux kernel.
• It basically limits the rights of a root user
to do modifications. It limits root access
to direct port access, direct memory
access, raw access, modification of log
files, limits access to file system. It also
prevents installation of sniffers or
changing firewall rules.
Tecniche di Sicurezza dei Sistemi
35
Detecting Intruders
LIDS
• An administrator can remove the
protection by giving a password to LIDS,
but if a hacker breaks into the root, he
cannot without LIDS password do much
damage.
• Is this good? it certainly makes the life of
a hacker more difficult, but what about a
hacker getting into the kernel?
• How nice it is being an administrator
using LIDS?
Tecniche di Sicurezza dei Sistemi
36