Intrusion Detection, Theory and Practice

Network security has been an issue almost since computers have been networked together. Since the evolution of the internet, there has been an increasing need for security systems.  One important type of security software that has emerged since the evolution of the internet is intrusion detection (ID) systems (IDS).

This review gives an overview of several types of IDSs, and introduces the reader to some of the concepts and practices involving ID.  Be aware that this review is only introductory, and while I have suggested a number of possible systems, further research should always be undertaken before trusting in the strength of your IDS.

What is Intrusion Detection?
ID is a type of security management system for computers and networks.  An IDS gathers and analyzes information from various areas within a computer or a network to identify possible security breaches, which include both intrusions (attacks from outside the organization) and misuse (attacks from within the organization). ID uses vulnerability assessment (sometimes referred to as scanning), which is a technology developed to assess the security of a computer system or network.

Intrusion detection functions include:

• Assessing system and file integrity
• Monitoring and analyzing both user and system activities
• Ability to recognize patterns typical of attacks
• Analysis of abnormal activity patterns
• Analyzing system configurations and vulnerabilities
• Tracking user policy violations

The safeguarding of security is becoming increasingly difficult, because the possible technologies of attack are becoming ever more sophisticated; at the same time, less technical ability is required for the novice attacker, because proven past methods are easily accessed through the Web.

What types of Intrusion Detection Systems are there?
Typically, an ID system follows a two-step process.  The first procedures are host-based and are considered the passive component, these include: inspection of the system's configuration files to detect inadvisable settings; inspection of the password files to detect inadvisable passwords; and inspection of other system areas to detect policy violations.  The second procedures are network-based and are considered the active component: mechanisms are set in place to re-enact known methods of attack and to record system responses.

• Host based systems. These types of systems actually run on the system being monitored.  These examine the system to determine whether the activity on the system is acceptable.
• Network based systems.  These types of systems are placed on the network, nearby the system or systems being monitored.  They examine the network traffic and determine whether it falls within acceptable boundaries.

We will also introduce briefly a more recent type of IDS know as Kernel Based ID:  Optimally resides in the operating system kernel and monitors activity at the lowest level of the system.  These systems have recently started becoming available for a few platforms, and are relatively platform specific.

Host Based Intrusion Detection

Once a network packet has arrived at the host that it was intended for, there is still available a third line of defence behind the firewall and network monitor. This is called "host based ID", and comes in several flavours.

The several types of host based ID are:

• Network Monitors:  Monitors incoming network connections to the host, and attempt to determine whether any of these connections represent a threat.  Network connections that represent some kind of intrusion attempt are acted on.  Note that this is different to network based ID, as it only looks at network traffic coming to the host it is running on, and not all traffic passing the network.  For this reason it does not require promiscuous mode on the network interface.
• Host Monitors:  This application monitors files, file systems, logs, or other parts of the host itself to look for particular types of suspicious activity that might represent an intrusion attempt (or a successful intrusion).  Systems administration staff can then be notified about any problems that are found.
• Monitoring Incoming Connections:  This mechanism attempts to protect a host by intercepting packets that arrive for the host before they can do any damage.
• Monitoring Login Activity:  This type of package monitors log-in and log-out attempts, and alerts the system administrator to activity that is unusual or unexpected.
• Monitoring Root Activity:  There is one more line of defence: Monitoring any actions performed by the root user or system administrator. Many unix systems allow logging or other monitoring of all activity by the root user, and packages such as Logcheck can then scan these logs for unusual activity and notify others about it.
• Monitoring the File Systems:  Programs such as Tripwire, fcheck, and AIDE are designed to detect when files change on the system, and alert the system administrator to any changes.  These programs also take care to make sure that the database of known cryptographic checksums itself isn't compromised in any way.

Network Based Intrusion Detection
Network based IDSs are those that monitor traffic on the entire network segment.  A network interface card (NIC) can operate in one of two modes, these being:

• Normal mode, where packets which are destined for the computer (as determined by the Ethernet or MAC address of the packet) are relayed through to the host system.
• Promiscuous mode, where all packets that are seen on the Ethernet are relayed to the host system.

Network based IDSs normally require that a network interface card is in promiscuous mode.

Network Based Intrusion Detection: The Evolution of the Packet Sniffer
Packet Sniffers and Network Monitors were originally designed to aid in the process of monitoring the traffic on an Ethernet network. The first of these were two products; Novell LANalyser and Microsoft Network Monitor.  These products basically capture all packets that they see on the network.  These tools can be used to do evil as well as good.  For example, packet sniffing can be used to find out someone's Unix password by sniffing telnet packets to the machine that they connect to.  Once an attacker has compromised your network, one of the first things they might install is a packet sniffer of some kind.

Unfortunately, from a security point of view, a packet sniffer is of limited benefit.  The task of capturing every packet on the network, disassembling it, and manually taking action based on the contents of the packet is far too time-consuming, even for a horde of specially trained network gnomes.  What if we were to have some software that automated the process for us (after all, that is what computers are for in the first place, is it not?).

Network based IDSs can intelligently “Sniff” your network as follows:

• Monitor the network for obvious port scans.  Before compromising a system, a hacker will often port scan the system to determine what vulnerabilities might exist.  Port scan attempts from a host on the internet can often be a signal that a person on such a host intends to damage your network.
• Monitor valid connections for well known attacks.  Accessing a web server host on the web server port (80) might be seen as a relatively harmless activity, but some access attempts are in fact deliberate attacks, or attempts at attacks.  For example, an access that looks like "GET /../../../etc/passwd HTTP/1.0" is probably a bad sign, and should be blocked.
• Identify IP spoofing attempts of various sorts.  The ARP protocol that is used to convert IP addresses to MAC addresses is often a target for attack.  By sending forged ARP packets over an Ethernet, an intruder who has obtained access to one system can also "pretend" to be operating as a different system.  This can lead to denial of service attacks of various sorts, as well as system hijacking, whereby an important server (such as a DNS server or authentication server) is "spoofed".  Hackers can use this "spoofing" to redirect packets to their own system, and perform "man in the middle" type attacks on what would otherwise be a secure network.  By keeping a register of ARP packets, a network based IDS can identify the source (Ethernet address) of a compromised system and flush out would-be hackers.

When unwanted activity is detected, network based ID can take action, including interfering with future traffic from the intruder, or reconfiguring a nearby firewall to block all traffic coming from the intruder's computer or network.

Kernel Based Intrusion Detection
Kernel based ID is a relatively new art form, and one that is starting to become prevalent, especially within Linux.
There are two main kernel based IDSs currently available for Linux.  These are OpenWall and LIDS.  These systems take the approach of preventing buffer overflows, increasing file system protection, blocking signals, and generally making it difficult for an attacker to compromise a system.  LIDS also takes steps to prevent certain actions by the root user, such as installing a packet sniffer or changing firewall rules.

Kernel Protection v File System Monitoring
Obviously, systems like LIDS and systems such as Tripwire take a rather different approach to attempting to achieve the same thing.  Both of these packages attempt to prevent a hacker from using the system for unauthorised purposes.

Conclusion
It is important in any environment to know what types of threats you might be facing.  Be aware of any potential security holes in your system, and take care to prevent attacks against these.  For example, a web server that is connected to the internet and placed behind a firewall may be reasonably secure against most packet based attacks, but a CGI program on the server might expose vulnerability.

Most importunately, keep all your systems Up-To-Date with all the latest signatures, security patches, and service pack.  An intrusion detection program between the firewall and the web server might be configured to throw out any accesses that are suspicious.

Watching certain mailing lists (such as BUGTRAQ) and security web sites can help you stay informed about the latest security issues affecting software that you have installed.  If you are alerted of vulnerability in a software package that you are using, or in a firewall product, or perhaps even in intrusion detection software package itself, then take the necessary steps to ensure you are current.