Towards a framework for the generation of enhanced attack/background network traffic for evaluation of network-based intrusion detection systems

Lo, O., Graves, J., Buchanan, W. (2010). Towards a framework for the generation of enhanced attack/background network traffic for evaluation of network-based intrusion detection systems. In: (Ed.) 9th European Conference on Information Warfare and Security, , () ( ed.). (pp. 190-200). Greece: . .



There are a multitude of threats faced in computer networks such as viruses, worms, trojans, attempted user privilege gain, data theft and denial of service attacks. To combat such threats, multiple lines of defence are applied to a network including firewalls, malicious software scanners and intrusion detection systems (IDS). IDSs are generally considered a last line of defence for the detection of attacks; therefore, it is vital for users to assess how well an IDS will perform through means of testing. Although various methodologies have been proposed for the evaluation of IDSs in the past there is still no widely agreed upon standard.
A framework which is capable of carrying out an evaluation of network-based intrusion detection systems (NIDS) is presented in this paper. The paper shows that such a framework requires the need for both realistic real-time network traffic and meaningful metrics when carrying out an evaluation of IDSs. Automation of the testing process is also emphasised - which provides for ease-of-use and simplicity in repetition when carrying out an evaluation.
The framework is evaluated against the NIDS Snort in order to show its capabilities. Through the use of pre-existing programs and utilities, the aim of generating real-time attack traffic is achieved whilst benign background traffic is generated using static data sets. The metrics of efficiency, effectiveness, packet loss, CPU utilisation and memory usage are derived and, finally, the goal of automation is achieved by implementing the framework as a singular application. The results of the evaluation show that, whilst Snort is highly effective in the detection of attacks (true-positives), its main weakness is the dropping of network packets at higher CPU utilisations due to high traffic volume.
Finally, the conclusion to this paper illustrates that the main weakness with current IDS evaluation methodologies is in the approaches used in the generation of benign background traffic. Whilst using static data sets is viable, the main argument against such an approach is that an IDS under evaluation will not react to the traffic in a real-time manner. Furthermore, the use of synthetic traffic generators also has limitations due to the fact that such traffic may not accurately reflect traffic seen on a live network. This paper proposes that further research and development must be applied in the area of benign traffic generation in order to achieve the aim of providing real-time generation of background traffic which realistically mirrors real-life networks when carrying out an evaluation of IDSs.
[Read More]


William Buchanan
Director of CDCS
+44 131 455 2759
Owen Lo
Research Fellow
+44 131 455
Jamie Graves
Affiliate Research Fellow
+44 131 455

Areas of Expertise

Electronic information now plays a vital role in almost every aspect of our daily lives. So the need for a secure and trustworthy online infrastructure is more important than ever. without it, not only the growth of the internet but our personal interactions and the economy itself could be at risk.