An Innovative Approach to Insider Threat Detection Using Data Mining and Machine Learning

Brown, M. (2017). An Innovative Approach to Insider Threat Detection Using Data Mining and Machine Learning (BEng (Hons) CSF Dissertation). Edinburgh Napier University (Sheykhkanloo, N., Russell, G.).



Insider threats accounted for a fifth of all data breaches in 2015, (Verizon, 2016).
With the introduction of stricter data controls such as General Data Protection Regulation, (, 2016) and many high-profile companies falling victim to data breaches. This project set out to produce an innovative approach to insider threat detection thought the use of data mining and machine learning tools.

The project achieved its aim by using a number of literature material to review and develop a novel approach to insider threat detection using machine learning and data mining.
A semi-supervised model is presented consisting of an unsupervised hybrid approach using a selection of anomaly detection algorithms within Rapidminer and a supervised approach using pattern recognition with MATLAB. ZoneFox (ZoneFox, n.d.), a market leader in user behaviour analytic and insider threat protection provided data collected from their system that imitated live user activity. This allowed for the testing of six realistic scenarios including: data theft, accessing sensitive files, security software tampering, unauthorised software installation, user behaviour and use of external contractors.

Evaluation of the presented model shows that all known threats were detectable from only 20% of the processed data after the entire dataset was analysed. A hybrid model that is explored produced highly accurate results when unrestricted and shows that using several algorithms is more accurate than a single algorithm in all test result. The supervised approach performed poorly in the project and suggestions are made for potential future work to improve and build upon the findings of this project. Due to limitations in the size of dataset available for testing, it is suggested that further testing is carried out to ensure accuracy is maintained. The implications of this approach could have a massive positive impact in industry when protecting against insider threats both pro-actively and re-actively.
[Read More]


Areas of Expertise

Associated Projects