Detection of Encryption using Raw Data as input to an Artificial Neural Network
Levick, D. (2017). Detection of Encryption using Raw Data as input to an Artificial Neural Network (MSc ASDF Dissertation). Edinburgh Napier University (Macfarlane, R.,
Detection of encryption is necessary for computer system security and forensics, and a variety of statistical pre-processing methods combined with machine learning (ML) have been applied to the detection task. However, the pre-processing limits the learning of the ML to the features picked out by the system designer and any other information potentially contained in the raw data is lost. There has been little research into the simple approach of supplying raw data as input to an ML model, and some research suggests that theoretically this approach will not work. No practical experimentation using the raw data approach has been attempted and this suggests a gap in the literature. Artificial neural networks which process raw information have been successfully used in many applications such as image recognition, speech recognition and general pattern matching. Also, with the development and availability of modern machine learning software libraries, it can be hypothesised that use of a state-of-the-art neural network model could find features in raw encrypted data that could allow classification. Therefore, this research was undertaken in order to evaluate the effectiveness of using a deep learning solution applied to detection of raw encrypted data. A neural network is implemented using Google’s TensorFlow machine learning library, and applied to encrypted data detection. It is discovered that some accuracy in data classification (95.0% to 98.9%) is achievable with this approach when data is dissimilar or header information is available. However, accuracy is much lower (65.5%) when attempting to classify fragments of similar data with no header information, especially with high entropy data formats. It is shown that extra information beyond the raw data is likely to be needed to reliably detect encryption in the most challenging circumstances, however the neural net with raw data input is effective in some scenarios and this method may be of use in certain applications or to stimulate further research.
Areas of Expertise
See all areas of expertise
Electronic information now plays a vital role in almost every aspect of our daily lives. So the need for a secure and trustworthy online infrastructure is more important than ever. without it, not only the growth of the internet but our personal interactions and the economy itself could be at risk.