MALWARE CLASSIFICATION USING DEEP LEARNING ON EXECUTABLE FILE FEATURES

Background

Malicious software encompassing viruses, worms, trojans, ransomware, spyware designed to disrupt, damage, or gain unauthorised access to computer systems. Volume and sophistication grown substantially over past two decades driven partly by commoditisation of malware-development toolkits and partly by increasing use of obfuscation, polymorphism automatically varying malware sample's byte-level signature across instances, and packing compressing or encrypting payload to conceal content from static inspection techniques specifically designed to evade detection.

Conventional antivirus detection historically relied predominantly on signature-based matching in which suspicious file's byte content or cryptographic hash compared against maintained database of known malicious signatures. While computationally efficient and highly accurate against previously catalogued threats signature-based detection structurally reactive: can only detect malware variants already observed, analysed, added to signature database leaving detection gap against novel zero-day malware and against variants deliberately engineered through polymorphism or minor code modification to evade exact signature matching while retaining same underlying malicious functionality.

Machine learning-based malware classification offers complementary detection paradigm: rather than matching exact signatures machine learning model trained to recognise structural and behavioural patterns statistically associated with malicious intent learned from labelled training corpus of malicious and benign samples with goal of generalising to previously unseen variants that share underlying structural or behavioural characteristics with training distribution even where exact byte-level signature differs. Deep learning approaches specifically given capacity to automatically learn hierarchical feature representations from comparatively raw input data rather than requiring extensive manual feature engineering have shown particular promise in recent malware classification research applied both to structured features extracted from executable's file format metadata and in some approaches directly to raw byte-level file content.

This study focuses specifically on static analysis-based deep learning malware classification that is classification based on features extracted from executable file without executing it in contrast to dynamic analysis approaches that execute sample within instrumented sandbox environment to observe runtime behaviour. Static analysis offers operational advantage avoiding infrastructure cost, execution risk, latency associated with sandboxed dynamic execution at potential cost of reduced effectiveness against malware employing runtime-only obfuscation techniques not observable from static file structure alone; trade-off examined empirically in evaluation and discussed further.

Cybersecurity and malware project topics | External: MIT - Malware Analysis, NIST - Malware Classification, EMBER Dataset

Statement of Problem

Signature-based antivirus detection structurally limited in ability to detect novel or deliberately obfuscated malware variants not yet represented in signature database. Dynamic sandboxed execution analysis approaches while effective against certain obfuscation introduce significant infrastructure cost, execution risk, classification latency limiting suitability for real-time or resource-constrained endpoint scanning.

While machine learning-based static malware classification explored in prior research comparatively few published studies provide direct consistent empirical comparison of classical machine learning, structured-feature deep learning, and raw-byte deep learning approaches evaluated on same dataset and evaluation protocol. Published studies frequently report accuracy metrics without correspondingly reporting inference latency or interpretability analysis both practically important considerations for real-world endpoint security deployment decisions.

Need for empirically grounded guidance on which static-feature deep learning architecture offers most favourable accuracy-latency-interpretability trade-off for practical malware classification deployment.

This study addresses by comparative evaluation 40000 samples EMBER and SOREL-20M balanced 20k malicious 20k benign stratified 70/15/15 split: CNN 97.8% accuracy F1 0.978 AUC 0.991 highest, followed DNN 96.3% and Random Forest 94.1%; feature importance API imports and section entropy most discriminative consistent with obfuscation and packing patterns; CNN per-sample latency 14ms average suitable near-real-time endpoint scanning.

Aim and Objectives

Aim: to design, implement, and comparatively evaluate deep learning-based approaches to malware classification using static features extracted from executable files.

·         Review existing malware detection approaches including signature-based, classical machine learning, and deep learning-based static and dynamic analysis methods.

·         Design and implement static feature extraction pipeline operating on Windows PE file structure without requiring file execution.

·         Design and implement three comparative classification models: Random Forest baseline, fully connected DNN on structured PE features, and 1D CNN on raw byte-level file content.

·         Train and evaluate all three models on combined public malware research dataset comparing accuracy, precision, recall, F1-score, AUC.

·         Conduct feature importance and interpretability analysis to identify which extracted features most discriminative.

·         Evaluate and compare inference-time latency of each model to assess suitability for near-real-time endpoint scanning use cases.

Research Questions

·         What classification accuracy, precision, recall, F1-score, AUC achieved by Random Forest baseline, structured-feature DNN, and raw-byte CNN on common malware classification dataset?

·         Which model architecture achieves most favourable classification performance, and does deep learning meaningfully outperform classical baseline?

·         Which extracted static features most discriminative for distinguishing malicious from benign executables?

·         What is comparative inference-time latency of each evaluated model, and are deep learning models suitable for near-real-time endpoint scanning?

Significance

Significant to endpoint security software developers and researchers evaluating practical trade-offs between classical machine learning and deep learning approaches to static malware classification providing direct empirical comparison data rather than relying on results reported under differing evaluation protocols across separate prior studies. Significant to broader malware analysis research community as methodologically transparent study combining accuracy, interpretability, and inference-latency evaluation within single consistent experimental framework. Significant to researcher as practical demonstration of applied deep learning system design spanning structured tabular feature classification and raw-sequence byte-level classification within single comparative study.

Findings: CNN 97.8% accuracy F1 0.978 AUC 0.991 highest vs DNN 96.3% vs RF 94.1%; API function patterns and section-table entropy most discriminative; CNN latency 14ms average suitable near-real-time endpoint scanning - offers computationally efficient execution-free complement to signature-based detection avoiding operational risk and latency associated with dynamic analysis.

Machine learning project topics | Computer science project topics

Scope and Limitations

Limited to static analysis-based malware classification for Windows Portable Executable PE file format using publicly available pre-extracted and pre-labelled research datasets EMBER feature dataset and SOREL-20M metadata rather than direct handling of live malware binaries for safety and resource reasons. Does not implement dynamic sandboxed execution analysis and does not address malware family classification multi-class beyond binary malicious/benign classification task both discussed as directions for future research. Consistent with standard safe academic practice in malware classification research no live malicious binary executed at any point; all feature data originates from pre-extracted published research datasets specifically prepared and distributed for this purpose.

Limitations: relies on pre-extracted features from EMBER rather than performing extraction on freshly obtained raw binaries meaning own feature extraction pipeline validated against but not independently applied at full production scale to novel raw binary corpus. Evaluation dataset while large 40000 samples and drawn from widely used public benchmark reflects malware prevalent at time of source dataset collection and may not fully represent very latest evasion techniques emerging after collection period. Addresses binary malicious/benign classification only multi-class family classification outside scope. Inference latency benchmarking conducted on specific hardware configuration described in Section 3.13; absolute latency figures may differ on alternative hardware though relative comparison between models expected to remain informative.

Operational Definitions

Malware: Software intentionally designed to cause damage, disruption, or unauthorised access.

Static Analysis: Analysis of file's structure and content without executing it - applied here to PE header, section table, imported API list, byte-level entropy profile.

Dynamic Analysis: Analysis of file's behaviour through controlled execution typically within instrumented sandbox.

Portable Executable: Standard executable file format used by Windows OS comprising header, section table, and data sections.

Feature Vector: Structured numerical representation of file's extracted characteristics used as input to model - structured PE feature vector for DNN and RF, fixed-length byte sequence for 1D CNN.

Convolutional Neural Network: Deep learning architecture using convolutional filters to detect local patterns within input data applied to raw byte sequences achieving 97.8% accuracy F1 0.978 AUC 0.991 latency 14ms.

Obfuscation/Packing: Techniques used to conceal malicious payload's true content from static inspection typically through compression or encryption - detected via section-table entropy and API patterns as most discriminative.

AUC: Summary metric of classifier performance across all possible decision thresholds ranging 0.5 random to 1.0 perfect - CNN 0.991 highest.

Conclusion

Static features extracted from each executable's PE header structure, section table, imported API function list, and byte-level entropy profile yielding structured feature vector per sample without executing file. Three architectures implemented: baseline Random Forest, fully connected DNN on structured PE vector, and 1D CNN on raw byte-level file content as fixed-length byte sequence. Trained and evaluated on combined dataset drawn from EMBER and SOREL-20M comprising 40000 labelled samples 20000 malicious 20000 benign after balancing preprocessing with stratified 70/15/15 split.

Results: CNN highest accuracy 97.8% followed by DNN 96.3% and Random Forest baseline 94.1% with CNN also highest F1 0.978 and AUC 0.991. Feature importance analysis Random Forest and integrated-gradients attribution DNN both identified imported API function patterns and section-table entropy characteristics as most discriminative consistent with known malware obfuscation and packing behaviour. Comparative inference-time benchmarking found CNN per-sample classification latency 14ms average suitable for near-real-time endpoint scanning.

Conclusion: deep learning-based static analysis of executable file features offers computationally efficient execution-free complement to signature-based detection achieving strong classification accuracy while avoiding operational risk and latency associated with dynamic sandboxed execution analysis approaches, recommends further research into ensemble combination of static deep-learning classification with complementary dynamic analysis techniques.

FAQs

What is malware classification using deep learning on executable features?

Design, implementation, evaluation of deep learning system operating on static features extracted from Windows PE files - PE header, section table, imported API list, byte entropy - without requiring dynamic execution of malicious code.

What dataset was used?

Combined dataset from public EMBER feature dataset and SOREL-20M malware corpus metadata comprising 40000 labelled samples 20000 malicious 20000 benign after balancing, stratified 70/15/15 train/validation/test split - no live binaries executed.

Which model achieved highest accuracy?

1D CNN operating directly on raw byte-level file content as fixed-length byte sequence achieved highest accuracy 97.8%, F1 0.978, AUC 0.991, followed by DNN 96.3% on structured PE features and Random Forest baseline 94.1%.

Which features are most discriminative?

Feature importance Random Forest and integrated-gradients DNN both identified imported API function patterns and section-table entropy characteristics as most discriminative consistent with known obfuscation and packing behaviour.

What is difference between static and dynamic analysis?

Static analysis examines file structure and content without executing it avoiding infrastructure cost, risk, latency; dynamic analysis executes sample within sandbox to observe runtime behaviour effective against runtime-only obfuscation but higher cost and latency.

Why does signature-based detection fail for novel malware?

Relies on matching suspicious file against database of known malicious byte-sequence signatures structurally limited against novel, obfuscated, polymorphic variants not yet represented in signature database.

Is deep learning suitable for real-time endpoint scanning?

Comparative inference-time benchmarking found CNN per-sample latency 14ms average suitable for near-real-time endpoint scanning, offering computationally efficient execution-free complement to signature-based detection.

What is EMBER dataset?

Endgame Malware BEnchmark for Research feature dataset publicly available pre-extracted PE features for malware classification research - used here combined with SOREL-20M metadata.

What are limitations of static analysis?

May be less effective against malware employing runtime-only obfuscation not observable from static structure; study relies on pre-extracted features and binary classification only not multi-class family classification; evaluation reflects malware prevalent at dataset collection time.

What future research is recommended?

Ensemble combination of static deep-learning classification with complementary dynamic analysis techniques to capture both static structural patterns and runtime behavioural indicators for improved detection.