Efficient Malicious Encrypted Traffic Detection via Multi-Scale Convolution-Augmented Transformer: The NetFlowClassifier Approach

With the widespread adoption of encryption technologies, an increasing proportion of Internet traffic is encrypted, which fundamentally weakens traditional traffic inspection mechanisms. Existing port-based and payload-based detection methods have become ineffective, while approaches relying on handcrafted features suffer from limited discriminative capability and poor generalization. To address the problem of malicious encrypted traffic identification, this paper proposes a lightweight hybrid model named NetFlowClassifier, together with a targeted data preprocessing pipeline. An encrypted traffic dataset is first constructed from CSE-CIC-IDS2018 for binary classification of benign and malicious flows. The proposed model integrates multi-scale depthwise separable convolution and an improved Transformer encoder to jointly capture local discriminative patterns and long-range dependencies. In addition, learnable feature position encoding and attention-weighted pooling are introduced to enhance feature representation. Experimental results show that NetFlowClassifier achieves an F1-score of 0.9661, outperforming baseline CNN and Transformer models by 1.8%–3.2%. With 11.2 million parameters and a throughput of 128 samples per second, the model effectively balances detection accuracy and computational efficiency. These results demonstrate the effectiveness of the proposed approach for malicious encrypted traffic detection in modern network environments.