针对日志序列数据存在不稳定性和数据间相互依赖等问题,提出了一种基于改进变分自编码器生成对抗网络（VAE-GAN）的无监督日志异常检测方法,所提出的模型结合了GAN和VAE的优势,通过将时间卷积网络模块嵌入编码器、解码器和判别器中,有效捕获日志序列数据的分布并优化潜在空间中的序列映射,从而实现高精度的正常日志序列重建。模型通过对抗训练机制不断提升变分自编码器的重建能力,使其能够更准确地识别日志中的异常模式。实验结果表明,与其他无监督方法相比,该方法在公开日志数据集上具有更好的性能。
Abstract
This paper proposes an unsupervised log anomaly detection method based on an improved variational autoencoder generative adversarial network (VAE-GAN) to address the issues of instability and interdependence in log sequence data. The proposed model combines the advantages of GAN and VAE by embedding the temporal convolutional network module into the encoder, decoder, and discriminator, effectively capturing the distribution of log sequence data and optimizing the sequence mapping in the latent space, thereby achieving high-precision reconstruction of normal log sequences. The model continuously improves the reconstruction ability of the variational autoencoder through adversarial training mechanism, enabling it to identify abnormal patterns in the log more accurately. The experimental results show that compared with other unsupervised methods, this method has better performance on public log datasets.
关键词
日志异常检测 /
生成对抗网络 /
变分自编码器 /
时间卷积网络
Key words
log anomaly detection /
generate adversarial networks /
variational autoencoder /
time convolutional network
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] 廖湘科,李姗姗,董威,等.大规模软件系统日志研究综述[J].软件学报,2016,27(8):1934-1947.
[2] 张颖君,刘尚奇,杨牧,等.基于日志的异常检测技术综述[J].网络与信息安全学报,2020,6(6):1-12.
[3] Li X,Chen P,Jing L,et al.Swisslog:Robust and unified deep learning based log anomaly de-tection for diverse faults[C]//2020 IEEE 31st International Symposium on Software Reliability Engineering,2020.
[4] ZHANG X,XU Y,LIN Q,et al.Robust log-based anomaly detection on unstable log data[C]//Proceedings of the2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering.2019:807-17.10.1145/3338906.3338931.
[5] DU M,LI F,ZHENG G,et al.DeepLog[C]//Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security.2017:1285-98.
[6] Yadav R B,Kumar P S,Dhavale S V.A Survey on Log Anomaly Detection using Deep Learning[C]//Proceedings of 8th International Conference on Reliability,Infocom Technologies and Optimization(ICRITO).IEEE,2020:1215-1220.
[7] 尹春勇,孔娴.基于双向时间卷积网络的半监督日志异常检测[J].计算机应用研究,2024,41(7):2110-2117.
[8] Niu Z,Yu K,Wu X.LSTM-Based VAE-GAN for Time-Series Anomaly Detection[J].Sensors,2020,20(13):3738.
[9] Gong X,Liao S,Hu F,et al.Autoencoder-Based Anomaly Detection for Time Series Data in Complex Systems[C]//2022 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS),Shenzhen,China,2022:428-433.
[10] Kulyadi S P,Mohandas P,Kumar S K S,et al.Anomaly Detection using Generative Adversarial Networks on Firewall Log Message Data[C]//2021 13th International Conference on Electronics,Computers and Artificial Intelligence (ECAI),Pitesti,Romania,2021:1-6.
[11] Lee C K,Cheon Y J,Hwang W Y.Studies on the GAN-Based Anomaly Detection Methods for the Time Series Data[C]//in IEEE Access,2021(9):73201-73215.
[12] 尹春勇,张杨春.基于CNN和Bi-LSTM的无监督日志异常检测模型[J].计算机应用,2023,43(11):3510-3516.
[13] He P,Zhu J,Zheng Z,et al.Drain:An Online Log Parsing Approach with Fixed Depth Tree[C]//2017 IEEE International Conference on Web Services(ICWS).Honolulu,HI,USA,2017:33-40.
[14] OULIN A,GRAVE E,BOJANOWSKI P,et al.Fasttext.zip:Compressing text classification models[J].arXiv preprint arXiv:161203651,2016.
[15] Yadav R B,Kumar P S,Dhavale S V.A Survey on Log Anomaly Detection using Deep Learning[C]//Proceedings of 8th International Conference on Reliability,Infocom Technologies and Optimization(ICRITO).IEEE,2020:1215-1220.
[16] Lin Q,Zhang H,Lou J G.Log Clustering Based Problem Identification for Online Service Systems[C]//Proc of The 38th IEEE/ACM International Conference on Software Engineering Companion,2016:102-111.
[17] Guo H,Yuan S,Wu X.LogBERT:Log Anomaly Detection via BERT[C]//International Joint Conference on Neural Networks(IJCNN),Shenzhen,China,2021:1-8.
[18] Lin S,Clark R,Birke R.Anomaly Detection for Time Series Using VAE-LSTM Hybrid Model[C]//ICASSP 2020 -2020 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP), Barcelona,Spain,2020:4322-4326.
[19] Xia B,Yin J,Xu J,et al.LogGAN:A Sequence-Based Generative Adversarial Network for Anomaly Detection Based on System Logs.In:Liu F,Xu J,Xu S,Yung M.(eds) Science of Cyber Security.SciSec2019.Lecture Notes in Computer Science,vol 11933.Springer,Cham.
基金
黑龙江省自然科学基金,项目编号：LH2022F008
