pipette
ENEnglish

Parameters efficient language model with latent cross-attention for IoT intrusion detection

Frederic Adjewa, Moez Esseghir, Leila Merghem-Boulahia, Cheikh Kacfah

PreprintUso en el mundo real

En palabras de los autores

Language models have shown strong potential for intrusion detection, but their high memory and computational demands make them difficult to deploy in resource-constrained IoT environments. In this work, we propose a compact Transformer-based architecture that compresses the input into a small latent space before processing it with a minimal Transformer core. We analyze nine compression configurations, compare parameter sharing with layer stacking, and evaluate our approach against SecurityBERT on two IoT intrusion detection datasets, Edge-IIoTset and CICIoT2023. With only 306k non-embedding parameters and 1.6M parameters in total (6MB), our model is about 7x smaller than SecurityBERT (11.2M parameters, 45MB). With the same privacy-preserving encoding, it outperforms SecurityBERT by about 2 points of macro F1-score on Edge-IIoTset and is within 0.7 points on CICIoT2023. It is also about 89x faster on a simulated edge device. We further show that a single Transformer layer is sufficient, and that parameter sharing matches layer stacking with up to 66% fewer parameters. These findings show that lightweight Transformer-based architectures can provide a practical and accurate solution for intrusion detection in resource-constrained IoT settings

Resultado principalEl resumen no menciona limitaciones.

Apareció: viernes, 23 de octubre. arXiv. Preprint, todavía sin revisión por pares.

Comentario de los autores: 14 pages