Edge AI-Based Multimodal Sensing for Real-Time Anomaly Detection in Intelligent Building Environments

Authors

  • Rochelle Hogg Department of Computer Science, University of New Hampshire, Durham, NH, USA. Author
  • Kiandra Yolanda School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, OR, USA. Author

Keywords:

edge AI; multimodal fusion; anomaly detection; intelligent buildings; real-time sensing; uncertainty calibration

Abstract

Intelligent buildings generate heterogeneous measurements from environmental sensors, power meters, occupancy detectors, cameras, acoustic devices, and building automation systems. The central problem is not simply detecting isolated outliers, but recognizing context-dependent and collective anomalies while respecting tight latency, energy, memory, privacy, and reliability constraints at the edge. This manuscript develops a methodological framework for edge AI-based multimodal sensing and real-time anomaly detection in intelligent building environments. The contribution is an integrated design-and-analysis model that connects sensing, synchronization, modality-specific feature extraction, uncertainty-aware fusion, anomaly scoring, risk-calibrated decision logic, and constrained deployment. The framework combines clock-aware preprocessing with temporal encoders, an adaptive gated fusion rule, a hybrid generative-predictive anomaly score, hysteresis-based alarm decisions, and a resource-constrained placement policy across sensors, gateways, and embedded accelerators. The analysis explains why static concatenation is fragile under missing modalities and sampling-rate mismatch, why asymmetric false-alarm and missed-fault costs require calibrated thresholds rather than fixed percentile cutoffs, and why occupancy and weather drift demand controlled adaptation. Boundary conditions include sensor drift, clock skew, label scarcity, model overconfidence, and privacy-sensitive visual data. The resulting framework provides a technically defensible basis for designing and evaluating edge-resident building anomaly detectors before field deployment.

References

1. Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM Computing Surveys, 41(3), Article 15. https://doi.org/10.1145/1541880.1541882

2. Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.

3. Shi, W., Cao, J., Zhang, Q., Li, Y., & Xu, L. (2016). Edge computing: Vision and challenges. IEEE Internet of Things Journal, 3(5), 637–646. https://doi.org/10.1109/JIOT.2016.2579198

4. Satyanarayanan, M. (2017). The emergence of edge computing. Computer, 50(1), 30–39. https://doi.org/10.1109/MC.2017.9

5. Baltrusaitis, T., Ahuja, C., & Morency, L.-P. (2019). Multimodal machine learning: A survey and taxonomy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(2), 423–443. https://doi.org/10.1109/TPAMI.2018.2798607

6. Atrey, P. K., Hossain, M. A., El Saddik, A., & Kankanhalli, M. S. (2010). Multimodal fusion for multimedia analysis: A survey. Multimedia Systems, 16(6), 345–379. https://doi.org/10.1007/s00530-010-0182-0

7. Kingma, D. P., & Welling, M. (2014). Auto-encoding variational Bayes. International Conference on Learning Representations. https://arxiv.org/abs/1312.6114

8. Ruff, L., Kauffmann, J. R., Vandermeulen, R. A., Montavon, G., Samek, W., Kloft, M., Dietterich, T. G., & Müller, K.-R. (2021). A unifying review of deep and shallow anomaly detection. Proceedings of the IEEE, 109(5), 756–795. https://doi.org/10.1109/JPROC.2021.3052449

9. Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

10. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (Vol. 30, pp. 5998–6008).

11. Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., & Bouchachia, A. (2014). A survey on concept drift adaptation. ACM Computing Surveys, 46(4), Article 44. https://doi.org/10.1145/2523813

12. Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., & Zhang, J. (2019). Edge intelligence: Paving the last mile of artificial intelligence with edge computing. Proceedings of the IEEE, 107(8), 1738–1762. https://doi.org/10.1109/JPROC.2019.2918951

13. Chou, J.-S., & Telaga, A. S. (2014). Real-time detection of anomalous power consumption. Renewable and Sustainable Energy Reviews, 33, 400–411. https://doi.org/10.1016/j.rser.2014.01.088

14. Liu, J., Luo, D., Fu, X., Lu, Q., & Kang, K. Y. (2022). Design strategy of multimodal perception system for smart environment. In Internet of Things for Smart Environments (pp. 93-115). Cham: Springer International Publishing.

15. Bonomi, F., Milito, R., Zhu, J., & Addepalli, S. (2012). Fog computing and its role in the Internet of Things. In Proceedings of the First MCC Workshop on Mobile Cloud Computing (pp. 13–16). ACM. https://doi.org/10.1145/2342509.2342513

16. Hong, T., Wang, Z., Luo, X., & Zhang, W. (2020). State-of-the-art on research and applications of machine learning in the building life cycle. Energy and Buildings, 212, 109831. https://doi.org/10.1016/j.enbuild.2020.109831

17. Kleiminger, W., Beckel, C., & Staake, T. (2014). Occupancy detection from electricity consumption data. In Proceedings of the 5th ACM Workshop on Embedded Systems for Energy-Efficient Buildings (pp. 1–8). ACM.

18. Wang, S., & Ma, Z. (2008). Supervisory and optimal control of building HVAC systems: A review. HVAC&R Research, 14(1), 3–32. https://doi.org/10.1080/10789669.2008.10390991

19. Alsheikh, M. A., Lin, S., Niyato, D., & Tan, H.-P. (2014). Machine learning in wireless sensor networks: Algorithms, strategies, and applications. IEEE Communications Surveys & Tutorials, 16(4), 1996–2018. https://doi.org/10.1109/COMST.2014.2320099

Downloads

Published

2026-10-22