Feature Restoration for Distributed Neural Network Inference in Edge Systems
DOI:
https://doi.org/10.31861/sisiot2026.1.01016Keywords:
distributed inference, edge computing, feature restoration, inpainting autoencoderAbstract
The growing volume of data generated by IoT devices has made deep neural networks an important tool in IoT and edge systems. The considerable computational and memory costs of deep neural networks made them difficult to run on constrained devices, which are limited by memory, computational power, and energy. However, IoT devices are usually connected to a network of other devices, which can help reduce the load on a single device and enable complex models to operate closer to the data source via distributed inference. Local networks used in such cases are usually wireless, so packet loss, interference, unstable bandwidth, and node inaccessibility are inherent to this system type. Such problems can lead to loss of the intermediate data representation during transmission and reduced model classification accuracy. This study suggests a lightweight restoration layer to reconstruct lost intermediate features during distributed neural network inference. The method defines feature loss as an inpainting problem. An autoencoder-based module reconstructs missing values using a binary mask indicating which components were received and which were lost and passes the results to the classifier. The method was tested by using a pretrained ResNet-18 model with frozen weights. The module was trained with error rates ranging from 0.0 to 0.5, in steps of 0.1, on a preprocessed 1000-class subset of the ImageNet-1k dataset. The Top-1 accuracy at the 50% feature-loss point, compared to a clean baseline, decreased from 69.73% to 52.84% without restoration, whereas the proposed layer increased it to 62.89%. Under the same circumstances, the Top-5 metric increased from 77.82% to 85.46% at the 50% feature-loss point, demonstrating a clear improvement. As shown by the results, the proposed layer exhibits a moderate reduction in accuracy under feature-loss conditions, compared to the larger drop observed in the model without feature restoration. The proposed approach can be used to improve model correctness by reducing losses in unstable wireless environments, which can help with implementing distributed inference in real-time systems with limited resources.
Downloads
References
I. Rodriguez-Conde, C. Campos, and F. Fdez-Riverola, “Horizontally Distributed Inference of Deep Neural Networks for AI-Enabled IoT,” Sensors, vol. 23, no. 4, Art. no. 1911, Feb. 2023, doi: 10.3390/s23041911.
K. M. Hou, X. Diao, H. Shi, H. Ding, H. Zhou, and C. De Vaulx, “Trends and Challenges in AIoT/IIoT/IoT Implementation,” Sensors, vol. 23, no. 11, Art. no. 5074, May 2023, doi: 10.3390/s23115074.
E. Baccour et al., “Pervasive AI for IoT Applications: A Survey on Resource-Efficient Distributed Artificial Intelligence,” IEEE Commun. Surv. Tutor., vol. 24, no. 4, pp. 2366–2418, 2022, doi: 10.1109/COMST.2022.3200740.
Y. Chen, T. Luo, W. Fang, and Neal. N. Xiong, “EdgeCI: Distributed Workload Assignment and Model Partitioning for CNN Inference on Edge Clusters,” ACM Trans. Internet Technol., vol. 24, no. 2, pp. 1–24, May 2024, doi: 10.1145/3656041.
F. Xue, W. Fang, W. Xu, Q. Wang, X. Ma, and Y. Ding, “EdgeLD: Locally Distributed Deep Learning Inference on Edge Device Clusters,” in 2020 IEEE 22nd International Conference on High Performance Computing and Communications; IEEE 18th International Conference on Smart City; IEEE 6th International Conference on Data Science and Systems (HPCC/SmartCity/DSS), Yanuca Island, Cuvu, Fiji: IEEE, Dec. 2020, pp. 613–619, doi: 10.1109/HPCC-SmartCity-DSS50907.2020.00078.
S. Itahara, T. Nishio, and K. Yamamoto, “Packet-Loss-Tolerant Split Inference for Delay-Sensitive Deep Learning in Lossy Wireless Networks,” in 2021 IEEE Global Communications Conference (GLOBECOM), Madrid, Spain: IEEE, Dec. 2021, pp. 1–6, doi: 10.1109/GLOBECOM46510.2021.9685179.
Y. Huang et al., “GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism,” 2018, arXiv. doi: 10.48550/arXiv.1811.06965.
F. Brakel, U. Odyurt, and A.-L. Varbanescu, “Model Parallelism on Distributed Infrastructure: A Literature Review from Theory to LLM Case-Studies,” 2024, arXiv. doi: 10.48550/arXiv.2403.03699.
Q. Yuan and Z. Li, “Distributed Inference Models and Algorithms for Heterogeneous Edge Systems Using Deep Learning,” Appl. Sci., vol. 15, no. 3, Art. no. 1097, Jan. 2025, doi: 10.3390/app15031097.
J. Du et al., “Model Parallelism Optimization for Distributed Inference via Decoupled CNN Structure,” IEEE Trans. Parallel Distrib. Syst., pp. 1–1, 2020, doi: 10.1109/TPDS.2020.3041474.
X. Guo, Q. Jiang, A. D. Pimentel, and T. Stefanov, “Model and system robustness in distributed CNN inference at the edge,” Integration, vol. 100, Art. no. 102299, Jan. 2025, doi: 10.1016/j.vlsi.2024.102299.
L. Gondara, “Medical Image Denoising Using Convolutional Denoising Autoencoders,” in 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW), Barcelona, Spain: IEEE, Dec. 2016, pp. 241–246. doi: 10.1109/ICDMW.2016.0041.
Y. Jiang, J. Xu, B. Yang, J. Xu, and J. Zhu, “Image Inpainting Based on Generative Adversarial Networks,” IEEE Access, vol. 8, pp. 22884–22892, 2020, doi: 10.1109/ACCESS.2020.2970169.
J. Deng, W. Dong, R. Socher, L.-J. Li, Kai Li, and Li Fei-Fei, “ImageNet: A large-scale hierarchical image database,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL: IEEE, Jun. 2009, pp. 248–255, doi: 10.1109/CVPR.2009.5206848.
O. Russakovsky et al., “ImageNet Large Scale Visual Recognition Challenge,” 2014, arXiv. doi: 10.48550/arXiv.1409.0575.
“resnet18 – Torchvision main documentation.” Accessed: May 5, 2026. [Online]. Available: https://docs.pytorch.org/vision/main/models/generated/torchvision.models.resnet18.html
Published
Issue
Section
License
Copyright (c) 2026 Security of Infocommunication Systems and Internet of Things
This work is licensed under a Creative Commons Attribution 4.0 International License.
