A Lightweight Hybrid Vision Transformer for Real-Time Surface Defect Detection in Industrial Imaging

Authors

  • Zhijuan Huang Author
  • Pengpeng Li Author
  • Hongping Song Author

Abstract

Vision transformers deliver strong global context modelling for surface defect detection, but their quadratic attention cost and deep stacks of large kernels keep them away from the production lines where defects must be localised within a few milliseconds. This paper presents LH-ViT, a lightweight hybrid vision transformer for real-time defect localisation. The backbone interleaves depthwise separable convolutions, which capture local texture cues at low cost, with windowed self-attention blocks whose shifted branches exchange information across windows; a channel-pruned feature-pyramid neck fuses multi-scale evidence, and a structured-pruning plus distillation recipe compresses the trained network for embedded deployment. The architecture is optimised under an explicit latency budget so that accuracy-critical blocks keep attention while texture-dominated stages remain purely convolutional. On three public industrial benchmarks (NEU-DET, GC10-DET and DAGM) LH-ViT reaches 79.4, 81.2 and 92.1 mAP, outperforming YOLOv8n by 6.8-11.2 points and full vision transformer baselines by a wider margin, while running at 63 FPS on an embedded Jetson Orin NX and using 4.1 million parameters, 38 percent fewer than the strongest compact baseline. An ablation shows that hybrid encoding contributes 3.9 mAP on average and that distillation recovers 86 percent of the accuracy lost to pruning. The results indicate that careful allocation between convolution and attention is the key to deploying transformer-quality detection on edge hardware.

Downloads

Published

2026-03-25

Issue

Section

Articles