CABiNet : Efficient Context Aggregation Network for Low-Latency Semantic Segmentation

Real-time semantic segmentation is a challenging task as the optimal balance between accuracy and efficiency (computational complexity, memory footprint and execution speed) is hard to achieve. Conventional lightweight and real-time semantic segmentation architectures usually address only one of the above perspectives, thereby making high-accuracy designs computationally expensive and high-speed models relatively inaccurate. In this research, we introduce an approach to semantic segmentation (for images), which successfully reduces the computational costs by almost 88% and increases the execution speed by a factor of 1.5 compared to the current state-of-the-art, while maintaining a comparable mean intersection-over- union score. Building upon the existing multi-branch architectures for high-speed segmentation, we design a cheap high resolution branch for effective spatial detailing and a context branch with compact asymmetric position and local attention (collectively termed as Context Aggregation Block), potent enough to capture both long-range and local contextual dependencies required for accurate semantic segmentation, at low computational costs. Specifically, we achieve 76.6% and 75.8% mIOU on Cityscapes validation and test sets respectively, at 76 FPS on a single NVIDIA RTX 2080Ti and 8 FPS on a Jetson Xavier NX. Our superior context aggregation techniques also outperform the current state-of-the-art on another public benchmark, UAVid dataset, by a significant margin of 14%. Codes and pre-trained models shall be made available at https://github.com/dronefreak/CABiNet.