Optimized Occlusion Handling in Human Detection by Fusion of Thermal and Depth Images for Mobile Robots

Abstract

In the domain of machine vision, surveillance systems serve as a security measure aimed at protecting public safety and properties. A key function of these systems is human detection. This paper introduces a human detection system that leverages thermal-depth information captured by a mobile robot in indoor settings. A novel fusion technique, termed Fusion of Thermal-Depth Information (FTDI), is proposed to enhance the segmentation process, ensuring robustness in various lighting conditions and improving processing speed. To address the challenge of occlusion, a new method known as the Occlusion Human Detector (OCHD) is introduced, which incorporates a pre-detector. This detector classifies occluded individuals using pixel codes derived from a candidate selection process. The results indicate that the proposed system achieves over 90% average accuracy across all datasets, outperforming state-of-the-art algorithms. Its innovative contribution is enhancing the classification of individuals and their occlusions. The proposed system is noted for being computationally efficient and maintaining high performance even in conditions of significant occlusion and low illumination.