Human Detection Combining RGB and Thermal Modalities

Chaymaa Fakrane, Kennedy O. S. Mota, Cristiano Premebida and Rui P. Rocha

In Proc. of 2024 IEEE 3rd International Conference on Intelligent Reality (ICIR), Coimbra, Portugal, Dec. 5-6, 2024. DOI: 10.1109/ICIR64558.2024.10976913

Abstract

This paper addresses a late-fusion approach that combines RGB and thermal (long-wave IR) modalities to enhance human detection in indoor environments for robotic perception domains. The study explores multimodal human detection using YOLO series (v5s-6s and v8s-11s models), demonstrating significant improvements in detection accuracy under challenging lighting conditions on a dataset collected by a mobile robot. The use of a Weighted-Mean (WM) as a late-fusion approach demonstrates favorable results over more complex methods such as neural networks or SVM. The WM approach is simple to implement and has a low computational cost. Additionally, it is more interpretable and modular and allows straightforward adjustments to the weights of the YOLO models and thresholds, facilitating adaptation to the target system's specific needs. The experimental results indicate that the implemented late-fusion approach improves the baseline performance in terms of mean average precision.

Index Terms — Thermal camera; RGB camera; late fusion; human detection; YOLO.

Full text

You may ask Rui Rocha for an electronic copy of this publication’s full text by e-mail:
.

Please select for your message’s subject ‘Requesting Rui Rocha’s electronic copy’ and include on the message’s body your full name, title and affiliation, why do you need to access the publication and the BibTeX information below.

BibTeX

@inproceedings{fakrane_et_al_2024,

author = {Fakrane, C. and Mota, K. O. S. and Premebida, C. and Rocha, R. P.},

title = {Human detection combining RGB and thermal modalities},

booktitle = {Proceedings of the 2024 IEEE 3rd International Conference on Intelligent Reality (ICIR)},

year = {2024},

month = {Dec.},

pages = {1--2},

address = {Coimbra, Portugal},

doi = {10.1109/ICIR64558.2024.10976913}

}

(top of the page)

Last update: 28/07/2025