ECCV 2026

Tricam-rPPG: A Multimodal Multispectral Dataset for Remote Photoplethysmography

Abhijit Sarkar^†,1,2,3, Surendrabikram Thapa¹, Ishtiaque Ahmed Khan³, Yogesh Deshpande², A. Lynn Abbott^†,2,3

¹Virginia Tech Transportation Institute ²Bradley Department of Electrical and Computer Engineering, Virginia Tech ³Department of Computer Science, Virginia Tech
^†Indicates Equal Advising

Paper Supplementary Material Video Data Access / DUL

participants

RGB + NIR

850 nm and 940 nm cameras

Aria

periocular video, IMU, gaze

BIOPAC

125 Hz fingertip PPG

Abstract

This paper introduces Tricam-rPPG, a multimodal dataset designed to support systematic studies of remote photoplethysmography (rPPG) using multispectral imaging. Remote, noncontact monitoring offers the potential for unobtrusive measurement of physiological signals related to health, cognitive load, and affect. However, most existing rPPG datasets rely primarily on RGB imaging, limiting the study of spectral effects, illumination variability, and sensing biases associated with differences in optical properties of the skin.

Tricam-rPPG provides synchronized recordings of 31 human subjects from three co-located cameras: a standard RGB camera and two near-infrared cameras operating at 850 nm and 940 nm, all captured under controlled illumination conditions. For fourteen participants, simultaneous recordings from Meta Aria glasses are also included. All cases are supplemented by reference blood volume pulse waveforms measured using a fingertip PPG sensor.

In addition to presenting the dataset, we establish baseline benchmarks with widely used rPPG algorithms to evaluate heart-rate estimation performance across different combinations of spectral channels. By providing synchronized RGB and NIR video together with physiological ground truth, Tricam-rPPG enables new research directions in multispectral physiological sensing, fairness-aware rPPG algorithms, and robust remote cardiovascular monitoring.

Video Overview

If the embedded player is unavailable, watch the video on YouTube.

Dataset Modalities

Raised hand sample frame — Face and hand recording

Meta Aria periocular NIR sample — Meta Aria periocular NIR

Collection Protocol

The acquisition system uses three co-located cameras mounted on rigid supports: one RGB camera and two monochrome NIR cameras at 850 nm and 940 nm. All camera streams are captured at 2046 x 2464 resolution and 60 fps under controlled illumination. Fingertip PPG is recorded with a BIOPAC MP46 at 125 Hz, and a subset of participants also wore Meta Aria glasses for periocular NIR video, IMU, and gaze data.

Recordings are organized into cool-light and warm-light sessions with tasks spanning resting baseline, mental arithmetic, calm and affective video viewing, side-view posture, raised-hand recording, and reading.

Tricam-rPPG recording setup — Controlled studio setup used for synchronized RGB, NIR, Aria, and PPG collection.

Sessions and Tasks

Tricam-rPPG captures physiological variation across controlled illumination, cognitive load, affective stimulation, pose, and body-region changes. Each recording segment lasts at least five minutes to support reliable heart-rate and HRV analysis.

Session	Lighting	Task	Description
S1	Cool, 8000 K	T1	Resting baseline with static fixation at a central cross.
S1	Cool, 8000 K	T2	Mental arithmetic for cognitive-load induction.
S1	Cool, 8000 K	T3	Calm video viewing to capture low-arousal responses.
S1	Cool, 8000 K	T4	Affective video viewing to elicit sadness.
S1	Cool, 8000 K	T5	Static side-view posture for profile-based facial analysis.
S1	Cool, 8000 K	T6	Resting baseline with raised hand beside the face for multi-region pulse analysis.
S2	Warm, 2700 K	T1	Reading task using a passage from Moby Dick.
S2	Warm, 2700 K	T2	Affective video viewing to elicit excitement.

What Tricam-rPPG Provides

RGB + NIR Sensing

Synchronized RGB, 850 nm, and 940 nm streams enable cross-spectral analysis and investigation of melanin-related attenuation in visible and near-infrared bands.

Reference Physiology

Every case includes fingertip PPG ground truth for heart-rate estimation, waveform recovery, and signal-quality analysis.

Viewpoint and Region Diversity

Frontal face, profile face, and raised-hand recordings support pose-robust rPPG and cross-region pulse recovery.

Metadata for Fast Experiments

The dataset includes facial landmarks, skin maps, PPG landmarks, patch intensity signals, and synchronized Aria data for a participant subset.

Benchmark and NIR Effectiveness

The paper evaluates widely used rPPG algorithms through the rPPG Toolbox and reports standard metrics including MAE, MAPE, correlation, and SNR. Experiments include aggregate performance, stratification by self-reported skin tone groups, cross-dataset generalization, and task-level analysis.

The multispectral experiments test channel combinations drawn from RGB, 850 nm NIR, and 940 nm NIR. Adding NIR channels improves signal quality and reduces heart-rate estimation error, with larger gains for higher-pigmentation skin-tone groups while preserving performance for lighter skin tones.

Five-channel RGB and NIR model schematic — Five-channel RGB + 850 nm + 940 nm architecture.

Three-channel RGB and NIR model schematic — Three-channel combinations used to evaluate multispectral rPPG retrieval.

Data Access

Tricam-rPPG is intended for research use under a data use license. The release includes synchronized camera recordings, PPG waveforms, Aria subset data, landmarks, skin maps, pulse landmarks, and patch intensity signals.

Because the dataset contains human-subject data, access is provided only after execution of a Data Use License (DUL). To request access, please email Abhijit Sarkar and Lynn Abbott. The DUL and access instructions will be sent to approved research requesters.

Email Abhijit Sarkar Email Lynn Abbott

Related Work From Our Group

Acknowledgements

We thank Ryan Mowri and Ryan Talbot from VTTI for helping with the hardware setup. We thank Gayatri Bhatambarekar, Joe Bekaronov, Jin Woo Baik, and Zeeshan Muhammad Karamat for additional help with data collection. Finally, we thank all the wonderful participants for taking part in the study.

Citation

@inproceedings{sarkar2026tricam,
  title     = {Tricam-rPPG: A Multimodal Multispectral Dataset for Remote Photoplethysmography},
  author    = {Sarkar, Abhijit and Thapa, Surendrabikram and Khan, Ishtiaque Ahmed and Deshpande, Yogesh and Abbott, A. Lynn},
  booktitle = {European Conference on Computer Vision},
  year      = {2026}
}