pipette
ESEspañol

Browser-Based Video Supports Scalable Computer Vision Analysis for Open Surgical Skill Assessment

A. Khan, S. Kulkarni, S.-L. Lai, X. Wang, I. Farber, G. Kugener, A. Pal, S. Volpi, O. Zohar, C.-S. Chen, S. Yeung-Levy, G. Brat

PreprintReal-world use

In the authors' words

Purpose: Objective assessment of open surgical skills remains limited by expert availability. Machine learning (ML)-based hand keypoint detection offers potential for automated, on-demand assessment, but performance in accessible browser-based recording environments (laptop-webcam video recorded in a web browser) is unknown. We hypothesized that browser-based recordings would achieve acceptable keypoint localization (percentage of correct keypoints [PCK] >70%). Methods: Retrospective technical validation of prospectively collected video from 14 PGY-1 interns at one academic center (July to August 2024). We annotated 203 frames (101 browser-based, 1280x720/854x480; 102 overhead, 3840x2160/1920x1080), yielding 6,232 ground-truth keypoints. An internally developed pipeline and MediaPipe Hands were evaluated. The primary endpoint was PCK normalized to the index-proximal bone; bounding-box-normalized PCK, detection rate, and per-region accuracy were also evaluated. Results: Primary PCK, normalized to a fixed anatomical hand scale, was 85.9% (95% CI, 80.3-91.0) for browser-based recordings and 81.5% (95% CI, 76.7-87.0) for overhead recordings. Secondary bounding-box-normalized PCK showed a similar pattern (91.6% vs 90.6%), and detection rates were high in both environments (94.9% vs 97.7%). In contrast, MediaPipe detection rates were only 5.3% for browser-based recordings and 15.4% for overhead recordings, resulting in substantially lower accuracy. Conclusion: Hand keypoint detection performance was comparable between browser-based and overhead recordings using the internally developed pipeline. These findings suggest that browser-based video acquisition can provide data of sufficient quality for ML-based hand keypoint detection, offering a scalable pathway for surgical skill assessment without specialized recording hardware.

Main resultThe abstract does not state a limitation.

Appeared: Wednesday, November 11. medRxiv. Preprint, not yet peer-reviewed.

DOI: 10.64898/2026.11.09.26367680