Medical Image Processing, Projects, Research Area

A Deep Learning Framework for Exercise-Based Rehabilitation Assessment via Markerless Motion Analysis using OpenCap

System Simulation
1. Introduction

In clinical practice, joint Range of Motion (ROM) and the quality of rehabilitation movements are traditionally assessed using manual goniometers or the visual estimation of clinicians. This approach is not only time-consuming and subjective, but it also lacks the quantitative framework necessary for tracking longitudinal recovery progress across multiple sessions.

While marker-based motion capture systems in biomechanics laboratories offer high precision, their prohibitive costs hinder widespread clinical deployment. Free tools such as Kinovea still require manual, frame-by-frame angle measurements and lack built-in mechanisms for continuous progress tracking.

Recently, markerless motion analysis frameworks such as MediaPipe, OpenPose, and notably OpenCap, have enabled the extraction of 3D kinematics from standard smartphone videos. However, integrating these tools into an automated, clinically intuitive system for healthcare professionals remains a significant research gap.

2. Project Objectives

  • Develop a markerless motion analysis system utilizing standard RGB cameras based on the OpenCap platform.
  • Create a robust tool for tracking rehabilitation progress on a session-by-session basis.
  • Compute quantitative clinical metrics, including ROM, Asymmetry Index, and Recovery Score.
  • Automatically generate visual reports and dashboards to illustrate recovery trajectories.
  • Establish the foundation for an AI-driven system capable of recommending exercise adjustments or providing early clinical warnings.

3. System Pipeline

The overarching workflow of the system comprises the following core stages:

  1. Data Acquisition: Utilize two tripod-mounted iOS smartphones and the OpenCap web application to record exercise sessions (e.g., squats, gait).
  2. Markerless Processing & 3D Kinematics: Employ computer vision and musculoskeletal simulation to estimate joint trajectories, forces, and moments.
  3. Feature Extraction: Perform data normalization, noise filtering, and the computation of ROM and symmetry indices.
  4. Deep Learning Assessment Model: Implement a Transformer-based architecture to:
    • Classify natural versus abnormal movement patterns.
    • Regress the clinical Recovery Score.
    • Forecast improvement trends.
  5. Dashboard & Reporting: Render intuitive graphical visualizations to support clinical decision-making.

System Pipeline

Figure 2: Data processing pipeline of the system.

4. Significance and Potential

This system is expected to standardize rehabilitation assessment towards a quantitative approach, mitigating the reliance on expensive laboratory equipment. In the long term, it will serve as a foundational Digital Assistant, empowering both clinicians and patients to optimize their rehabilitation regimens.

One thought on “A Deep Learning Framework for Exercise-Based Rehabilitation Assessment via Markerless Motion Analysis using OpenCap”

  1. Pingback: DO NGUYEN ANH DUC - iBME Lab

Comments are closed.