https://www.nedo.go.jp/news/press/AA5_101824.html
Remote rehabilitation projects using online video calls and non-immersive VR are increasing, and XR rehabilitation is also progressing overseas, but there are not many international cases where remote and XR have been combined. The reason cited for this is that the user experience, such as wearing XR equipment such as head-mounted displays, using rehabilitation apps, and participating in the metaverse, does not fully meet the value expected by users. In addition, remoteness has weakened the presence of trainers, doctors, nurses, etc., and made it difficult to precisely assess the physical and mental functions of rehabilitation service users (including patients) with actual illnesses, making it difficult to motivate them to continue exercise training, which was a major issue.
Against this background, AIST, Kyoto University, the University of Tokyo, Seiko Epson, and Everyha have been working on the construction of a multisensory XR-AI technology platform for remote rehabilitation and mutually beneficial care.
A MR3 wear was developed which incorporates a group of highly sensitive, low-hysteresis strain sensors and a hanger reflex device for the purpose of assessing the user’s movement and presenting a sense of force, and the application of the hanger reflex to remote upper limb rehabilitation and grasping scapular movement with a wearable device. In addition, an AI for motion evaluation was developed that inputs measurement data obtained from the group of strain sensors in order to quantify the magnitude of the scapular movement and estimate the angles of each joint in the upper limb. Furthermore, a hand redirection was developed, which is attracting attention as a means of increasing self-efficacy and contributing to motivation, to upper limb rehabilitation, and the evaluation of the effect of remote reciprocal care in improving the continuity of exercise training.
By refining the measurement and understanding of the quality and quantity of the user’s movement, it is possible to realize a precise assessment of the mental and physical functions of remote rehabilitation service users, which can contribute to motivating them to continue their exercise training. To achieve this, it is necessary to lower the threshold for technological development, performance evaluation, and practical application. Therefore, the world’s first open dataset of motion capture data on upper limb and scapular movement was developed, and provided to companies and organizations in Japan that can submit experimental plans that have been reviewed and approved based on the “Ethical Guidelines for Life Science and Medical Research Involving Human Subjects”.
The dataset includes 18 types of upper limb and scapular movement
- Shoulder flexion and extension (up to 90 degrees)
- Shoulder flexion and extension (up to maximum range of motion)
- Shoulder abduction and adduction (up to 90 degrees)
- Shoulder abduction and adduction (up to full range of motion)
- Shoulder horizontal abduction and adduction
- Shoulder external and internal rotation (1st position)
- Shoulder external and internal rotation (2nd position)
- Shoulder external and internal rotation (3rd position)
- Elbow flexion and extension
- Forearm pronation and supination (elbow flexed 90 degrees)
- Forearm pronation and supination (elbow extended)
- Reaching (inner)
- Reaching (forward)
- Reaching (outer)
- Moving the hand from the knee to the side of the ear
- Touching the back of the waist
- Touching the back of the head
- Wiping a desk with a cloth
18 types of open dataset of upper limb and scapular movements and some images
AIST will take the lead in forming a community with universities, research institutes, rehabilitation providers, and other private companies based on the release of this dataset, aiming to promote collaborative research and development on remote XR rehabilitation and contribute to market development. In addition, by working on standardization to facilitate the implementation of avatar control and hand redirection in the metaverse, and the release and refinement of various usage guidelines including reciprocal care, we will make remote XR rehabilitation easier to use and more attractive, and contribute to solving issues in its spread.
