Recent Projects

Nowadays it is common to find cameras everywhere, e.g., surveillance cameras on the streets or inside stores, dash cameras mounted on cars, etc. The captured information can help detecting and alerting potential crime actions thus enhancing the safety of our society. In order for us to advance towards a smarter city, intelligent methods are required to analyze the big data captured from these sensors. In this project we focus on predicting human behavior and understanding group interactions through social motion capture. Multiple cameras can be used to capture information with multi-modal input data such as normal RGB cameras together with depth sensors. Skeleton trajectories of the human subjects present in the scene can then be extracted from the input frames. One research issue is to explore an effective way to fuse the multi-view and multi-modal input data in order to extract accurate features from each type of input data to be combined, under challenging conditions from occlusions or missing information. We hope to decode some meaningful cues from social interactions by understanding one’s physical body language. Another research issue is to predict human behavior which is useful for autonomous driving and human-robot interaction but is challenging due to human consciousness movement and difficulty of long-term kinematics modeling.

With the motion capture technology, we have developed a performance training tool to provide a virtual training environment for dancing and martial arts. In the environment, the student’s performance is analyzed in our system by evaluating the posture similarity between the student and the virtual teacher then feedback will be given for his/her improvement. In addition, we have proposed a new perceptual similarity measure for comparing two motions. We also studied the problem of real-time recognition of the user’s live dance performance in order to determine the interactive motion to be rendered by a virtual dance partner. Besides, we have proposed a system for automatically building dance lessons given an input dance motion sequence by extracting the repetitive patterns and computing the prerequisite relations.

We have developed computerized tools to analyze the digital ink of the handwriting trajectory captured from a student’s writing task in order to screen out students with handwriting difficulties. This is a multi-disciplinary research in which we have collaborated with occupational therapists and school teachers and our work has attracted some industrial collaborations. The research outcomes include applications that have been used by healthcare professionals, occupational therapists and teachers to help them determine children's handwriting performance and identify their potential problems.