What is Motion Capture?
Motion capture covers a wide range of technique for tracking motions in the real world and transferring them into a digital representation. It has classically been implemented by analysing images taken by normal cameras, which were then analysed to detect how foreground bodies (usually people) moved in a room. More advanced motion capture systems such as the MoCap studio operated by UiO's fourMs lab uses very exact rigging and timing to track special markers that can be placed on anything - people or objects.
For this thesis, we use the Rokoko Smart Suit II. It is essentially an overall and a pair of gloves that are equipped with several inertia measurement units, which report their relative position in space to a software that comes with the suit. The positions can then be seen directly on a screen, and be translated to an avatar (for example in Unreal Engine), or they can be sent over the network.
It is this latter application that we care about.
Specific question for this thesis
In this thesis, you will explore how we can predict body motion. We know that the motion of the human body is constrained by our bones and muscles, and unless we hit an obstacle, we can generally not stop a motion immediately.
That allows us to predict the trajectory of fingers, hands, arms or the whole body with very high confidence for a short period of time. And we don't need much time to compensate for the delays of computers and networks: typically less than 100 milliseconds. But these 100 milliseconds must be predicted correctly, and that is the task of this thesis:
- create a digital collection of human body motions in an enviroment where you can walk around freely
- use reinforcement learning to develop a prediction model that takes an arbitrary position and predicts the motion of every limb for up to 100 milliseconds
- explore how you can improve your model by removing a dependency on the speed of the body's motion (which the initial model will likely have).
Finally, you should use Unreal Engine 5.2 to illustrate your results subjectively. Rokoko's software provides an Unreal plugin, which will assist in this endeavour. You should measure the objective deviation between real and predicted motion.
Learning outcome
Experience in
- in formulating, investigating and answering research questions
- in using machine learning techniques for event prediction
- collecting dataset of appropriate size and homogeneity
Conditions
We expect that you:
- have been admitted to a master's program in MatNat@UiO - primarily PROSA
- take this as a long thesis
- will participate actively in the weekly SINLab meetings
- are present in the lab and collaborate with other students and staff
- are interested in explore machine learning techniques
- are interested in exploring motion capture hardware in the lab
- are willing to share your results on Github
- include the course IN5060 in the study plan, unless you have already completed a course on classical (non-ML) data analysis