Research Problem
A lot of research articles attempt to use artificial intelligence to understand the content of low-density 3D scans in terms of high-level abstractions. For examples, work exists to find cars, trees, people and trashcans in a 3D scan.
We believe that these attempts are feasible for some application, such as automatic cruise control, but that the idea is a dead end for general applications.
The reason is that the number of abstract things that can exist in a 3D scan is infinite. Whenever you believe that you have trained you AI for every possible concept in the world, someone will come and build something with a size and shape that you've never seen before. This is an unsolvable problem because all creative fields have as a goal to create something that hasn't existed before - and they use other AIs to help them.
Instead of this dead end, we will understand basic geometric elements that exist in a scene. Later, someone can group these into abstract concepts if that is desired. We want to find easily described geometric 3D structures: planes, balls, cylinders, pyramids, and very simple deformations of these. This is a powerful representation that can be later be used for efficient compression, simplification, the creation of CAD models, or a second-stage interpretation of the scene's content. Even the power cable of your computer can be described by as many deformed cylinders (or one deformed cylinder, but that's hard).
The Thesis
The literature review of this thesis explores the related work about
- how simple geometric 3D shape can be represented very easily,
- how they can be discovered in pointclouds using brute-force, non-ML techniques,
- and how ML is used today to find "things" in 3D pointclouds.
The research objectives are as follows:
- Find several datasets of 3D scenes where 3D elements can easily be described by their simple geometric 3D shapes, and which can be downloaded and loaded into Blender. A suitable source of such scenes are CAD models.
- Write a program that creates several geometrically realistic low-density pointclouds from every 3D model, tagging every point with the shape it belongs to. This makes it possible to create the ground truth.
- Show that brute-force algorithm can find the expected 3D shapes in the pointclouds you created. Note that this has been done many times for planes - you must find other shapes as well.
- Develop an AI model that divide a pointcloud into its 3D shapes with similar or better properties than the brute-force algorithms.
Potential Contributions
The creation of the dataset where all basic 3D shapes are labelled will by itself be an important contribution for the research community.
Collecting brute-force algorithms for finding 3D shapes in point clouds (whether they are self-written from algorithm descriptions or imported from open source projects) will be a contribution.
Developing an explainable AI approach that makes use of knowledge from brute-force detection of 3D shapes will be the main contribution of the thesis.
All software should be released on Github.
Learning outcome
Experience in
- in formulating, investigating and answering research questions
- knowing about 3D modeling in general and in particular in Blender
- knowing how machine learning can be used to identify structure in 3D scenes, which is a hot topic
Conditions
We expect that you:
- have been admitted to a master's program in MatNat@UiO - primarily PROSA
- take this as a long thesis
- will participate actively in the weekly SINLab meetings
- are present in the lab and collaborate with other students and staff
- are interested in and have some knowledge of Python programming
- are willing to share your results on Github
- include the course IN5060 in the study plan, unless you have already completed a course on classical (non-ML) data analysis