Current methods on Computer Vision rely on square-patch tokens as inputs. However, what if you could have semantic-aware tokens, as superpixels, as inputs to then do common tasks on Computer Vision. What would these neural network architectures look like? To what extent will they be better than the existing models?
This project aims to answer these questions. The possible areas of work are the application of superpixels to:
-
Develop new and more powerful superpixel model tokenizers
- Panoptic (Instance/Semantic) Segmentation
- Object Detection and Recognition
- Action Detection and Recognition
- Feature extraction and representations
These areas contain many subproblems that can be discussed and tackled independently. The final application area and the task will be defined based on the interests, background, and capabilities of the candidate.
Qualifications:
- Excellent programming skills in Python and Pytorch
- Deep learning background
- Image processing or analysis background
- Knowledge of Git