Self-Supervised Learning

Background Self-supervised learning has become a major paradigm in modern representation learning, including for molecular and materials data. The central idea is that useful representations can be learned from structure, context, or predictive objectives without requiring dense task labels. In atomistic systems, this is particularly appealing because labels can be scarce, expensive, or difficult to define consistently across chemistry and materials domains. A growing body of work explores contrastive learning, BYOL-style self-distillation, masked prediction, and joint-embedding predictive objectives as ways to learn representations that preserve chemical and geometric information. Yet, a key open issue remains: what exactly do these learned representations encode, and which objective is most appropriate for preserving the right information for scientific applications? This question is not only about downstream benchmark performance, but also about whether the representation reflects chemically meaningful structure, geometric information, rare environments, and robust variation across the data manifold. ...

Background 3Shape developes hardware and software for intraoral scanners. This project will revolve around the 3D mesh data of intraoral scans (scan of people’s teeth). You will have the opportunity to work part-time from their offices in Kongens Nytorv. Deep self-supervised learning has been established as a great foundational model for a plethora of downstream tasks. Especially SimCLR has been shown to perform very well on downstream tasks. However, these empirical results have not been replicated in the 3D domain. In this project, we would like to explore how to apply this to the 3D models of teeth that 3Shape works with. Initially, we would like to benchmark some established methods on our data and then we would like to explore how to translate these methods to the 3D space. ...