Background
When designing and training machine learning architectures for medical data, we often neglect prior knowledge about the data domain. In this project, we want to investigate whether we can find shared information between medical signals and their corresponding textual descriptions. Current advances in the field of NLP have made it possible to learn rich contextual information from textual data. Given cross-modality observations, Relative Representations make it possible to compare and align latent spaces of different modalities or models.
One suggestion would be to perform knowledge scraping of textual data from Wikipedia, scientific articles or medical textbooks to create a knowledge base or knowledge graphs, to be used as prior general knowledge guiding the embedding of the textual meta data associated with medical data.
Our suggestion would be to work on the ECG data from Liu et al, 2023, but the data source is not set in stone, and we are open to working with your data, if you have something available of the nature that we are describing.
{width=“4.363017279090114in”
height=“1.9224015748031495in”}
Objective(s)
i) Investigate the relation between medical signal embeddings and > textual signal embeddings using relative representations
ii) Create a new self-supervised learning model using text embeddings > to contextualize the signal embeddings.
Requirements
Need to have:
Strong programming skills in Python
*Experience working with deep learning models in frameworks such as
TensorFlow or PyTorch*
Nice to have:
*Experience working with medical time series signals (EEG/MRI/ECG
etc)*
Experience working with natural language processing
Maximum number of students
1-2 pr. group
Supervisors
Qianliang Li, Postdoc ([glia@dtu.dk]{.underline})
Thea Brüsch, PhD ([theb@dtu.dk]{.underline})
Fabian Mager, PhD ([fmager@dtu.dk]{.underline})
Contact information
Building 321
Section for Cognitive Systems
DTU Compute
References
Moschella et al, 2023
([https://arxiv.org/pdf/2209.15430.pdf]{.underline}) - Relative representations for comparing embedding spaces from different models
Liu et al, 2023
([https://arxiv.org/pdf/2309.07145.pdf]{.underline}) - Text guided learning of ECG encoders