Dementia is hard to diagnose. And there is no known cure, maybe because it too late by the time the diagnosis is confirmed. It is understood that the development of dementia commences 10 to 15 years before the symptoms first appear. Also, language, especially spontaneous speech, is a promising indicator/biomarker for diagnosing dementia and other cognitive disorders.
2. Language
The model will be trained on English audio.
3. Model
As far as I know, wav2vec2 is the best candidate.
4. Datasets
There is dementia classification from dementia bank. However, it is a binary classification dataset for dementia and no dementia.
To predict dementia 10 to 15 before the onslaught of symptoms, we would need longitudinal data on individuals who develop dementia. The only such available datasets I know is the Framingham heart study, and it is text only.
I have been building a list of public figures diagnosed with dementia and scrapping videos of youtube into different categories; after symptoms, two years before symptoms, five years before symptoms etc. Over the next week, I will build a streamlit app to extract 8 to 10 secs of audio files of the person of interest from the video.
Maybe we can use data from dementia bank or other sources for no dementia class??
Possible links to publicly available datasets include:
Dementia Bank
Google sheet with list of public figures with dementia and YouTube urls
5. Training scripts
TODO
6. Challenges
The dataset is too small, too noisy?
No dataset on time for no dementia
7. Desired project outcome
A proof of concept streamlit app that this works?
8. Reads
The following links can be helpful to better understand the project and
what has previously been done.
IBM efforts using Farmingham data for dementia prediction
Awesome two people is enough - let’s create the project!
Think this is a really cool & interesting project and we should be able to demo it well! Here I think it’ll actually make most sense to fine-tune a pretrained Wav2Vec2 model, maybe this one: facebook/wav2vec2-base-100k-voxpopuli · Hugging Face ?
There is currently no official fine-tuning script for Wav2Vec2, but it should be relatively easy to adapt the pretraining script: https://github.com/huggingface/transformers/pull/12271 or the official one in PyTorch.
Let’s create the project - very excited about this one
I would love to be a part of this project. I actually used wav2vec2 embeddings for participating in a Alzheimer dementia challenge. The wav2vec2 embeddings didn’t work so well for the task but this might be because there wasn’t enough data. I’m more then happy to try this out again.
Hey @shreyasgite I would love to join and contribute to the project.
If it is late can I follow the group’s progress. Really want to get going with a small project in the area
also @shreyasgite , make sure you right click on your server icon → invite and copy the link provided. You cannot access a discord server via the link in your browser.
