Skip to content
Artificial intelligence (AI) is on everyone’s mind these days. In healthcare and biotech, AI is addressing some of the industry’s biggest challenges, as well as enabling clinicians to care for their patients in unprecedented ways.
San Francisco-based Modality.AI is assessing patients with neurological and psychiatric conditions, ranging from Parkinson’s disease to schizophrenia, to measure disease progression and the effect of treatments, thus providing invaluable insights for the development of life-changing therapies.
Modality.AI was one of the companies supported by the National Institutes of Health (NIH) SEED program chosen to present on the NIH Innovation Zone Stage at the 2024 BIO International Convention. Bio.News interviewed Modality.AI and the company’s founder and CEO, Dr. David Suendermann-Oeft, about their work and what their vision is for the future of AI-based neurological assessments.
Q: For those less familiar with the details of different AI programs, can you explain what a multimodal system is and why it is so effective for remote neurological and psychiatric assessment?
A multimodal AI system can process and analyze multiple types of data inputs, such as text, images, video, audio, and sensor data. This approach is highly effective because it is modeled on the standard-of-care clinical practice of observing and listening to patients, as well as analyzing what they say and how they say it.
Modality.AI’s multimodal system enables remote assessments, making frequent monitoring accessible from the patient’s home through a computer, tablet, or phone. Traditional in-clinic assessments limit the frequency of evaluations, and interpretation can vary widely between clinicians. By using a virtual guide named Tina we standardize the administration process leading patients through the assessment.
Analysis is performed automatically by Modality’s AI models, providing the results, along with a segmented and cataloged recording of the assessment immediately. This innovative approach combines the benefits of multimodal data capture with the convenience and scalability of remote, AI-powered assessments in the comfort and convenience of a patient’s home.
Interview with Dr. David Suendermann-Oeft, Founder and CEO, Modality.AI
Q: What are the target patient groups for Modality.AI’s platform? And what could patients anticipate when it comes to the clinical dynamic between them, physicians, and the AI system? Would there be any learning curve for patients to get over?
Modality is being used in clinical trials of new treatments for neurodegenerative conditions such as Amyotrophic Lateral Sclerosis (ALS) and Parkinson’s. We’re also working with academic and non-profit organizations on natural history studies in autism, schizophrenia, and mild cognitive impairment. There’s virtually no learning curve for patients as Modality works on a web browser, so there’s no app to install or login to remember. Anyone who can use Facetime or Zoom, can use Modality.
Q: You work with Parkinson’s patients a great deal at Modality.AI, how did you develop your platform to best monitor and measure their deterioration?
We worked with academic research partners at Purdue to develop a dataset of multimodal data from Parkinson’s patients, including speech and video recordings of facial expressions and body movements. We also collected verbatim reports of patients’ most bothersome problems from over 35,000 participants through our collaboration with the Michael J. Fox Foundation.
By analyzing these modalities we can capture a comprehensive picture of disease manifestation and progression. Over the past five years we published and presented more than forty papers and posters about our data and findings in Parkinson’s disease and almost one hundred across all target diseases (see: https://modality.ai/publications).
Q: How is Modality.AI’s growing database of information helping neurologists and psychiatrists better understand diseases like Parkinson’s disease, ALS, or depression? What insights have you found that might have gone unnoticed without the use of AI in this space?
Our analysis of extracted multimodal features like vocal characteristics, facial muscle movements, and body pose/movement patterns has shown utility in differentially diagnosing neurological and psychiatric disorders like schizophrenia, depression, and ALS. We can detect subtle differences in how these conditions manifest through speech impairments, facial expressions, and motor dysfunctions.
Through speech and natural language processing (NLP) of patient recordings, Modality’s platform can identify changing patterns of communication, cognitive deficits associated with neurodegeneration often leading to slowed speech and difficulties with word retrieval, or dysarthria which often affects the motor aspects of speech production in diseases like ALS or Parkinson’s. Facial masking as a symptom in Parkinson’s disease, lack of emotional expression as seen in schizophrenia or autism, or facial muscle weakness and impaired control in ALS can be assessed using Modality.AI’s facial expression analysis capabilities. This allows tracking of condition-specific progression and differentiation between diseases.
Q: As many have noted, AI technology is growing at an incredible rate. What is the future of Modality.AI and how do you anticipate your machine learning systems developing in the coming years?
Today, Modality is used exclusively in clinical trials to help bring new treatments to the market, and enable researchers to better understand diseases, especially in neurology and psychiatry. Thanks to the simple and accessible nature of Modality’s assessments, which are administered by the virtual guide Tina and processed in the cloud, they can be provided to anyone who would benefit from accurate, real-time tests in the privacy of their homes. This is valuable for early diagnosis, disease monitoring, and measurement of treatment efficacy. By expanding into healthcare in the future, Modality.AI will help address the urgent need for better neurological and psychiatric care combined with precision medicine.
Learn more about the NIH Small business Education and Entrepreneurial Development (SEED) funding at seed.nih.gov.
