Background: Communication disability resulting from stroke or traumatic brain injury affects the speaking, reading, writing, and comprehension abilities of a person depending on the type and severity of the condition. Current tools and services do not support an automated approach to evaluate the abilities of a person affected by this condition.
Aims: This project aims to offer an AI-driven automated platform that extracts linguistic and audio features to generate insights into a person's speech disabilities through conversation recordings. This platform will support profiling individuals based on their speech abilities to compare and evaluate them with others and themselves over time.
Methods: Audio clips were extracted from the conversation videos, and transcripts were generated using transcribers finetuned to process non-fluent speech data. Audio signal processing, Natural Language Processing, Large Language Models, and other related machine-learning techniques were used to extract speech-related features from the data and build feature profiles for people. Novel machine learning algorithms were implemented to cluster these feature profiles which enable to compare and evaluate the current state or the progress of a person with communication disabilities.
Results and Discussion: An online tool is produced with an easy-to-access interface to analyze speech data in terms of language usage, fluency, and relevant speech features, enabling health professionals and researchers to get valuable insights about the person's condition. The profiling tool positions the person’s speech abilities respectively to other people, which can help compare the status and improvements over time. Future work will involve refining the platform and evaluating its effectiveness through an upcoming pilot trial. "