Toward a Context-Aware Augmentative and Alternative Communication System with Vocabulary Prediction

Abstract: Persons who rely on Augmentative and Alternative Communication (AAC) systems to communicate, face many difficulties when they try to maintain a dialog, in part because of the poor output rate offered by current systems. To minimize this problem, this thesis proposes and evaluates applying the wholeutterance approach to vocabulary prediction. One of the techniques studied was sentence prediction, a technique that may complement word prediction to assist users compose their messages faster. Since prediction of sentences seems to be a very context-dependent process, we also evaluated use of context-awareness to improve vocabulary prediction. For AAC users who can only communicate using pictograms, or images, we also evaluated counterpart techniques of word and sentence prediction, namely single pictogram and pictogram sentences prediction.Our approach to context-awareness consisted of using specific user profiles for different communication contexts (location, time, and speaking partner). Depending on the context, the AAC device can configure itself using data from the associated user profile, to adapt its interface, and language models for vocabulary prediction, to the current communication context. To evaluate this approach we developed, in conjunction with rehabilitation professionals four context-aware AAC solutions for real AAC users. Using these real AAC solutions, we were able to collect corpora representative of this type of communication, train language models for vocabulary prediction, and carry out user tests and software simulations to emulate ideal performance.In user tests, combining word and sentence prediction were obtained statistically significant higher words per minute (WPM) than using only word prediction (18.8 WPM vs 8.3 WPM). In another study, using location-specific language models for word and sentence prediction were achieved mean improvements of 2.4% in words per minute, but differences were not statistically significant. Software simulations showed that location-specific language models tended to perform better than a single language model under high sentence reuse scenarios.Concerning pictogram prediction, in user tests, the condition combining single pictogram and pictogram sentence prediction obtained the best results (6.2 WPM). The condition with no pictogram prediction achieved the worst result (4.5 WPM). However, there were not statistically significant diferences between conditions. There were though, statistically significant increases in keystroke savings. In a subsequent study, simulations a pictogram corpus showed that location-specific language models could outperform a single language model, in a statistically significant way, when the sentence reuse rate was greater than 75%. Very interestingly, analysis of the corpus produced by a real AAC user, we followed-up during one year, showed high sentence reuse rates, ranging from 73.5% to 99.0%, depending on user location. Globally speaking, results obtained show that the proposed techniques can improve the AAC users’ performance.

Tópicos: