Sonya Nikolova will present her preFPO on Wednesday December 2 at 10AM in Room 402.

The members of her committee are:  Perry Cook, advisor; Marilyn Tremaine (Rutgers) and

Christiane Fellbaum, readers; Adam Finkelstein and Szymon Rusinkiewicz, nonreaders. Everyone

is invited to attend her talk.  Her abstract follows below. 

*****

Title and Abstract:

Improving Vocabulary Organization in Assistive Communication Tools: A Mixed-Initiative Approach

Navigating a vocabulary consisting of thousands of entries in order to select appropriate words for building communication is challenging for individuals with lexical access disorders like those caused by aphasia, a cognitive disorder. Most existing assistive communication vocabularies have a lexical organization scheme based on a simple list of words. Some word collections are organized in hierarchies which often leads to deep and confusing searches; others are simply a list of arbitrary categories which causes excessive scrolling and a sense of disorganization. Ineffective vocabulary organization and navigation hurt the usability and adoption of assistive communication tools and ultimately fail to help users engage in practical communication.

We have developed a multi-modal visual vocabulary that enables improved navigation and effective word finding by modeling a speaker’s “mental lexicon”, where words are stored and organized in ways that allow efficient access and retrieval. Due to impaired links in their mental lexicon, people with aphasia have persistent difficulties accessing and retrieving words that express intended concepts. The Visual Vocabulary for Aphasia (ViVA) attempts to compensate for some of these missing or impaired semantic connections by organizing words in a dynamic semantic network where links between words reflect semantic association measures based on WordNet, human judgments of semantic similarity, and past vocabulary usage. 

ViVA is both adaptable, able to be customized by the user, and adaptive, able to dynamically change to better suit users’ past actions and future needs. This mixed-initiative approach enables the user to feel in control by making changes and anticipating ones that have been initiated by the system while still allowing adaptive methods to help determine where and when changes are required. ViVA’s adaptable component allows the user to add and remove vocabulary items, and group them in personalized categories. The adaptive component updates the semantic network according to vocabulary usage, user preferences, and a number of semantic association measures. Thus, ViVA tailors the vocabulary organization according to both user-specific information and general knowledge of human semantic memory. 

We report results from two studies evaluating how ViVA compares to a vocabulary access system in which words are organized hierarchically into common categories. Since our target user population is people with cognitive impairments, further work involves studying the effect of adaptation on users’ cognitive load and investigating the tradeoff between improved navigation and changing vocabulary structure due to adaptation.