[talks] X Ma general exam

[Melissa M Lawson]

Xiaojuan Ma will present her research seminar/general exam on Wednesday 
May 9 at 10 AM in Room 301 (note room!).  The members of her committee 
are Fei-Fei Li, Szymon Rusinkiewicz, Tom Funkhouser.  Her advisor is Perry 
Cook.  Everyone is invited to attend her talk, and those faculty members wishing
to remain for the oral exam following are welcome to do so.  Her abstract and 
reading list follow below.
-------------------------------------------- 

Abstract:

Assistive devices and tools for persons with aphasia, a debilitating condition that robs
an individual of the ability to communicate, have traditionally depended on icons to
convey the meaning of words that the individuals have lost. Despite the predominant use of
icons, which are created by artists, speech language pathologists suggest that personal
digital photographs may be easier for people to map to the real phenomena. Also, compared
to other visual art resources such as icons, images are cheaper, ubiquitous, numerous, and
varied. A concept of picture-as-language, which uses digital photographs as a language has
shown some significant and long-lasting effects on the learning for people with mental
retardation, developmental disabilities, or learning disabilities.

Designing for a population that relies heavily on picture representations, it is important
to understand to what extent the choice of icons and images affects the adoption and
usability of assistive technology. A study with unimpaired members of an older demographic
likely to be affected by aphasia shows that, for sentences with words replaced by either
images or icons, the word-level and sentence-level meaning are retained equally well for
both images and icons. It is also supported by a further study with our target group,
people with aphasia, on the effectiveness of icons and images on conveying noun concepts. 
However, compared to concrete nouns, both studies showed that neither icons or images
performed as well on abstract nouns. What's more, the study with senior citizens revealed
that both icons and images did a poor job of embodying verbs and attributes. A third study
evaluating the efficacy of four different visual representations of verbs for subjects
from the general population addressed the question of the best representation of both
motion and abstract verbs. Related video and image database for verb illustrations is
under construction.


Books:

Preece, J., Rogers, Y., and Sharp, H. Interaction Design: Beyond Human-Computer
Interaction. New York: John Wiley & Sons, Inc., 2002.

Shneiderman, B. Designing the User Interface: Strategies for Effective Human-Computer
Interaction. Boston: Addison Wesley, 1997. Chapters: 1-4, 10, 11



Papers:

Newell A.F., Carmichael, A., Gregor, P., and Alm, N. Information Technology for Cognitive
Support. The Human-Computer Interaction Handbook, Jacko, J. and Sears, A. (Editors).
Mahwah, NJ: Lawrence Erlbaum Associates, Inc, 2002. 464-481.

R. D. Steele, M. Weinrich, R. T. Wertz, M. K. Kleczewska, and G. S. 
Carlson. Computer-Based Visual Communication in Aphasia. 
Neuropsychologia, 27(4):409-426, 1989.

van de Sandt-Koenderman, M. W. M. E. High-tech AAC and aphasia: widening horizons?
Aphasiology (2004), 18 (3), 245-263.

Baecker, R., Small, I., and Mander, R. Bringing Icons to Life. 
Proceedings of CHI '91 Human Factors in Computing Systems. New York: 
ACM, 1991, pp. 1-6.

J. L. Boyd-Graber, S. S. Nikolova, K. A. Moffatt, K. C. Kin, J. Y. Lee, L.W. Mackey, M. M.
Tremaine, and M. M. Klawe. Participatory design with
proxies: Developing a desktop-PDA system to support people with aphasia. 
In Proc. CHI2006, pages 151-160. ACM Press, 2006.

K. Tee, K. Moffatt, L. Findlater, E. MacGregor, J. Mc-Grenere, B. 
Purves, and S. S. Fels. A visual recipe book for persons with language impairments. In
Proc. CHI 2005, pages 501-510. ACM Press, 2005.

Danielsson, H. and Svensk. A. Digital Pictures as Cognitive Assistance. 
In Proceedings of the 6th European Conference for the Advancement of Assitive Technology,
2001. (5 pages)

R. Mihalcea and B. Leong. Toward communicating simple sentences using pictoral
representations. In Proc. of the Conf. of the Assoc. for Machine Translation in the
Americas 2006, Boston, MA, 2006.

T. Takasaki. Pictnet: Semantic infrastructure for pictogram communication. In P. Sojka,
K.-S. Choi, C. Fellbaum, and P. Vossen, editors, Proc. Global WordNet Conference 2006,
pages 276-289. 
GlobalWordNet Association, January 2006.

Pettersson, Rune. Interpretation of Image Content. Educational Communication and
Technology Journal, v36 n1 p45-55 Spr 1988.