University of Ulster

It is apparent that mobile technologies are demonstrating great potential to aid and assist us in our everyday lives. Mobile phones and handheld computers allow us to communicate and gain access to information anywhere and at any time while gaming devices are proving to be successful and a popular entertainment medium especially amongst children. This research investigates the use of multimodal interfaces on mobile handheld devices. More specifically, the use of a mobile multimodal learning environment for children. The primary objective of the research is the design and development of AmbiLearn, a multimodal learning environment based on a mobile handheld device. Using speech and 2D gesture as input and providing speech, graphics and an embodied agent as output, the development of AmbiLearn will address the interface design and usability issues of a multimodal system for children. Addressing the role of serious games in education, an application namely TreasureLearn will be developed to investigate whether an educational game integrated with elements of a virtual learning environment based on a handheld gaming device can have a positive impact on children’s education at a primary level.