Tutorials

Interspeech 2020 Tutorial on Spoken Language Processing for Language Learning and Assessment (with Klaus Zechner and Keelan Evanini)

This tutorial provides an in-depth survey of the state of the art in spoken language processing in language learning and assessment from a practitioner’s perspective. The first part of the tutorial will discuss in detail the acoustic, speech, and language processing challenges in recognizing and dealing with native and non-native speech from both adults and children from different language backgrounds at scale. The second part of the tutorial will examine the current state of the art in approaches to automated scoring of monolog speech data along various dimensions of spoken language proficiency. The final part of the tutorial will look at hot topics and key challenge facing the field at the moment – that of automatically generating targeted feedback for language learners that can help them improve their overall spoken language proficiency. We also present current hot topics in the field such as the automated scoring of dialog and multimodal data for language learning and assessment.

The presenters, based at Educational Testing Service R&D in Princeton and San Francisco, USA, have more than 40 years of combined R&D experience in spoken language processing for education, speech recognition, spoken dialog systems and automated speech scoring.


Interspeech 2018 Tutorial on Spoken Dialog Technology for Educational Domain Applications (with Keelan Evanini and David Suendermann-Oeft)

This tutorial introduces participants to the basics of designing conversational applications in the educational domain using spoken and multimodal dialog technology. The increasing maturation of automated conversational technologies in recent years holds much promise towards developing intelligent agents that can guide one or multiple phases of student instruction, learning, and assessment. In language learning applications, using spoken dialogue systems (SDS) could be an effective solution to improving conversational skills, because an SDS provides a convenient means for people to both practice and obtain feedback on different aspects of their conversational skills in a new language. These allow learners to make mistakes without feeling incompetent, empowering them to improve their skills for when they do speak with native speakers. From the assessment perspective, well-designed dialog agents have potential to elicit and evaluate the full range of English speaking skills (such as turn taking abilities, politeness strategies, pragmatic competence) that are required for successful communication. Such technologies can potentially personalize education to each learner, providing a natural and practical learning interface that can adapt to their individual strengths and weaknesses in real time so as to increase the efficacy of instruction.

The tutorial assumes no prior knowledge of dialog technology or intelligent tutoring systems and demonstrates the use of open-source software tools in building conversational applications. The first part of the tutorial covers the state of the art in dialog technologies for educational domain applications, with a particular focus on language learning and assessment. This includes an introduction to the various components of spoken dialog systems and how they can be applied to develop conversational applications in the educational domain, as well as some advanced topics such as methods for speech scoring. The final part of the tutorial (not fully represented in the slides below) is specifically dedicated to a hands-on application building session, where participants will have a chance to design and deploy their own dialog application from scratch on the HALEF cloud-based dialog platform using open-source OpenVXML design toolkit, which will allow a better understanding how such systems can potentially be designed and built.