The Indian Institute for Human Settlements (IIHS, www.iihs.co.in) and the SpokenMedia (spoken- media.mit.edu) team from the MIT Office of Educational Innovation and Technology (OEIT) have been discussing how SpokenMedia technologies might be used by IIHS to provide cost effective ways of making video/audio course materials accessible to the diversity of students expected by IIHS. This presentation provides a case study of the proof-of-concept demonstration SpokenMedia developed for IIHS. Presented by Brandon Muramatsu at OCWC Global 2010, Hanoi, Vietnam, May 5, 2010.
Judging the Relevance and worth of ideas part 2.pptx
Opening Up IIHS Video with SpokenMedia
1. Opening Up IIHS Video with SpokenMedia Brandon Muramatsu Andrew McKinney Peter Wilkins May 2010 Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License ( creativecommons.org/licenses/by-nc-sa/3.0/us/ ) Citation: Muramatsu, B., McKinney, A., Wilkins, P. (2010). Opening Up IIHS Video with SpokenMedia. Presented at OpenCourseWare Consortium Global 2010: Hanoi, Vietnam, May 7, 2010.
2.
3. The Indian Institute for Human Settlements (IIHS) will… “create India’s first independent National Innovation University focused on the challenges and opportunities of its urbanisation.” – Indian Institute for Human Settlements: Curriculum Framework Version 3.0 January 2010
4. “ The IIHS Website is our commitment to a different way of looking at things.” – Aromar Revi 5 January 2010
5. “ The Institution will fail or scale based on language.” – Aromar Revi 5 January 2010
6. What did we do? Auto Transcribe Edit Translate Present
12. How did we do it? Auto Transcribe Edit Translate Present
13. Edit & Translate: Accuracy Automatic Transcription Hand Transcription Time Adjusted Translated Hindi I I I मेरे खयाल से think think think once one one नयोजन की एक मुख्य चुनौती है and central so challenge central the of challenger planning challenge of planning is planning nice legitimacy is legitimacy of legitimacy of of government government सरकार की एक ऐसी मुख्य संस्थान के रूप में वैधता government as as
14.
15. How did we do it? Auto Transcribe Edit Translate Present
16.
17.
18.
19.
20. Thank You! Brandon Muramatsu, [email_address] Andrew McKinney, [email_address] Peter Wilkins, [email_address] ° Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License ( creativecommons.org/licenses/by-nc-sa/3.0/us/ )
Editor's Notes
Citation: Muramatsu, B., McKinney, A., Wilkins, P. (2010). Opening Up IIHS Video with SpokenMedia. Presented at OpenCourseWare Consortium Global 2010: Hanoi, Vietnam, May 7, 2010. Unless otherwise specified, this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
Four step process (simple) First we used the SpokenMedia to do automatic transcription. Next we did hand edit and translation steps. Then we created a player for the presentation of the video and transcripts…
Demo
First we used the SpokenMedia to do automatic transcription.
Lecture Transcription Jim Glass and his group have years of research experience for spoken languages Lectures are a different type of spoken language Much of the speech recognition research has focused on real time transcription of news broadcasts, or interactive voice response systems (telephone) Broadcast news has something like 300 unique words in an hour long broadcast Broadcast news is well structured, prepared copy (in studio via teleprompters), clear transitions between speakers, etc. Lectures are conversational and spontaneous Can use highly specialized vocabularies, engineering, physical sciences, mathematics
Spoken Lecture Project Supported by iCampus Includes the browser (which was just demo’d) the processor (back end lecture transcription) and a hand workflow to do the processing Approximately 400 hours of video indexed
We only used part of the process due to time constraints. Audio separation, speech processing, time-coded transcript, and then presentation through a SpokenMedia player.
Next we did hand edit and translation steps.
For this demo, we did computer-based automatic transcription, sent a file to IIHS for “editing” that consisted of performing a hand transcription (due to the format we sent, and the low accuracy of the automatic transcription in this case), a time alignment (though I actually feel that it’s “off” or “slow”) and then a hand translation by IIHS.
Recognizer Accuracy Base accuracy is approximately 50% (generic domain and speaker models) Increase accuracy with speaker model up to 80-85%, and specific domain model This approach is good for courses with multiple lectures by the same speaker Domain models get more useful as more relevant text documents are indexed (keyword/noun phrase extraction) Initial results indicate that doing one 99% accurate (by hand/manual) transcript can help immensely for additional lectures by the same speaker Better use of limited resources Search accuracy is closer to 90%, searches tend to be for unique words which the processor is better at recognizing
Then we created a player for the presentation of the video and transcripts…
What we have today It’s not perfect, but a pretty good start Prototype has a number of useful features that demonstrate search interfaces and interaction interfaces
Where are we heading? Transition from research project to service Explore new interactions—what we’re calling Rich Media Notebooks
Citation: Muramatsu, B., Kumar, M.S., Merriman, J., Wilkins, P. (2010). Project Greenfield: A New Way of Thinking about OpenCourseWare and Open Educational Resources for MIT. Presented at OpenCourseWare Consortium Global 2010: Hanoi, Vietnam, May 7, 2010. Unless otherwise specified, this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License