Please use this identifier to cite or link to this item:
Title: Robotic gesture-speech synchronization
Authors: Lim, Keck Soon.
Keywords: DRNTU::Engineering::Mechanical engineering
Issue Date: 2011
Abstract: This report provides a detailed description of the steps taken to build a text to speech to gesture engine. The building of the engine is separated into three main segments. The first segment involves gesture planning. This includes generating a blending motion between the first and subsequent motions. The blending motion is created to prevent jerking motions from occurring during the translation of motion. Two methods were introduced to remove the need of having a huge library of pre-recorded gestures. The first method is to introduce coherent noises into existing gesture file to produce gesture of similar type yet different movement which can be perceived. The second method is to generate beat gestures during run time which can totally remove the need of a gesture library. The next segment of the project is to create and link various engines. The method used for communication was via transmission control protocol/internet protocol. A text engine was created to prepare the modified text format to be processed by the text to speech engine. Timings are obtained from the engine and sent to the gesture engine for synchronization. The gestures files are created from the gesture engine and the simulation can be seen. To make it more user friendly for non programmers, a graphical user interface was created. In addition, an auto tagging function was included to automatically determine the start and end time of gesture. The last segment of the project is to implement the engine into an actual robot, Olivia 2.1. The challenges faced include keeping the speech and gesture in sync while respecting the robot’s physical constraints. Also, to determine the coefficient that affects the sampling rate of the robot.
Schools: School of Mechanical and Aerospace Engineering 
Organisations: A*STAR Institute for Infocomm Research
Rights: Nanyang Technological University
Fulltext Permission: restricted
Fulltext Availability: With Fulltext
Appears in Collections:MAE Student Reports (FYP/IA/PA/PI)

Files in This Item:
File Description SizeFormat 
Keck Soon FYP Report.pdf
  Restricted Access
1.23 MBAdobe PDFView/Open

Page view(s)

Updated on Jun 14, 2024


Updated on Jun 14, 2024

Google ScholarTM


Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.