Developing an Autonomous Cognitive Assistant in Three Modalities of Data
School Name
Governor's School for Science and Mathematics
Grade Level
12th Grade
Presentation Topic
Computer Science
Presentation Type
Mentored
Written Paper Award
4th Place
Abstract
Due to advances in computing power and increased access to large datasets, machine learning has become a viable way of fostering productive and intuitive interactions between humans and computers. However, previous research and product developments, such as Apple’s Siri and Amazon’s Alexa, have been primarily limited to processing single modalities of data. This project sought to develop a more flexible and human-like cognitive assistant capable of making decisions based on its understanding of auditory, visual, and natural language input. This involved creating software that identifies songs based on short clips, recognizes the faces of users, and understands and produces language using a combination of statistical analysis and machine learning. To facilitate smooth user interface, the front end of the application used Amazon’s voice recognition software hosted on the Echo Dot. Systems of this kind have widespread applications, including cognitive assistants for personal use and various professional uses.
Recommended Citation
Tennis, Jaden, "Developing an Autonomous Cognitive Assistant in Three Modalities of Data" (2018). South Carolina Junior Academy of Science. 38.
https://scholarexchange.furman.edu/scjas/2018/all/38
Location
Neville 206
Start Date
4-14-2018 9:00 AM
Presentation Format
Oral and Written
Developing an Autonomous Cognitive Assistant in Three Modalities of Data
Neville 206
Due to advances in computing power and increased access to large datasets, machine learning has become a viable way of fostering productive and intuitive interactions between humans and computers. However, previous research and product developments, such as Apple’s Siri and Amazon’s Alexa, have been primarily limited to processing single modalities of data. This project sought to develop a more flexible and human-like cognitive assistant capable of making decisions based on its understanding of auditory, visual, and natural language input. This involved creating software that identifies songs based on short clips, recognizes the faces of users, and understands and produces language using a combination of statistical analysis and machine learning. To facilitate smooth user interface, the front end of the application used Amazon’s voice recognition software hosted on the Echo Dot. Systems of this kind have widespread applications, including cognitive assistants for personal use and various professional uses.
