Title

Developing an Autonomous Cognitive Assistant in Three Modalities of Data

Author(s)

Jaden Tennis, GSSM

School Name

Governor's School for Science and Mathematics

Grade Level

12th Grade

Presentation Topic

Computer Science

Presentation Type

Mentored

Written Paper Award

4th Place

Abstract

Due to advances in computing power and increased access to large datasets, machine learning has become a viable way of fostering productive and intuitive interactions between humans and computers. However, previous research and product developments, such as Apple’s Siri and Amazon’s Alexa, have been primarily limited to processing single modalities of data. This project sought to develop a more flexible and human-like cognitive assistant capable of making decisions based on its understanding of auditory, visual, and natural language input. This involved creating software that identifies songs based on short clips, recognizes the faces of users, and understands and produces language using a combination of statistical analysis and machine learning. To facilitate smooth user interface, the front end of the application used Amazon’s voice recognition software hosted on the Echo Dot. Systems of this kind have widespread applications, including cognitive assistants for personal use and various professional uses.

Location

Neville 206

Start Date

4-14-2018 9:00 AM

Presentation Format

Oral and Written

COinS
 
Apr 14th, 9:00 AM

Developing an Autonomous Cognitive Assistant in Three Modalities of Data

Neville 206

Due to advances in computing power and increased access to large datasets, machine learning has become a viable way of fostering productive and intuitive interactions between humans and computers. However, previous research and product developments, such as Apple’s Siri and Amazon’s Alexa, have been primarily limited to processing single modalities of data. This project sought to develop a more flexible and human-like cognitive assistant capable of making decisions based on its understanding of auditory, visual, and natural language input. This involved creating software that identifies songs based on short clips, recognizes the faces of users, and understands and produces language using a combination of statistical analysis and machine learning. To facilitate smooth user interface, the front end of the application used Amazon’s voice recognition software hosted on the Echo Dot. Systems of this kind have widespread applications, including cognitive assistants for personal use and various professional uses.