An Interactive Whiteboard that Leverages Mathematical Handwriting Recognition to Assist in Understanding Math

School Name

Spring Valley High School

Grade Level

11th Grade

Presentation Topic

Computer Science

Presentation Type

Non-Mentored

Abstract

Handwritten Mathematical Expression Recognition (HMER) remains a longstanding challenge in computer science due to the two-dimensional structure, symbol ambiguity, and spatial relationships inherent in mathematical notation. While recent advances in machine learning have substantially improved off-line recognition of static handwritten expressions, a gap persists between state-of-the-art HMER models and their integration into real-time, user-facing applications. This study investigates whether an HMER-enabled interactive digital whiteboard can effectively support the solving, graphing, and visualization of handwritten mathematical expressions in real time. To address this question, a web-based digital math whiteboard was designed and implemented using the VueJS frontend framework. The system integrates the TexTeller model for handwritten expression recognition, the Desmos API for graphing, and symbolic computation libraries (Nerdamer and Math.js) for solving and manipulation. Users write mathematical expressions directly on a canvas using stylus or mouse input, select regions for recognition, and receive immediate typeset expressions in the form of interactive widgets. These expression widgets can then be converted to graphs or solutions. Results demonstrate that the application successfully recognizes handwritten mathematical expressions and supports flexible manipulation, solving, and visualization of those expressions. While recognition accuracy proved strong, recognition latency emerged as a key limitation affecting usability and workflow. The findings suggest that HMER-powered whiteboards are a viable and promising tool for mathematical learning and exploration, particularly in educational contexts, though future work must focus on improving recognition speed through more efficient models or more powerful hardware to enable seamless real-time interaction.

Location

Furman Hall 109

Start Date

3-28-2026 9:45 AM

Presentation Format

Oral and Written

Group Project

No

COinS
 
Mar 28th, 9:45 AM

An Interactive Whiteboard that Leverages Mathematical Handwriting Recognition to Assist in Understanding Math

Furman Hall 109

Handwritten Mathematical Expression Recognition (HMER) remains a longstanding challenge in computer science due to the two-dimensional structure, symbol ambiguity, and spatial relationships inherent in mathematical notation. While recent advances in machine learning have substantially improved off-line recognition of static handwritten expressions, a gap persists between state-of-the-art HMER models and their integration into real-time, user-facing applications. This study investigates whether an HMER-enabled interactive digital whiteboard can effectively support the solving, graphing, and visualization of handwritten mathematical expressions in real time. To address this question, a web-based digital math whiteboard was designed and implemented using the VueJS frontend framework. The system integrates the TexTeller model for handwritten expression recognition, the Desmos API for graphing, and symbolic computation libraries (Nerdamer and Math.js) for solving and manipulation. Users write mathematical expressions directly on a canvas using stylus or mouse input, select regions for recognition, and receive immediate typeset expressions in the form of interactive widgets. These expression widgets can then be converted to graphs or solutions. Results demonstrate that the application successfully recognizes handwritten mathematical expressions and supports flexible manipulation, solving, and visualization of those expressions. While recognition accuracy proved strong, recognition latency emerged as a key limitation affecting usability and workflow. The findings suggest that HMER-powered whiteboards are a viable and promising tool for mathematical learning and exploration, particularly in educational contexts, though future work must focus on improving recognition speed through more efficient models or more powerful hardware to enable seamless real-time interaction.