Incorporating AI to SenseBound

June 6, 2025

SenseBound is taking a big step and incorporating AI into our games. We believe that with the use of technology and AI, blind students will be able to learn programming even more efficiently without needing an actual computer science teacher to guide them through every single step. With the guidance of Professor Ramayya Krishnan, the Dean of Carnegie Mellon’s Heinz College, SenseBound is working on the digital section of this project.

Currently, the job of AI in the game is to read and analyze the player’s moves, including their efficiency and the steps taken to reach the target. We are testing the data analysis capabilities of OpenAI, which are currently working well. By focusing only on the digital side of the game for now, we are testing move analysis and adjusting it based on inputs made online. Possible options are “up,” “down,” “left,” and “right.” Once the AI is finalized, the goal is to connect the two components, the board game and the AI, using sensors on the board as well as some kind of link between the board and an application accessible on devices. As seen here, the plans for future development are still vague, since we are currently focused on refining the AI.

Furthermore, the OpenAI Large Language Model (LLM) is capable of giving feedback based on the player’s moves. Right now, there are generally two types of feedback: feedback after every single move, hinting whether the player is getting closer or farther from the target, and the second type, where the AI summarizes the player’s moves at the end and gives overall feedback. The style we are going for is similar to how teachers give comments on small assignments along the way compared to feedback after a test covering all the topics. Currently, the AI is able to analyze the moves clearly, but we are still working on finding ways for the AI to give the perfect feedback, one that doesn’t give away too much of the solution but still provides clear and helpful hints.

We are meeting with Professor Krishnan one to two times every month, currently updating him with our progress and getting feedback to grow even more with this project.