An Open-Source Reproducible Chess Robot for Human-Robot Interaction Research

arXiv:2405.18170 · cs.RO, cs.HC · Submitted 2024-05-28 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Today's paper: "An Open-Source Reproducible Chess Robot for Human-Robot Interaction Research".

Dev: An open-source, reproducible chess robot for human-robot interaction research is presented, integrating computer vision, chess engine evaluation,

Rosa: First, who's behind it and why it matters.

Paper summary: Dev: So, thinking about the full picture of "An Open-Source Reproducible Chess Robot for Human-Robot Interaction Research," it really seems like this work is significant because it provides a blueprint that others can use to replicate and study HRI effects in a very controlled manner.

Rosa: I agree; the title itself emphasizes that reproducibility is key, which means the open-source nature isn't just about sharing code, but about establishing a reliable methodology for testing these specific interaction models.

Taro: What I see as important is how they bridge the gap between complex AI algorithms and observable human responses, giving us data on those perceptions they’re measuring with the five hundred ninety-seven participants <ref:2405.18170#pg2>.

Dev: And from an engineering viewpoint, the paper's contribution lies in detailing a specific architecture—Perception through ArUco markers to Analysis through Stockfish—that you can actually follow and try to build upon for your own interaction studies.

Rosa: The implication for the broader research field is that chess isn't just a game; it’s a powerful, standardized tool for creating measurable data on how embodied AI influences human decision-making in structured settings.

Taro: It suggests that future work should focus on extending this to more dynamic, unstructured environments where the system has to react not just to the board state, but to unpredictable human behavior as well.

Dev: I think for practical application, we need systems that can manage those latency issues and failure modes they mentioned so that the interaction feels fluid rather than broken during gameplay.

Rosa: So, ultimately, "An Open-Source Reproducible Chess Robot for Human-Robot Interaction Research" offers a concrete platform to rigorously test the social dynamics between humans and AI in a way that is both controlled and transparent.

Conclusion: Rosa: So, to wrap up this discussion about "An Open-Source Reproducible Chess Robot for Human-Robot Interaction Research," we've seen how this platform connects computer vision, chess engines, and direct human feedback to study AI behavior in action.

Dev: Yeah, it’s definitely a solid setup on the hardware side, but I’m still thinking about how reliably that whole loop runs when you get real-world input; the latency between seeing a move and the robot executing it is something we need to nail down.

Taro: From my angle as someone who looks at autonomy, this isn't just about playing chess; it’s about how a system with physical presence and complex decision-making communicates its strategy non-verbally, which opens up a whole new way to model human perception of AI.

Rosa: Exactly; the authors are giving us a clear roadmap here for anyone wanting to use this as a testing ground, and I’m curious if this kind of controlled environment could translate into more complex physical interactions outside of just chess.

Dev: If we can stabilize the execution loop like they’re aiming for, then maybe we could push these systems into more dynamic physical tasks where the robot has to adapt its strategy on the fly, which is where I see the real engineering challenge.

Taro: And that adaptability is what makes it interesting; when things get messy or unpredictable in a physical interaction, how does this AI system handle those deviations from a perfect plan?

Rosa: It seems like they’re setting the stage for us to really dig into those failure modes and see if the human reaction changes based on how well the robot handles unexpected situations.

Dev: I think that’s what makes it valuable; we need to understand not just when it works perfectly, but precisely what happens when its perception or planning module hiccups during a game.

Taro: That opens up avenues for developing more robust AI that can manage ambiguity in physical environments rather than just following pre-programmed chess sequences.

Delft University of Technology

cs.RO, cs.HC

Submitted: 2024-05-28

Updated: 2025-04-04

Journal ref: Frontiers in Robotics and AI (2026)

DOI: 10.3389/frobt.2025.1436674

Code: https://github.com/renchizhhhh/OpenChessRobot

Project page: https://frankaemika.github.io/docs

License: http://creativecommons.org/licenses/by-nc-sa/4.0/

Importance score: 83/100

The gist: An open-source, reproducible chess robot for human-robot interaction research is presented, integrating computer vision, chess engine evaluation, and both verbal and non-verbal interactions to study

Key concepts

Franka Emika Panda robot arm
This is the physical robotic arm used by the system, featuring seven degrees of freedom. It is equipped with a Franka Hand, which allows the robot to physically interact with and move pieces on the chessboard during gameplay.
ZED2 StereoLabs camera
This camera provides visual input for the robot. It is used by computer vision modules to capture images of the chessboard, enabling the system to perceive where pieces are located and identify their types.
Stockfish 15 (Stockfish, 2022)
This is a powerful chess engine integrated into the robot's software. It analyzes game positions by predicting the best moves and assigning scores to candidate moves based on established chess principles.
Verbal and non-verbal interactions
The robot communicates with humans in two ways. Verbally, it uses ChatGPT to explain strategies in a tutor-like tone. Non-verbally, it uses physical gestures like nodding or shaking when its evaluation of the game changes significantly.

Terminology

Summary

An open-source, reproducible chess robot for human-robot interaction research is presented, integrating computer vision, chess engine evaluation, and both verbal and non-verbal interactions to study how embodied AI influences human behavior. The gist is that the OpenChessRobot is an open-source platform designed to evaluate the impact of a robot's behavior on humans through verbal and non-verbal interactions in a controlled chess environment.

Hardware and Software Architecture

The robot's physical setup consists of several key components, including a Franka Emika Panda robot arm equipped with a Franka Hand, which has seven degrees of freedom. For sensing, it utilizes a ZED2 StereoLabs camera. The computing power is provided by an NVIDIA Jetson Nano running on a Linux PC with Ubuntu 20.04 and an Intel I7-8700K processor. Communication between the arm and the control PC is managed through the Franka Control Interface (Franka Robotics GmbH, 2023) using the Libfranka library and ROS Noetic.

The software architecture is built upon ROS Noetic, divided into four core modules: Perception, "Analysis & Evaluation," "Motion Planning & Execution, and Interaction." The Perception module uses the ZED2 camera SDK to capture images and employs a neural network-based chess identifier to translate game images into text descriptions. The Analysis & Evaluation module feeds the game annotation to a chess engine in order to get predicted moves and their corresponding scores.

Perception Module

The ability of the robot to perceive the chessboard is crucial, utilizing computer vision methods. This module consists of two distinct classifiers: one for occupancy and one for piece classification. To train these classifiers, researchers synthesized chess positions in the NVIDIA Isaac Simulator based on previous games and assigned ground truth labels in Forsyth-Edwards Notation (FEN) and pixel coordinates.

The system uses four ArUco markers to indicate the 3D position of the chessboard, allowing for 3D board localization. A non-linear least squares algorithm is used to optimize a grid corresponding to the chess squares based on marker positions. Piece detection is an extension of a CNN-based model, fine-tuned using both synthetic and real-world datasets. The model was trained on 5,000 game positions synthesized from grandmaster games and further adapted by manually iterating over all pieces on the board to create a piece-square dataset.

Analysis & Evaluation Module

The processed chess FEN is forwarded to a chess engine wrapper that utilizes the Universal Chess Interface (UCI) protocol. The system integrates Stockfish 15 (Stockfish, 2022) as the default engine, using ten CPU threads. The chess engine assigns scores to candidate moves, which are monotonically related to the player’s win rate.

The robot's move execution is guided by the predicted move and its 3D chessboard localization results, which are used by the MoveIt! motion planner (Coleman et al., 2014) to plan and execute chess moves. The system also incorporates a legality check that verifies if a game position conflicts with chess rules, requesting new images or changing camera angles if the predicted state is illegal.

Interaction Modules

The robot implements an interactive gameplay pipeline to manage verbal and non-verbal feedback. For verbal interaction, the system connects ChatGPT to the chess engine output via a prompt wrapper. This allows the robot to explain strategies by feeding it the move history and the current game FEN and generating responses in a tutor-like tone.

Non-verbally, the robot expresses its evaluation through posture feedback. It enacts nodding or shaking gestures when the reduction in win probability due to the latest move exceeds a predefined threshold. The system is also programmed to interpret human behaviors, such as gazing or asking questions, to provide a responsive and engaging experience.

Evaluation of People’s Views

The robot's efficacy was assessed through two main methods: an interview with an expert chess player and an online survey of 597 participants across six countries. The online study used a between-subjects design, presenting participants with three demo videos representing Robotics Education, Chess Coach, and Home Entertainment scenarios.

The results showed that acceptance was highest for the Robotics Education scenario, where the robot was praised for its Advanced technical ability. The Chess Coach scenario received high acceptance, noted as an “effective teaching approach,” but participants mentioned “limited interactivity” as a downside. The Home Entertainment scenario received the lowest scores, with concerns focused on its inability to replace screen-based entertainment and its lack of human-like emotional elements.

Performance Metrics

The study also measured specific performance metrics.

Improvements for AI systems

Here are specific improvements for AI systems based on the OpenChessRobot paper, focusing on enhancing its capabilities in Human-Robot Interaction (HRI) research and application:

  1. The current system relies heavily on a fixed set of chess pieces and a pre-defined environment setup (ArUco markers).

  2. The AI system can be improved by integrating a more dynamic, generalized perception module that utilizes self-supervised learning or meta-learning to rapidly adapt piece recognition and 3D localization to novel, uncatalogued board setups and piece variations without extensive retraining on new datasets.

  3. The robot's current verbal interaction relies on a static prompt structure with ChatGPT. This can be upgraded by fine-tuning Large Language Models (LLMs) specifically on chess strategy databases (e.g., PGN files combined with expert commentary) to move beyond general knowledge and provide genuinely deep, context-aware strategic analysis, similar to the suggested need for explainable AI in the discussion section.

  4. The non-verbal feedback mechanism (nodding/shaking) is currently binary based on a fixed threshold of win probability reduction. This can be enhanced by implementing a reinforcement learning (RL) framework where the robot learns an optimal expressive strategy based on the human opponent's observed cues (e.g., gaze, facial expressions if integrated later), allowing for nuanced, context-sensitive emotional signaling rather than simple pass/fail indicators.

  5. The motion planning and execution module can be improved by incorporating predictive modeling of human movement or intent (if eye-tracking data is integrated). This would allow the robot to anticipate the human player's next move and proactively adjust its trajectory to be ready for a capture or defense, increasing perceived responsiveness and fluidity beyond current reactive planning.

  6. The system's performance in physical manipulation (grasping) shows sensitivity to piece placement (edge vs. center). The grasping AI can be improved by integrating tactile sensing or vision-based grasp quality assessment models that dynamically adjust the gripper force and approach based on real-time visual feedback of the piece's exact orientation and position relative to the gripper's tolerance zone, aiming for near 100% success rates across all positional shifts.

  7. The system should incorporate an explicit Legality Verification Module that uses a symbolic representation of chess rules (rather than relying solely on image classification) to pre-filter predicted moves from the chess engine wrapper, drastically reducing unnecessary computation time and improving robustness against perceptual errors, as suggested in Section 3.1.2.

  8. The overall platform should be enhanced by developing a modular middleware that allows for easy swapping of perception models (e.g., switching from RGB CNNs to point cloud processing) and interaction backends (e.g., replacing the ChatGPT wrapper with other generative AI models), directly addressing the need for an open-source reproducible platform that is adaptable to future advancements in AI, as highlighted by the paper's primary goal.

Abstract

Recent advancements in AI have accelerated the evolution of versatile robot designs. Chess provides a standardized environment for evaluating the impact of robot behavior on human behavior. This article presents an open-source chess robot for human-robot interaction (HRI) research, specifically focusing on verbal and non-verbal interactions. The OpenChessRobot recognizes chess pieces using computer vision, executes moves, and interacts with the human player through voice and robotic gestures. We detail the software design, provide quantitative evaluations of the efficacy of the robot, and offer a guide for its reproducibility. An online survey examining people's views of the robot in three possible scenarios was conducted with 597 participants. The robot received the highest ratings in the robotics education and the chess coach scenarios, while the home entertainment scenario received the lowest scores. The code is accessible on GitHub: https://github.com/renchizhhhh/OpenChessRobot

Sources

Related papers