Students: Shiyar Jamo, Ixent Cornella Vilana, Guillermo Forcén Ordovás
The Smart Manufacturing and Robotics minor (SMR) from The Hague University of Applied Sciences (THUAS), Delft faculty, announces the release of a demonstration robot to promote the minor and attract more students.
The problem:
The SMR minor staff intends to increase the amount of students enrolled in their minor. It has been found that the best way to achieve this result is to design and implement a promotional robot setup that will be located at the entrance of the campus.
The robot needs to be interactive with the user, it needs to be eye-catching. Two Kawasaki robots, FS03N and FW03N, are available to use for the project.
The Solution:
To address the need for an user-friendly and interactive setup, a two robot interactive system has been developed.
First, we began designing a setup that would be as attractive as possible, with maximum visibility and mobility. To achieve this, we decided it should have wheels (which could be raised to ensure the structure remained completely stable) and curved sides to allow maximum visibility from the greatest number of positions.
Next, we assessed the costs of materials and delivery times. Realizing that it wasn’t entirely feasible to complete the entire project within the available time, along with our client we decided to focus on creating a provisional setup and set aside the construction of a professional setup. This would allow us to focus on programming the robot accordingly, as we set aside the idea of building a complete setup for the upcoming time. The original concept can be seen in figure 1, and the final result for the provisional setup can be seen in figure 2.
Figure 1: ”Promobot” setup concept, designed in late April 2024, later scrapped.
Figure 2: ”Promobot” provisional set up, finally made in June 2024.
For this project, we started with the premise that we had to use the Kawasaki FS03N and FW03N robots. To make the robots more versatile, we needed to find end effectors that were both visually appealing and allowed us to interact with the spectators—in other words, the future students of the minor. Therefore, we decided to choose a robotic hand capable of playing rock-paper-scissors and a camera to facilitate interaction with the audience.
To further enhance user-friendliness, a custom UI screen has been developed. This UI has some possible functions which upon selection them activates different robot modes. Each possible operation for the robot is explained with detail in the next section.
How the robots operate:
The setup consists of two Kawasaki robots, a TV, a laptop with a touchscreen, an Arduino controller, a robotic hand and an Intel Depthsense camera. The laptop works as the server and interactive touch screen, connected to both robots, the camera, a TV, and an Arduino.
For the movement of the robots, the laptop must be powered on, and a Python file must be executed through the user interface (UI). In the case of the robotic hand, it must also be connected to the Arduino. This is because to move the robotic hand, the Python code needs to trigger the code embedded in the Arduino.
A user interface has been programmed to include the following functions:
- Rock-Paper-Scissors game: The right robot does the rock-paper-scissors motion three times, and then the robot’s choice is seen on the TV and the hand does the selected pose.
- Dance mode: The robots can dance in a synchronized motion, to the beat of a family-friendly
song. - Pong game (optional feature): The robots can play a classic game of pong if the voice command ”Play Pong” is called. This feature is labeled as optional because it was not planned to be developed, but it was kept as an ”easter egg”.
- People detection: With the use of AI and the Intel camera, the detection of people in front of the robot can be done and shown in the camera for the user to see itself.
- Movement detection: With the use of AI and the Intel camera, the user can see their exoskeleton outlined on the TV.
- Feeling detection: Again, using AI, the detection of people’s emotion is done for the user to interact again with the robot.
- TV Mode: Finally, we have the TV Mode, which makes both robots go to the corner and straighten themselves, so that the TV is fully visible and advertisements can be shown.
The other functions that are not written on the touchscreen but are rather built implicitly are:
- Speech Recognition: We’ve implemented speech recognition and voice commands so that the robot can be used without using the touchscreen. Captions are also written down on the TV.
- Emergency mode: An emergency mode (which puts the robot on hold) can be pressed and also called with speech recognition, and then unpressed so the robots can resume their motions.
- Hello robot: We have implemented the ’Say Hello’ feature in the robot. Now, the robot
automatically greets people when it detects someone saying ’hello’.
Figure 3: ”Promobot” custom created UI.