Students: Ian de Boom, Jaap Knoop, Zarif Osmani and Diederik Ploem
KLM Catering Service
KLM Catering Service (KCS) is responsible for the production, packaging and delivery of food for all planes of KLM. For the business class passengers food is delivered on the planes using trolleys which contain trays with a variation of dishes. These trays with their corresponding dishes are prepared by hand at the airport, on a busy day this can add up to 3000-4000 trays per day.
The operators that produce the trays are repeating the same repetitive tasks for 8 hours a day, over the years this can contribute to minor or in some cases major health problems. To combat this, KCS wants to lighten the load of their operators by using a collaborative robot to support them.
The project
KCS has little knowledge about the applications and work process of cobots. For that reason KCS has given us the objective to build a proof of principle (see image 1), using a cobot, that is able to produce the trays. The proof of principle needs to be able to produce a variation of trays, which come with different sizes and shapes of dishes. Using machine vision the dishes will be detected and pick and placed using the cobot.
Besides the proof of principle, KCS also wants a detailed advisement about the obstacles we faced and the choices we made. So that KCS will be prepared in the future to start their own project using cobots.
Proof of principle
Here below in image 1, a diagram of the proof of principle can be found.
(image 1: Proof of principle)
To be able to give KCS a definitive advise, it was decided to work out multiple solution to be able to differentiate and choose which is best applicable. For that reason we worked out 3 different kinds of machine vision; using a Cognex camera, using OpenCV and using machine learning. In the final proof of principle only the Cognex camera and machine learning are used, so only these will be explained here further.
Cognex camera
Cognex is a camera with integrated vision software which allows the user to detect objects without or with minor programming knowledge. Underneath the Cognex camera, there is a basket filled with empty dishes. Using the Cognex software we can detect them and determine their centerpoints, which we can use to pick them up.
MAchine learning
Machine learning is a software practice in which you train a model to be able to predict certain tasks or information, in this case its used to differentiate between the dishes. . By training a model using the dishes for each tray, the camera now has knowledge which dish to pick up. For that reason it doesn’t matter what dishes are placed where under the camera, it can detect the correct one and pick it up.
The Grippers
After the detection by machine vision, the robot needs to pick up the dishes. Because there is a big variation between the shape and sizes of the dishes, it isn’t possible to pick everything up with only one gripper. Which is why three different grippers (see image 2 here below) are designed and fabricated, these will be explained furthermore here below.
image 2: The grippers
Gripper 1
Gripper 1 is a mechanical gripper. Out of all the dishes there are two which have the same size, which means they can both be picked up with the same gripper. The gripper works with a Festo gripping module, which under pressure extends or retreats. On this module we designed gripping arms, which hold the dishes on the side, to be able to grip the two dishes.
Gripper 2
Gripper 2 is a suction cup. Some dishes can be empty, for example the dishes with hot food, the hot food is placed on board on the plane. Because there is no food on this dish, it can be picked up using a vacuum sucker.
Gripper 3
Gripper 3 like gripper 1 is an mechanical gripper. As apposed to gripper 1, this gripper is fitted with a spring between its Festo gripping module. With two different pressure settings the spring gets compressed in two different ways, which allows it to be able to pick up two different sizes, with the lower pressure a higher size and vice versa.
Advise
The advise that was delivered to KLM, is that a project using Cobots has high potential. There are a lot of repetitive tasks the operators need to do which would easily be able to do by a Cobot.
Apart from that it has been concluded that machine learning is the best applicable machine vision method. It is able to make a clear distinction between the dishes and delivers a point of center which is accurate enough. It does require some programming knowledge (if yolov5 with Roboflow is used) but not as much as writing your own complete program.
For the grippers, it is advised that KLM seeks contact with companies which focus and sell grippers for robot applications. A gripper which also has high potential are soft grippers, these are flexible grippers which can move around an object. Because of the great variation in sizes, this might be a solution.