Authors: Thomas van Mil, Steven Vijverberg, Barry van Leeuwen en Marwin Bokkers
This article describes the broccoli project that ran during the Smart Manufacturing and Robotics 1 (SMR1) minor of fall 2016. This minor is given at The Hague University of applied sciences and technology in Delft, by coordinator Thijs Brilleman.
The project members are bachelor students of the earlier mentioned school one from Mechatronics Thomas van Mill, and three Mechanical Engineering students Steven Vijverberg, Barry van Leeuwen en Marwin Bokkers. The project assignment is attained from industry in this case a broccoli grower.
The client René Verdonk is a broccoli grower in the Netherlands. At this farm a lot of the work is done by manual labour. The broccoli is harvested and put in boxes of a cubic meter in volume. Then it is cooled and stored before packaging. At this moment the boxes are tipped on a sorting belt. Next workers pick the broccolis from the belt. They check size (weight), quality and for debris. Then they orient it in a specific way to prepare it for packaging and weighing. After labelling the broccolis are boxed ready for shipping to supermarkets or auction houses.
Our client asked the project group to concentrate on pre-sorting the broccoli by size (weight estimation) before it is actually packed and weighed. The reason here for is that the broccoli when packed is not as long-lasting as when stored unpacked. It is mainly a study of the possibilities of vision application.
The supporting personnel from the minor, Mathijs van der Vegt and Guus Paris already started on sorting broccolis with the use of vision applications. The database and program that resulted from their work was used by us for learning the In-Sight Explorer software. It was also a starting point for an improved sorting program.
Analysis
We started with an analysis of the relevant production line. This resulted in the figure underneath.
| Legend | 4 Conveyor belt for packing machine |
| 1 Container Tipper | 5 Conveyor belt for packing machine |
| 2 Conveyor belt 1 | 6 packing machine |
| 3 Conveyor belt 2 | 7 packing machine |
The pre-sorting should be somewhere between conveyor 2 and the packaging machines.
The black box that describes our system shown below.
Inside this black box we distinguished between inspection and sorting inspection contains the detecting an analysis of the broccolis while the sorting positioning and the orienting the broccolis. The system is displayed in the flow diagram underneath. The scope is shown inside the red framework.
Design
The project group was assigned a Universal Robot (UR10), a cognex camera and a Siemens PLC. To be included in tot the design of the prototype for educational purposes.
The next step in the project was the rough design of out prototype setup. This was straight forward, a conveyor with a light box for the camera and on the end the UR with a gripper attached. This design was cut op in pieces for each member a piece to focus on
We defined:
- Vision;
- Communication and interface (PLC);
- Robotic programming ;
- The gripper.
Vision
The vision is part of the project is very important as it give all the information needed by the robot to operate. It also includes the sorting program that differentiates between weight classes.
The start was finding the correct lens and camera height to get and photo of the correct size that a broccoli always would fit.
Second was the lighting to get the clearest picture and thus clearest information in all circumstances. There was a light box created to filter out all false light and to gain a constant amount of light with each picture.
Next was giving the light a filter to let only the green light through and make the light bundles diffuse.
The programming side of the vision contains the standard blocks that are available in the In-Sight explorer software from Cognex. Using these made it possible to distinguish between the flower and the stem of the broccoli. Of these two distinct parts of the broccoli are parameters taken, for instance the area perimeter and width.
Using a database and logistic regression we made an algorithm that made it possible to reach higher weight sorting accuracy than that was initially reached at the starting point of the project.
Other information that is attaint form the software is the placement of the broccoli on the conveyor, the main interest was in de position of the broccoli on the width (x direction) of the conveyor. And the angle it had. These data points together give the pose of the broccoli. With this the robot knows where to pick it up, and how to turn it for the packaging line.
PLC
The PLC was used as server. Is controls the conveyor, ties the camera and the robot together and contains the Human Machine Interaction (in other words the start and stop button).
Robotic programming
The robotic program is used to interpret the data give to it by the camera, conveyor encoder and gripper to find the broccoli and to activate the gripper. It is in a sense the only active component of the system.
Gripper
The first idea of the project group was to pick the broccolis up by the stem. But on analysing the different shapes that broccolis grow in and are cut to at harvesting, concluded that this would not be a robust solution. As the next figure demonstrate, the first one clearly has a stem were as the last one has not.
The following research of the existing possibilities led to the field of soft robotics. This is in the basis a very compliant type of robotics constructed of flexible material as silicone. The way in which we use this is with silicone and air pressure.
Basis of this design are four digits called tentacles that are constructed in a way that when they are pressurized they curl in a specific way.
The design above describes this is the green material; it is flexible and elastic, while the red part is constructed of a flexible but non-elastic material.
Four of these tentacles are mounted on a mounting plate, and this plate is also bolted to the robot. Last part of the gripper is a sensor, to measure the height of the gripper to the broccoli when this sensor is activated it activates a valve to pressurize the tentacles these then grip the broccoli form the conveyor.
End result
We ended up with a system that is able to distinguish the size of the broccoli and is able to pick the irregular shape up. And put it in the correct way for further processing.
