Students: Gijs van Haeff, Jip Rasenberg, Marten Haaksema and Wendy Exterkate. (2022)
Introduction:
BSL Bricks is a LEGO store in Hardinxveld-Giessendam in the Netherlands. They buy second-hand LEGO from all over Europe and sell it in their store or on BrickLink.
When BSL Bricks buys the LEGO, it is often a big container full of all kinds of LEGO. They sell it by brick, so the LEGO gets sorted in their store by hand. They believe this can be done faster and would like to have a machine to sort the second-hand LEGO. They would like to have the LEGO sorted into 5 categories, blocks (2 by something bricks), bricks (1 by something bricks), sheets (plates of 2 by something or bigger), plates (1 by something plates) and tiles (flat plates or sheets).
Difficulties:
There were several challenges while making the machine. The machine was to be made as inexpensive as possible. This meant the first concept of multiple conveyor-belts was too expensive. So it was necessary to get creative in order to make the machine.
The different sizes, very large or super small, of the LEGO also proved to be a problem. If everything was put in the machine, it would jam the machine. It was decided to sort the LEGO in size before putting it in the machine. A wooden sif to sort the LEGO in 6 different sizes was made.
To recognize the LEGO machine learning was used. The first cameras were to cheap and could only make blurry pictures. More advanced cameras were ordered, which were able to take clear enough pictures to train a machine learning model.
After we made the pictures we started annotating them in Roboflow and train them in Google COLAB. After making a first model a second model was made with different settings and more pictures. Unfortunately the accounts of both sites got blocked due to being a big user. New accounts were made, but all 2.500 images had to be annotated again.
Another difficulty of machine learning was the model. The used model made with YOLOv5, but after some time it became clear YOLOv5 was not the best model to use for this problem. Due to time limitations it was decided to continue with this model.
Our solution:
Our solution consists of 2 solutions, a sif and a machine. Before putting the LEGO in the LEGO sorter it is sorted by size by using a manual sif.
In the HMI you can decide what category of LEGO you want to sort and in which container.
The LEGO machine consists of 4 components, a hopper, a feeder, a vision box and a separation station.
The LEGO is poured into a hopper which will push it upwards in batches via a small conveyor-belt. To make the machine low-cost a LEGO conveyor-belt was used.
From the hopper it falls into the feeder. This contains two vibrating v-gullies, which singulate the LEGO. They vibrate thanks to small motor with an off-centre mass and leaf springs. In between the gullies, slides are mounted to guide the LEGO from one gully to the other. The feeder delivers the singulated LEGO to a conveyor-belt. Two IR sensors start and stop the hopper and feeder to provide distance between LEGO and a constant feed of LEGO. This ensures that only one LEGO block enters the vision box at the same time.
When the IR sensor in the vision box is triggered, pictures of the LEGO will be taken with 3 different cameras. All three pictures are put in a machine learning model. All pictures are classified and compared. If all three pictures are classified as the same piece, it is easy to define what category this brick is. If the three classifications are not the same, the classification with the highest percentage is taken. Anything else will be classified as unrecognised.
If the LEGO is recognised as a certain category, the separator of the corresponding container indicated by the HMI will open and grab the LEGO. The separators are clad in TPU to maximize the grip of the separator. If the LEGO is not selected to be sorted in a container or is not recognised it falls of the end of the conveyor-belt into a miscellaneous container.
Recommendations:
This machine is a proof of concept that can also be used immediately in the store. Though this machine can definitely be upgraded.
First of all, the machine learning model isn’t optimal and can be improved by adding more pictures and using another more suitable model like a convutional network.
Secondly, the feeder works well but has difficulty with tires because of the frictional difference. An optimazation is required in order to make the machine truly universal. It is also beneficial to do a more in depth investigation about the vibrations of the feeder. Sometimes the feeder has a cluttered up batch of LEGO or a big gap.
Thirdly the belt of the conveyor has to be replaced. The standard black belt did not go well with the taking of the pictures, so a white film was glued onto the conveyor. This film is prone to cracking, so this is not a long term solution. The solution would be to replace the belt on the conveyor with a white one.
The last recommendation is improve the separation station. The separators sometimes don’t grab the LEGO well enough and they lose the LEGO or the LEGO gets stuck in between the separators.
Conclusion:
Despite the fact we ran into some problems we were able to make a machine which can sort LEGO into the five main categories asked by the client. We are happy to provide BSL Bricks with a machine that can sort their second-hand LEGO, so this doesn’t have to be done by hand anymore.
Epilogue:
We would like to thank BSL Bricks for giving us this great project. Likewise we would like to thank our supervisor Jan Baan for his great input to this project. Jan is really creative, and experienced as we have noticed and it has been great working with him throughout the whole project. Also we would like to thank the staff of the workshop from the Hague University of Applied Sciences for using their facilities and tools during this project. And of course we would like to thank our teachers Thijs Brilleman, Mathijs van der Vegt and Guus Paris for their technical support and feedback.