Sign in to follow this  
Hanso

[EV3] Automatic candy sorter (needed for Lego Warehouse)

Recommended Posts

For the large candy container warehouse that I am building (click here to read more details), I need three more builds:

  1. A candy feeding mechanism. I need a mechanism that feeds one candy at a time to a conveyor belt.
  2. Color detection. The color of the candy needs to be determined.
  3. Systems that puts the candy into a container.

The container can then be conveyed to the warehouse where it can be stored sorted by color. The color as determined in step 2, needs to be passed to the warehouse brick so it can store the container on the right level.

Color detection (step 2)

I started with step 2. To realize the candy sorter, I needed to know if a Pixycam (see www.charmedlabs.com/default/pixy-lego/) can distinguish the colors of the candy. The colors are not bright (like the Lego bricks), so I wasn't sure if it was possible.

I created a quick setup to test the color detection:

36440858165_c97cbe660f_c.jpg

Next, I placed the four different candies (Fruit-tella) on the conveyor belt and programmed the camera for the four colors.

This is the image of the camera, from left to right: (1) raw video, (2) cooked video (= mix of raw video + color detection) and (3) detection only video.

36272525442_1e86477e78_n.jpg  36303826111_6ef263de73_n.jpg  36272525312_9226cf5eae_n.jpg

Finally, I made a simple test EV3 program. With the addition that the detected block should be more than 20 pixels in width, it detects the candies in the correct color.

 

Candy feeding mechanism

Ok, so I have a conveyor belt with a candy on it. But how do I get the candy, one by one, on the conveyor belt. This is where the feeder comes in. I searched the internet for 'real life' solutions. I found out that a rotary feeder or bowl feeder would do best. I tried to make a vibratory bowl feeder (out of 16 circle gear rack elements 24121), but that didn't work. However, rotating the bowl seemed to work so I made a prototype based on rotation and gravity. The bowl is placed at an angle of +/- 20 degrees and then it just rotates in a steady pace.

The result was good: it never happened that two candies were fed to the conveyor at the same time.

The next step is to make a more solid version and add the color detection unit to it. I'll keep you posted.

Enjoy, Hans

Edited by Hanso

Share this post


Link to post
Share on other sites

The feeder should ensure that 1) there will be never two candies released at the same time and 2) that there will be a (kind of) constant pace in the feed. I have tested different configurations of the bowl.

36334329552_965ec087d7_c.jpg

On the video, you can see that most important change is the rotation direction (now anti-clockwise). The last configuration (number 7, also on the picture above) gives the best results until now.

Enjoy watching and drop me a note if you like it.

Best, Hans

Share this post


Link to post
Share on other sites

More photos of the work-in-progress just uploaded to Flickr (click on photo to go there) and a video to Youtube.

35740888253_364718734e_c.jpg

Enjoy, Hans

Share this post


Link to post
Share on other sites

I'm really excited to see this when finished!

seems to work quite well when full up with candy.

how will you keep feeding the candy sorter? is that the train setup etc? 

 

 

Share this post


Link to post
Share on other sites
4 minutes ago, MangaNOID said:

I'm really excited to see this when finished!

seems to work quite well when full up with candy.

how will you keep feeding the candy sorter? is that the train setup etc? 

 

 

Thanks MangaNOID.

 

With respect to your question: no, the idea that I have is that the bowl will be filled by hand, but from that point everything goes automatic.

After the color has been determined, the candy will be placed in a container and then conveyed to the warehouse where it will be stacked.

The warehouse delivers the candy on request to the pickup station, where the train will be loaded.

The warehouse + candy sorter will not be part of the layout 2017.

Edited by Hanso

Share this post


Link to post
Share on other sites
14 hours ago, OneMoreRobot said:

After viewing three prototype videos - I'm now dizzy ? 

Ha ha. You can rotate a bit more when you watch my latest video:

 

Share this post


Link to post
Share on other sites

The configuration and calibration off the Pixycam camera is one of the most important parts in the reliability of the automatic candy sorter. For example, take a look at the image that the Pixycam reads:

36412152372_3fff7d797b_z.jpg

 

The camera should read a red candy (color 2, in my case). Since this is the largest object, it will result in a positive reading. However, if the red candy would not be in place (yet), the camera would probably read the reflection as a blue reading (color 4), i.e. it would result in a false reading of a blue candy. The blue color is the reflection on the roof tile bricks. I found out that the (reddish) brown color gives the least reflection, so I ordered some more reddish brown bricks.

Another way to minimize the reflection, is the small roof (in gray, but will also be replaced by reddish brown bricks). On the other hand, it should not be to dark inside, so I've kept some openings to get enough light for the reading.

35726188264_fe3b22c323_c.jpg

Last but not least: when I get a reading from the Pixycam camera, I multiply the height and the width of the reading. The object should be 'large' enough (I have now set the threshold to a minimum of 1000, that seems to work fine) to distinguish the candy from background noise.

 

35771459613_00f308df25_c.jpg

As you can see on the picture above, the candy is aligned to the center of the conveyor belt. This makes sure that the camera gets the 'largest' surface to scan (which again is needed to meet the threshold of 1000).

 

The combination of the threshold, the alignment of the candy to the center of the conveyor, the brown roof tile bricks and the roof, result in almost 100% positive readings and 0% false readings.

The video below shows the first test run:

 

I have now ordered the missing bricks / correct color bricks. I will do a second test run when they have arrived and keep you posted.

Enjoy, Hans

Edited by Hanso

Share this post


Link to post
Share on other sites

Very interesting to watch advances. I still think a better way to sort can be done, but it works, so I will hold back my thoughts.

Share this post


Link to post
Share on other sites
4 hours ago, Aventador2004 said:

Very interesting to watch advances. I still think a better way to sort can be done, but it works, so I will hold back my thoughts.

Thanks. And please don't hold back your thoughts. Always interested in a discussion about other possibilities.

Regards, Hans

Share this post


Link to post
Share on other sites
Just now, Hanso said:

Thanks. And please don't hold back your thoughts. Always interested in a discussion about other possibilities.

Regards, Hans

I think I may have an idea then. I will think on it.

Share this post


Link to post
Share on other sites
On 8/15/2017 at 6:53 PM, Aventador2004 said:

I think I may have an idea then. I will think on it.

Hi Aventador2004,

Any ideas to share?

For myself, I saw two other possibilities (both copied from the GBC's):

1) A mechanism like the axle sorter of Akiyuki (see timestamp starting at 0:20):

 

2) A mechanism like the rotating filter of BrickIT (see timestamp starting at 1:43):

 

I am really curious about the other possibilities you had in mind.

Regards, Hans

Edited by Hanso

Share this post


Link to post
Share on other sites

O will try and make a ldd, if I don't respond, it didn't work.

Definitely try the first one to see.

Share this post


Link to post
Share on other sites

In the starting of this thread, I mentioned three builds to be engineered:

  1. A candy feeding mechanism. I need a mechanism that feeds one candy at a time to a conveyor belt.
  2. Color detection. The color of the candy needs to be determined.
  3. Systems that puts the candy into a container.

The first and second item have been designed, but the third one still needed to be done. One of the possibilities was to have a container 'waiting' at the end of the conveyor belt and once the candy has been dropped into the container, it needed to be lifted on the rollers that would move the container-with-candy to the warehouse.

A proof of concept can be viewed on Youtube:

 

In one of our meetings, one of the team members came with the idea to use a Trebuchet (see Wikipedia: https://en.wikipedia.org/wiki/Trebuchet) to throw the candy into a container. The main problem was, how to make the trebuchet that raising the beam could be done completely automatically. Connecting a motor to the beam is the easy part, but how to release the beam?

I found the solution in the elements that are typically used in technic models to make a gear box or to have several functions with one motor: the change-over catch (6641), driving ring (6539) and clutch gear (6542). In the picture below you see how I made the beam completely working automatically:

39232429941_675f1dae15_c.jpg

A first prototype was quickly made:

 

And I connected the second version to the conveyor belt of the candy sorter:

 

The only part that I now have to make, is a funnel to catch the candies. I'm thinking of the elements (32939) that are used in the Carousel 10257. In fact, exactly like the roof but then upside down. But maybe I may end up with a completely different funnel. Will keep you posted.

 

Enjoy,
Hans

 

Share this post


Link to post
Share on other sites
21 hours ago, Aventador2004 said:

Nice, your design team is very good at this.

I have made a more attractive version (i.e. to my opinion ;-) of the Candy Trebuchet.

25391638678_f732e343de_c.jpg

 

Please let me know what you think of it.

Best,
Hans

Edited by Hanso

Share this post


Link to post
Share on other sites
On 23/12/2017 at 8:13 AM, Hanso said:

The only part that I now have to make, is a funnel to catch the candies. I'm thinking of the elements (32939) that are used in the Carousel 10257. In fact, exactly like the roof but then upside down. But maybe I may end up with a completely different funnel. Will keep you posted.

The funnel is ready.

39374514022_08b3aaf51b_c.jpg

A video of the trebuchet and the catching funnel:

 

I have tested the trebuchet and funnel for +/- 50 times. In three cases, the candy didn't end in the container (because it hit the container that hard, that it bounced out of it). I already made a sliding part to slow down the candy, which works fine but obviously not 100% guaranteed.

24538934737_8fe5405907_c.jpg

 

So suggestions to slow the candy even more are welcome.

 

Edited by Hanso

Share this post


Link to post
Share on other sites

To make sure that there is only and exactly only one container under the funnel, I extended the conveyor rollers so that there will be a queue of empty containers.

39444634431_276e128456_c.jpg

The conveyor rollers consist of three sections A, B and C. I use two color sensors to detect a container, sensor #1 (directly under the funnel) and #2 (middle section):

  1. When an empty container is needed under the funnel, section B and C are powered. The containers now move from the right to the left, towards the funnel.
  2. When sensor #2 reads a container, section C is stopped. This means, there is now only exactly one container in the middle section B.
  3. Section B continues to be powered, until sensor #1 reads the container. Because section A is unpowered, the container is always moved exactly under the funnel.
  4. After the candy has been thrown into the container, section A is powered. Now the filled container moves to the left (and to the warehouse, see this thread)

 

And the video:

Enjoy, Hans

 

Edited by Hanso

Share this post


Link to post
Share on other sites

I proudly present: the complete and working Automatic Candy Sorter.

From left to right:

  1. The candy is dispensed from the Candy Dispense Unit, one candy at a time.
  2. The conveyor belt moves the candy to the Color Check Unit (a Pixycam) to determine the color. This information will be passed to the PC application, in this subsystem it has no further function
  3. After the color has been determined, the candy is moved to the bucket of the Trebuchet.
  4. The Trebuchet launches the candy into the Catching Funnel.
  5. Earlier, a candy container has been placed directly under the Catching Funnel by the Container Conveyor.
  6. The container-with-candy is then moved to the Warehouse section. This will be my next project.

Automatic Candy Sorter on Flickr

(click on the picture to see more photos on Flickr)

 

Please note: I have already a Proof of Concept of the Warehouse, you can read more about this here.

 

Enjoy,
Hans

Share this post


Link to post
Share on other sites

*** IMPORTANT UPDATE ***

In my last writing of two months ago, you have seen the complete and working candy sorter.

39465479701_1a60b43588_c.jpg

 

But when finalizing the details and discussing the software interface with the team, I needed to solve two more issues:

  1. When the dispense unit (on the left) dispenses a candy to the conveyor belt, it takes a couple of seconds before the candy reaches the color check unit. At this moment, the dispense unit stops rotating. Although there was little chance to happen, but in those few seconds a next candy could have been released. Meaning that the trebuchet might end with more than one candy. So, issue #1: how to ensure that only one candy would end up in the trebuchet.
    And although not really a big problem, how to ensure that a candy will end up in the trebuchet. Sometimes, it could take 'ages' before a next candy was released.
     
  2. Another issue is that a candy color needed to be rejected. The reason is that the containers-with-candy are stored in the warehouse (not in this picture). The warehouse can store 6, maybe 12 containers per color. But when the warehouse is full of, let's say red candies, the color sorter will get the message 'do not send me containers with red candies'. So, after the color check unit, it should be possible to return the candy to the candy dispense unit again. Or at least, remove it from the belt.

 

Issue #1: Find a solution to dispense one candy at a time
I have looked at many Youtube videos to get inspiration how to ensure 'one candy at a time'. How is it done in the real world? I ended up with  'bowl feeders', 'singulation conveyors' and many more ingeneous types of solutions. But nothing that helped me. Another inspiration was (of course) the Lego GBC builds. There are many solutions for feeding the balls one by the time. But the square shape of the candies make all of the Lego GBC solutions not an option for our situation.

The following Youtube video that I saw, stayed somehow in my mind:

 

eureka.png

I don't know when, I don't know how, but suddenly I had this "EUREKA" moment. You can see in the proof-of-concept that I re-used the idea of the two belts rotating in opposite drections.

Using a slide with an opening at a angle, so that only one candy can pass and other candies will be moved to the other belt when the door closes, is my own invention ("the eureka moment").

So, hooray, the concept works!

Interesting detail: You would expect that the ultrasonic sensor is the best way to detect if a candy has passed the gate. No, that doesn't work at all. The distance is too small to detect a candy and you end up in all kind of false readings. The color sensor however, as you can see in the video, works perfect when you use reflection mode.

After the proof of concept, I build a better one with a faster sliding mechanism and with the color detection unit included.

 

Issue #2: Returning a specific color to the candy storage
Coincidentally, the solution of the first issue gave me immediately also a solution for the second issue. By using a two step 'pass-through' with the color detection unit in the middle, it would be simple to return a 'wrong' color. At the end of conveyor belt 1, the trebuchet would be connected. The other conveyor belt that is used in the circular candy section, can be extended and then act as the 'returning belt' (see picture below). I just needed to add a second 'door' to be able to pass the candy to the trebuchet or to move it to the returning conveyor belt.

39699003315_29cf9e116b_c.jpg

 

And of course a video to show how it works.

 

Please let me know what you think of it.

Enjoy, Hans

Edited by Hanso

Share this post


Link to post
Share on other sites

Finished the build completely.

Photo's can be found on Flickr (click on the picture below):

Automatic Candy Sorter Photos

 

One of the major updates in this final version, is the end of the conveyor belt. When a candy needs to return to the storage, sometimes the candy kept on rotating on the two belts. By adding an extra sprocket (57520), the conveyor belt is lifted a bit. And it makes the belt a bit more wobbly. That results in a candy ending up on the returning belt with more certainty. This is shown at the end of the video (starts at timestamp 2:00):

 

Enjoy, Hans

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.