osullivp

Lego 51515 Bluetooth Object Detection Robot

Recommended Posts

Hi all,

I've been working on a Lego 51515 robot that uses computer vision to detect an object and then pick it up. Here is a video:

P1010645_reduced.jpg

The robot works as follows:

1) Objects are detected by a free TensorFlow Android App, available on Google Play (ArdObjectTracker)

2) The detection data is sent from the mobile via Bluetooth Classic and picked up on the Arduino board by an HC-05 module

3) The object detection data is then processed and converted to robot movement instructions which are then sent to the Lego hub via the BluetoothLE module (HM-10)

4) Finally, the Lego hub (running a Python program) receives the instructions via a BluetoothLE notification and then moves to pick up the object

I have shared the code and more details on this Github page:

https://github.com/osullivp/Lego51515BLEArduinoTensorFlow

Hope you all find this helpful!

Regards,

Paul.

 

 

Edited by osullivp

Share this post


Link to post
Share on other sites
7 hours ago, Gimmick said:

I realy like it! Great work!

Thanks!

I wanted to demonstrate an object detection capability without the need for TensorFlow programming.

I also wanted to demonstrate connecting to an Arduino board using the ubluetooth python module on the Lego Hub.

I've shared the full code for both the Lego Hub and the Arduino elements in the Github repo here:

https://github.com/osullivp/Lego51515BLEArduinoTensorFlow

Any questions, please let me know :classic:

Edited by osullivp

Share this post


Link to post
Share on other sites

Why is the arduino needed in between? The Hub has BT as well, right? So can't we just let the hub receive the data directly from the Android app via BT and process it in python on the Hub?

Share this post


Link to post
Share on other sites

Hi @BrickDesignerNL,

The reason for the Arduino board is that the particular Android app I am using transmits data via Bluetooth Standard whereas the Lego Hub uses BluetoothLE via Micropython. Possible alternatives would be:

  • Find a TensorFlow Android app on Google Play or elsewhere that transmits object detection data via BluetoothLE. I wasn't able to find one but if one exists that would simplify things
  • Code a TensorFlow Android app myself that implements the TensorFlow API and transmits object detection data via BluetoothLE. I could do this but it would be more time consuming

Regards,

Paul.

Edited by osullivp

Share this post


Link to post
Share on other sites
On 5/6/2021 at 9:15 AM, osullivp said:

Hi @BrickDesignerNL,

The reason for the Arduino board is that the particular Android app I am using transmits data via Bluetooth Standard whereas the Lego Hub uses BluetoothLE via Micropython. Possible alternatives would be:

  • Find a TensorFlow Android app on Google Play or elsewhere that transmits object detection data via BluetoothLE. I wasn't able to find one but if one exists that would simplify things
  • Code a TensorFlow Android app myself that implements the TensorFlow API and transmits object detection data via BluetoothLE. I could do this but it would be more time consuming

Regards,

Paul.

There is a post by Nard on the Spike/Mindstorms FB groups that does exactly that.   You may want to check it out.

Share this post


Link to post
Share on other sites
On 5/8/2021 at 7:08 PM, ninoguba said:

There is a post by Nard on the Spike/Mindstorms FB groups that does exactly that.   You may want to check it out.

Hi @ninoguba

Thanks for this. For everyone's reference, Nard's post can be found here:

https://www.facebook.com/groups/SPIKEcommunity/permalink/1357979457913616/

In the post there is a demo of a TensorFlow Android app talking directly to the Spike Prime hub (presumably via Bluetooth LE). The demo shows a Spike Prime robot tracking a cup as the cup is moved around. It's not clear if it's an off the shelf app or developed by Nard as there are currently no details in the post. I would ask myself but I don't do Facebook :-)

Hope this post helps other people looking for a TensorFlow solution that can be used with Spike Prime/51515.

Regards,

Paul.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.