Control+ General Discussion

[Gimmick]

The last point is what they announced. I understood that as "upload a PU program as firmware to the hub". Which is by no means trivial if you want to support every option in PU (multiple hubs/controllers...).

Edit2: And maybe it's not even possible and this feature needs its own interface.

Edited August 22, 2021 by Gimmick

[Ngoc Nguyen]

@kbalage Let's say I wanna use Pybrick for the 42100, then how does Pybrick allow for control multiple hubs with one controller? Or do I need two controllers?

[howitzer]

4 minutes ago, Ngoc Nguyen said:

@kbalage Let's say I wanna use Pybrick for the 42100, then how does Pybrick allow for control multiple hubs with one controller? Or do I need two controllers?

I don't think a single controller can handle that, so you'd need two. But considering the functions of 42100, you're probably better off with a smartphone control anyway.

[Gimmick]

Hub-to-Hub communication is planned for Pybricks 3.3. Until that you need one remote per hub.

[kbalage]

1 hour ago, Gimmick said:

The last point is what they announced. I understood that as "upload a PU program as firmware to the hub". Which is by no means trivial if you want to support every option in PU (multiple hubs/controllers...).

Considering the pace of development so far, what are the chances that all the functions will be implemented in the VM plus the controllers etc. in a timely manner? I wouldn't hold my breath... Pybricks is here, already useable and has a pretty well defined development path. Don't get me wrong, I would also love to see a solid official solution, but based on the past years I'm a bit skeptical about the timelines TLG can follow. They are still focusing on the experience tied to products (Control+, Mindstorms) and does not seem to have much interest in investing resources for generic solutions. 

[Gimmick]

1 hour ago, kbalage said:

Considering the pace of development so far, what are the chances that all the functions will be implemented in the VM plus the controllers etc. in a timely manner?

I don't know. I just mentioned what they announced, that a PU-based solution is not easy and maybe not do-able if the logic as it is now has to be preserved. I do not want to argue against pybricks.

[Tcm0]

3 hours ago, kbalage said:

If I think about it, to beat Pybricks they'd really need to release a physical controller with proportional joysticks, then simplify the code blocks and make it possible to create a configuration for the controller and hub(s) that can be saved on the device(s) to run it without the smart device. 

I think that the best solution for that would be support of pybricks for BLE controlles (like the xbox series gamepad). No toy company will be able to make a gamepad with a good ergonomic feeling and all of the inputs/outputs (analog and digital) for a comparable price.

 

For anyone who wants to try the direct connection between pybricks and the powered up remote: I wrote programms for almost all of the official sets: https://github.com/Tcm0/PybricksRemoteLayouts

Edited August 22, 2021 by Tcm0

[AVCampos]

What TLG could do that improved on Pybricks regarding physical remotes (for which Pybricks is a bit overkill IMHO) would be an interface like BrickController 2: press a button on the gamepad, and assign it to a hub output. Then that table of assignments could be downloaded to the hub, which would then connect directly to the gamepad without further need of the mobile device.

[howitzer]

3 minutes ago, AVCampos said:

What TLG could do that improved on Pybricks regarding physical remotes (for which Pybricks is a bit overkill IMHO) would be an interface like BrickController 2: press a button on the gamepad, and assign it to a hub output. Then that table of assignments could be downloaded to the hub, which would then connect directly to the gamepad without further need of the mobile device.

I don't think TLG will add an official support of third party game controllers, as they would then have to answer all of the inevitable problems that will arise, or at least they'd have to establish a partnership with a certain manufacturer and create together the interface in such a way that it's really easy to set up. I'd sooner see them produce their own gamepad-like controller but even that is unlikely. Another possibility is an extension or upgrade for the train remote, so that it would enable more controls and perhaps a proportional control.

[AVCampos]

I don't know if it's technically possible, but if the hub can connect to an HID-compliant controller there should be no compatibility issues.

[Gimmick]

@howitzer

 

They already did for Mindstorms and Xbox/PS controller.

Edited August 23, 2021 by Gimmick
Posting buged

[howitzer]

25 minutes ago, Gimmick said:

@howitzer

 

They already did for Mindstorms and Xbox/PS controller.

Oh, do tell more, please?

[kbalage]

51 minutes ago, howitzer said:

I don't think TLG will add an official support of third party game controllers, as they would then have to answer all of the inevitable problems that will arise, or at least they'd have to establish a partnership with a certain manufacturer and create together the interface in such a way that it's really easy to set up.

They managed to do it for the Mindstorms robot inventor with the Xbox One and PS4 controller support.

59 minutes ago, AVCampos said:

What TLG could do that improved on Pybricks regarding physical remotes (for which Pybricks is a bit overkill IMHO) would be an interface like BrickController 2: press a button on the gamepad, and assign it to a hub output. Then that table of assignments could be downloaded to the hub, which would then connect directly to the gamepad without further need of the mobile device.

Adding a support for the controllers mentioned above would not be a big deal in the Powered Up app, but this would still mean that a smart device is required to run the code and do the assignment of controls. BC2 works the same way, you need the app to run constantly. 

There are multiple challenges with the controls themselves. Mapping buttons / joysticks to PU motors require a lot of parameters, it's not a simple X axis = Port A assignment. Is it a servo (needs calibration, limits, power, etc), if it is a button then what does it do (max power/0 power, stepper, etc). This configuration needs to happen on the smart device with the code itself that is executed by the remote, then the whole configuration has to be able to run on the hub in standalone mode with the remote being connected to the hub directly. Due to the limitation of the BT protocol used it would only work with the Xbox One controller so it is doable, but definitely not a trivial task.

@howitzer - here is a video demonstrating the concept for 51515:

As you can see this one only works in streaming mode with the hub, so it cannot work without the smart device.

[Gimmick]

4 minutes ago, howitzer said:

Oh, do tell more, please?

There is not much more to tell. The Robot-Inventer-Software has dedicated blocks for the Xbox and PS4 controller.

But that works only in streaming mode, since these controllers do not support BTLE.

[Tcm0]

1 hour ago, kbalage said:

There are multiple challenges with the controls themselves. Mapping buttons / joysticks to PU motors require a lot of parameters, it's not a simple X axis = Port A assignment. Is it a servo (needs calibration, limits, power, etc), if it is a button then what does it do (max power/0 power, stepper, etc). This configuration needs to happen on the smart device with the code itself that is executed by the remote, then the whole configuration has to be able to run on the hub in standalone mode with the remote being connected to the hub directly. Due to the limitation of the BT protocol used it would only work with the Xbox One controller so it is doable, but definitely not a trivial task.

Small correction: it would only work with the xbox series controllers and other ble controllers like the steam gamepad, not with the xbox one controller. That one uses BTC. The earlier XBox One controllers don't even have any bluetooth (connection to xbox is done with a proprietary protocol).

LEGO didn't want to mess with different gamepad input styles (XINPUT vs DINPUT etc) so they rely on the android/windows/whatever operating system to handle the gamepads. It's by far the simplest thing to do but it has it's own limits. On the other hand it would probably be hard to tell the customer that he can't use the PS4 controller he has because of technical reasons but the xbox series controllers work.

[kbalage]

2 hours ago, Tcm0 said:

Small correction: it would only work with the xbox series controllers and other ble controllers like the steam gamepad, not with the xbox one controller. That one uses BTC. The earlier XBox One controllers don't even have any bluetooth (connection to xbox is done with a proprietary protocol).

Interesting, I have an "Xbox One S" controller and I had the impression that it supports BLE but apparently it is BTC only. This makes the BLE-compatible hardware list pretty limited, I'm not sure if it is worth for TLG to develop a solution if only a handful of compatible 3rd party controllers exist and it is quite difficult to identify which ones would actually work with the VM directly while a wider range is supported through the app. BLE compatibility is not something that is actively advertised by the manufacturer.

The solution available for the Mindstorms 51515 set btw has its own flaws, the reaction time highly depends on the smart device used and possibly on the complexity of the code. I did some tests when I did my first video about the set and sometimes the response was very laggy, and there were other issues like multiple motors not starting/stopping the same time when controlled by the joystick so running the robot straight was not always successful. 

I still think a native solution with proportional joysticks would be the best, that could work through the PU app and could also connect directly to the hubs. Mould King could make such remote so it should not be a challenge for TLG :) They'd only need the will to do it and then dedicate the proper resources. 

 

[Tcm0]

23 minutes ago, kbalage said:

Interesting, I have an "Xbox One S" controller and I had the impression that it supports BLE but apparently it is BTC only. This makes the BLE-compatible hardware list pretty limited, I'm not sure if it is worth for TLG to develop a solution if only a handful of compatible 3rd party controllers exist and it is quite difficult to identify which ones would actually work with the VM directly while a wider range is supported through the app. BLE compatibility is not something that is actively advertised by the manufacturer.

Yep. I use https://github.com/ndeadly/MissionControl/issues?q=is%3Aissue+is%3Aopen+label%3A"BLE+controller" as a list. Lego would probably have to write support/ button mapping for each of them (if they wanted to support all of them)... not going to happen.

24 minutes ago, kbalage said:

work through the PU app and could also connect directly to the hubs. Mould King could make such remote so it should not be a challenge for TLG :) They'd only need the will to do it and then dedicate the proper resources. 

It would have many advantages, yes. But it would have other limitations. Gamepads had a long path of evolution and they are pretty perfect when it comes to inputs (analog and digital) and outputs. A toy company probably wouldn't come close to that perfection and a "good" controller by lego would be very expensive especially if it has some kind of configurability.

[kbalage]

8 minutes ago, Tcm0 said:

It would have many advantages, yes. But it would have other limitations. Gamepads had a long path of evolution and they are pretty perfect when it comes to inputs (analog and digital) and outputs. A toy company probably wouldn't come close to that perfection and a "good" controller by lego would be very expensive especially if it has some kind of configurability.

I don't think it is such a big effort, TLG already made remotes with joysticks, dials and buttons. If Chinese companies can produce a generic BT controller for $15-20 with 2 joysticks, a D-pad and a few buttons that work totally fine with BrickController2 then LEGO should be able to do it as well. There's no real need for super advanced ergonomics or anything extra to have something good enough in a LEGO "suit". Even with studded parts or pin holes it would be far-far better and easier to use than touch control. I'm sure it would be more expensive than the PU train remote, but at least it'd be an option to use.

[JaBaCaDaBra]

32 minutes ago, kbalage said:

I don't think it is such a big effort, TLG already made remotes with joysticks, dials and buttons. If Chinese companies can produce a generic BT controller for $15-20 with 2 joysticks, a D-pad and a few buttons that work totally fine with BrickController2 then LEGO should be able to do it as well. There's no real need for super advanced ergonomics or anything extra to have something good enough in a LEGO "suit". Even with studded parts or pin holes it would be far-far better and easier to use than touch control. I'm sure it would be more expensive than the PU train remote, but at least it'd be an option to use.

True,
but TLG puts a lot of energy in testing and safety of their devices.

That mouldy king almost set my house on fire with that piece of cada junk that exploded here.
Neither are they interesed in choking kids etc.

[kbalage]

1 hour ago, JaBaCaDaBra said:

True,
but TLG puts a lot of energy in testing and safety of their devices.
That mouldy king almost set my house on fire with that piece of cada junk that exploded here.
Neither are they interesed in choking kids etc.

It's not like TLG would need to reinvent the wheel. They have decades of experience with electronics, remotes, child safety, all the know-how that is required to produce such remote is there. They already made one remote for PU, nothing would prevent them to make another one.

The only thing missing apparently is the will to make it. As I see all the PU-based smart solutions (Control+, City PU sets, Mindstorms etc) were created fully with touch control in mind. The train remote was designed for the City trains because they had something similar for PF and train fans are loud, but according to TLG that should work for everything else as the small PF remote had the same abilities and that was also enough for all the PF vehicles they ever made. 

TLG fully focuses on the sets and hardware is created based on their needs. An app-based touch interface will always beat physical controls from the designers' perspective since the app can offer a "rich experience" compared to a simple remote. Any generic hardware that would improve the user experience of a smaller amount of users might be at the end of the priority list, if it even happens to be included in the list.

Honestly I don't have high hopes, the best I expect is the implementation of the Xbox/PS4 remotes in the PU app like it works in the Mindstorms app. The VM functionality is fully experimental and apparently does not have much priority either, so a dedicated remote designed to work with that has almost zero chance in the next 2-3 years. Hopefully we'll have a working 3rd party solution way sooner than that.

[Tcm0]

19 minutes ago, kbalage said:

The only thing missing apparently is the will to make it. As I see all the PU-based smart solutions (Control+, City PU sets, Mindstorms etc) were created fully with touch control in mind. The train remote was designed for the City trains because they had something similar for PF and train fans are loud, but according to TLG that should work for everything else as the small PF remote had the same abilities and that was also enough for all the PF vehicles they ever made. 

I think that the big question is what another remote should be used for and if the market is big enough. Which inputs are required/necessary? I agree with TLG that the PUP remote is enough for almost everything. Just an official way to connect the remote directly to the hubs is missing.

The remote can be used to control more or less all of the control+ vehicles available today. Yes, it has limitations because the buttons are digital only but there can be workarounds for that.
Two ideas for analog-like steering:
1. move the motor slowly into the direction as long as the button is beeing pressed and move slowly to the center if the button is beeing released
2. move the motor slowly into the direction as long as the button is beeing pressed but only return to center if red button is pressed

Similar things can be done for speed. Stuff like simulated gears with the red buttons.

So how would another remote improve on that, how specific would it be (the power functions train remote was the only official analog remote but the layout was really bad for vehicles) and would it be worth it? I think that the powered up vm would be a part of the solution but then they don't really get their stuff together. Maybe an individual app that simplifies the use of pybricks would be a good solution (connect to the hub, maybe autodetect model, upload firmware together with right python code embedded).

Edited August 23, 2021 by Tcm0

[kbalage]

@Tcm0 controlling LEGO cars with the buttons only on the PU remote is possible, but it is far from being convenient for most of them. I suggest to test it with a few vehicles and compare the experience with a game controller using BC2. Proportional control for steering and throttle is a must if you want fast and precise control. It is worth to check the virtual controls of the different Control+ profiles and see how the functions can be translated to physical controllers. A few examples:

- a faster car (e.g. the Off-road buggy) needs the ability to control speed. You want to be able to select your speed and maintain it, quickly accelerate, stop, reverse etc, and for this purpose a joystick is the best controller. Although a single joystick has 2 axes to use, the default Control+ profile for the buggy showed how bad it is to assign both throttle and steering control to a single joystick, so another joystick is required for steering, see e.g. the Control+ profile of the Zetros. Steering also requires proportional control, the servo gives us the ability for precision control so let's use it :)

- If you check the Control+ profile of the Liebherr, it has 2 joysticks (all axes are used for the different movements), there are 2 sliders for the tracks that can be replicated with pressure sensitive buttons, and other functions like opening/closing the bucket or predefined activities could be triggered with buttons. Here's an example how I configured the Xbox controller with BC2 for this set.

I would say that a LEGO controller would need 2 joysticks, 4 buttons on the top next to the joysticks and 2x2 pressure sensitive buttons on the front like on the Xbox controller. These would more or less cover the requirements, and if more buttons are needed then a train controller could be attached any time.

[Tcm0]

1 hour ago, kbalage said:

- a faster car (e.g. the Off-road buggy) needs the ability to control speed. You want to be able to select your speed and maintain it, quickly accelerate, stop, reverse etc, and for this purpose a joystick is the best controller. Although a single joystick has 2 axes to use, the default Control+ profile for the buggy showed how bad it is to assign both throttle and steering control to a single joystick, so another joystick is required for steering, see e.g. the Control+ profile of the Zetros. Steering also requires proportional control, the servo gives us the ability for precision control so let's use it :)

Normally, you'd use the "pressure sensitive buttons" (also called triggers) for acceleration/reverse. It's the default for most racing games. Personally I think that sticking to default video game controls makes sense because that's what many kids are used to. None of the games use the other joystick for acceleration/reverse. That's mainly used for camera controls, nothing you need for a remote controlled vehicle. On the other hand, cranes sometimes use both joysticks in videogames.
But speed could also be controlled with a virtual gear shift similar to the top gear car control+ profile. That might actually make more sense because holding a joystick/trigger in a position inbetween is inconvenient for a longer period of time, at least for me.

2 hours ago, kbalage said:

I would say that a LEGO controller would need 2 joysticks, 4 buttons on the top next to the joysticks and 2x2 pressure sensitive buttons on the front like on the Xbox controller. These would more or less cover the requirements, and if more buttons are needed then a train controller could be attached any time.

Actually, most common gamepads including the xbox one controller only have 2 pressure sensitive buttons on the front/top, both triggers (RT/LT). The bumpers are digital only (RB/LB). The nintendo switch controllers don't even have analog triggers. They only have digital triggers and bumpers. I'm not sure if it would be possible to create 2 relieable analog triggers per side. Companies get really creative when it comes to the design of the analog triggers.

What about motion controls? Many modern controllers have them and some gamers prefer to use them for precision.

But a complicated controller might also make problems with copyrights/patents.

[kbalage]

6 hours ago, Tcm0 said:

Normally, you'd use the "pressure sensitive buttons" (also called triggers) for acceleration/reverse. It's the default for most racing games. Personally I think that sticking to default video game controls makes sense because that's what many kids are used to. None of the games use the other joystick for acceleration/reverse. That's mainly used for camera controls, nothing you need for a remote controlled vehicle. On the other hand, cranes sometimes use both joysticks in videogames.

A real life RC vehicle is not a video game :) You don't see any RC car controllers with triggers. There are 2 big camps, first is controllers using sticks (joysticks) and the other is the pistol grip style with a wheel for steering on the side. When I was a kid we had mostly stick controllers, but nowadays the pistol grip is more widespread for cars. For anything that flies or needs more channels, the stick controller is still the way to go.

A pistol grip controller has limited number of channels so it is not universal enough to cover most LEGO RC vehicles, that's why I'd go with the classic controller with 2 joysticks. Throttle and steering can be separated with this configuration and sticks are convenient for precision control. Front triggers could be added for the video game generation if they really prefer throttle and brake control there, otherwise they can be useful e.g. for tracked vehicles as additional channels.

6 hours ago, Tcm0 said:

Actually, most common gamepads including the xbox one controller only have 2 pressure sensitive buttons on the front/top, both triggers (RT/LT). The bumpers are digital only (RB/LB).

Well that shows how frequently I use these controllers in video games :) The RC excavators I've seen either use buttons on the front or extra levers to control the tracks. I think front buttons are more convenient since that way it is possible to drive and operate the sticks the same time. 

6 hours ago, Tcm0 said:

What about motion controls? Many modern controllers have them and some gamers prefer to use them for precision.

Based on my experience motion control it is far from being precise. If someone prefers it then it is already available in the PU app and can be used with a smart device.

[Tcm0]

2 hours ago, kbalage said:

Based on my experience motion control it is far from being precise. If someone prefers it then it is already available in the PU app and can be used with a smart device.

Often physical buttons are beeing used for unprecise control and then motion controls are used for the last bit of precision. It's not one or the other but a combination of both.

2 hours ago, kbalage said:

A real life RC vehicle is not a video game :) You don't see any RC car controllers with triggers. There are 2 big camps, first is controllers using sticks (joysticks) and the other is the pistol grip style with a wheel for steering on the side. When I was a kid we had mostly stick controllers, but nowadays the pistol grip is more widespread for cars. For anything that flies or needs more channels, the stick controller is still the way to go.

A pistol grip controller has limited number of channels so it is not universal enough to cover most LEGO RC vehicles, that's why I'd go with the classic controller with 2 joysticks. Throttle and steering can be separated with this configuration and sticks are convenient for precision control. Front triggers could be added for the video game generation if they really prefer throttle and brake control there, otherwise they can be useful e.g. for tracked vehicles as additional channels.

For normal RC controllers for remote controlled vehicles, the joysticks often are for one direction (top/down OR left/right), not both at the same time. And a configuration would make everything more complicated.

I agree that I wouldn't want a pistol grip controller. I don't understand why they are successful.