How Exactly do Focus Test Groups Work?

[Lego David]

We all know about the fact that TLG often does focus test groups in order to determine how kids react to certain types of sets, but not much else. How exactly do they select the kids that are going to be tested? How much do those focus groups influence the final products? If kids react negatively, what do they do? Which sets specifically go through focus test groups, and which don't?

By any chance, has anyone here ever been involved in any of those focus groups, in order to answer some of those questions? I am really curious, since I didn't even know about the existence of focus test groups until recently. 

[Mylenium]

35 minutes ago, Lego David said:

How exactly do they select the kids that are going to be tested?

Such stuff is typically handled by companies/ agencies specializing in product testing, not necessarily the companies themselves. They provide a pool of candidates that fit a profile and the company then picks whichever ones they deem suitable. No idea with whom LEGO are working, though.

39 minutes ago, Lego David said:

How much do those focus groups influence the final products? (...) Which sets specifically go through focus test groups, and which don't?

Very little and very few I would argue, given how obviously some products flopped despite allegedly having been tested. Of course they're always going to test completely new product lines, but it is my impression that they are not having a very broad basis. Also somehow a lot of the marketing feels very skewed toward specific groups, further making me think that they are missing a lot of beats by limiting the test groups too much.

45 minutes ago, Lego David said:

If kids react negatively, what do they do?

Hard to tell. You would have to know how early they are testing and whether there is time for an iterative approach, how many new ideas they develop in parallel and so on. I'm sure they have their graveyard of never released stuff, but at the same time I don't believe in massive piles of scrapped ideas. Their creative output is high enough to quickly fill gaps if something gets cancelled, it would seem.

Mylenium

[AmperZand]

@Lego David, Those involved either as participants or organisers are most likely under strict NDAs (non-disclosure agreements), so cannot reveal anything.

[that_lego_builder]

Legodavid i know this is has nothing to do with the topic but, ay I like our youtube content

[Lyichir]

Lego keeps a lot of the specifics of their focus-group testing close to their chest, but a few things I've picked up about it include:

• The selection of participants is guided by what's being tested—it's not entirely random. Lego often focus tests somewhat broadly (among multiple demographic groups and in multiple countries), but for certain themes there is a particular demographic that is emphasized—examples include girls for Lego Friends and children from China for Monkie Kid. For themes like these, that are meant to target particular demographics, failing to capture the demographic in question would be a major misstep, even if the appeal of the themes in question will likely extend beyond just those groups. The feedback Lego gains from different countries is known to vary culturally—I've heard that German children are often particularly finicky about how a Lego fire station or police station "should" look, for example.
• Feedback from focus groups is not limited to the initial testing period, nor is it based solely on direct feedback. Children are observed playing with the sets so that how they play with the sets can be analyzed beyond just what they verbally respond to. For example, if they go back to a feature again and again, it can likely be considered successful. If something fails for them or frustrates them, it might still need work. Even after the groups have gone home for the day, they and their parents are followed up with for feedback on what aspects of sets stuck with them. One example of this I've heard related is that kids often love when a toilet is included in a set and will excitedly talk about it with their parents even days afterward, hence the uptick in the number of times sets have included that feature as of late.
• Sometimes, multiple variations of a set (such as versions with different color schemes) are tested against one another, with the feedback compared to help settle on a final design.

[Lego David]

1 hour ago, AmperZand said:

Those involved either as participants or organisers are most likely under strict NDAs (non-disclosure agreements), so cannot reveal anything.

How exactly is that possible? If this were about testing with adults, it would make sense, but with kids? How exactly are they gonna force a strict non-disclosure agreement to little kids who just came to play with some new sets? 

[Lyichir]

15 minutes ago, Lego David said:

How exactly is that possible? If this were about testing with adults, it would make sense, but with kids? How exactly are they gonna force a strict non-disclosure agreement to little kids who just came to play with some new sets? 

Doesn't sound the hardest to me, especially if the kids aren't online yet. Hypothetically, there might be some conditional rewards for keeping their participation a secret, such as sending the kids copies of the sets once they're released (sort of like the classic marshmallow experiment—delay the mild gratification of talking about details with schoolmates or friends for a greater reward in the future). Whatever the NDA involves, it seems to work considering that there rarely seem to be any significant "leaks" from that process (the only times I can recall people claiming to have inside info on upcoming sets from focus testing of that sort, it turned out to be a hoax).

[dr_spock]

They are fun to participate. 

I suppose if you break your NDA, they won't invite you again.

 

 

 

[AmperZand]

4 hours ago, Lego David said:

How exactly is that possible? If this were about testing with adults, it would make sense, but with kids? How exactly are they gonna force a strict non-disclosure agreement to little kids who just came to play with some new sets? 

What @Lyichir said.

It would not surprise me if some of the participating kids are the children of LEGO employees which would help to keep the lid on any disclosure.

Not that it makes any difference to the disclosure aspect, but I believe that LEGO uses observation studies, not just focus groups, which may be what you are talking about. The two are not the same. In the former, a single child (or sometimes two kids who already know each other such as friends or siblings) is observed interacting with a prototype product. In the latter, a small group of kids of the same age or year group are asked about or given a number of tasks (disguised as games) relating to one or more prototypes or actual products.

Edited March 9, 2021 by AmperZand

[koalayummies]

57 minutes ago, dr_spock said:

They are fun to participate. 

Yeah they're interesting. I participated in some for major beverage companies; decent money for about 30 minutes of tasting stuff. Sometimes they keep the company conducting the test a mystery, other times they'll say a company but whether that's really the case is unknown. Most of the drinks were terrible. So bad. But I don't think they listened to me, all that stuff is on shelves now. I tried to stop it. I really did.

[Lyichir]

1 hour ago, AmperZand said:

What @Lyichir said.

It would not surprise me if some of the participating kids are the children of LEGO employees which would help to keep the lid on any disclosure.

Not that it makes any difference to the disclosure aspect, but I believe that LEGO uses observation studies, not just focus groups, which may be what you are talking about. The two are not the same. In the former, a single child (or sometimes two kids who already know each other such as friends or siblings) is observed interacting with a prototype product. In the latter, a small group of kids of the same age or year group are asked about or given a number of tasks (disguised as games) relating to one or more prototypes or actual products.

Lego definitely uses focus groups, at least sometimes. One of the most visible examples (and the only time, to date, that I think Lego has released footage of one of these exercises) can be seen in the documentary Lego had made about the making of the Monkie Kid theme—snippets from a focus testing session, including both the reactions of a group of kids and the observation of the Lego designers, are featured near the end of the documentary. Unfortunately, like the TV series for the theme, that documentary has yet to get a "legit" release outside of Asia. I'm hoping it will arrive sooner or later, but in the meantime, you can stream it here: https://www.bilibili.com/video/BV13z411B7Hj?from=seopage&fbclid=IwAR3hmGYmRwPJ7Bi_nHJrL-jwPDhm1b21vyuNm-MbRIaZgHJPqG9OTxRF2Ks
 

[Peppermint_M]

In addition to that above, in the UK there was a documentary about LEGO on Channel 4 that mentioned some focus groups were local children in the area who were the target demographic for City and Friends. This was just one of the groups though, and it was mentioned external agencies also did focus group testing. It was called the Secret World of LEGO and I can't find it online right now...

However if I could just leave this here:

Focus groups do not get it 100%, they are just one of many tools to use when developing something.

 

[Tube Map Central]

Deleted

Edited March 10, 2021 by Tube Map Central
Identical point already made by someone else