Many Crummy Trials BEAT Deep Thinking: Discussion

 Two related posts in the last month have generated a fair amount of interest:

  1. “Many Crummy Trials BEAT Deep Thinking” (4/20)
  2. “the cost of trying is lower than the cost of analyzing.” (4/1)

An executive in charge of online services at a major Wall Street firm sent an email which included:

[I] sent this article onto a few of my staff – it happened to hit at a particularly interesting time.  I have a project with very complex business logic and the analysis phase is quite painfully slow.  I had been contemplating this very issue – as the article puts it, “is the cost of detailed analysis greater than the cost of iterative trials.” 

I like the notion of “cost” here – either time or money.

One comment on the site said that trials instead of analysis was already a key part of innovation in his bank, but that better techniques to manage and prioritize the trials was needed.

In two comments and five emails (I hope ratios will reverse some day: WordPress makes it easy to post) NO ONE found the idea of trying before analyzing to be strange. I think this is a result of interest in its own right.

One financial service executive did note that at her firm development teams tended to BELIEVE in analysis or prototyping/Beta culture with religious zeal that is not impacted by empirical data. In her firm the North American team believe in getting Beta version out quickly, the European group believed in analysis, and the two Asian teams were also split into one each. Therefore the development process followed for a new service depends on which regional group is in charge of the project.

One commentor questioned whether crummy trials had the potential to hurt the brand name.

I indicated that in my experience (B2B) we picked the testers carefully and indicated that we were Beta testing even if it was really Alpha or first time testing.

B.J. Fogg, the author of the original study of Facebook applications cited and the source of the “Crummy trials” quote, also responded to that comment, indicating that “As of right now there are no major brands in the top 50 applications on Facebook. It’s likely they are afraid of hurting their brand by doing something innovative and new – but eventually if they want to play the game they’ll have to figure out the rules. It takes a completely different way of thinking about innovation and most major brands haven’t learned to do that yet. Regarding crummy trials hurting brand, it’s an interesting question. There is certainly that potential… I think people are learning not to expect perfect execution all the time.” BJ Fogg Stanford Persuasive Technology Lab

Crummy trials beat deep thinking. Ready, Fire, Aim! Probe and Learn. Experiment. Prototype. Beta Culture.

However you express it — just do it.

— But lets work to find a way to manage, prioritize and sell the experiential methods.

The original postings are below (4/20).

The link to the BJ Fogg Stanford study is

This entry was posted in Co-creation or User collaboration, Customer Research Methods, experiential innovation, Experiment, financial services, NSD Process and tagged , , , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s