BC MK-803 product use testing 3-15-12

of 52 /52
© 2012 Steve Herbst Confidential and Proprietary Steve Herbst Developer of Topical Storm software -a web-based outlining program that offers suggestions to overcome writer’s block This presentation for MK-803: My product development story and what I know about Product Use Testing SM
  • date post

    11-Sep-2014
  • Category

    Technology

  • view

    636
  • download

    0

Embed Size (px)

description

 

Transcript of BC MK-803 product use testing 3-15-12

PowerPoint PresentationSteve Herbst
Developer of Topical Storm software
a web-based outlining program that offers suggestions to overcome writer’s block
This presentation for MK-803:
My product development story and what I know about Product Use Testing
SM
Cartoon by Barbara Smaller, The New Yorker 4/26/10
© 2012 Steve Herbst Confidential and Proprietary
Names don’t work that well compared to product. Google – we worry about the name … will the world accept … or Yahoo. Or Lotus. How silly. Lotus. Apple. And yet it’s the product. The product really, really matters.
Dan Bricklin, co-founder of VisiCalc
Interview May 7, 2004
1998
Groundwork:
1998
Groundwork:
© 2012 Steve Herbst Confidential and Proprietary
© 2011 Steve Herbst Confidential and Proprietary
Hurricanes start when warm, moist air from the ocean surface begins to rise rapidly, where it encounters cooler air that causes the warm water vapor to condense and to form storm clouds and drops of rain. The condensation also releases latent heat, which warms the cool air above, causing it to rise and make way for more warm humid air from the ocean below.
As this cycle continues, more warm moist air is drawn into the developing storm and more heat is transferred from the surface of the ocean to the atmosphere. This continuing heat exchange creates a wind pattern that spirals around a relatively calm center, or eye, like water swirling down a drain.
Daughter’s 6th grade essay topic:
Warm air causes hurricanes
Done!
Warm air causes hurricanes
?
What causes warm air?
What do hurricanes cause?
What else causes hurricanes?
Can warm air be used to predict hurricanes?
Can preventing warm air prevent hurricanes?
Do different kinds of warm air cause different kinds of hurricanes?
And so on.
© 2012 Steve Herbst Confidential and Proprietary
Warm air causes hurricanes
How does warm air cause hurricanes?
Evidence that warm air causes hurricanes
What causes warm air?
What do hurricanes cause?
What else causes hurricanes?
Can warm air be used to predict hurricanes?
Can preventing warm air prevent hurricanes?
Do different kinds of warm air cause different kinds of hurricanes?
And so on.
Warm air causes hurricanes
How does warm air cause hurricanes?
Evidence that warm air causes hurricanes
What causes warm air?
What do hurricanes cause?
What else causes hurricanes?
Can warm air be used to predict hurricanes?
Can preventing warm air prevent hurricanes?
Do different kinds of warm air cause different kinds of hurricanes?
And so on.
Warm air causes hurricanes
How does warm air cause hurricanes?
Evidence that warm air causes hurricanes
What causes warm air?
What do hurricanes cause?
What else causes hurricanes?
Can warm air be used to predict hurricanes?
Can preventing warm air prevent hurricanes?
Do different kinds of warm air cause different kinds of hurricanes?
And so on.
Too much television causes ADHD
Noticing the pattern “A causes B”, the computer asks:
How does too much television cause ADHD?
Evidence that too much television causes ADHD
What causes too much television?
What does ADHD cause?
What else causes ADHD?
Can too much television be used to predict ADHD?
Can preventing too much television prevent ADHD?
Do different kinds of too much television cause different kids of ADHD?
And so on.
Too much television causes ADHD
Noticing the pattern “A causes B”, the computer asks:
How does too much television cause ADHD?
Evidence that too much television causes ADHD
What causes too much television?
What does ADHD cause?
What else causes ADHD?
Can too much television be used to predict ADHD?
Can preventing too much television prevent ADHD?
Do different kinds of too much television cause different kids of ADHD?
And so on.
Too much television causes ADHD
Noticing the pattern “A causes B”, the computer asks:
How does too much television cause ADHD?
Evidence that too much television causes ADHD
What causes too much television?
What does ADHD cause?
What else causes ADHD?
Can too much television be used to predict ADHD?
Can preventing too much television prevent ADHD?
Do different kinds of too much television cause different kids of ADHD?
And so on.
What evidence do you have that market share is important?
What other factors are also important?
Are there any unspoken assumptions behind this statement?
What steps can we take to focus on market share?
What is the purpose of market share?
Who knows the most about market share?
Who might disagree with this statement?
How can we research this recommendation further?
© 2012 Steve Herbst Confidential and Proprietary
1998
Groundwork:
2010
2010
© 2012 Steve Herbst Confidential and Proprietary
2010
Lesson learned:
Hold steady to the original vision. Don’t rush to change the
product in response to trends.
Positive outcome: clarity
Plenty of existing tools for sharing content. But there may be
value in helping people create good written content to share.
© 2012 Steve Herbst Confidential and Proprietary
1998
Groundwork:
Showed a civics demo to a history professor friend.
Me: Isn’t this technology super cool? My program generates all these ideas with just a few powerful rules!
John: This is bad grammar. And this question is vague. Teachers won’t accept it.
Reality check 2010
Lesson learned:
Positive outcome: user focus
Make it simple; hard-code the answers. Don’t mention “rules”.
Concentrate on what people want from the product.
Reality check 2010
1998
Groundwork:
Douglas School Acton MA
May 2011
Survey comments from 6th grade writing students:
Yes I think it is a very successful website. Could be a great tool for kids writing essays.
i really do enjoy filling in all of the checklists things and answering the questions. i really think that topical storm is a very productive and fun way to write an essay.
It helps writers that have writers block and are unable to organize there ideas while trying to write there essay or story.
I would definetly use a program like this again because it really makes you think about what you are writing about.
I would use this program again because it comes up with ideas that i would have not come up with.
I would use this web site agen because never befor have i been able to make a first draft of an essay in a single day before. Thank you!
© 2012 Steve Herbst Confidential and Proprietary
Lessons learned:
They don’t want happy cartoon characters.
They love the scoreboard.
Simplify it beyond belief.
May 2011
1998
Groundwork:
Software as Show Business
even for 5 dollars.
Early morning realization:
Nobody wants this.
© 2012 Steve Herbst Confidential and Proprietary
“Enlightened persons who know the True Self work for the welfare of the world, unattached, ever helping to point humanity toward dharma (right action, living a truth-based life).”
- THE BHAGAVAD GITA, 200 BCE – 200 AD
© 2012 Steve Herbst Confidential and Proprietary
Product Use Testing
drive a vehicle
Things to find out:
How accurately
How they feel about the experience afterwards
Would they buy? Recommend?
© 2012 Steve Herbst Confidential and Proprietary
Method
Simple:
See when they get confused or lose interest
Ask them for suggestions afterwards
Use this priceless information to improve the product
Testing is useful at every stage of development
© 2012 Steve Herbst Confidential and Proprietary
Conventional view of testing
Reality of testing
Test early & often
Test a mockup
For software, focus test a “wireframe”:
© 2012 Steve Herbst Confidential and Proprietary
Testing in large organizations
Statistical analysis of results
thinkeyetracking.com
Simpler non-fanatic approach
Think, fix
Step by step
2. Choose a realistic task
3. Select participants
6. Run the test
8. Report results to the participants
© 2012 Steve Herbst Confidential and Proprietary
1. Define the goal of this test
Find out how to improve the product:
See where people get stuck
Gauge emotional reactions
Compare several products or concepts
(“Use each of these machines to make espresso”)
© 2012 Steve Herbst Confidential and Proprietary
1. Define the goal of this test
Note:
A Product Use Test tests the product, not the participant
Task should not be overly difficult
Everything the user does is right; the product is wrong
© 2012 Steve Herbst Confidential and Proprietary
2. Choose a realistic task
Realistic: Something that people need to do
and that your product needs to facilitate
Examples of tasks for a website:
Create a document
Make a purchase
Sign up (how enticing is the lead-up?)
© 2012 Steve Herbst Confidential and Proprietary
3. Select participants
Who:
People
It takes only five users to uncover 80 percent of high-level usability problems.
Jakob Nielsen
Why do the test subjects know better than us?
They know very little about our product
(just like all the people who will use it someday)
They’re detached
They’re focused on their own needs and couldn’t
care less how much more work we have to do.
They’re in a hurry
(just like all the people …)
© 2012 Steve Herbst Confidential and Proprietary
4. Prepare the participants
Set expectations, give motivation:
If you will be recording, tell them
Guarantee confidentiality
Promise a reward: cash, iTunes card, free product or subscription
Ask them to sign a consent form
Let them view the test results later!
© 2012 Steve Herbst Confidential and Proprietary
5. Questionnaire beforehand
(How) do you currently use this type of product?
What difficulties or needs do you have?
What other similar products do you know?
Blue sky: What product would you love to have?
© 2012 Steve Herbst Confidential and Proprietary
6. Run the test
While performing the task(s):
What they are about to do
How they plan to do it
What step they are trying now
What they think a feature does
What surprises them
7. Ask for reactions, suggestions
Post-interview or questionnaire:
Summarize this experience
What was difficult?
What was confusing?
© 2012 Steve Herbst Confidential and Proprietary
8. Report results to participants
This is one of the strongest incentives to participate
Get contact info or post on a website
Report what most people liked & hated
Report any interesting ideas that came up
© 2012 Steve Herbst Confidential and Proprietary
Today’s Product Use Test is different
Everybody all at once
We can’t do think-aloud (unfortunately)
The program will save your work and comments, and I will look at them later
We’ll discuss when done
I will post a summary of the results
© 2012 Steve Herbst Confidential and Proprietary
Today’s Product Use Test
1. Log in with your user name & password
Pretend you want to write a college application essay
Start by knowing nothing about the product
3. See how far you get (that’s the goal of this test)
4. Click this button to enter your comments:
5. Whole class discussion at 9PM
My email: [email protected]
This PowerPoint at: www.topicalstorm.com/BC/slides