Hypothesise like you Mean it!
•Download as PPTX, PDF•
2 likes•277 views
We designed a hypothetical civic art installation for Cambridge with interactive features representing local history. We formed an initial hypothesis that adding a phone interaction feature would increase awareness of the installation. However, upon diagnosing errors in our experimental design, we realized our assumptions were flawed and risks were high. We then created a new, better-informed hypothesis to test a different feature with the goal of removing uncertainty and validating our assumptions through an improved experimental approach.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Hypothesise like you Mean it!
Similar to Hypothesise like you Mean it! (20)
Recently uploaded
Recently uploaded (20)
Hypothesise like you Mean it!
- 1. Hypothesize like you mean it The Essentials of Metrics, Hypotheses, and Iteration
- 2. Greetings :)
- 3. Chris Massey Product lead @ Mind the Product chris@mindtheproduct.com @camassey
- 4. Lisa Long Founder @ Before You Code lisa@beforeyoucode.com @beforeyoucode
- 6. Metrics & Hypotheses & Prioritisation & Iteration ● What makes a good metric ● How to design an effective hypothesis ● The essentials of Prioritisation ● Experimental Iteration
- 7. Let’s go!
- 8. What is a Metric? Meaningful metrics (data+context) drive targeted, effective actions that create tangible value
- 9. Good metric / Bad metric Does your metric suggest success without being directly tied to the outcome you care about? Is your metric a placeholder for something that is hard to measure?
- 10. What is a hypothesis? A statement of the specific (& testable) impact you believe a proposed change will have.
- 11. The purpose of a hypothesis To structure thinking To reveal assumptions To direct resources To avoid waste To help you prioritise
- 12. Null Hypotheses Assume your worldview wrong !
- 13. High vs Low-level Hypotheses Understand your chain of value Make sure you can isolate your variables / metrics Pick one
- 15. Goals HIGH-LEVEL MISSION, VISION OR OBJECTIVES DESIGN PARAMETERS Metrics Framing Context Features SPECIFIC, MEASURABLE, CONTEXTUALISED ACTION- ORIENTED ENVIRONMENT, AUDIENCE, TIMEFRAMES ... HIGH-LEVEL FEATURE CONCEPTS, SPECIFIC IMPLEMENTATIONS ...
- 16. Goals ● Tourist attraction ● More awareness of Cambridge history / significance DESIGN PARAMETERS Metrics Framing Context Features ● Tour company inclusion ● Pre-test / Post-test of local history ● Traffic-flow assessment ● Social media buzz ENVIRONMENT, AUDIENCE, TIMEFRAMES ... HIGH-LEVEL FEATURE CONCEPTS, SPECIFIC IMPLEMENTATIONS ...
- 17. Instant Art Generator ™ 1. Kinetic 2. Projection 3. Light-based 4. Physical interaction 5. Multiple pieces 6. Historical 7. Phone interaction 8. Large scale 9. Persistent changes 10. Consistent start state 11. All-weather 12. University collaboration
- 18. Go design
- 19. Activity: Design a civic art installation 15 mins Goals Brainstorm a potential design for an informative civic art installation Activity ● Use google to get inspiration for art installations that use your constraints ● Consider what historically significant or helpful facts about Cambridge could be represented in a public installation ● Come up with as many ways as possible to represent those facts ● Converge on a set of design “features” for your installation Deliverable An informative civic art installation, described in “features”, potentially supported by rough sketches
- 21. To the hypothesis canvas!
- 22. We believe that ... For ... Will lead to ... Because... SOME FEATURE OR CHANGE AN ENVIRONMENT OR CONTEXT SOME SPECIFIC, MEASURABLE CHANGE YOUR RATIONALE HYPOTHESIS CANVAS
- 23. Go Hypothesise
- 24. Goals Form a testable hypothesis around one aspect of your art concept Activity ● Pick the metric you want to effect ● Pick your riskiest assumption / feature ● Select your experimental context ● Think through your experimental setup ● Articulate your experimental rationale Deliverable A well-formed, testable hypothesis, allowing you remove uncertainty / validate an assumption around one “feature” of your design product Activity: Form a hypothesis 10 mins
- 25. How’d it go?
- 26. Designers & Diagnosis & Dragons
- 27. Some Role-play Heads = Failure Tails = Ambiguous
- 28. What impacts experimental outcomes? ● Bias ● Confounding factors ● Chance! ● True effect
- 29. Common mistakes ● Not everything deserves a hypothesis ● When to abandon (sunk cost) ● Check sample size & source ○ (67% conversion on 3 actions out of 10 million users) ● Are you asking the right question? ● Is every test succeeding / failing? ○ (check for bias or poor systematic design) ● Bad data foundations ● Technical challenges
- 30. What happened?
- 31. Activity: Diagnose errors in your experiment 10 mins Goals Brainstorm a wide range of possible ways your experiment might have yielded an ambiguous or unsuccessful result Activity ● Revisit & reassess your underlying assumptions ● List any ways your assumptions could be wrong ○ Don’t underestimate human stupidity ● Step through your experiment setup ● List the possible errors in that setup Deliverable A list of possible incorrect assumptions and experimental errors
- 32. NOW what’s your riskiest assumption?
- 33. Let’s talk about prioritisation ● Go back to your metrics ● What are your assumptions? ● What are you uncertain about? ● What are the risks? ● What are the costs to test?
- 34. Uncertainty Risk TRACK TEST IGNORE MITIGATE Low High High
- 35. Risk Cost TEST ASSESS ASSESS IGNORE Low High High
- 36. What’s the next Hypothesis?
- 37. We believe that ... For ... Will lead to ... Because... SOME FEATURE OR CHANGE AN ENVIRONMENT OR CONTEXT SOME SPECIFIC, MEASURABLE CHANGE YOUR RATIONALE HYPOTHESIS CANVAS
- 38. Goals Create a better-informed, testable hypothesis around one aspect of your art concept Activity ● Consider what you’ve learned thus far about your context ● Pick the metric you want to effect ● Pick your riskiest assumption / feature ● Select your experimental context ● Think through your experimental setup ● Articulate your experimental rationale Deliverable A well-formed, better-informed, testable hypothesis, allowing you remove uncertainty / validate an assumption around one “feature” of your design product Activity: Form a new hypothesis 5 mins
- 40. What did we learn?
- 41. Metrics & Hypotheses & Prioritisation & Iteration ● What makes a good metric ● How to design an effective hypothesis ● The essentials of Prioritisation ● Experimental Iteration
- 43. Thanks! Chris Massey & Lisa Long @camassey | @beforeyoucode chris@mindtheproduct.com lisa@beforeyoucode.com
- 44. Bonus round
- 45. How did it go this time?
- 46. Some more role-play Rock = Success Paper = Failure Scissors = Ambiguous
- 47. Presentation to Design Council … in one week … to secure more £££
- 48. What do you do?
- 49. How to present experimental findings Context, Context, Context Your sources Your sample size Your assumptions Your experiment design Your findings
Editor's Notes
- 50:50 - activities to talking 10-15 minutes of lecture (10m without interactivity), then 15-20m activity