in Uncategorized

Metrics Gathering notes from Workshop by Aaron Ginn

Aaron Ginn from StumbleUpon.com
October 10th, 2013 @ Miami Science Museum / Refresh Miami

Most important metrics?
Number of installations.
Revenues.

Types of metrics:
– Quantitatie vs -Qualitative.

Idea:
– On Wizard, let them know how to be in touch for Tech Support
via @frostwire and facebook.com/FrostWire Official.
Task for Marcelina to design that screen on the wizard.

QUANTITATIVE
The funnel (Pirate Metric)
Acquisition > Activations > Retention > Referral > Revenue.

QUALITATIVE
User Testing (In Progress)
Surveys
Study Groups
User Interviews

Stages for companies:

Early Stage involves discovery, validation. Then get Seeded out of pocket and angels.
Qualitative metrics help here.

Later Stage creation and building – Series A, profitability.
Quantitative metrics.

During creation, you want to start doing A/B to grow and see what works.

PRODUCT DASHBOARD
Engagement
User’s worth (LTV)
Growth rate

Qualities of a Good Metric
Measurements that are aggressively attacking your product.

Actionable – short feedback loop.
Product Centric – attack core assumptions of your product
Ratio – Compare against competitors performance, industry numbers.
Explainable – simpler the better
Timeless
(Checkout “Flurry” reports

Key Metrics per business

Marketplace: fulfillment rate, WOW (week over week) growth on both sides, % repeat purchase

Social Product: DAU/MAU, WOW daily registration, % ad impressions

B2B – sales funnel metrics, LTV

B2C – sales funnel metrics, LTV, WOW growth

LTV = churn * avg revenue per customer

Virality = measure the share cycle time is important. The faster, the deeper impact at sub-levels. Time between invites and activations.
Look for vitality equations.

Create your own AARRR

Acquisition – Number of new downloads, newsletter signups, twitter followers, Facebook page likes, community forum signups, Number of displays of our thank you page
Activation – Number of first uses. Displays of our promotions.
Retention – Number of sessions, Number of core actions, Newsletter unsubscribes, Facebook unlikes
Referral – Measure social activity, buzz, cult (pictures, fan art)
Revenue – Daily reports from all revenue channels (some hourly)

Qualitative Testing Schedule

One user a day. Watch them.
Two user sessions a month.
Three usability studies a month.

Important Factors in Quantitative Testing
is it Statistically Significant?
sample size.
sensitivity of the test.
duration of the test.
velocity. How many tests can you run in a period, pretty important. You need to test and get results as fast as possible, to learn more and more.
conversion.

Good habits
Test the hell out of MVPs.
Kill tech that slows you down.
Validate Instrumentation First. (make sure your test is running well)
The Goal is testing velocity.
Think about your next developer/employee.

Tools we use
Qualitative Testing Tools:
FT User Researcher
UserTesting.com
MouseOver Videos
User Studies

Quantitative Testing Tools
AB testing
Daily metrics emails
Product dashboards
Team metrics

Rapid A/B Testing Workflow

Test Iteration Implementations -> Rapid implementations.
Full Implementations -> Winner only (clean losers)
Code Fragmentation -> Clean unused test code.

Branch Based Workflow (git based)

Test Iteration Implementation -> Rapid
Full Impl -> Winner only

What will we do differently after this?
Added some new metrics I was missing thanks to this in relationship to acquisition and activations.
Will try new tools like UserTesting.com
Will look at what tech might be slowing us down.
Will not build more visualization tools of test data.

Write a Comment

Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.