Metrics Gathering notes from Workshop by Aaron Ginn

Aaron Ginn from
October 10th, 2013 @ Miami Science Museum / Refresh Miami

Most important metrics?
Number of installations.

Types of metrics:
– Quantitatie vs -Qualitative.

– On Wizard, let them know how to be in touch for Tech Support
via @frostwire and Official.
Task for Marcelina to design that screen on the wizard.

The funnel (Pirate Metric)
Acquisition > Activations > Retention > Referral > Revenue.

User Testing (In Progress)
Study Groups
User Interviews

Stages for companies:

Early Stage involves discovery, validation. Then get Seeded out of pocket and angels.
Qualitative metrics help here.

Later Stage creation and building – Series A, profitability.
Quantitative metrics.

During creation, you want to start doing A/B to grow and see what works.

User’s worth (LTV)
Growth rate

Qualities of a Good Metric
Measurements that are aggressively attacking your product.

Actionable – short feedback loop.
Product Centric – attack core assumptions of your product
Ratio – Compare against competitors performance, industry numbers.
Explainable – simpler the better
(Checkout “Flurry” reports

Key Metrics per business

Marketplace: fulfillment rate, WOW (week over week) growth on both sides, % repeat purchase

Social Product: DAU/MAU, WOW daily registration, % ad impressions

B2B – sales funnel metrics, LTV

B2C – sales funnel metrics, LTV, WOW growth

LTV = churn * avg revenue per customer

Virality = measure the share cycle time is important. The faster, the deeper impact at sub-levels. Time between invites and activations.
Look for vitality equations.

Create your own AARRR

Acquisition – Number of new downloads, newsletter signups, twitter followers, Facebook page likes, community forum signups, Number of displays of our thank you page
Activation – Number of first uses. Displays of our promotions.
Retention – Number of sessions, Number of core actions, Newsletter unsubscribes, Facebook unlikes
Referral – Measure social activity, buzz, cult (pictures, fan art)
Revenue – Daily reports from all revenue channels (some hourly)

Qualitative Testing Schedule

One user a day. Watch them.
Two user sessions a month.
Three usability studies a month.

Important Factors in Quantitative Testing
is it Statistically Significant?
sample size.
sensitivity of the test.
duration of the test.
velocity. How many tests can you run in a period, pretty important. You need to test and get results as fast as possible, to learn more and more.

Good habits
Test the hell out of MVPs.
Kill tech that slows you down.
Validate Instrumentation First. (make sure your test is running well)
The Goal is testing velocity.
Think about your next developer/employee.

Tools we use
Qualitative Testing Tools:
FT User Researcher
MouseOver Videos
User Studies

Quantitative Testing Tools
AB testing
Daily metrics emails
Product dashboards
Team metrics

Rapid A/B Testing Workflow

Test Iteration Implementations -> Rapid implementations.
Full Implementations -> Winner only (clean losers)
Code Fragmentation -> Clean unused test code.

Branch Based Workflow (git based)

Test Iteration Implementation -> Rapid
Full Impl -> Winner only

What will we do differently after this?
Added some new metrics I was missing thanks to this in relationship to acquisition and activations.
Will try new tools like
Will look at what tech might be slowing us down.
Will not build more visualization tools of test data.

#IWasThinking About the future of society #S02E01

How do you see things unfolding in a world where technological advancements replace every major sector of the economy in which human beings play a major role?

No more humans necessary for manufacturing, health care (yes robot nurses and doctors for emergency surgeries, organ and tissue regeneration thanks to advanced stem cell technology, nanotechnology+biotech, custom dna drugs), transportation (self driving vehicles with 0.00000000001% error rates, no more taxi drivers, bus drivers, truck drivers, plane pilots, all the texting you want, no more designated drivers)…

what will we do with millions of useless people?

Please comment, subscribe and like this video. Trolls welcome.