Sew Make Do – A Lean Startup Experiment – Part 2, Metrics

Mrs Fragile recently bought a hand made lamp shade online and was disappointed with the results, as a keen crafter she wondered if she could do better, and perhaps even sell some of her own creations.

In doing so I thought it would interesting to incorporate ideas from the Lean Startup Movement as popularised by Eric Ries and document progress through Fragile. The project is named Sew Make Do.

Metrics

A key idea in lean startups is that metrics ought to be actionable. On his blog Ash Maurya defines explains Actionable Metrics 

An actionable metric is one that ties specific and repeatable actions to observed results.

The opposite of actionable metrics are vanity metrics (like web hits or number of downloads) which only serve to document the current state of the product but offer no insight into how we got here or what to do next.

Tracking sales is of course an obvious thing to do but it is a very coarse measure. A more interesting metric is to look at how easy it is to convert a potential customer into a real customer. Over time we not only expect sales to increase but also expect to get better at selling such that our conversion rates also increase.

In an ideal world I would like to perform Cohort Analysis. This means tracking individual user behaviour and using it determine key actionable metrics. While more commonly applied in medical research in order to study the long term affects of drugs, common examples in the context of Lean Startups might be tracking user sign up and subsequent engagement over time. If it can be shown that 2 months after sign up users generally cease to engage in a service, it provides a pointer to what to work on next, as well as a clean means to determine if progress is being made.

The in-house analytics provided by Etsy do not provide the means to track the habits of specific users, but they do allow for aggregations of periods of time. This means that some level of analysis is still possible, though cannot be describes as true cohort analysis.

I’ve modelled my funnel like so:-

Of those that viewed the shop

  • What percentage favourited the shop or a product. There is no reason to assume that someone buying the product will also favourite it, though at this point it is reasonable to assume some level of correlation.
  • What percentage bought a product for the first time
  • What percentage are returning paying customers buying a subsequent item.

As you can see from the graph, there is not a lot of data. Throughout the process our absolute views and favourites have increased, though it is interesting to see that our favourited percentage has improved. We put this down to improving the pictures and copy, though without more data it’s hard to make any firm statements.

What I’ve not done is break this down on a per product basis, right now we do not have enough products or traffic to justify it but we’re certainly noticing that some products are more popular.

In a few months times I’ll revisit this post and let you know how things are going. With a bit of luck there’ll be some yellow and green on there.

Sew Make Do – A Lean Startup Experiment

I’ve been an advocate of applying lean thinking to software for some time, and learnt a lot form Eric Ries’s blog. I’ve just finished Ries’s book ‘The Lean Startup’ and naturally am looking for opportunities to apply its ideas in my own work place. However doing so will take time and more immediately I wondered what would happen if I started on something smaller.

Mrs Fragile recently bought a hand made lamp shade online and was disappointed with the results, as a keen crafter she wondered if she could do better, and perhaps even sell some of her own creations. While initially suspicious of my gallant offers to help her run things on lean startup lines so far she’s tolerating my efforts.

I thought it would interesting to document progress through fragile and perhaps receive some feedback/advice along the way. The nice thing is that since this is not a serious venture it should be possible to be more open then would other wise be possible. The project is named Sew Make Do.

Assumptions

We started with the following assumptions to test.

  1. People would like to buy Mrs Fragile’s lamp shades
  2. The people that would like to buy the lamp shades are female and in their late 20’s to early 40’s.
  3. 30cm and 20cm drums will be most popular.
  4. People will pay ~£28 for a 30cm shade
  5. People will pay ~£22 for a 20cm shade
  6. People will suggest custom fabrics to drive product development.

Of these assumptions by far the most risky is No 1. We have no idea if anyone will actually want to buy them. Therefore it makes sense to prioritise testing this assumption. To this end Mrs Fragile set up a shop on Etsy and presented a choice of 3 lamp shades offering a range of styles and sizes. This is our MVP for assumption 1. There is no reason to assume that long term Etsy will be the main distribution channel but it does provide a very quick way to put the product in front of potential customers.

Once, assumption 1 has been tested sufficiently to give us hope to persevere it will be easier to address the remaining assumptions, since all are dependent on sales.

Thoughts on metrics

The lamps shades have been up for a few days now, so far there have no sales but a good number of people have ‘admired’ them. It will be interesting to see if there is a link between the number of views, the number of admires and the number of sales. Longer term it would be interesting to perform cohort analysis on these indicators.

For now though we’re just hoping for the first sale – or possibly our cue to try something else…..