Twitter Clone Tutorial in Web2py Part 1 – Getting started

Welcome to my getting started with Web2py tutorial.

This tutorial assumes very little on the part of the reader, some knowledge of a programming language will certainly help, but if not don’t worry. I’ll take you from installation through to v1 of your application.

If you don’t want to copy from the tutorial, the full source is hosted on github.

Why Web2py?

Web2py is a fantastic web development framework for Python, it focuses on

  • Ease of use
  • Rapid development
  • Security

Additionally it had a excellent documentation and a highly responsive mailing list, these factors combine to make it a great place to start for those new to web development and programming in general.

To give an idea of just how easy it is to get going this tutorial we will build a twitter clone from scratch – nothing too flashy, but enough to demonstrate some core ideas to get you going.

The Twitter Clone – Witter

We’ll call the app Witter and steal other Twitterisms such as weets and weeting.

The app will be made up of three pages

  • A home page, that displays all the weets of the user as well as those of anyone they follow
  • A wall page, that displays profile details of a given user as well as their weets
  • A search page that can be used to find others users and follow and unfollow

 

Home_reduced

Getting Web2Py

There are a number of ways to do this, the easiest is to download the binary packages available for OS X and Windows. The binaries includes everything you need including a Python interpreter and an integrated development environment. Linux users will need to download the source, if you are comfortable with Linux I’m going to assume that you are comfortable with installing web2py.

Once downloaded you have everything you need and it is time to start web2py for the first time.

On OS X you’ll want to copy your newly downloaded file to your Applications folder, then double click just as you would any normal application to launch.

On Windows, copy web2py.exe to Program Files and double click to launch:

When prompted enter an admin password, and click ‘Start Server’ and soon after you will be presented with the web2py welcome app.

This is great place to start, including some useful examples and documentation, for now just click on the ‘Administrative Interface’ button on the right hand side to reach the site view of the administrative interface.

site_admin_reduced

Creating your new application

The first thing to do is to create the new application, on the right hand side under ‘New Simple Application’ enter ‘witter’, this will be the name of our Twitter clone. With a bit of luck you are now looking at the Witter administrative interface – we are going to spend some time taking a tour of this page, but before we do that let’s head to the Witter home page. In a separate tab point your browser to

http://127.0.0.1:8000/witter
Witter_default_index_reduced

Congratulations  – this is your brand new application, you’ll notice that it looks very similar to the welcome application, it is just using the same default content. We soon get rid of that in the next post as we start in earnest on Witter.

What ‘How your company treats exiting staff’ says about you

Nothing impacts your working life like company culture and yet culture is very difficult to assess. This problem keeps me up at night and I’ve written about it before in ‘Why work at your company?

The trouble is that in many cases, the company itself does not really know that much about its own culture and even if it did, expressing it in a sincere manner is challenging, in fact most companies don’t even try at all.

But how about this as a question to quickly determine a how a company feels about its staff?

“HR hoops aside, what happened the last time someone left your company?”

Seems pretty innocent, no? Every company can answer that question in a positive manner, but I think the variations speak volumes.

For instance, recently we said goodbye to junior (ish) developer, he’d been with the company 18 months and was leaving to pursue a childhood dream to work as an air traffic controller (actually true).

On his last day there was a leaving ceremony where a few words were said in the office and a leaving gift presented, immediately afterwards there was a trip to the pub for those wishing to see him off.

Nothing unusual so far.

But it was the whole office of eighty people that turned out – the parting gifts, while inexpensive, were completely personalised – in this case it was a ‘make your own picture book’ with each picture and audio presenting some sort of in joke – jokes understood by the whole group. In similar situations, gifts have included a ‘time of day’ vs ‘date’ commit plot for a notoriously nocturnal team member, a Wordle of IRC logs for an especially chatty colleague and even a framed 3ware card (his nemesis) for a long suffering sys admin.

The pub trip was not just his team or engineering, but had attendees from across the entire company, and at all levels of seniority. Numbers were high, especially given it was a Friday night.

It was a sad occasion, but in many ways felt like a celebration. ‘Great working with you, can’t wait to see what you do next’ – it’s one thing for your immediate work mates to say this, but how about a whole company?

What struck me was how normal this felt, this is completely standard practice for us. Why wouldn’t we act in this way? On reflection I think that this would be considered unusual at most organisations. Such sends off are not unheard of but usually reserved for especially treasured members of the team. This difference say something about our culture.

By comparison, at my friend’s organisation, a senior (much loved) member of staff of 20 years service cannot even expect their manager to drop into their leaving ceremony. Their reluctance to attend driven by fear of tacit endorsement. It’s the same set up, leaving ceremony followed by pub, but a completely different feel.

How a company treats its exiting employees, speaks volumes to remaining staff. The actions of individuals in response to the leaver, reflect the company’s culture. Since the leaver no longer has any direct value to the company, these actions speak honestly about how the company values people.

So ask yourself, how does your company treat exiting employees and what does it say about the culture? What message are you sending to remaining staff? It is not for management to mandate a leaving procedure, but it is for management to create an environment where people matter.

How to answer ‘What is your greatest weakness?’

So, Mr Johnson, what would you say is your greatest weakness?

Worst. Interview. Question. Ever.

In isolation, it’s an interesting question to ask yourself, it requires a great deal of honesty and openness on the part of the individual and it requires trust that there will not be repercussions for the answer.

Sadly, an interview is a place for none of those things. Candidates need, and are expected to, go out of their way to impress their interviewers and actively admitting weakness is a high risk strategy.

And yet it’s popularity remains, company’s still ask it and candidates still find a thousand and one tortured methods to say “I’m a workaholic”. In a novel attempt to get around this, one company used to ask for your three greatest weaknesses. I’d love to know what they gained from this.

So how should this question be answered? From personal experience, responding ‘My left elbow’ sadly does not work.

In comes down to framing, strengths and weaknesses are pretty much alternate views on the same characteristic. Above a certain level of competence it is rare to find people truely good at certain pairings.

For instance, think those people who you admire for their vision. These people are inspiring, charismatic they lift you up and show you a world that you dimly knew existed but couldn’t quite realise. Wonderful people, but tell me, how are they on detail? How good are they are dotting the Is and crossing the Ts and meticulously going through the minutiae of a project, ensuring that everything is as it should be. Not so much huh?

Clearly there will exceptions but Richard Branson, just isn’t a details kind of guy and nor should he be. Conversely I want an Accountant to be exactly the opposite, they need to get down to the details.

Since the question is now framed in terms of strengths it is much easier to answer. Just figure out what strength of yours you wish to highlight, determine how that makes you weak and then go for it.

Not convinced? Here is an example.

Imagine for instance the Great Remallo, Lion Tamer extraordinaire, sitting for an interview. His CV/Resume contains plenty of past experience dealing with many different types of lion and as the inventor of the lion proof cape he clearly knows his stuff. As the final question of the interview the Circus owner, Mr Top, fires out the big kahuna:-

Mr Top: Mr Remallo I have one final question for you. Tell me, what is your great weakness?
The Great Remallo: Well, Mr Top, if I had one weakness it would be that I seem to have trouble accurately assessing the level of danger I might be in at any given time, even as a small child when I used to play in traffic. Actually it’s something that really helps me as a lion tamer, I should think that if I ever truley understod the peril I put myself in everyday I would be petrified and never even enter the ring.

Still not convinced?  Let’s try the same question for a Software Developer.

My biggest weakness is that I can’t stand not knowing how things work, it used to drive my family crazy as took everything I owned (and somethings I didn’t) apart in order to see what was going on under the hood. To this day I really struggle to take things at face value if I can’t look and see what is going on. I suppose it’s one of the reasons I became an engineer in the first place, this drive to understand how things works stands me in good stead when wrestling with a gnarly code base or diving deep into library code, bug hunting.

I’m sure you can come up with something better yourself. Just remember if you reframe the question as an opportunity to talk about strengths, it’s then just a question of determining the flip side to that strength and using it as an in.

Anyone else got a good answer to this question?

 

The Complex, the Complicated and what it means for NASA

When people say ‘Simplifying Complexity’ I hear ‘I don’t know what these words mean’. This isn’t entirely fair, as we shall see, but it gives you an idea of my preconceptions before attending a training session of that title.

The facilitator started off by inviting each member of the group to talk about the influence of complexity in their professional lives. As we went around the room it became clear that for most of the group complexity was a synonym for hard or complicated. Many would agree, including the Oxford English dictionary. However, here was a group dominated by senior members of a very large IT professional services company. These are people for whom complexity science is highly relevant to their professional lives and people for whom there ought to be value in defining Complexity with a big C and in particular differentiating the Complex from the Complicated.

Cynefin

The Cynefin Framework characterises systems (and problem spaces) into five classes – Simple, Complicated, Complex, Chaotic and Disordered. Once the class of a system is identified the framework provides guidance on how to work most effectively with the system. On a personal level I find it very useful in explaining to me not only why an incremental and iterative approach generally works well in software development, it also suggests to me under what circumstances such as approach does not hold. You won’t see NASA using Scrum for instance, more on that later.

Cynefin can be expressed visually like so:-

 

cynefin_as_of_1st_june_2014-1

Credit Wikipedia

The key distinction I see is that of Complex and the Complicated.

  • Complicated

An example of a complicated task is building a submarine. It’s definitely not trivial, but if the tasks are broken down sufficiently the problem is tractable and predictable – essentially it becomes a series of Simple tasks. By contrast a Complex task retains its complexity even after being broken down.

  • Complex

An example of a Complex task could be to ask a room of people to arrange themselves such that the closest person to them is exactly half the distance from them as that of the 2nd closest. It would be extremely difficult for an individual to orchestrate this task, instead each individual actor must make a small change, observe the consequence and then make a further change based on the feedback.

What does this mean for Software?

Taking a manufacturing production line as an example, once it reaches steady state the system can be described as Complicated and is predictable. This is important because for many years Software Development looked to the world of manufacturing for guidance. In doing so, implicitly defining software development as a Complicated task. The thinking being that with sufficient up front analysis the problem could be solved without a line of code being being written. It is for this reason that the Waterfall model rose to such prominence and was so readily adopted.

More recently manufacturing has been shown to be an inadequate metaphor and that in fact software development is more akin to product design, thereby  inhabiting the Complex quadrant. This means that the correct approach, according to Cynefin, is to probe, sense and respond. In effect the iterative and incremental approach practiced by those inspired by Agile and Lean thinking.

So what does this mean for NASA?

If what I say is correct, what does this mean for NASA? After all they have some of the smartest brains on the planet available to think about this sort of thing and yet their processes appear to assume a Complicated rather than a Complex environment.

NASA is an extreme organisation and it is only natural that what works for them will deviate from the common case. Despite the challenging nature of their work I would argue that their domain is Complicated rather than Complex.

Complex systems are characterised by there being many unknowns and an inability to determine the nature of those unknowns at the beginning of the process. In NASA’s case they are genuinely in a position where, for software at least, the problem space is well defined.

For instance, the software runs upon hardware based on Intel’s 386 architecture, hardly cutting edge, but whose behaviour (warts and all) is well understood and has not changed in years. This means that the inputs to the system are well understood and the behaviour deterministic. NASA has managed to turn what would ordinarily be a Complex system (that of software development) into a Complicated system and in doing so their software teams work to vanishingly small defect rates, albeit at enormous cost and at the expense of delivery time. I found this article to be a fascinating insight into their working practices.

In conclusion, Complexity science provides a means to characterise systems, the Cynefin framework provides definitions to differentiate the Complex from the Complicated. Software development is almost always Complex though it was originally assumed to be Complicated. This is why agile and lean approaches have been shown to much more effective than traditional methods that assume a Complicated system such as Waterfall.

Identity and Narrative – Managing Change

People hate change, and the reason they hate change is that they really hate change, and that change is hated because they really hate change…….

I’d love to know who said this

All teams are subjected to continuous environmental change, but it tends to be gradual and hard to perceive at a week by week level. I want to talk about the sharp, often unexpected step changes and go into some strategies to guide a team through the worst.

Before diving in, I want to introduce a model for characterising teams. There are two attributes that I consider critical in determining a team’s ability to function.

  • Identity – Who the team perceive themselves to be, what they value.
  • Narrative – Why the team exists, what value they bring.

I’m unaware of anyone else talking specifically in these terms but similar thinking appears in Daniel Pink’s ideas of Autonomy, Mastery (both mapping to Identity) and Purpose (narrative) as well as echoes in the higher levels of Maslow’s hierarchy of needs.

Ordinarily, definition of identity and narrative is straight forward. The team will arrive at their own identity over time, while the narrative, for the most part, comes from the commercial arm of the company. In times of change there are no such guarantees. I’ll look at each in turn.

Identity

As an individual, our identity is in part context specific and a function of those around us. The same is true for teams. This means that when the environment changes quickly, it can be difficult for a team to define itself. Definition means identifying the skills and attributes that set it apart and most importantly what it values when compared to those around it.

A manager can help speed this process. They have a birds eye view, they know how their team have defined themselves in the past and have more opportunities to interact with the broader business. The manager ought to be able to spot and highlight specific points that will go and form part of the team’s new, long term identity.

Additionally during upheaval it is for the manager to contextualise the actions and focus of other teams/departments. It’s all too easy to enter into a spiral where ‘everyone apart from us is an idiot’. A team needs to understand how they are different, but they also need to collaborate and work effectively with those around them.

Narrative

Narrative is interesting in that it should be easy to identify. The business is willing to invest in the team for some purpose and that purpose ought to be the team’s narrative.

During times of upheaval this is not a given, and it could take months for a clear narrative to emerge, as the dust settles and the business redetermines the best way for the team to add value.

But waiting months for the new vision is not an option. Put bluntly, if the business cannot provide a compelling narrative quickly then the team manager must arrive at one. Once again it is time to make use of the manager’s elevated view of the organisation to sift through the confusion and draw out something tangible that resonates.

Conclusion

All teams need a sense of identity and a sense of narrative in order to be productive. During times of significant change both of these characteristics come into question. It is up to the team’s manager to act as the catalyst, as the team aims to arrive at new definitions.

Sew Make Do – A Lean Startup Experiment – Part 2, Metrics

Mrs Fragile recently bought a hand made lamp shade online and was disappointed with the results, as a keen crafter she wondered if she could do better, and perhaps even sell some of her own creations.

In doing so I thought it would interesting to incorporate ideas from the Lean Startup Movement as popularised by Eric Ries and document progress through Fragile. The project is named Sew Make Do.

Metrics

A key idea in lean startups is that metrics ought to be actionable. On his blog Ash Maurya defines explains Actionable Metrics 

An actionable metric is one that ties specific and repeatable actions to observed results.

The opposite of actionable metrics are vanity metrics (like web hits or number of downloads) which only serve to document the current state of the product but offer no insight into how we got here or what to do next.

Tracking sales is of course an obvious thing to do but it is a very coarse measure. A more interesting metric is to look at how easy it is to convert a potential customer into a real customer. Over time we not only expect sales to increase but also expect to get better at selling such that our conversion rates also increase.

In an ideal world I would like to perform Cohort Analysis. This means tracking individual user behaviour and using it determine key actionable metrics. While more commonly applied in medical research in order to study the long term affects of drugs, common examples in the context of Lean Startups might be tracking user sign up and subsequent engagement over time. If it can be shown that 2 months after sign up users generally cease to engage in a service, it provides a pointer to what to work on next, as well as a clean means to determine if progress is being made.

The in-house analytics provided by Etsy do not provide the means to track the habits of specific users, but they do allow for aggregations of periods of time. This means that some level of analysis is still possible, though cannot be describes as true cohort analysis.

I’ve modelled my funnel like so:-

Of those that viewed the shop

  • What percentage favourited the shop or a product. There is no reason to assume that someone buying the product will also favourite it, though at this point it is reasonable to assume some level of correlation.
  • What percentage bought a product for the first time
  • What percentage are returning paying customers buying a subsequent item.

As you can see from the graph, there is not a lot of data. Throughout the process our absolute views and favourites have increased, though it is interesting to see that our favourited percentage has improved. We put this down to improving the pictures and copy, though without more data it’s hard to make any firm statements.

What I’ve not done is break this down on a per product basis, right now we do not have enough products or traffic to justify it but we’re certainly noticing that some products are more popular.

In a few months times I’ll revisit this post and let you know how things are going. With a bit of luck there’ll be some yellow and green on there.

Sew Make Do – A Lean Startup Experiment

I’ve been an advocate of applying lean thinking to software for some time, and learnt a lot form Eric Ries’s blog. I’ve just finished Ries’s book ‘The Lean Startup’ and naturally am looking for opportunities to apply its ideas in my own work place. However doing so will take time and more immediately I wondered what would happen if I started on something smaller.

Mrs Fragile recently bought a hand made lamp shade online and was disappointed with the results, as a keen crafter she wondered if she could do better, and perhaps even sell some of her own creations. While initially suspicious of my gallant offers to help her run things on lean startup lines so far she’s tolerating my efforts.

I thought it would interesting to document progress through fragile and perhaps receive some feedback/advice along the way. The nice thing is that since this is not a serious venture it should be possible to be more open then would other wise be possible. The project is named Sew Make Do.

Assumptions

We started with the following assumptions to test.

  1. People would like to buy Mrs Fragile’s lamp shades
  2. The people that would like to buy the lamp shades are female and in their late 20’s to early 40’s.
  3. 30cm and 20cm drums will be most popular.
  4. People will pay ~£28 for a 30cm shade
  5. People will pay ~£22 for a 20cm shade
  6. People will suggest custom fabrics to drive product development.

Of these assumptions by far the most risky is No 1. We have no idea if anyone will actually want to buy them. Therefore it makes sense to prioritise testing this assumption. To this end Mrs Fragile set up a shop on Etsy and presented a choice of 3 lamp shades offering a range of styles and sizes. This is our MVP for assumption 1. There is no reason to assume that long term Etsy will be the main distribution channel but it does provide a very quick way to put the product in front of potential customers.

Once, assumption 1 has been tested sufficiently to give us hope to persevere it will be easier to address the remaining assumptions, since all are dependent on sales.

Thoughts on metrics

The lamps shades have been up for a few days now, so far there have no sales but a good number of people have ‘admired’ them. It will be interesting to see if there is a link between the number of views, the number of admires and the number of sales. Longer term it would be interesting to perform cohort analysis on these indicators.

For now though we’re just hoping for the first sale – or possibly our cue to try something else…..

 

 

Too Much Trust

Trust trust trust trust trust trust trust trust trust trust
Excerpt from the management book I wish someone would write

 

A central theme in agile software development is that of trust. The agile (small a) movement speaks of openness, collaboration and collective responsibility – none of which are possible without trust. As a manager my team cannot be effective if they do not trust each other nor can I bring about anything but the most superficial change if they don’t trust me.

I’m not the only one who feels this way, turns out I’m in good company 1 2 3

So I like trust and consider it to be a ‘good thing’ but the point of this post is not to talk about how great it would be if there was more trust in the world. In fact I want to talk about situations where increasing trust can actually be destructive.

The total level of trust is undoubtedly important, but equally important is the distribution of that trust. The greater the differential between the relationship containing the most trust and that containing the least the less chance that the overall group can act as effective team.

A good high level example might be an engineering org and a sales org. It doesn’t matter how much internal org trust exists – if org to org trust is low the company will not perform as well. In fact the lack of inter org trust will felt all the more keenly in contrast to the strong internal trust that exists.

Applying this idea to a single engineering team, if a team has high trust for one another and a new member joins then it will take time for that new member to earn the group’s trust and be accepted as part of the team. This healthy and only natural. However if the team is split down the middle with two groups of high internal trust who do not trust one another then strengthening internal group trust will only entrench the distrust of the other group. In this case increasing trust can actually be harmful.

What I’m saying is that the effectiveness of a group to act as a team can be characterised by the weakest trust links in the group. If the differential between relationships is high then increasing trust in already strong relationships can actually hinder rather than help the team.

From a practical perspective, the manager’s job is always to create an environment where trust can grow, but it is important to focus on the low trust relationships since they are the ones that characterise the effectiveness of the team.

Worried about candidates googling during a phone screen? You’re doing it wrong.

Interviewing is time consuming, companies have a finite amount of time to dedicate to recruitment and inevitably some capable candidates are turned down at CV stage without ever having a chance to shine.

Phone screens are a great way to address this problem, they are typically shorter and often run solo. They allow a company to take more risks and consider candidates from further afield.

My company is still pretty new to phone screening, we’ve been trialling it out in cases where it is difficult for the candidate to attend in person – perhaps they are based overseas. As a result I’ve been doing a lot of reading on how best to construct a decent phone screen. By far the best writing I’ve found is Steve Yegge’s take. I’m not sure how practical it is to fit everything Yegge mentions into a 45 minute call, but I consider it an excellent resource.

A common fear I have seen in other discussions seems to be that candidates will use google to somehow game the system. If this is a genuine concern then one of two things has gone wrong. Either:-

  • The questions are purely fact based and will tell the interviewer nothing about how the candidate thinks.
  • Or, the questions are fine but the interviewer is focusing on the wrong part of the answer.

A question like ‘In Java what is the difference between finally, final and finalize’ will tell you very little about the candidate. Plenty of terrible programmers could answer that without problem and what’s worse, a talented but inexperienced developer might stumble. In short these type of quick fire questions add little value to the overall process.

Something like ‘How does a Hash Map work? How would you write a naive implementation?’ is more interesting, it’s open ended but forces the candidate to talk about a specific area of knowledge – even if they don’t know, you’ll learn how good they are at thinking things through from first principles. The only way that it can be gamed through googling is if the interviewer simply waiting to hear specific terms and is not asking free form follow ups.

I’ve just googled Hash Maps on wikipedia and could probably quickly extract ‘Associative array’, ‘key-value pair’, ‘Collision’ but really if that’s all the interviewer wants to hear then the question is of limited value.

So what I’m saying is that if you’re concerned about googling, then it’s probably the questions or desired answers that are the problem. Furthermore if one in a hundred people do manage to game the system you’ll pick them up in the face to face in an instant.