The 30 second guide to planning fallacy and what you can do about it

At the digital agency I help run, we work hard to try and ensure projects are delivered on time, to a high quality and with minimal stress for all concerned. We are, I’m pleased to say, good at delivering projects on time. This is in part because we put quite a lot of emphasis on estimating and planning projects (although there is always room for improvement of course).

When estimating it’s important to be aware of something called optimism bias. This is when people underestimate their own but not others’ completion times for tasks.  Even with tasks similar to things we’ve done before, time and time again we still tend to make optimistic estimates. This is also referred to as Planning Fallacy (pdf). Consciously or subconsciously, we kid ourselves.

This has been studied by more clever people than me and here are some headlines:

  • People focus on the most optimistic scenario for the task, rather than using their full experience of how much time similar tasks require.
  • The bias only affects predictions about one’s own tasks; when uninvolved observers predict task times, they show a pessimistic bias, overestimating the time taken.
  • Anecdotally, it appears that although people fail to meet their predictions, they do typically meet important deadlines.
  • One experiment found that when people made their predictions anonymously, they do not show the optimistic bias.

This last point suggests that people may make optimistic estimates so as to create a favorable impression with others – colleagues, managers or clients and possibly themselves.  In competitive environments and industries, there is clearly pressure to give optimistic estimates and costs for work. We’re geared towards doing everything efficiently and with minimal waste. Inefficiency is weakness.  Delivery and timescales are the priority.  This is true in both our professional and personal lives.

Do we really estimate it will take 8 weeks to create this thingy? We can’t go back to the client with 8 weeks, they’ll think we’re idiots. It must have been overestimated. We can do it faster this time I’m sure.

There is a fear that we, our team, our company might be seen as being inefficient. In competitive arenas, perhaps through procurement or in pitches, that is a cardinal and potentially business losing sin.

So if we accept this planning fallacy exists, what can we do?

Step 1 – be aware this happens. Make other people aware it happens.

Step 2 – get an external view of the estimate. A view more removed from the project. If you have a system which tracks timesheets and gives you stats on how similar’ projects fared, make use of it and let it influence you. If you feel something is overestimated, get someone more distant and ideally more experienced to check it. (The bias only affects predictions about one’s own tasks) There are two outcomes from this:

Step 3 – you may get two quite different estimates. Don’t treat this as a bad thing, it is valuable. This indicates different interpretations from different people. There is no smoke without fire, something, somewhere is unclear – scope, scale, experience, requirements whatever. If you can identify the source of the smoke and take steps to remedy it, you may avoid a fire.

Step 4 – you may find that both estimates do indeed indicate a project longer than you’d thought. Again, this is a valuable and possibly face saving early warning system. You may still have time to avert disaster by exploring options – reducing scope, phasing delivery, adding people etc. It’s better to know about the problem early when you may still be able to remedy it.

Step 5 – control your projects. The project you deliver is not the one you originally estimated.  An accurate initial estimate is useless if you don’t have good project controls in place (it appears that although people fail to meet their predictions, they do typically meet important deadlines). I will revisit this point in a future post.

Step 6 and perhaps the most important. Create an environment where people feel they can give honest objective estimates. Yes there will always be commercial pressures, but underestimating projects causes pain for you, your teams and your clients. DO NOT put pressure on your staff to promise delivery to optimistic deadlines. Try to separate the estimate and the commercials.

Step 7 – trust your gut instinct and intuition.

These steps are important whether you are agency or client side. If as a client, an agency is telling you they can deliver the project in half the time that all other agencies say they can, then they are either inexperienced, falling victim to planning fallacy or telling you what they think you want to hear. All of these are going to lead to a place of unhappiness.

If you have other steps to add then let me know. Hopefully you got the mild joke in the title.

References

The New York Time – The Busy Trap

Exploring the “Planning Fallacy”: Why People Underestimate Their Task Completion Time (pdf)

Psychologists call it the Planning Fallacy. Nobel laureate Daniel Kahneman explains why people so often get it wrong. (video)

Why things takes longer – probability hates you

When you think about why things often take longer than you originally thought, there are some reasonably obvious reasons which jump out. But there is one hugely significant reason which most people don’t consider. Understanding this reason is important for anyone interested in getting things done.

Originally I titled this “variance is not symmetric” but that wasn’t as catchy as “probability hates you”. Variance is not symmetric, may sound a little complex but it is in fact rather simple (although a little difficult to explain…bear with). Essentially when we break things down into individual tasks, we expect that some tasks will take a bit longer and some a bit less. Overall we hope they’ll balance out and you’ll finish on time. Graphically this would look like this (apologies for my rusty photoshop skills):

Bell Curve 1Importantly the area to the left of the line (early) is equal to the area of the right (late) which basically means the probability of being a bit early is equal to the probability of being a bit late. It’s all in balance. 

The truth is not symmetrical 

In real life though, the graph is not symmetrical. Consider: There is a limit to how early a task can be, but no limit to how late a task can be. 

Or put another way: There is no maximum limit to how long something can take, but there is a limit to how little time something can take. When applied to lots of tasks I’m afraid this works against us.

Imagine you estimate two tasks will take two hours each. Total time is four hours. 

So you crack on and Task One goes badly and takes five hours, so overall you are running three hours behind. Ouch, maybe Task Two will make up the time. Fortunately, Task Two goes very well indeed, but it can never take the minus one hour you need to get your three hours back in order to get back on schedule. So no matter how well Task Two goes, you will be late. So I think the bell curve actually looks more like this:

bell-curve 2The area under the right side (late) is greater than the area to the left (early) which is bad news and why probability hates you. Even worse, theoretically the long tail probably reaches to infinity! There is absolutely no limit to how long a thing can take. 

So often tasks which take longer than expected are not balanced out by other tasks which take less time than expected. Any single task which takes a lot longer than expected requires many other tasks to go well in order to get you back on schedule.

Sorry.

Why we should ‘create’ not ‘build’ websites

Before I delve in, let me state that I am not a developer. However I studied physics and engineering for four years, have done some coding and have a good sense of logic.  I’ve worked with developers for 14 years and so I think I have reasonable qualifications to comment.

Development is not assembly

Often I hear people discussing the process of creating something digital using terms connected to construction and manufacturing – ‘build a site ‘ and  ‘wireframes are like blueprints’ are common examples.

While I understand this, I believe it is misleading and actually damaging. Construction and manufacturing are about assembling components and repetition. You try to make the same task highly repeatable so you can do it again and again and again – reducing variation (and therefore cost) as much as possible.

When creating digital solutions (i.e. websites, apps etc) the possibilities are endless and so the solution is almost always new and unique. It is this uniqueness which actually gives the product its value.  If you want the product to be the same or similar to the last one we made, then yes you can have it cheaper and faster. But you don’t.

Development is invention

I believe this to be particularly true during development. Development involves solving many problems which have many possible solutions. If you consider that development is perhaps more akin to design or invention than assembly, you may start to understand why estimating how long development tasks will take is difficult.  An inventor will find it difficult to tell you what percentage of his invention is done. Yet this is a question that clients and project managers ask developers every day. But, we live in a commercial world and so it is both an unfair and fair question.

The only way to fully understand with 100% accuracy how long it will take is to solve the development problems is do the actual work. The very process of creating the product actually defines what the product is. Catch 22.  This makes estimating how long something will take at the outset rather difficult. Having a separate scoping and production phase helps a lot, but you cannot completely separate design and production –  the agile approach embraces this.   

You could argue that ‘creating the product actually defines the product’ could be said to some degree for almost everything. But it is the uniqueness in digital which is the big factor. Presumably if you asked Barratt Homes how long to make their “Canterbury” model executive home they could be pretty accurate. If you asked a builder how long to build a home which is quite different to the last one they built, then even if it has full wireframes (sorry I meant blueprints) then they’ll come back with a range of 4 to 6 months. When you then consider that humans are very experienced at making buildings, that’s a pretty wide estimate range. 

So what to do?  

If you are building something similar to something you’ve made before, on a platform you’ve been working on for a long time and with an experienced team then your estimates should be accurate. Fortunately this is the case where I spend my working days, which makes my role as Head of Production somewhat easier.  There will be inaccuracies of course, but within acceptable tolerances.   Any good project lead should be able to bring you home safely.

If you are building something new, on a new platform with a new development team then your estimates are going to be very inaccurate. You are fairly stuffed I’m afraid. Try to remove the word ‘new’ from as many aspects as possible. Get in as many brilliant people with relevant experience as you can. If you have to commit to a timescale, cover your arse and put in as much contingency as you can possibly get away with. Alternatively go for an agile approach if you can sell it in, or frankly – consider a different job.

And if you are talking to clients (and indeed developers), use terms like create, shape or develop, not build or construct.

Tips to run 520 miles in a year

At the start of 2013 I set myself an ambition to run 520 miles before the end of December. I’d been running on and off for a year, completed a half marathon but was erratic and inconsistent. In the summer months I’d run lots and in others I’d do little – basically going back to square one several times a year.

Setting myself a target for the year I hoped might force me into better habits. I set the target at a level which meant averaging 10 miles a week, which is only a couple of shortish runs a week. Quite achievable. Low enough that if I got a little behind I could make it up without too much trouble.

I won’t bore you with the details, but I got waaaaay behind. I had a few months where I had constant colds and a month where I really knackered my leg with a stress fracture and did nothing for weeks. By that point I’d given up, I was too far behind to ever catchup. But following a 12 mile run at Tough Mudder I realised I was still in relatively OK shape and got a calculator out. I did the sums and at 17 miles a week for the rest of the year I could do it. So I dug in and did that weekly mileage through the dark months of October, November and December. I finally ran the last few miles in the Canadian Rockies on December 30th cheered on by my wife and parents in a stunning snowy landscape. I felt like I’d just won the Olympics. All the Olympics.

Which was quite amazing because not only had I achieved something I wasn’t sure I was capable of, but I’d fitted it into my day to day life with a wife and 2 year old. I hadn’t spent lots of cash and I hadn’t had to take 2 months off work to do it.

So this year I’m doing it again. I’m optimisically aiming for 838 miles (John O’ Groats to Lands End by Google maps) which will be difficult (about 16.5 miles a week). There are a group of people on Facebook I’ve managed to cajole into trying it and so far they seem to be doing really well. Some are doing 520, some more and some less. In truth, the number doesn’t matter. What matters is getting out and doing something you thought would be hard, finding out it is quite hard but that you can still do it.

monthlymileage2013

February and March were full of colds. Not sure what happened in May and in August I injured myself.

520.56 miles equals about

  •  108 runs
  • 80,000 calories
  • 78 hours

Tips for doing 520 miles in a year

1. Choose a total which is achievable

10 miles a week is good if you don’t have much time. If you are just taking up running then go for 260 miles or 5 miles a week. That is still an amazing achievement. If you choose a number which is too high, if you fall behind you will really struggle to make up the lost miles. It has to be hard but achievable.

2. In the winter just try to keep yourself in the game

It’s cold and dark. If you’re losing a few miles a week don’t worry. You’ll hit spring in good shape and make up the miles easily.

3. Learn to fit it into your life with minimal impact

If it’s an inconvenience it makes it twice as hard. Especially in wintertime. If you can do a run at lunchtime once a week then you’re half way there without having to go out in the evening. If you run at work then keep washing stuff there so you don’t have to constantly lug it all to and from work. Minimise the inconvenience on yourself and your family as much as possible.

4. Tell people what you are doing

It will sound cool. It will make you want to succeed if you’ve told people.

5. Use a GPS device

Use an app on a phone (runkeeper, strava, Nike+ etc) or a GPS watch. You don’t want to have to measure the miles with a piece of string and a map.

6. Countdown the miles each week

Every time you run, count down the miles remaining that week – “Only 4 miles left his week”. Plan when you’re going to fit the remainder in and try and get into a weekly routine.  Look at the forecast to figure out when it’s going to be the best weather.

7. Sign up for a couple of races

Book yourself onto some 5Ks, 10Ks or half marathons. They give you something to focus on and will boost your confidence. You’ll go faster than you think.  One day you might find yourself in a race where there’s not a single person in fancy dress and everyone has t-shirts saying Flosbury Flyers and the like. Then you’ll feel proud of yourself. Those people are in proper running clubs and you’ll beat some of them. If you can, try and find someone you know who enters such races and go with them, it’ll make them seem less daunting.

8. Increase your mileage slowly

If you are behind, don’t overstretch yourself just to make up the miles. People don’t get injured, they injure themselves. I did just that by trying to catch up on lost miles too quickly. And it’s sore.

9. Reward yourself with chips

You’ll burn a lot of calories. Reward yourself with Chinese takeaway, wine and pizzas.  That goes for kit too, you don’t need much equipment, but you MUST buy a high vis jacket for winter running. Light yourself up like a bloody Christmas tree. In wintertime get some of those running tights and hat and gloves – you look a bit stupid but they make it much more warm and pleasant.

10. Don’t get bored

You’ll get bored doing the same routes. Try different places and routes, circular routes, out and back etc. Mix it up. Look at training plans for different events to try and increase your pace.

11. Get proper shoes

Or you’ll get sore legs or feet.

12. Take your shoes on holiday

I really love running on holiday, especially abroad. You get to see new places you’d never have seen otherwise. It’ll be hot, it’ll be cold, it’ll be different from your neighborhood.

13. Cancel your gym membership

You won’t need it.

14. You can run when you are sick

I had to run with colds because it was December and I couldn’t lose the miles. It turns you can run with a cold, but very very slowly and stopping regularly. You might actually feel a little bit better afterwards. So if you possibly can, get a few miles in.

15. Say thank you to the loved ones in your life

Minimise the impact it has on their lives. They’ll see that you are benefiting and you have more energy and so on, but nonetheless they’d probably rather see you than not see you.

If you are interested then come and join us here on Facebook: https://www.facebook.com/groups/520milesin2014/

Everyday Hacks: Use Google Feeling Lucky in Chrome’s search bar

There is and has been for a while a definite trend of people using Google rather than URLs to get to sites.  You often see search terms being advertised not rather than URLs now.

I have noticed this behaviour in myself.  To get to The Daily Mail (obviously) I just type Daily Mail into Chrome’s address bar and almost subconsciously click on the top search result in Google.  This makes sense when the URL is a little bit hard to remember like Google Maps at maps.google.co.uk.  But I do that now for easy to remember destinations like the BBC or Etsy.

Anyway, I’ve found a neat hack that helps with this way of thinking and does it all in one easy step! – without having to go via the Google search results page! I know I know, I can hear you squealing with excitement from here. If you follow the steps below you’ll be able to make use of the Google Feeling Lucky feature directly in your Chrome browser. This will automatically take you to the top search result in Google, which if you’re mildly sensible with your terms, will get you to your destination.

1) Go to the Chrome Settings page and choose Search > Manage Search Engines

2) Go to add new search option. This might vary depending on what Chrome version you’re on, but you get the gist.

3) Give it a name, it doesn’t really matter what, I use “Go straight to”

4) Give it a shortcut key. This will tell Chrome you want to use the ‘Go straight to’ option. I use backslash “\”. So when you want to use Feeling Lucky, you type “\” followed a space and whatever name or term you want to go straight to.

5) In the third box for the URL, paste http://www.google.com/search?ie=UTF-8&oe=UTF-8&sourceid=navclient&gfns=1&q=%s

6) Click the make this my default and you’re ready to go. You might need to close and reopen Chrome.

Power Up:

In the address bar at the top type “\” and space and Go straight to appears.

1

Then type in your terms and hit return.

2

And you’ve guessed it, this will take you straight to the Wiggle helmets page.

3

Now you’ve probably saved yourself a fraction of a second, but don’t you feel just a little bit clever?

Running the GR-92 around Calella de Palafrugell

There is a beautiful and easy to reach coastal area in northern Spain that is perfect for combining family holidays and some breathtaking running.

The area is around the coastal town of Palfrugell which is 1.5 hours north of Barcelona and less than an hour from Girona. Both of which are Ryan ‘we don’t give a crap about you or your bags’ Air destinations. This is not the Spain of massive hotels and pubs called ‘The Red Lion’, this is the Spain of rocky coves, myriad little beaches and wealthy Barcelonés. Fortunately if you go just off season in September the temperature is not too hot  and the prices are such you can afford the finest of views.

Calella

Calella de Palafrugell

The area is peppered with enough beaches to visit a different one each day. From Calella de Palafrugell many are accessible by foot on the coastal path. The path forms part of  GR-92 Camino do Ronda a long distance route which runs from Blanes in the south all the way north to the French border. From Calella de Palafrugell this path offers great walks and runs both north and south. Like the coastline, the path is VERY hilly and in the backwoods can be quite badly signposted (little red and white marks on trees which are often intermittent). But if you like hills, exploring beautiful countryside and cooling down in the clear Mediterranean afterwards then you won’t find a better place to stretch your legs. I’ve run the sections around here many times and because information is surprisingly scarce, I thought I’d share these fantastic routes and maybe stop you getting lost quite as much as I did.

North of Calella through Llafranc to Sant Sebastia lighthouse (and maybe beyond)

North from CalellaThe path north from Calella passes impressive villas as it hugs the coastline from Calella to the next resort of Llafranc. The path is well marked and very easy to follow and not too hilly at this stage. Look for the lighthouse on the top of the hill on the other side of the beach above the harbour as you come round to Llafranc. You’re going up that! As you enter Llafranc you will drop down a lot of steps the beach front, enjoy that because you’ll be coming back up them later I’m afraid. Head along the beach road past the restaurants and cafes and as you are about to enter the harbour area look for the steps on the left. There are a lot of steps. If you can run up them you are doing well. You’ll emerge on a road, and it’s easy to see which way to head – uphill of course. The road winds upwards and gets steeper at the end, but the view at the top is worth the effort, you can see for miles and the scenery is stunning.

Now you can either turn around here (of which you should not be ashamed) or head further north into the woodland. There is a hotel above the viewpoint, go along the side of this (there are loos on the coast side of the building at the far end where you can top up your water if you need to). From here you start entering scrubland, with glimpses down over the cliff to the rocks below. The path twists and turns and you’ll have to do the best you can to follow it.  You’ll emerge back onto a tiny road where I turned right by mistake, I think the official route is almost straight across the road, but you’re on your own from this point.

View the route in detail on Garmin

South of Calella towards Palamos

This is a more off road run through woodland tracks. It’s not shy of hills. The path initially again hugs the coastline through Calella  and then ventures off road into the countryside. There are big climbs in both directions, but longer and gentler than the climb to the lighthouse in Llafranc.

right turn

Beware the right turn or face scraped legs and thorns

Head south along the coast on the well signposted camino path. You’ll pass above Golfet beach which is an amazing spot for a swim. The path folds into a maze of suburban streets, you don’t need signposts here, just head uphill. You can follow directions for the botanic garden (which are amazing to visit) if you get lost. You’ll come out of town to see a path stretching out across some open fields towards woodland – still going upwards I’m afraid.  The path through the trees is marked by red and white blocks and it gets narrower and rockier until is seems to vanish on something of a summit. At this point STOP and look behind you to the right. Go that way. If you miss this sharp turn you’ll be sliding down a steep hill through scrub and bushes which is really not much fun. Last time I ran it there was stone on the ground with the red and white marker to mark the turn. Stones move though.

You should then pass an unfinished building and from here it’s more or less downhill all the way. The trail is relatively easy to follow and gets wider until becoming a rough road. You’ll then emerge onto a tarmac road and from here you can head left to a beach or carry on across an open field towards Palamos. This section is fine for a while but then starts to get into suburbs where I lose interest.

South of Calella

South of Calella (waypoints in miles)

On the return leg, the path back is harder to follow and I’ve never gone the same route twice. Once you’re off the tarmac and back amongst  the trees, make sure you spot the left turn off the rough track. After that good luck, if you get lost follows signs for Castell Cap Roig and that will bring you back onto better signposted tracks. There is of course the obligatory hill climb, but at least it’s in the trees so there is some shelter.

If you want to do a shorter run, I would recommend driving towards the Botanic gardens and starting from where the path head out across the field. This way you get the countryside section and not the long hill climb through the suburbs of Calella.

View the route in detail on Garmin

Palafrugell to Palamos

palamos

I’ve only done this route once while staying in Palafrugell town which is a mile or two inland from the coast. There is a cycle

and walking path which goes from the edge of Palafrugell to Palamos. Once you find the start of the path then the route is easy to follow and flat all the way. There is no shade so if it’s sunny you will really feel the heat. The distance from the north side of Palafrugell to the very tip of the harbour (turn left once you hit the beach in Palamos) and back to Palafrugell is more or less 13.1 miles. If you’re going to do that distance I would take a lot of water to get you there and back. I took one bottle, late afternoon on a sunny day in September and I really struggled (although it was my first half marathon distance).

View the route in detail on Garmin

There is not much information about the GR-92 on the web, here’s some stuff I’ve found:

Route map: http://www.ikimap.com/map/gr-92-catalunya-sender-de-gran-recorregut

Some chap’s blog about walking this stretch (with pics): http://www.ottsworld.com/blogs/camino-ronda-hiking-spain-costa-braval/

Account of hiking along the route: http://easyhiker.co.uk/la-fosca-on-the-cami-de-ronda/

Everyday Hacks – Copy and Pasting Without the Formatting

Ever get frustrated when you’re copy and pasting and the formatting gets copied as well. You have a nicely formatted document which have to send out in 5 minutes and you copy in something from another document , or a spreadsheet or and BOOM, it’s made the font red, underlined and Bold. Then you have to fiddle about in Word trying to get it to remove the copy.

That doesn’t work because, well it’s Word isn’t it. So then you end up having to find notepad and copy and paste via notepad to strip out the guff. Not any more….

If you install this tiny bit of software Puretext then you can both copy and paste and strip out the formatting all in one easy step. When running it sits passively in the background and all you have to do is use slightly different shortcuts and it does all the hard work for you. Enjoy…