Archive

Lean

More than few years ago I’ve read the book Agile Modeling, by Scott Ambler, and it was quite a revelation for me. I was beginning to look into extreme programming and TDD at the time, and the fact that you could (and should) write software in an evolving manner instead of the usual big architecture up front that I had studied in university was quite refreshing and empowering.

(as a side note, I actually was going to use the title Agile infrastructure for this, but I’ve promised I’m not to use the word anymore)

Not sure how many people have read the book, but it basically goes through principles and techniques that enable you to write software one piece at a time, solving the problem at hand today, and worrying about tomorrow’s one tomorrow.

If I remember correctly, there was a sentence that went something like this (please don’t quote me on that):

Your first responsibility is to make the code work for the problem you are solving now. The second one is the problem you are solving next.

Many years have passed since the book has been written. Nowadays growing software while developing is what (almost) everyone is doing. The idea of drawing some kind of detailed architecture that will be implemented in a few months or years is completely foreign in most sensible organisations.

Basically, evolving software is almost not interesting anymore. People do it, and know how to do it (as I wrote this I’ve realised it isn’t actually true, a lot of companies don’t do it or know how to, but let’s keep in mind the ones that do…).

In the meantime, a lot has evolved and new areas that were completely static in the past are becoming increasingly dynamic, the current trendy one being IT infrastructure.

The uprise of virtual infrastructure and the so called devops movement have developed tools and practices that make it possible to create thousands of instances on demand and automatically deploy packages whenever and wherever you want. However the thinking behind infrastructure within most IT departments is the equivalent of waterfall for software.

I’m not just talking about auto-scaling here, since that seems to be a concept that’s easy to grasp. What I don’t quite get is why the same thinking that we have when writing software can’t be applied when creating the servers that will run it.

In other words:

  1. Start writing your application in one process, one server*, and put it live to a few users.
  2. Try to increase the number of users until you hit a performance bottleneck
  3. Solve the problem by making it better. Maybe multiple processes? Maybe more servers? Maybe you need some kind of service that will scale separately from the main app?
  4. Repeat until you get to the next bottleneck

* ok, two for redundancy…

The tools and practices are definitely there. We can automate every part of the deployment process, we can test it to make sure it’s working and we can refactor without breaking everything. However, there are a few common themes that come back when talking about this idea:

“If we do something like this we will do things too quickly and create low quality infrastructure”

This is the equivalent of “if we don’t write an UML diagram, how do we know what we are building?” argument that used to happen when evolving software was still mystery to most people. It’s easy to misunderstand simplicity as low quality, but that doesn’t need to (and shouldn’t) be the case. As with application code, once you put complexity in, is a major pain to take it out, and unnecessary complexity just increases the chance for problems. Simple solutions are and will always be more reliable and robust.

“We have lots of users so we know what we need in terms of performance”

If a new software project is being developed, it is pretty much understood nowadays that nobody knows what is going to happen and how it is going to evolve over time. So pretending that we know it in infrastructure land is just a pipe dream in my opinion.

“We have SLA’s to comply to”

SLA’s are the IT infrastructure equivalent of software regulations and certifications, sometimes useful, sometimes just a something we can use to justify spending money. If there are SLA’s, deal with it, but still in the simplest possible way. If you need 99.9% uptime, then provide 99.9% uptime, but don’t do that and also use a CDN to make things faster (or cooler) just in case.

As it’s said about test code, infrastructure code is code. Treat it the same way.

To close my participation at LAST Conference, I’ve presented a follow up of the talk I’ve done at LESS 2011, talking about why I believe most organisations are not set up for learning.

In the presentation I’ve explained my thoughts on why I believe change programs are often unfair to employees, asking them to embrace change, but only the one that management is putting forward.

I’ve also talked about learning within organisations, product teams, and how management teams should step back and understand their new role as leaders instead of controllers of the company.

If it sounds interesting to you, there is more info here.

Last week I’ve presented with Herry at LAST Conference about our experience of helping forming a new distributed team in Melbourne and Xi’an while transferring the knowledge of an existing product.

It was an interesting challenge since what we had a not so simple task to do, which was to

  • finish a major release of an existing product
  • ramp down the existing team in Sydney
  • create a new team distributed between Melbourne and Xi’an
  • transfer the knowledge as smoothly as possible, so the new team could start delivering new functionalities as soon as possible

It was great to talk about our experience for the first time, and the amount of questions that came from the audience showed us that it is a interesting (and controversial) topic for other people as well.

The slides are available here, unfortunately it’s hard to get all the content just based on them, but just get in touch with us if you have any.

Last Friday I’ve participated in the Lean, Agile & Systems Thinking conference, which was a one day event organised by Craig and Ed  with the intention to be a day with content “from practitioners” and “to practitioners”.

I have told both of them and a bunch of other people, but it won’t hurt to say again that I believe Australia was in need of an event like this, organised by the community and with focus in providing useful content more than anything. Its success was definitely proven by the attendance and also the twittersphere on the day, so if you haven’t congratulated them yet, don’t wait any longer!

I’m going to try to share what Ive heard around the event here, but there are definitely more places to look for, and videos of some sessions should be available soon.

Storytelling by Shawn Callahan

I’ve started the day by going to the double session from Shawn from Anecdote on business storytelling. I’ve been reading on the subject for a while and the session was very interesting.

After proving in a story telling exercise that everyone has a story to tell (and that we start telling them after we hear other people doing it!), shawn spoke about the storytelling spectrum and how we should keep the business stories on what he called the small ‘s’ version, avoiding the risk of telling something so epic that it makes us lose the engagement of our colleagues.

He also spoke about anecdote circles, where people gather to tell stories about the environment they are in, and gave a few examples and tips on how to create questions that will spark stories from people:

  • Never ask why questions
  • Use when and where
  • Make open questions
  • Ask for an example if answer is too narrow
  • Include emotions, as in “when did you feel like … ?”

Some examples of questions:

  • What have you seen lately that has surprised you?
  • When was the last time that something small made a big difference?

He finished the session talking about a story narrative and how you can construct a story, showing video examples of great story tellers and how they used the elements he was talking about.

Overall a great session, definitely recommended if you have the opportunity!

Cynefin Model by Kim Ballestrin

Kim gave an overview of what the Cynefin model is and how she is using that in her current work.

Having had a brief understanding of the model beforehand, it was really useful to see someone talking about it in action, and what it could be used for.

She gave an example of how the model can be used in classifying types of work, and then using the classification to execute them differently. She used three categories for that:

  • Complex Work (with uncertain outcome) – Create experiment and test it the cheapest way possible, to verify if it’s worth being done
  • Complicated Work (where analysis is required)  – Analyze
  • Simple Work (where outcome is certain)  -Build

She spoke about what an experiment actually is in that context and how it could be just a conversation, giving the example of a kids’ party, where parents are constantly assessing risk and acting on weak signals.

Live Below the Line by Claire Pitchford

Claire spoke about her experience as a business analyst in the Live Below the Line project, where they had to deliver a campaign website in six weeks.

She talked in some detail about the inception process that was performed with the client and the whole team during one week, helping everyone get to a shared understanding of the project, and how different tools as personas, storyboards, stories and estimation where used in the process.

It was a very practical talk about what was done, and also quite impressive that she was happy to talk about everything that went wrong, things she tried and didn’t work but also what she learned during it.

As I mentioned before, it was a great day above all. I also have presented on two sessions during the day, and will write about it in separate posts.

It is well known that Agile (not sure about the big/small “A” thing, it always gets me confused) has gone mainstream nowadays. With big companies and conferences endorsing it, long gone were the days when you actually had to convince people this was a good idea… and that’s great!

However, with the early/late majority now adopting Agile, we are not talking about small companies anymore, and that means that the challenge is not how to get teams delivering better, but whole departments and organisations. So yes, scaling agile is one of the current challenges, with the preferred approach being the dreaded (at least by me) “Change Programs”.

I don’t believe agile is a fad or just a small team thing, since empowerment, short feedback loops and delivery of results it’s never going to be a fad and isnt IT specific either. There are many other examples, in different industries (errm… Toyota?) which show that the same principles can be applied successfully in a much larger scale, on complete different problems.

The problem begins though, when by scaling agile, companies try to scale the practices, keeping the control mindset, instead of scaling what’s important, i.e. the principles.

It is common to see companies, for example, adopting a hierarchical structure to create multiple agile teams, all “reporting” to a main office, killing most of what was good about the idea.

If agile in the organisation is the challenge, we need to think about empowerment, short feedback and delivery of real results for the whole company. Unfortunately, the more common behaviour is to talk about bigger walls, how to standardise velocity across teams and synchronisation of iterations.

And if we go back to the roots, and look at the manifesto, I believe we can find a good guidance of what can be done:

Individuals and interactions over processes and tools: Instead of scaling processes and tools to control teams, why not increase the ability of individuals to traverse the organisation, making software developers talk to real customers and sales people talk to qa’s?

Working software over comprehensive documentation: Scale by making a working product the final goal, removing the necessity of internal product briefs, extensive training of sales staff and long delivery cycles.

Customer collaboration over contract negotiation: It is actually impressive how this still makes sense now. We just need to change the customer. It’s not the product owner anymore, it’s _the_ customer, the one who buys your product and pays the bills.

Responding to change over following a plan: Feedback, feedback and feedback. Not from a showcase though, but from the market.

Yes, if it feels like you’ve heard these ideas before, it’s because they are not new. Product teams are the way to scale agile in my opinion. Small and independent groups of people that can get to the best results in short iterations with feedback from the market, in a Lean Startup style. The game shouldn’t be about execution anymore, it is about innovation.

Ive been involved in quite a few discussions about shared code lately, mostly because I’ve been working in a team that has developed a set of internal tools that are going to be used across the company. One of the common topics to debate in this area is what happens next?

After the main part of development has been done, who is responsible for mantaining and evolving this shared codebase?

I’ve written a little bit about shared code and I don’t think it is the best solution for most cases, but in a few situations sharing is necessary, so how should we handle it?

As any popular topic, opinions are diverse, but here are my two cents to the conversation:

It depends : )

The way I see it, there are two main types of shared codebases that can exist in a company, and each of them should be treated in a different way.

Components that are not critical for any team

This is the case of the famous “utils” package, or any other component that was once written because someone needed it and then was reused since it was useful for other people. The main characteristic here is that this code doesn’t sit on the critical path of any team/application. In other words, I could choose to use it or not if it fills my needs.

In this situation, I believe that using an open source model is just fine. Think about your packages as an open source library, that someone can use it or not if they want, and if at some point it needs to be maintained or evolved, the people using it can spend some time doing it.

It doesn’t matter that much if they break things or take the codebase in a direction no one expected, since people can always use older versions or just find a replacement for it.

Components that are critical to teams 

Now this is a more delicate situation. If teams depend on your component working all most of the time, I believe the open source model is not appropriate anymore.

As I mentioned before, developing internal applications should also use a customer-centric view, and that takes time and effort. As the number of users grow and diversify, it’s important to have people in your team that can steer the development in the right direction, thinking about the roadmap for the future and also development quality.

Being a critical component, the open source model makes it hard for anyone to own the responsibility to maintain the code, and hard decisions that will have to be made start to get postponed, while patches start happening more and more.

It doesn’t take long until everyone is attached to a codebase that no one really understands and is afraid to change. In other words, legacy is written.

In this situation, the biggest step is to realize you have a product to maintain, and that will cost, as it costs to maintain any product. There is the need for a focused team that can plan the future and guarantee that users are happy with what they are receiving, and avoiding the pain will just guarantee a greater amount of it in the future.

A common topic when discussing IT in organisations is how to structure software teams, more specifically, how do we divide work when a company grows enough that one team is not enough anymore, and how do we deal we shared code?

This is not a simple question and probably deserves a few blog posts about it, but now I wanted to focus on one specific aspect of it, which is the duplication of effort.

A simple example would be if my company has multiple teams delivering software, they will eventually find the same problems, as in how to deploy code, how to do logging (the classic example!), monitoring and other things like that.

And in almost all organisations I have been, the response is unanimous: we should invest in building shared tools/capability to do that! It’s all in the name of uniformity and standardisation, which must be a good thing.

Well, as you might imagine at this point, I disagree.

Building shared applications in IT is the equivalent of economies of scale for software, and it is as outdated in IT as it is in other industries.

In knowledge work (as we all agree software development is at this point), no effort is really duplicated. You can develop the same thing 100 times and you are most likely getting 100 different results, some quite similar but yet different. And that small difference between each other is where innovation lies.

This is the trade off that companies should be aware of. If people don’t have freedom to experiment and look for different solutions for problems, is very easy to get to a place where everyone is stuck in the old way of thinking. And that’s what companies are really doing when making people use common tools, shared applications, etc.. They are putting cost-saving in front of innovation, a lot of times without realising it.

And I don’t mean we can’t all learn from each other and reuse solutions when they are appropriate. After all, there is still a place for economies of scale when appropriate.

We are all standing on the shoulders of people that came before us and should keep doing it, even internally to an organisation. But that should be an organic and evolutionary process, not one that is defined by an architecture team.

I’ve been thinking about and researching the topic of organisational learning for some time now. Last year, I presented on the topic at LESS 2011, but realized it was time to take some time and write about it. So this is the first post of a series to explore the topic, which I’m also hoping will help me clarify my view on the subject.

Organisational learning. Should anyone care ?

My hope is that this goes without saying, but I guess that the first question to be asked is why should anyone care about learning? Some businesses make money staying still, while the world keeps moving. Full stop.

While this could be true for established business in the past, it’s something that is not so clear anymore. Gary Hamel makes a good point in his “Reinventing the Technology of Human Accomplishment” presentation: Change is changing, and it’s much faster now than it has ever been.

The world is changing, and faster than ever, so (every) company has to adapt, not only once but constantly.

Why is it so hard ?

A company has to learn. If this was an easy task, it wouldn’t be so rare, but creating a learning organisation is one of those things that gets harder the more you try and control it. If you want to form a learning organisation, a few questions you might think of are:

  • What does my company need to learn ?
  • If I had to setup the body of knowledge necessary to run my organisation, what would it be ?

While these could have been easy questions to answer 50 years ago, nowadays, in this ever changing (specially IT) world, whatever you write as being essential today might not be useful at all in one year’s time.

And the big issue is that most companies are not setup to deal with this problem. Instead of an evolving structure, modern organisations still have a very hierarchical structure based on top down control, with the vision of a CEO (put any other senior role here) that drives the strategy while most of the company just follows.

They have Project Management Offices that dictate how to deal with projects, architecture teams that say what can and cannot be done and common policies that dictate behaviour for everyone. Bjarte Bogsnes  has a good tale about how companie’s policies try to establish control over the employees:

“A friend of mine works at SAS. Even if he is trusted to fly planes around the world, if he wants to change his shirt more often than what the policy states, he needs a written authorization”

So the common alternative to evolution in this kind of structure are the well known “Change Programs”. They try to implement a top-down approach to change, that can be explained in two simple steps:

  1. Senior people decide how to move forward
  2. They teach the rest of the company how to do it

While step 1 is always a success, step 2 is usually more troublesome that people would like to think. And it’s not hard to understand why, since the whole idea of a change program is quite an unfair proposition to the affected employees, which could be described like this:

“You have to embrace change, but only the change we want”

As expected, people resist. Peter Senge has a quote that explains this situation quite well – “People don’t resist change, they resist being changed” – It’s natural that most employees will doubt and resist something that they haven’t been asked about. I would, and I’m quite confident that most people would too.

And as a result, money is spent, time and most importantly people are consumed through the process, and not much is learned in the end.

So is there an alternative to this situation ? Do you agree with this view ? Let me know what you think

If you have heard about Lean and Kanban software development, one thing you might be doing is measuring the cycle time for the cards in our project, meaning the time between when a card is started until it’s finally done.

I’ve been doing it for a while, and have tried a couple of ways to actually get the data so far, going from having a spreadsheet where I would keep all the dates when cards transitioned (very detailed and very complex to maintain) to the usual noting down those dates in the card itself (less complicated but still troublesome, since people would forget to write on the card when they moved it).

Since I’ve started my last project, Herry introduced me to a new way, which as most good ideas, is so simple and good that I wonder how I had never thought about it.

The way we have been doing now is using stickers to keep the count of the number of days a card has been on the board. Whenever it starts to get played, we apply one _put your favourite colours here_ sticker to it for every day it is in play until it’s done. We currently just have two states in our board (doing and accepting) so we have two different colours to track the days in each specific stage.

The reason I find it much simpler is because you don’t have the problem of getting dates wrong anymore, or have to remember to note things on cards when they move. The only action needed is to apply a sticker to every card being played after the standup.

One of the things that attracted my interest to LESS 2011 was the Beyond Budgeting track. Having read the book a couple of years ago, it is definitely a topic that catches my interest. And it was even better that the first keynote was presented by Bjarte Bogsnes, talking about how beyond budgeting is used at StatOil

As any presentation write-up, this is just my understanding about what was presented, so please don’t hold the presenter on to what I’m writing here :)

Why ?

The first question Bjarte addressed was why the need for something different. And I believe this sentence explains it all

January-December is artificial for business and just works for accounting. But this is not accounting

He followed by explaining that StatOil is always trying to get the best performance they can (who isn’t!), and coupling everything they do to an accounting mechanism (the budget) doesn’t make sense for them. He illustrated it with a comparison between the use of traffic lights and roundabouts. While the first is easier to use and provides more control, the latter actually provides better results because it relies more in the current situation than on statistical analysis.

We need to find more self regulating ways to manage a business, and beyond budgeting is one of them.

Bjarte used Douglas McGregor’s Theories X and Y to explain how we traditional management doesn’t usually trust people and relies on a stable environment to succeed, but in a dynamic environment in which most companies are inserted today, we need to start trusting our employees much more, and that is where beyond budgeting stands.

He had a nice example to show the lack of trust some companies impose, about a friend who is a SAS pilot, and even when he is trusted to fly planes full of people around, if he wants to change his shirt more often than it’s stated in the company’s policy, he needs a written authorization for it.

How does it work ?

According to Bjartes, the problem with budget is that it’s a single tool used for three separate purposes: Setting targets, creating forecats and allocating resources. Since these results are usually different from each other (targets are what we want, forecasts are what we expect to get), we won’t get an optimal result from using it.

In StatOil they have developed an alternative format of planning called Ambition to Action. These are a few principles that are used when cdefining it:

  • Good performance is being better than those we compare ourselves with
    • It could be external companies or even ourselves, which means we are learning.
  • Do the right thing
    • Every new joiner receives a StatOil book (a thin one!) with some of the principles that guide the company, so they can all use their business judgement and make decisions when needed.
  • Resources are made available and allocated on a case by case basis
    • Bjartes compared having an yearly budget to having a bank that opens for only a month throughout the year. How can anyone make timely decisions with that ?
  • Business is forward looking and action oriented
  • Performance measurement is holistic, and composed by 50% results and 50% behaviour

These principles are used to define what to do in the company, in a process that goes like this:

1. Strategic Objectives -> 2. KPI’s -> 3. Action & Forecast -> 4.Individual goals

So the company is going to set strategic objectives which are then converted in KPI’s. These are used to define the things that need to be done (Ambition to action), which reflect on how performance is measured for everyone in the company.

Some interesting thoughts he shared when explaining the process were:

  • The perfect KPI doesn’t exist, since not everything that counts can be counted.
  • StatOil creates around 1100 Ambition to action plans, and these are not a reporting tool, but more a guidance on how you as an employee should manage him/herself. Apart from that, everything is open, so everyone can see everyone’s else goals.
  • Performance measurement is usually based on teams and, as said before, divided between results and behaviour. Assuming that the environment very often changes, how someone behaved in that occasion is as important as the results he/she delivered.

The last point that was made during the presentation is how StatOil is moving from a calendar-driven model to a business-driven one. There are no more annual versions of Ambition to Action being created, and they can be changed at any time. The performance review is still happening annually, but Bjartes mentioned they are currently revisiting it now.

Overall it was a great introduction to Beyond Budgeting and how it is applied at StatOil. If you are interested, there is more information here and here.

Bjartes is speaking in the Thoughtworks Live Australia event. If you have a chance to go, don’t miss it.

Follow

Get every new post delivered to your Inbox.

Join 836 other followers