Archive

Tag Archives: Development

Who owns the codebase?

04/07/2012
Agile, Development, Lean, Management
2 Comments

Ive been involved in quite a few discussions about shared code lately, mostly because I’ve been working in a team that has developed a set of internal tools that are going to be used across the company. One of the common topics to debate in this area is what happens next?

After the main part of development has been done, who is responsible for mantaining and evolving this shared codebase?

I’ve written a little bit about shared code and I don’t think it is the best solution for most cases, but in a few situations sharing is necessary, so how should we handle it?

As any popular topic, opinions are diverse, but here are my two cents to the conversation:

It depends : )

The way I see it, there are two main types of shared codebases that can exist in a company, and each of them should be treated in a different way.

Components that are not critical for any team

This is the case of the famous “utils” package, or any other component that was once written because someone needed it and then was reused since it was useful for other people. The main characteristic here is that this code doesn’t sit on the critical path of any team/application. In other words, I could choose to use it or not if it fills my needs.

In this situation, I believe that using an open source model is just fine. Think about your packages as an open source library, that someone can use it or not if they want, and if at some point it needs to be maintained or evolved, the people using it can spend some time doing it.

It doesn’t matter that much if they break things or take the codebase in a direction no one expected, since people can always use older versions or just find a replacement for it.

Components that are critical to teams 

Now this is a more delicate situation. If teams depend on your component working all most of the time, I believe the open source model is not appropriate anymore.

As I mentioned before, developing internal applications should also use a customer-centric view, and that takes time and effort. As the number of users grow and diversify, it’s important to have people in your team that can steer the development in the right direction, thinking about the roadmap for the future and also development quality.

Being a critical component, the open source model makes it hard for anyone to own the responsibility to maintain the code, and hard decisions that will have to be made start to get postponed, while patches start happening more and more.

It doesn’t take long until everyone is attached to a codebase that no one really understands and is afraid to change. In other words, legacy is written.

In this situation, the biggest step is to realize you have a product to maintain, and that will cost, as it costs to maintain any product. There is the need for a focused team that can plan the future and guarantee that users are happy with what they are receiving, and avoiding the pain will just guarantee a greater amount of it in the future.

Where to put my build tasks ?

21/06/2012
Development, software, Technical
Leave a Comment

In my current project we are working with automation of deployments, so a lot of our work involves automating builds and (trying) to create a pipeline using build tasks (currently using Jenkins).

One of the things that started to annoy us is that as a standard build tasks are part of the main code repository for an application, even when the tasks itself don’t depend at all on the codebase. In our case, we are using Rake with Ruby, and using RPM’s to deploy, so our Rakefiles would contain tasks that involved testing and deploying RPM’s, but still were included in the application.

I can see the point of keeping the everything related to an app inside the same codebase, but add that to a slow git and gem server, and our build tasks would spend 90% of their time downloading and bundling the application.

So one simple practice that we are trying now is to create a separate repository for build tasks. If all you need is a Rakefile, why do we have to download more than that ?

Just to exemplify how simple it gets…

That’s it. Quite simple, but thought it was worth mentioning!

Developing internally ? You are not special

21/06/2012
Development, Management, software
1 Comment

In different opportunities IT organisations need to develop internal tools/code to be used for its developers. The reasons for it are a lot of times questionable, like in some cases of creating shared code, but there are moments when they are the right option. With the rise of continuous delivery, deployment automation tools is one of the examples where common tools can be useful for a company.

When that happens, internal teams are often assembled to deal with this problem, and create solutions to be used internally. Unfortunately, these teams seem to suffer from a “too much introspection problem”, and often forget lessons learned about product development when going on their mission.

There is a perception that because something is developed internally and “approved” by the head office (after all, they are putting money on it), other people will be much more receptive and understanding about problems and poor customer service.

Since there are no real paying customers and everyone will have to use the tool at the end (at least we hope for), the need for short product iteration and constant feedback doesn’t seem that necessary, plans are made to architect something that could solve all the problems, just not the one that the users are having.

The problem is that, in reality, market laws operate internally to a company as much as they do in the outside world, and poor over architected solutions that are forced upon users are still not a big hit, even between the organisation’s walls.

For real products (the ones that people pay for), the current solution are lean startup techniques and product teams, and I believe the frame of mind when entering the world of internal development should be the same.

Start with a vision, not a complete solution - Even when the problem is supposed to be known and understood, it usually isn’t as much as we hope. Over-architecting is a problem here as in any other software project and the only way to avoid the plan to fail when facing reality is to start small and validate your idea as you go along. Yes, Im talking about a MVP.

Get out of the building - In this case, your users are probably inside the same building, so there are really no reasons to avoid them. Walk around the floor, find the people that you will help with your product and get them to help defining what needs to be executed from your vision.

Iterate fast, close to the customer - Get your solution out there to be used from the beginning. Sit with your users, help them adopt it and treat feedback appropriately (as in hear it and do something about it).

Use metrics, track adoption and problems - Metrics are there to be used, so why not get them to help understanding how your tool is being used and adopted by your colleagues (or not)?

In summary, despite not having paying users, internal software is also supposed to help somehow, and if it doesn’t, it will be left behind and forgotten, so companies should give it the importance it deserves.

Letting go of agile

19/06/2012
Agile, Management
1 Comment

That’s it!

“Software, agile and some nonsense” - That used to be the description of my blog.

With the extreme popularisation and use of the word “agile” in the IT industry from anyone who wants to be part of the in crowd, sometimes referring to practices and thoughts that are well beyond what I believe is correct, I’ve decided to just stop using the word.

I believe the risk of being misinterpreted when you mention “agile” nowadays is bigger than the benefit you get out of using it, so let’s start talking about feedback, transparency and delivering value and see where that will lead!

Duplication? Yes, please!

14/06/2012
Agile, Development, Lean, Management, software
2 Comments

A common topic when discussing IT in organisations is how to structure software teams, more specifically, how do we divide work when a company grows enough that one team is not enough anymore, and how do we deal we shared code?

This is not a simple question and probably deserves a few blog posts about it, but now I wanted to focus on one specific aspect of it, which is the duplication of effort.

A simple example would be if my company has multiple teams delivering software, they will eventually find the same problems, as in how to deploy code, how to do logging (the classic example!), monitoring and other things like that.

And in almost all organisations I have been, the response is unanimous: we should invest in building shared tools/capability to do that! It’s all in the name of uniformity and standardisation, which must be a good thing.

Well, as you might imagine at this point, I disagree.

Building shared applications in IT is the equivalent of economies of scale for software, and it is as outdated in IT as it is in other industries.

In knowledge work (as we all agree software development is at this point), no effort is really duplicated. You can develop the same thing 100 times and you are most likely getting 100 different results, some quite similar but yet different. And that small difference between each other is where innovation lies.

This is the trade off that companies should be aware of. If people don’t have freedom to experiment and look for different solutions for problems, is very easy to get to a place where everyone is stuck in the old way of thinking. And that’s what companies are really doing when making people use common tools, shared applications, etc.. They are putting cost-saving in front of innovation, a lot of times without realising it.

And I don’t mean we can’t all learn from each other and reuse solutions when they are appropriate. After all, there is still a place for economies of scale when appropriate.

We are all standing on the shoulders of people that came before us and should keep doing it, even internally to an organisation. But that should be an organic and evolutionary process, not one that is defined by an architecture team.

Test Driving Spikes

21/03/2012
Agile, Development, Estimation, software, Tests
Leave a Comment

If you have ever been in an Agile project, or something that looks like it, you should have heard about the concept of spikes.

For those who haven’t, spikes are the agile response to reducing technical risk in a project. In case you are not sure how to build something or fear that some piece of functionality might take much more than expected, and you have to gain more confidence about it, it’s time to run a spike. Usually timeboxed, spikes are technical investigations with the objective of clarifying a certain aspect of the project.

In my current team, we are developing a few tools that are quite specific to our context and were not sure on how to solve a few issues, so we have been quite frequently playing spike cards.

This is all not new and I’m sure most of you have done that before. If you did the way I used to do, you would write a spike card, something as “investigate technology X for problem Y“, would spend 2 days doing it and would have a response for it in your head once you were finished.

In our current context, team members were rotating quite quickly, so we were worried that any knowledge we would get from spikes could have been lost if it was just…let’s say.. in our heads.

Not wanting jut to write the findings up, as we first thought about doing, we decided to tackle the problem with the golden hammer of agile development: tests!

So, instead of writing tests to decide how we should write our code, we started writing tests to state the assumptions we had about the things we were investigating, being able to verify them (or not) and have an executable documentation to show to other people.

For example, here is some code we wrote to investigate how ActiveRecord would work in different situations:

it 'should execute specific migration' do
  CreateProducts.migrate(:up)
  table_exists?("products", @db_name).should be_true
  table_exists?("items", @db_name).should be_false
 end
it 'should execute migrations to a specific version' do
  ActiveRecord::Migrator.migrate(ActiveRecord::Migrator.migrations_paths, 02) { |migration| true }
  table_exists?("products", @db_name).should be_true
  table_exists?("items", @db_name).should be_true
  table_exists?("customers", @db_name).should be_false
 end
it 'should not execute following migrations if one of them fails' do
  begin
    ActiveRecord::Migrator.migrate(ActiveRecord::Migrator.migrations_paths, nil) { |migration| true }
  rescue StandardError => e
      puts "Error: #{e.inspect}"
  end
  table_exists?("invalid", @db_name).should be_true
  m = ActiveRecord::Migrator.new(:up, ActiveRecord::Migrator.migrations_paths, nil)
  m.current_version.should == 3
  table_exists?("products", @db_name).should be_true
  table_exists?("items", @db_name).should be_true
  table_exists?("customers", @db_name).should be_true
  table_exists?("another_table", @db_name).should be_false
 end

We have used this technique just a few times and I won’t guarantee it will always be the best option, but so far the result for us is having code that can be executed, demonstrated and easily extended by others, making it easier to transfer knowledge between our team.

Can I just throw it over the wall ?

15/02/2011
Agile, Development, software
Leave a Comment

One of the good things I’ve took from one of the recent projects I’ve been in was the way the handover of stories from Dev to QA was done.

I mean, the handover topic is something that Im usually asked about, and I’ve never had a decent answer for it, since in my experience it has always been something like this…

Is this story ready to Qa ? Well, I think so,  it must be !

However, in the project mentioned above, at some point we had lot of bugs. Not really serious ones that would make us worry about the quality of the code we were writing, but a great number of small and annoying things that showed us we weren’t being really careful when developing a story.

Most of the stories that we “delivered” (yeah, right…) came back with a considerable number of small fixes to be done. Needless to say, we were losing a lot of time reworking things that should have been caught before finishing it, and not getting much code across the line because of it.

So we had some discussions about the problem and what we came up with was an amazing… checklist.

If we, as developers, were being sloppy when finishing the stories, why didn’t we have a list of what we had to do before considering it ready ?

As you can see above, there is nothing incredible about it,  just the fact that it made us remember what to do, something that is a lot of times easier said than done. Looking at this simple list I can make sure I always test the story against all the browsers we support, as well as verifying that the acceptance criteria were really met.

Talking about acceptance criteria, another goodness that came with it and facilitated the whole process was having mind maps describing what QA’s were going to be testing in every story.

Having this information available and easy to understand was miles ahead from the traditional “I’m going to find bugs in your story” mentality that dominates some teams. As usual, cross-role collaboration was a huge win for productivity.

mindmap

Knowing what was going to be tested enabled developers to take a look at the story with different eyes, verifying if all cases were accounted for before actually handing it over. As expected, we delivered more.

Software (Engineering or Development?)

06/04/2008
Agile, Development
2 Comments

One thing that always interests me about software is the eternal discussion, that has been going for some years now, about the relationship between software development and other engineering disciplines.

On one side stand the traditional approach folks, claiming that software projects are the same as any other engineering projects, and because of that, should be managed as other engineering projects (the favorite one is civil engineering, with house building…)

On the other, the agile practitioners don’t accept software development to be called “engineering”, and say that this is probably one of the reasons software development is in this situation nowadays (not a good one, as you may conclude).

So, what is the right answer? In my opinion, both! To the explanation…

Martin Fowler wrote something about it in this article (everybody knows which side he is in :-) ), where he explains why the traditional civil engineering approach can not work in software development. Quoting one of his sentences

Can you get a design that is capable of turning the coding into a predictable construction activity? And if so, is cost of doing this sufficiently small to make this approach worthwhile?

And I totally agree with him. If you look at UML models and other software design methodologies, there is no way to design a software in such detail so that the construction phase could be a less skilled activity, with predictable results. All you get in projects with long design phase is a lot of rework in the construction phase.

Ok, so software is not civil engineering, but is it not engineering at all? That’s what got clearer after reading the appendix F of the Rogers Commission Report, written by Richard Feynman, which investigated the Challenger Space Shuttle disaster, in 1986.

Now, if you are still paying attention, you should be asking: “What space shuttles have to do with software?”

Simple, it’s engineering. And not just that, it is high technology engineering.

Take a look at what Dr. Feynman said in his report about the space shuttle main engine (quoted from this other blog, where I’ve found all the material):

The Space Shuttle Main Engine was handled in a different manner, top down, we might say. The engine was designed and put together all at once with relatively little detailed preliminary study of the material and components. Then when troubles are found in the bearings, turbine blades, coolant pipes, etc., it is more expensive and difficult to discover the causes and make changes.

Anything rings a bell? How about that?

The usual way that such engines are designed (for military or civilian aircraft) may be called the component system, or bottom-up design. First it is necessary to thoroughly understand the properties and limitations of the materials to be used (for turbine blades, for example), …

With this knowledge larger component parts (such as bearings) are designed and tested individually. As deficiencies and design errors are noted they are corrected and verified with further testing. Since one tests only parts at a time these tests and modifications are not overly expensive. Finally one works up to the final design of the entire engine, to the necessary specifications. There is a good chance, by this time that the engine will generally succeed, or that any failures are easily isolated and analyzed because the failure modes, limitations of materials, etc., are so well understood.

Well, I don’t know what are you thinking, but when I read that, it seemed to me that somebody was talking about iterative software development. And that’s the whole point about it, software development can be compared to an engineering discipline, but to an advanced engineering discipline.

If you try to compare developing software to building houses, you try to compare 2000 years of experience to something we started to do in the last century. But even in software, if you go to well known domains, with well known technologies (not easy to find projects like that… :-) ), you can probably automate the process with different tools, making it a “less skilled task” and also a lot easier.

But in the great majority of projects, developing software is still about messing with recent discovered technologies and changing domains, and that what makes it so hard.

So, just to make a final point, software development is rocket science!

Cheers!

Follow

Get every new post delivered to your Inbox.

Join 835 other followers