Features vs. Infrastructure

21 December 2009 by Stuart Cam

Your software development team likely has a process for dealing with requirements, estimating, allocating tasks and completing coding work. After all, developing software is just about providing application features isn't it?

Each and every software project has an internal work stream, which if neglected can start undermining your ability to produce quality software. I've heard this type of work referred to by a number of different names: core, infrastructure, supporting code or framework. I propose that it can be summed up as:

Work that needs to be done to support getting the primary work done.

I'll refer to this type of work as infrastructure throughout the rest of this post.

Let's look at a typical enterprise software project, what are some examples of infrastructure work?

Physical infrastructure

  • Day one experiences
    • Seating
    • Workstation setup
      • Installed software
      • E-mail setup
      • Network permissions
    • Team introductions
    • Code & business walkthroughs
  • Development servers
  • Software licenses

Development infrastructure

  • Source control
  • Logging & monitoring
  • Security & Permissions
  • Continuous Integration Setup
    • Centralised compilation
    • Automated tests
    • Code inspection and reporting
    • Scripted deployment
      • Code + configuration
      • Dependencies such as IIS, MSMQ
  • Automated backups and disaster recovery
  • Living documentation - e.g. wiki.

I'd even go so far as classifying refactoring and unit testing as a form of on-going infrastructure work.

Physical infrastructure work usually consists of one-off items which can sometimes be solved with a credit card. If you can solve the problem with money then take advantage! It's the problems which can only be solved with time which grow into ugly monsters.

As a software developer I experience most problems (unsurprisingly) around development infrastructure - in particular around refactoring and continuous integration systems. What happens when we start neglecting such items of work? Firstly, we should recognise why infrastructure tasks often take a back seat:

  • Lack of time. Ever heard "we don't have enough time to setup a repeatable build and deployment system", only for the project to spend months in the integration and roll-out phase? Or my personal favourite "we don't have time for backups", only for progress to grind to a halt when the shared development server overheats and cooks the hard drive? I've experienced both; the latter resulting in the calamity of attempting to rebuild the server from copied files on each developer's machine.
  • Lack of money. The customer is only interested in paying for software features. Management therefore decides to concentrate resources on developing features exclusively. Everything else is labelled as non-billable and therefore not justifiable as legitimate work items.
  • Lack of experience. Sometimes a development team, in particular management, just doesn’t have the experience required to recognise that infrastructure work exists. If you don't know that work exists how can you possibly plan and allocate resources to it?
  • Lack of skills. You may have a crack team of developers writing code but development infrastructure tasks require a holistic overview of the project. Infrastructure work often touches all areas of a system, meaning that people skills are just as important as technical ability.
  • Lack of discipline. Given the choice most developers would like to spend their time writing code and not spending time on those other "boring tasks".
  • Lack of tools. I've participated in teams where the purchasing of software licenses is handled by a far removed finance department. This often results in the development team waiting weeks and weeks for a piece of software that would otherwise dramatically improve their productivity. In one case an order was rejected because the finance department didn't understand what the software was for! This is a slight variation on the cost-cutting behaviours in the Short Pencil project pattern.

So, we start our project and spend our time writing features and neglecting infrastructure work:

Prioritising Features Over Infrastructure

Look at all of those features we are getting done! Unfortunately this graph doesn’t show the consequences of such an approach, for that we need to look at the overall team velocity:

Poor Velocity From Neglecting Infrastructure

It's possible for the team to achieve a pretty impressive velocity during the early stages (or iterations) of the project. The problem is that this approach is unsustainable. At some point the technical debt from neglecting infrastructure tasks will begin to overwhelm the team. Examples of such technical debt are the accumulation of spaghetti code from a lack of refactoring, brittle code from a lack of testing, a painful manual integration or deployment step or attempting to retrospectively apply logging and permissions across the entire application.

The end result is that progress can slow to where it's possible for your project to remain 90% complete for 90% of its lifecycle - killing team morale in the process. Your project has derailed into a Death March. Any attempts to predict a completion date go out of the window and you'll join project Chandler and many other software projects at the bottom of Fred Brooke's tar pit.

By ignoring infrastructural tasks you've created a massive amount of undone work. To make matters worse you are faced with the monumental task of having to complete the undone work at the end of project where it is much more difficult.

A better method is to prioritise both feature and infrastructure work based on the projects lifecycle:

Features And Infrastructure Prioritised

What we see is that during the initial phases of the project a lot more effort is spent on infrastructure tasks than on developing features. The development team will firstly spend their time creating an environment which enables them to ship features quickly, but spend only a small amount of time actually developing such features. Such work items could involve setting up a CruiseControl server, creating virtualised development environments, implementing and agreeing on automated code inspections. As the project progresses we see items such as unit tests, security, logging and code inspection taking a more prominent role.

The idea is to even out the team velocity to a sustainable and predictable pace.

Predictable Velocity

The failure mode for this infrastructure-first approach is when the project manager measures progress by the number of completed features. The problem with infrastructure tasks is that they are much less visible than software features and therefore more difficult to manage. Since less time is being spent on features it can appear (from the outside) as if the project is in trouble from the beginning. The temptation is then to add more resources to the project which is the easiest way to make a bad situation worse.

Technical Debt has a less known cousin, Technical Credit. By prioritising sound software engineering principles and paying attention to infrastructural work it's possible to reap massive benefits in later iterations.

Tags: , ,

Categories: Agile | Books | Management | Scrum

I Read... There... I Said It!

03 July 2009 by Stuart Cam

I am sometimes asked "so how do you keep up-to-date with all that's happening in the industry?".

Well, I...

The problem is making time for these endeavours. Usually I will alternate between them, much like a buffet, occasionally gorging on a particular topic when I am in the right frame of mind. Obsessive? possibly... but I think it's important to have a varied diet to keep it exciting.

Online articles and coding websites are great for picking up the odd skill or two but they often lack the depth required for some topics.

I had attempted to reconcile my love of technical books and mitigate some of their downsides (size, weight, cost and errata) by purchasing a Sony PRS-500 and seeking out digital versions of titles. I bought the unit when it first launched a couple of years ago and since then it has seen relatively little action. It's been quite a disappointment. The main problem is screen size, or lack thereof. It's way too small to render diagrams and code snippets with clarity. If your primary goal is to read technical books then I'd suggest avoiding the unit altogether and consider a tablet PC instead. That, or take a look at the much larger 9.7" Kindle DX.

I have recently purchased a round of new books in dead-tree format, which I intend to read over the coming weeks...

Software Development

97 Things Every Software Architect Should Know - Various

97 Things Every Software Architect Should Know - Various

I originally found out about the book through Udi Dahan's blog. Udi has made a couple of article contributions and figured it would be worth a read since I respect his opinions on software development and admire his NServiceBus framework.

The book contains small snippets of non-technical advice from a variety of architects - experience which, in some cases, has been hard won.

The unedited contributions from each author are available for free.

SOA in Practice: The Art of Distributed System Design - Nicolai M. Josuttis

SOA in Practice: The Art of Distributed System Design - Nicolai M. Josuttis

What are the two rules of distributed computing?

1. Do not distribute, it's difficult
2. See #1 (asynchronously)

I know, I know, it's a terrible joke! I had been looking around for an authoritative book on SOA and distributed systems and this book came highly recommended for cutting through the hype. The accompanying website can be found here.

.NET Framework

Effective C#: 50 Specific Ways to Improve Your C# - Bill Wagner

Effective C#: 50 Specific Ways to Improve Your C# - Bill Wagner

I had originally found out about this book through my professional network on LinkedIn via Chris Fulstow. He had mentioned that he was reading the book so I figured it must be worth a look.

It's full of best practices specifically for C#, covering some of the more esoteric language 'features' and how to avoid digging yourself into a big hole with them.

Take a listen to the interview with Bill Wagner on .NET Rocks!

More Effective C#: 50 Specific Ways to Improve Your C# - Bill Wagner

More Effective C#: 50 Specific Ways to Improve Your C# - Bill Wagner

I figured I ought to buy the sequel as it contains information on how to get the best out of the newest C# features such as LINQ and Lambda Expressions.

I heard about this book through Chris Fulstow (again) and subsequently through another .NET Rocks! podcast with Bill Wagner.

ASP.NET MVC 1.0 Quickly - Maarten Balliauw

ASP.NET MVC 1.0 Quickly - Maarten Balliauw

I've been aware of the ASP.NET MVC framework for quite some time. Pretty much the day after Scott Guthrie announced a prototype at the ALT.NET conference. I was looking for a short book which would assume knowledge of ASP.NET and cut straight to the .NET MVC implementation.

Since v1.0 is fairly new there are numerous online tutorials for earlier versions and some great MVC blogs, but not much in the way of printed books. This looked to be the best of a small bunch.

Windows Presentation Foundation Unleashed - Adam Nathan

Windows Presentation Foundation Unleashed - Adam Nathan

I have a minor confession. I have zero WPF experience, which is a little embarrassing since it replaced GDI+ long, long ago.

Unlike many other technical books this one is printed on glossy paper in full colour and totally jam packed with pictures and diagrams. Perfect presentation for learning a presentation framework!

An ex-colleague, Jack Ukleja, recommended this book.

Other Technologies

The Definitive ANTLR Reference - Terrence Parr

The Definitive ANTLR Reference - Terrence Parr

I purchased a digital copy of this book some time ago and have nearly finished reading it... in the bath... on my PRS-500... wrapped rather optimistically in cling film. This time I figured I'd buy the paper version and risk the £16.88 it would cost me to replace if I dropped it!

Bath + electrical items != mix.

Compilers: Principles, Techniques and Tools - Various

Compilers: Principles, Techniques and Tools - Various

Probably the densest, in both physical and subject matter, of all the books I purchased. I had a quick flick through some of the material and it looks pretty heavy going and full of maths. I anticipate that this will take a long time to read and no doubt expose many other holes in my knowledge along the journey.

My primary motivation was to supplement the ANTLR book with other compiler topics.

Collective Intelligence in Action - Satnam Alag

Collective Intelligence in Action - Satnam Alag

A rather interesting book intended for a Java audience, but which contains mathematics and algorithms suitable for implementation elsewhere. I discovered the book whilst surfing the internet for recommendation engines, which itself was a spur from reading about a recommendation engine that Joel Pobar had written in F#.

Hopefully this book will shed some insight on writing a sophisticated Web 2.0 application.

NHibernate in Action - Various

NHibernate in Action - Various

Another re-purchase of a book I own in digital form. I have followed it through the Manning Early Access Program.

My main gripe is that it doesn't cover the newest version of NHibernate, but given the time it takes to write a book and the speed at which frameworks evolve it's forgivable.

One of the best references for NHibernate available today.


Here Comes Everybody - Clay Shirky

Here Comes Everybody - Clay Shirky

If I had a dollar for every time I heard Jeff Atwood mention Clay Shirky on the StackOverflow podcast I'd probably be able to buy this book without opening my wallet! Instead, I decided to spend my own money and discover the material for myself.

It comes highly rated and promises some insight on the huge changes we are seeing on the internet today with social networking and the wisdom (or madness) of crowds.

The Paradox of Choice: Why More is Less - Barry Schwartz

The Paradox of Choice: Why More is Less - Barry Schwartz

This is the least technical book of the bunch, cited as a reference in The Cult of the Amateur - a book I really enjoyed reading on my travels.

I would probably consider myself a 'maximiser' when it comes to making purchases - perhaps this book will change the way I rationalise my spending decisions? Perhaps it will change the way I buy books? :)

Simply put, read books and treat your education as a #1 priority.

Bill Hicks would probably agree (just with more swearing):

Tags: , , ,

Categories: .NET | Books | C Sharp | General | MVC | SOA | Web

© Codebrain 2021. All Rights Reserved. Registered in England: 07744920. VAT: GB 119 4078 13