BLOG

Eric Evans DDD Presentation

12 June 2010 by Stuart Cam

Another fantastic presentation from Eric Evans which resonates strongly with me:

http://domaindrivendesign.org/node/198

During the presentation he talks about documenting the process he has been using over the last 6 years. You can download a copy of the draft write-up too.

http://domainlanguage.com/processdraft/

Is this the possible start of a new book?

Tags: , ,

Categories: Agile | DDD

Kanban Experiment

27 February 2010 by Stuart Cam

Problem

What do you do when your development process is in pieces, hundreds of defects are being raised against your code, ad-hoc work requests are piling up, the developers are groaning 'if we only had the time to make these improvements life would be easier' whilst the fixed deadline is looming.

If you find yourself in this situation the simplest and easiest thing to do is walk out the door.

It may be that the project you are working on is so fundamentally broken that even the most talented development team in the world still has no chance of succeeding. Hundreds of decisions are made during the project inception stage and each one can affect the overall chances of success. By the time the developers arrive the problems have solidified, and given their nature (financial, contractual, resourcing, managerial) the developers are, for political reasons or otherwise, unable to tackle them. Another death march begins.

Adding developers to a project with fixed deadlines, fixed deliverables in a fixed price contract rarely produces good software and in the worst case can leave the company responsible for delivery in ruins, perhaps even bankrupt.

I've been contracting long enough to realise that the majority of the problems I face as a software developer are not software development related, but both project management and business related. I'm not saying that all companies are like this, but that some companies are more deadly than others.

Acceptance

If you find yourself on a challenged project and are courageous enough to go the extra mile you can try and introduce some order onto the chaos, patching up the major holes as best you can. A do-it-or-die attitude is required because you'll probably ruffle somebody's feathers in the process.

Given the above situation, I decided go back first principles and implement a Kanban system with the help of my team, in the face of some serious project challenges.

Solution

First, you'll need a large wall space, some marker pens, coloured tape, sticky dots and post-its. If walls with whiteboards are in short supply then the back of some borrowed cupboards will suffice.

We loaded the Kanban with our backlog, giving each type of work a different coloured post-it:

Use Case

A large chunk of work which needs to be broken down into smaller tasks. Many tasks go together to form a complete Use Case.

Task

A task for a particular Use Case of sufficient size to be estimated and worked on by a developer(s).

Improvement

Improvements are internally generated within the development team. An example may be a largish refactoring opportunity, automated process or tools upgrade. The point here is that time spent on an improvement will increase the flow of other post-its across the board.

Defect

Raised by the test team on the current release of code. These defects are raised and entered through an external system, the post-its are placed on the Kanban to track the work in progress.

Ad-hoc

Externally generated work, perhaps as a result of a customer demonstration.

Problem

Can be placed on any other post-it to denote that there is an outstanding issue, complete with a description of the problem.

As well as colour coding each card contained the following information:

  • Date the post-it arrived on the board.
  • A description of the work item.
  • Priority denoted with a coloured sticky dot, for defects this dot represented severity.
  • Initials of the current owner.
  • Time spent on the work item. We would dot all items that were in progress every half day.
  • Optional. Date the post-it needs to be completed by.
  • Optional. Identifier from the external system, e.g. Defect Id.

The team were educated on the system in under half a day and by week 4 our Kanban had grown to incorporate another cupboard and many more modifications:

Kanban - Week 4

We cleared the finished section of the board (far right) every week. The board above is telling us that we spent a lot of our time addressing failure demand in the form of defects. The concern here is that the actual estimated and paid for work is represented by the pink post-its on the far left and they didn't move for over a month!

Retrospective

The Good

  • The teams focus was aligned to completing work.
  • Encourages the team to recognise, record and make improvements.
  • Anybody can write up a post-it and add to the backlog, I'd often say "don't tell me, tell the board".
  • Single visualisation of the work backlog, work in progress and work completed.
  • See different types of work at a glance.
  • The team has autonomy over pulling new work, creating a sense of ownership which is lost when work is assigned.
  • Managers can prioritise work in the backlog.
  • Colourful and tactile, giving a sense of fun to the project.
  • Flexible and adoptable. We altered the board as our understanding improved.
  • Simple and cheap. The total cost of implementation was less than £30.
  • Easy to gather metrics about how much work the team can get done.
  • The simplicity of the solution promotes the idea that it is easy to change and customise.

The Bad

  • Doesn't work with distributed development teams for obvious reasons.
  • It's simplicity can be interpreted as amateurish which results in the idea being written off.
  • Easy to forget to mark post-its with key information, particularly dates and owner information.
  • Can be difficult to find a particular post-it, the problem somewhat compounded by using identifiers from external tools.
  • No backup, although conversely it's less likely to crash and lose your data!

The Impact

  • The board showed us that there was more work that needed doing than was planned or imagined.
  • The previous development approach had led to a lot of defects.
  • Developers can self-organise and work honestly if they are given the chance.
  • The role of management is to remove problems (purple post-its) and to optimise the system to improve the flow of work through it.

The Kanban board provided us with a greater level of insight into what was happening on our project. The visualisation of work enabled us to make better decisions about how to utilise our time. By recognising that we have a finite number of resources and a large amount of work, we can make team decisions about how best to proceed.

In some ways the Kanban board affirmed what we already knew, that our project was seriously challenged, but it also gave us a framework to work within. Understanding and learning about your situation is simply the first step in escaping it, without walking through the door!

Tags: , , ,

Categories: Agile | Management

Development Models

31 January 2010 by Stuart Cam

Agile / Iterative Development

Iterative Development

Waterfall / Predictive Development

Predictive Development

Tags: , , ,

Categories: Agile | Management

Features vs. Infrastructure

21 December 2009 by Stuart Cam

Your software development team likely has a process for dealing with requirements, estimating, allocating tasks and completing coding work. After all, developing software is just about providing application features isn't it?

Each and every software project has an internal work stream, which if neglected can start undermining your ability to produce quality software. I've heard this type of work referred to by a number of different names: core, infrastructure, supporting code or framework. I propose that it can be summed up as:

Work that needs to be done to support getting the primary work done.

I'll refer to this type of work as infrastructure throughout the rest of this post.

Let's look at a typical enterprise software project, what are some examples of infrastructure work?

Physical infrastructure

  • Day one experiences
    • Seating
    • Workstation setup
      • Installed software
      • E-mail setup
      • Network permissions
    • Team introductions
    • Code & business walkthroughs
  • Development servers
  • Software licenses

Development infrastructure

  • Source control
  • Logging & monitoring
  • Security & Permissions
  • Continuous Integration Setup
    • Centralised compilation
    • Automated tests
    • Code inspection and reporting
    • Scripted deployment
      • Code + configuration
      • Dependencies such as IIS, MSMQ
  • Automated backups and disaster recovery
  • Living documentation - e.g. wiki.

I'd even go so far as classifying refactoring and unit testing as a form of on-going infrastructure work.

Physical infrastructure work usually consists of one-off items which can sometimes be solved with a credit card. If you can solve the problem with money then take advantage! It's the problems which can only be solved with time which grow into ugly monsters.

As a software developer I experience most problems (unsurprisingly) around development infrastructure - in particular around refactoring and continuous integration systems. What happens when we start neglecting such items of work? Firstly, we should recognise why infrastructure tasks often take a back seat:

  • Lack of time. Ever heard "we don't have enough time to setup a repeatable build and deployment system", only for the project to spend months in the integration and roll-out phase? Or my personal favourite "we don't have time for backups", only for progress to grind to a halt when the shared development server overheats and cooks the hard drive? I've experienced both; the latter resulting in the calamity of attempting to rebuild the server from copied files on each developer's machine.
  • Lack of money. The customer is only interested in paying for software features. Management therefore decides to concentrate resources on developing features exclusively. Everything else is labelled as non-billable and therefore not justifiable as legitimate work items.
  • Lack of experience. Sometimes a development team, in particular management, just doesn’t have the experience required to recognise that infrastructure work exists. If you don't know that work exists how can you possibly plan and allocate resources to it?
  • Lack of skills. You may have a crack team of developers writing code but development infrastructure tasks require a holistic overview of the project. Infrastructure work often touches all areas of a system, meaning that people skills are just as important as technical ability.
  • Lack of discipline. Given the choice most developers would like to spend their time writing code and not spending time on those other "boring tasks".
  • Lack of tools. I've participated in teams where the purchasing of software licenses is handled by a far removed finance department. This often results in the development team waiting weeks and weeks for a piece of software that would otherwise dramatically improve their productivity. In one case an order was rejected because the finance department didn't understand what the software was for! This is a slight variation on the cost-cutting behaviours in the Short Pencil project pattern.

So, we start our project and spend our time writing features and neglecting infrastructure work:

Prioritising Features Over Infrastructure

Look at all of those features we are getting done! Unfortunately this graph doesn’t show the consequences of such an approach, for that we need to look at the overall team velocity:

Poor Velocity From Neglecting Infrastructure

It's possible for the team to achieve a pretty impressive velocity during the early stages (or iterations) of the project. The problem is that this approach is unsustainable. At some point the technical debt from neglecting infrastructure tasks will begin to overwhelm the team. Examples of such technical debt are the accumulation of spaghetti code from a lack of refactoring, brittle code from a lack of testing, a painful manual integration or deployment step or attempting to retrospectively apply logging and permissions across the entire application.

The end result is that progress can slow to where it's possible for your project to remain 90% complete for 90% of its lifecycle - killing team morale in the process. Your project has derailed into a Death March. Any attempts to predict a completion date go out of the window and you'll join project Chandler and many other software projects at the bottom of Fred Brooke's tar pit.

By ignoring infrastructural tasks you've created a massive amount of undone work. To make matters worse you are faced with the monumental task of having to complete the undone work at the end of project where it is much more difficult.

A better method is to prioritise both feature and infrastructure work based on the projects lifecycle:

Features And Infrastructure Prioritised

What we see is that during the initial phases of the project a lot more effort is spent on infrastructure tasks than on developing features. The development team will firstly spend their time creating an environment which enables them to ship features quickly, but spend only a small amount of time actually developing such features. Such work items could involve setting up a CruiseControl server, creating virtualised development environments, implementing and agreeing on automated code inspections. As the project progresses we see items such as unit tests, security, logging and code inspection taking a more prominent role.

The idea is to even out the team velocity to a sustainable and predictable pace.

Predictable Velocity

The failure mode for this infrastructure-first approach is when the project manager measures progress by the number of completed features. The problem with infrastructure tasks is that they are much less visible than software features and therefore more difficult to manage. Since less time is being spent on features it can appear (from the outside) as if the project is in trouble from the beginning. The temptation is then to add more resources to the project which is the easiest way to make a bad situation worse.

Technical Debt has a less known cousin, Technical Credit. By prioritising sound software engineering principles and paying attention to infrastructural work it's possible to reap massive benefits in later iterations.

Tags: , ,

Categories: Agile | Books | Management | Scrum


© Codebrain 2017. All Rights Reserved. Registered in England: 07744920. VAT: GB 119 4078 13