By Sjoerd Jan Ter Welle

The average software development project runs twice as long as initially intended or estimated. More than 60% of the features build in software is hardly or never used even once by the client after the software is delivered. Almost 70% of all software development projects run substantially out of budget. Estimating software development projects is traditionally done at the beginning of the project before the actual start of developing the software. First of all, the functional and nonfunctional requirements are gathered. The effort and costs are estimated and a quote is send to the client. Ultimately he signs a contract. This established way of estimating software development projects often does not satisfy.

Once the project is started the customer tends to push new requirements to the project making it harder to deliver on time and stick to the contract. If the client gets to see a prototype sometime during the development phase, he will then suggest new features, request changes or might even drop functionality. New ongoing insight or changing market conditions lead to problems if the project stays fixed to the requirements agreed upon up front and put in the contract. One way to cope with these changes is to incorporate a change management procedure. Alternatively, making projects agile can provide a more flexible and satisfying software delivery model. However, the question arises as to how to estimate project costs and delivery dates with a fair degree of accuracy with a continuous flow of new features being requested and other functionality maybe even dropped? How to quote an agile project to a customer and cope with the changes? How to stay in control of the project budget while offering agility?

Agility helps reduce project failure

In 2001 a group of software gurus got together and formed the Agile Alliance and published the Agile Manifesto. This manifesto states that individuals and interactions go above processes and tools, working software comes over comprehensive documentation, customer collaboration over contract negotiation and responding to change over following a predefined plan. Some of the agile software development methods around today are SCRUM, DSDM Atern, Feature Driven Development (FDD) and Extreme Programming (XP). What all these agile methods have in common is the idea that not all the variables in a software development project can easily be fixed up front. The idea is to fix costs and time on the project, but to leave the number of features to deliver flexible. Often new insight or ideas are found only while the project is already underway. The market is moving as well, while the project is in progress, which means that necessary adjustments to initial plans and ideas often need to be made. Fixing the scope of the project at the start does not necessarily provides a software solution that suits the client needs the best. Implementing a formal change management strategy might not always be the best solution to the challenges at hand.

One of the most common agile practices is to work in iterations, also called timeboxes or sprints, of two to four weeks. Another practice is to continually re-prioritize the requirements at the end of each iteration. All features still to implement are stored in a feature backlog, sorted according to customer value and the features implemented in the next iteration are the features with the highest client value. At the end of each iteration working functionality or completely developed features are delivered. The client is involved from the start, sees the results of each iteration and is motivated to suggest improvements or provide even new feature requests to the team. Testing is done within the iterations themselves and the next iteration delivers new features not worked on before or not fully completed and tested in the previous timebox. Project planning is focused more on delivering working features at the end of each timebox than on completing a list of tasks.

The advantages of these agile practices are many. Ongoing insight and new ideas are more easily welcomed while the team continues to work first on the features with the highest priority in the backlog. Project progress is made visible to the client through completed features at the end of each iteration. Agility helps thus reduce the risk of focusing too much on delivering functionality agreed upon at the start but not needed anymore. However, questions remain as to how to stay in control over the project costs if clients are so easily allowed to add new features while the project is running. How can a development project stay within budget and still deliver the right functionality in time and within costs?

Estimating agile software development projects

A practical but effective way to manage agile software development projects is by estimating the size of all known features or user-stories in the backlog relative to each other using story-points. User-stories are simplified use-cases, written in a simple text format and are easy to comprehend by non-technical project participants as well. They describe the feature to build. Story-points are numbers assigned to each feature or user-story to indicate the size or effort of building the feature. For example, if you estimate that building one feature takes twice as much time and effort as another feature, than the first feature gets twice as many story-points as the second feature. Right now you do not want to link effort in time directly to user-stories yet. When assigning story-points it is best to stick to a simple list of possible numbers like 1, 2, 3, 5 or 8 story-points. Begin with the smallest user-story or start with a user-story of average size and work from there. Estimate how one user-story relates in size and effort to another and give each feature the relative amount of story-points. Lets assume that during the first iteration of two weeks several user-stories are developed into working software functionality, which features were on the top of the feature backlog and were highlighted by the client as having the highest value to him at that moment. After 2 weeks three user-stories were completely developed and tested. Each of these three user-stories were previously estimated at 5 story-points each. This results in having the current rate of progress, productivity-rate or also called the velocity of the development team at hand. The velocity on this project is 15, that is, 15 story-points per iteration.

Velocity shows how to stay within project budget

Velocity is calculated by estimating all known features to be implemented in the project, assigning story-points to all the features and relating these estimated values to the number of story-points the team in average can develop during the course of one iteration. As new features are added to the feature backlog and maybe other features are being dropped from the same list, the total number of story-points already developed and still to develop gives a clue as to where the project is heading and when the project is expected to be completed. As you progress and more features are completely developed into software the velocity is fine-tuned automatically as it is adjusted at the end of each subsequent iteration. Note that the functionality delivered at the end of the project is not fixed. However, the software being developed during the project runtime is probably software of practical use to the client, where the client was involved already early in the project and fully tested features were delivered already early on. Most likely the core of the software will have been delivered within the project timeframe. Maybe some of the features were not included or postponed to a later release.

Yes, agile projects can be estimated. The velocity gives insight in when the project can be delivered if the rate of development goes on unchanged and with a known set of features. If the velocity shows that the project cannot be delivered in time, action can be taken to add more resources and people to the project or to drop specific functionality. Using the productivity-rate of previously completed but similar development projects the project manager has valuable information available which he can use to estimate new projects, even enabling him to give customers a valid insight in the amount of functionality which can be produced in a fixed amount of time.

Sjoerd Jan ter Welle has years of experience in managing software development projects. He has his own offshore software development firm now for over ten years in Romania and is an active board member of the Agile Consortium Benelux and initiator of the Dutch Process Improvement Consortium. He is author of books and articles on process improvement, project management and compliancy. He is CEO of Bright55 http://www.bright55.com – Bright55 provides BPM and project management solutions.