Managing Budgets and Billing while Practicing Agile Development

When we started transitioning into using an Agile development method just over a year ago, one of the first and most constant challenges we ran into was how to make it work for our clients. Agile has been a fantastic tool for defining internal processes that really work for us at Affinity Bridge. Many of our clients, being non-profit organizations and academic institutions, however, are accountable to boards who have to review and approve their budgets ahead of time. They're not able to bill according to work done during agile sprints, without having budgeted for the work ahead of time. Here are a few tips from the lessons we've learned for doing agile development while managing estimates and budgets in a way that works for our clients.

Fixed bid Request For Proposals (RFPs) are still a fact of life

We've been lucky enough to have a few ongoing projects where we have been able to do truly agile billing. The client knows how much our work costs, we ballpark for them how much work the features they're requesting will cost, and then we get to work with this understanding. But more often than not, when an organization wants to do a redesign of their website or begin a new web development project, they often need to know how much that is going to cost upfront so that they can request budget from whoever approves spending. When working with corporate clients, sometimes the Product Owner (the person who represents the client when working with us) is the person who manages spending, and has a lot more flexibility to decide how budget is managed. But with non-profits, government funded projects, and academic institutions, this is almost always not the case.

How to contend with this? Use estimates, not quotes. Estimates mean there are factors you are unsure of that might cause the final cost to be higher or lower than your best guess. Prospective clients should be made aware that oftentimes research is necessary prior to being able to give a well-educated estimate for a project... sometimes a lot of research. If you are reviewing an RFP, and get that gut feeling that the project's developer hours (hours of actual development work, not including meetings, documentation, etc.) could be anything between 150 and 700, stop right there! Never act like you can guarantee an estimate when you are this unsure.

If there are complicated parts to this project, such as database migrations from unknown sources, or major API integrations that you have not had a lot of experience with, be honest about this! Tell the client that it would be dishonest to give a very concrete estimate without first digging into the database or coding a proof-of-concept for the API integration. Don't just tell them what you think they want to hear to land the contract. Honesty is always the best policy, and will prevent you from breaking your word down the road.

If they are extremely wary of having a separate initial project phase that entails only research, give them an escape route: don't make them commit to having you do the build. If you trust your technical and interpersonal skills, and your team, this is a good way to build trust with the client, and show them that you are 100% confident you will do an excellent job completing the project. 

After you've done your discovery work, then you can go ahead and provide a much more detailed and accurate estimate for completing the project. This estimate for the second phase (the main development phase) is what you can substitute as your "fixed bid" estimate. But it's not really a fixed bid, because you're working within an agile framework. What this number represents, is a concrete cost for the main build phase. Which brings us to...

Flexibility and the three legs of the project sustainability stool

There's a widely used concept in sustainable development, the "three pillars" of sustainability, which are society, economy, and the environment. You can think of it like a three legged stool; if one leg is weak, you're sure to fall down, ie. lack long term sustainability. 

In web development, the three pillars tend to be seen as scope (the feature set and complexity of the project), cost (actual cost in dollars), and time (when the project is completed). Whereas in most fixed bid projects these are all set in stone from the beginning, and changes are a process in and of themselves, with agile development, we are much more flexible. That said, if there is a request to change one of these, the others will need to bend and accommodate. If you are actively accommodating your client's requests to modify the initial technical specifications (what features there will be and how they'll be built) during the course of a project, they need to fully understand that no change comes without a consequence.

If they want to add features (scope), they will either need to cut other features, or the project will cost more and take more time to complete. If they want to cut costs, they will need to cut features to accommodate. If they want a quicker completion, they may need to both increase budget (so you can hire extra developers) and cut features. We give our clients a fair amount of control over these things (within reason) as long as they understand the interplay of these three factors. This can only work when you have a solid relationship with your client, that is based in honesty and trust. If either of you doesn't hold up your end of the deal, this method will be a lot more risky.

Some clients may not want to contend with the risk and complexity this adds, and won't ask for any changes during the course of the project. This is totally okay, and makes for a quick and tidy project. But because we do some fairly complex projects at Affinity Bridge, many of the clients we've worked with find they start discovering new needs they did not know they had, or different needs than what was in their RFP. This is especially common once they start seeing exactly what it is we're building; we demo our in-progress builds to our clients early and often, and this leads to more discovery on their side about their business needs.

Some companies are completely against this approach, because obviously scope creep can be risky business. But we've found that the flexibility agile provides becomes a huge asset, and helps our clients to be more satisfied with, and have more of their needs met by, the completed project.

Phased development helps bridge the gap

One of the best ways we've found to contend with flexibility, while meeting the expectations and needs of both ourselves and our clients, is working with phased development. Breaking a big project into a few smaller phases helps cut down on the risk involved on both sides, while providing the time and structure to build trust with your client. 

We tend to work with three main phases:

1. Discovery
2. Build
3. Additional Features

Sometimes in bigger projects, there are more than one "Build" phase, for example if there are huge custom feature sets, or a big database migration to tackle. The "Additional Features" phase is a nice way to contend with scope creep (ie. client requests for additional features to be pushed into the project without any give on the time or cost). It allows us to evaluate feature requests, consider whether they are a high enough priority to be accommodated in the main "Build" phase. And if not, push them into an upcoming phase that the clients know will actually happen (ie. not just brushing off the requests).

Finally, we almost always (unless the client has an internal developer/team to hand the project over to upon completion) follw the final phase of development with ongoing maintenance, that might sometimes involve future "Phases" if there is a larger feature/change needed. All Drupal sites require a small amount of maintenance to keep modules and the core system up to date on security updates. Often, there are also smaller follow-up requests from the clients as they use their new site and discover needs they weren't anticipating initially.

Resources

A lot of these lessons have come purely through experience, but in talking with other project managers, I've found that this is a system that is becoming fairly common among Drupal webdesign and development companies, especially those experimenting with using agile development practices. Here are some general resources I found helpful early on in the transition, as well as articles more specifically on the topic of agile estimating:

• Agile Estimating and Planning, by Mike Cohn (There's a lot of pretty intense information here about how to do the math of agile planning and estimation, but regardless of whether you want to get that in depth, it's got a lot of good ideas about how to estimate when doing more pure agile.)
• Video of Mice Cohn talking at Bay XP Meeting: Part 1 and Part 2 (He starts getting into estimating around 20:40 of Part 1)
• Scrum Alliance: Why fixed bids are bad for clients, by Andy Brandt (General commentary)
• Dr. Dobbs Agile Newsletter: The consequences of fixed-bid IT projects, by Scott Ambler (Good summary of the risks and benefits involved in fixed-bid vs. variable-priced estimating.)
• Agile Modeling: Examining the "Big Requirements Up Front (BRUF) Approach" (Gives some overviews of rationales behind moving towards this sort of development, estimation, and billing.)
• Ambysoft: Agile Project Planning Tips, by Scott Ambler (Another general article about agile project planning and estimating, lots of good links and references listed at the end.)
• Agile Software Development: 10 Contracts for your next Agile Software Project, by Peter Stevens (Examples of various models for pricing)

If you're confused about what Agile Development is, or by some of the language I've used, I highly recommend reading Practices of an Agile Developer or at least Wikipedia's page on Agile software development.