Drupal 8: git + sprints = efficient + smooth

At DrupalCon Chicago this past week, there was a "Core Conversations" session track, made up of sessions pitched by contributors to the core Drupal project. A wide range of topics were covered from the Butler project (a new system for context and blocks), to the built-in Help system, to Deployment strategies, to redesigning the issue queue. These sessions were shorter presentations followed by a discussion period for the attendees to give input on the topics.

The final conversation on Thursday by Dries Buytaert (Drupal's project lead) focused on discussing the development process for Drupal 8. (It was also exciting to witness the Drupal 8 branch being created live during the session!) During the presentation portion, Dries described in more detail the process changes he had suggested during his keynote. He then opened the floor up for everyone to bring up issues they felt needed some attention/discussion.

Discussion points

Here is the list of discussion points (that Dries noted during the talk) we core enthusiasts came up with:

  • More structured sprints - project management
  • Sandboxes (aka. Samboxes, since Sam Boyer was a huge contributor to Drupal's CVS to git migration) and locations
  • Timeline of release cycle
  • Hybrid development cycle
  • Ubuntu model?
  • Gate details; performance testing, etc.
  • Roles of initiative owners
  • Backporting
  • Non-initiatives / small changes / bugfixes -- different process?
  • Tools for usability/accessibility
  • Process around new initiatives / proposal
  • Documentation gate - workflows

Many, but not all points were discussed, and as we progressed through the conversation, I began to see parallels between some of the process changes we've implemented here at Affinity Bridge and what's going on in the Drupal development process. When I first began my position as Project Manager here, the first task I was assigned was to figure out how to make our development process work better, especially for larger, ongoing projects. This was sparked by the pain of working on very large projects, with huge issue backlogs and many feature requests, and no set launch date. 

After a lot of reading and research, our team started experimenting with a more agile process. Picking smaller chunks of work, completely finishing them, picking a new chunk of work, completely finishing it, etc. helps to make progress faster and at a better quality. Many times now, the benefits of this process have proven themselves, and I am beginning to see how some of the main points we discussed at the core conversation could be addressed with some incremental changes towards a more agile development process. 

A sidenote, as I was discussing this over the weekend, Bruno De Bondt from DeWereldMorgen.be pointed me at a blog post from one of his old coworkers from Krimson (a development shop in Belgium), Kristof De Jaeger (aka. swentel). The post is from about a month and a half ago, and outlines a proposal for an agile Drupal 8 development process. He goes into much more detail about the process and different roles involved than I will here, but it is well aligned with my thoughts (and potential long term vision), so I recommend you read it!

Specific ideas about sprints and agile for core

Before I delve into details here, I would like to preface this by saying I am not a religiously devoted agile follower. So far, what I've seen work is using elements of the agile methodology that fit for the team and project. Some devoted capital-A "Agile" followers might find that to be a flaw, but I don't think it's necessary to shoehorn people into a process. It's important that anyone adopting this feel comfortable with it, and not forced into it. Also, speaking briefly with Dries after the core conversation, he noted that it's important with such a large community not to try and undertake too much change at once (referring to the git migration and also the new idea of core "initiatives"). I very much believe those are wise words, and so am more keen to suggest some practices that the community can experiment with. If these are successful, then perhaps they can be applied more overarchingly.

Now, to go through a few of the main discussion points and others that came up, and relate them to how a more sprint-based/agile process might help address them.

New opportunites from sandboxes and git

The move to git is obviously going to open up many doors as far as more collaborative work and changes to workflows. Any Drupal.org user can now have their own development sandbox, and as Dries mentioned anyone can now fork core (ie. make a duplicate version and add improvements to it), though it's not always recommended! 

As many others noted in the session, this is going to allow multiple people to easily work on different parts of core and then merge their work back together. It could also allow multiple people working on different approaches to the same parts of core, to then compare and combine the best pieces. In any case, it will allow for a lot more safe experimentation and collaboration.

Keeping the criticals count below 15

This was something Dries brought up in both his keynote and the session. The Drupal 7 release cycle was very drawn-out because of the large quantity of criticals accrued through the three years of work. Longer release cycles tend to lead to burn-out and work that doesn't get fully completed.

Having the sandboxes and git will allow for a totally new approach here, one which is common to a more "agile" process, which prioritizes keeping the master branch clean and ready to launch at any time. To work like this, the master branch for core would be kept in a launchable state, and any changes or new work would be done in branches/sandboxes which would have to be completely done to be merged back into core. The "gates" would protect the master branch from bugs or incomplete work, so that we could be relatively certain that core is always ready to ship.

As a result, we wouldn't be aiming for under 15 criticals, but rather a constant of zero criticals in the master branch.

Small changes and bug fixes

Small changes and bug fixes would also be done in branches, but merged back in more frequently. There would be no necessity to save up batches of small fixes, though they could be done in batches related to different pieces of functionality. 

We would want to aim to avoid incurring technical debt, ie. fix problems as we go, as much as possible. Never delay working on bugs that will need to be fixed before launch. If that's the case, the work is not "done" and should not go into the master branch. This may mean more refactoring, but it helps keep the master branch ready to ship at all times.

Gates

The "gates" Dries talked about are still being defined, but the general idea is that there would be some steps that need to be completed before any branch would be merged into the master. Examples of what gates there might be include:

  • Documentation: making sure all of the code/API documentation is complete/updated, and ideally make sure the online documentation is also complete/updated. Possibly could go through review by Documentation Team members or other developers.
  • Accessibility: making sure that basic accessibility standards are met. Possibly could go through review by Accessibility Team.
  • Testing: making sure automated tests are written and pass. Possibly could go through manual testing as well.
  • Performance: making sure performance meets certain standards.
  • Design/UI: having design or usability reviews.

Making this concept of "gates" work will require defining a set of requirements and standards, and possibly finding people willing to do frequent reviews. But it will also mean that this additional work is really done before merging into the master branch, and that "done" refers not only to code being complete, but a more holistic interpretation of when work is ready to ship.

Timeline of release cycle

Dries suggested a shorter release cycle for Drupal 8, with likely a more focused and smaller set of initiatives (which he's outlined as areas of focus that will have initiative leaders). Keeping the master branch ready to ship at all times will mean that the release cycle can be as short or long as we want, and that we are not limited by half-finished functionality in the master branch (since only finished functionality would ever be merged in).

More structured sprints and phases

A lot of these goals and ideas lend extremely well to working within more structured sprints, and using phased development for larger initiatives. My initial suggestion (Kristof's post suggests two month sprints) would be for one month sprints. Despite the issue of working with a spread out team of mostly volunteers, I find that shorter sprints lead to better momentum. I also feel that it will be easier to pick out clear sets of issues to work on per sprint if they are relatively short.

Another benefit to shorter sprints would be the potential to attract more help from people who aren't interested/able to work on core more consistently. It's a lot easier for someone to commit to working on some functionality for a month rather than for three years. I would bet that once someone helped with one sprint, they would be far more likely to end up helping on another one down the road.

For larger pieces of work, we'd want to work in phases that are several sprints long. Each phase would have a set of functionality that can be merged with the master branch when complete. And the end of each phase would be a great place to work through the "gates". It would be nice to think we could go through the gates at the end of each sprint, but I don't think we currently have the resources to do this. It might take many phases to get an initiative fully complete, but the length of the initiative wouldn't necessarily delay a potential release.

A note on infrastructure...

Structured sprints would be much easier to do if we were able to add "sprint" and/or "phase" fields to the issues, but even without this we can always organize what is in each using tags, as was done with the git migration.

It would also be ideal to be able to relate issues in a richer way like in some issue tracking systems such as Unfuddle (see image below). Despite Unfuddle not being created specifically for agile development, it's easy to use it this way. I tend to create "parent" tickets for meta-issue/discussions and then create specific work item tickets as "children". You can also mark "related" issues, which aren't part of the batch of tickets, but are somehow related. And you can have parent tickets that have child tickets that are parents to others, so it's possible to create a hierarchy of what needs to be done to move onto another piece of work. Then, I create a "Milestone" for each sprint, with start and end dates, and add and prioritize tickets in the milestone appropriately after discussing priorities with the product owner and the development team.

If this interests you, there are issues filed for redesigning the issue queue and creating functionality to support meta-issues that you can add your opinions to.

Project management

This all seems to beg the question: does the Drupal project need a dedicated project manager (or scrum master)? 

At this point, I would say no. Between Dries (who tends to fill the product owner role), the core maintainer he appoints (who tends to act somewhat as a project manager, and does a lot of QA), and now the new Initiative leads, I don't feel that this would be necessary during a period of more experimental adoption.

If teams and individuals try this method, and find it works well enough that it could be adopted fully for all core development, it would be a very good idea. A much more structured process takes a lot of work to keep organized and stick with, and I believe it's best to keep developers' time free to be dedicated to development as much as possible. So at this point we would want to create some overall structure in the issue queues to be able to manage sprints efficiently, and have someone who can oversee the project's organization as a whole.

Realistically, I don't think this would be necessary very soon. For now, I would suggest anyone interested start with:

  • Running set sprints (recommending one month). Meeting at least once per sprint (on IRC or by voice) to review and align priorities amongst whoever is working on a given issue/set of issues/initiative. Communicating frequently on IRC (like we always do!).
  • Defining work to be done at the start of the sprint, and doing your best not to introduce new work into the in-progress sprint. In the beginning, underestimate how much can be done, until you get a feel for the momentum possible in a single sprint. If some things don't get finished, they go back into the "backlog" (or main queue of issues) and are reevaluated in regards to whether they will be in the subsequent sprint.
  • Making sure at the end of the sprint (or phase for larger initiatives), that you pass through the "gates" and deliver functionality that is completely "done" and ready to merge into the master branch.
  • Making sure that no bugs get into the master branch so it can always be ready to ship. As a result being able to make the release cycle any length, and launch on short notice if desired.
  • Avoiding technical debt (things that you put off that will need to be done later); making time for refactoring or bug-fixing as you go. Possibly even having a technical debt recovery sprint early on.
  • Defining the "gates" clearly and deciding who is responsible for signoff before merging into the master branch of core.
  • During this experimental phase, making sure there are some resources that teams and individuals can refer to and learn more about these methods. Possibly looking for project managers in the Drupal community who are familiar with agile development, and open to informal advising on IRC.
  • Remembering that this is a pilot project of sorts. Not putting too much pressure to follow capital-A Agile strictly. Letting this be an organic process to see what fits the community best, while leading to a more efficient, clean, and smooth process.

I was extremely happy to hear how open and interested the other attendees of Dries' core conversation were, and hope this helps clarify some ideas. This should be an open discussion rather than anything too prescriptive, so I would love to continue the conversation and hear your feedback, questions, and concerns below.

(ps. The Drupal 8 wordmark used with this post on the homepage is from Dries' slides, just for the record!)

Comments

Fantastic proposal

This is really exciting stuff and as far as I know rather revolutionary among open source projects. Most open source projects do the BDFL + Company sponsored devs + up-and-coming geeks with spare time thing.

It's okay, but I think the other side benefit of structure is community building and opening up the process to people who want to be continuously involved, but can't afford to spend the intense amount of time required to do so currently.

I support you +100

-Jacob

Hey Jacob - that is awesome

Hey Jacob - that is awesome to hear, coming from another agile/PM geek. I wouldn't mind hearing your thoughts on whether the project actually needs a dedicated PM (or might down the road) if you catch this. I'm pretty on the fence right now...

This is fantastic

As a veteran of both the d.o redesign implementation and the Git migration, this is music to my ears. IMHO, having focused sprints with clear goals and finite timeframes would be a big win over our current practices for core development.

In fact, I basically organized a quasi-sprint like this at the very end of the D7 lifecycle to try to get the Update manager to not completely suck before we shipped 7.0 (I can say that, I'm the maintainer). ;) It basically worked -- I posted a clear statement of the problems, maintained a list of issues we needed to fix, and some folks came out of the woodwork to really help crank through the list. It literally was a sprint -- we only had about 2 weeks before 7.0 shipped, so it was definitely finite time to work with. ;) In the end, I didn't have to write the "rip the Update manager out of core" patch (although there's still lots to do.

Having a bit more structure for those sorts of efforts, a culture in the community that this is a good way to do things, and some more support from our collaboration tools to make it easier would be great. Plus, with Git and the concept of targeted core initiatives with their own maintainers (some officially bless by Dries, others self-organized), it'd be a lot easier for people to coordinate and organize their own sprints.

Probably the main thing to get right here is that we need a clear, obvious place to document the initiatives and sprints so people who want to know what's going on and how to help don't have to be following dozens of g.d.o groups, IRC, blogs, twitter, etc, etc. I believe that place should be:

http://drupal.org/community-initiatives

In terms of core, that'd be:

http://drupal.org/community-initiatives/drupal-core

Both of these pages (and their children) need some help (content-wise) to really be the best place for this, but in terms of a URL and a toolset (wiki pages with children, the [#nid] issue link filter, and comments), I think these pages are exactly what we need. And d.o already considers this part of the "Get involved" section of d.o, which is perfect. We just need to get the IA right so it's more obvious how to find them from http://drupal.org/getting-involved and http://drupal.org/community ...

Speaking of our collaboration tools (in particular, the issue queues), the best places to talk about fixing those are:
A) The new Prairie initiaitive group on g.d.o.
B) The Redesign the issue queue community initiative wiki page.

Thanks for getting this discussion rolling!

Cheers,
-Derek (dww)

Thanks Derek - such great

Thanks Derek - such great input. :)

Indeed, I don't think how much of an impact it can have becomes clear until you actually get into the habit of working like this. Then it's hard to imagine how you ever worked through a huge backlog of issues over years of a release without it!

It seems like there's a lot of interest in this, but I definitely don't have a completely clear vision of how well it would work for smaller chunks of work (or how self-motivated people will be to set up and follow sprints.

I might go ahead an flag this on the PM and core g.d.o groups for some more input before starting an initiative section. Once there's been a little more discussion definitely the initiatives section should be where this all gets documented (as far as process etc).

try this on the help system

Hi Ariane,

I really like the idea and if Jennifer agrees I think we should definitely try this out on the help initiative.

That sounds like a great

That sounds like a great idea. Once I get through some post-con decompression, I'd like to try running a sprint for the online docs as well. Maybe finish off a bunch of what people were working on at the sprint.

Release early and release often?

Ariane, Excellent post, as a developer new to drupal, starting to contribute is currently a tough slope to climb. What you and Dries and others are proposing seems like it'll make it much easier to get started ...

My one concern is why no point releases? Why are people so focused on Drupal 8 (which I assume will contain breaking changes, and deprecations to the APIs) and no one is talking about Drupal 7.1, 7.2, 7.5, etc.? Shouldn't SimpleTest make doing point releases the extend and refactor the core while maintaining capability possible?

I've seen a lot of comments about features and API extensions that didn't get into core for Drupal 7 so it's sitting in Contrib modules, can't we move some of that into Core as part of point releases. I understand that having any incompatibility in the point releases would mess up the contrib eco system, but the module projects already support targeting 7.x verse 7.3 (if it relies on a new API feature), etc.

Thanks!
Dustin.

That is really something you

That is really something you should ask the long-time core devs themselves to explain (they'll be able to do a much better job than me on that one). My understanding is that it is hugely frowned upon to break APIs after a major version is released, and also due to the needs of the translation teams, we cannot change UI text after the major version is released (aka. "string freeze").

Some things can be done in point releases, and there are point releases, but for any major work, the focus will be on D8 from the get go. The one workaround is that anything added to D8 may optionally be backported to D7, so there is a bit of a loophole (though I imagine the same restrictions about breaking APIs or changing UI text apply).

That's all I really feel solid enough to say about it, hope it helps a bit!

Thanks Ariane, the API item

Thanks Ariane, the API item is pretty standard, but a lot of work can be done to improve API's without breaking compatibility ... adding new methods, marking existing methods as Deprecated, etc., but the sting freeze issue hadn't registered with me.

awesome post

Ariane,

A belated thanks for this post. It's a great summary of the conference. I'm very happy to see that the key areas needing attention / issues for Drupal and drupal.org at large are being directly addressed. I'm impressed by the willingness of Drupal people to be critical of the weak points in Drupal/drupal.org whilst remaining positive and constructive. This is something which elevates open source way, way beyond the traditional corporate model where weaknesses get glossed over or outright ignored in the name of sales.

It was because of the sheer awesomeness of the community that I first got involved with Drupal. I could see from the dynamics of the dialogue that people were embracing the open source concept at it's visionary source, yet also pragmatically. (I think we owe Dries a lot for that balanced mix). Drupal is truly up there with Linux, Ubuntu, and Firefox as one of the greatest open source projects ever.

Keep the conference updates coming, as you go to them. You're amazingly plugged in, and it's great to be able to get the goods so finely summarized, especially as a telecommuter that needs creative ways to stay plugged in.

Yer co-worker,
Shiraz

Thanks Shiraz! I agree, we

Thanks Shiraz!

I agree, we are quite open to criticism - I think at the end of the day most of us are perfectionists and want everything to constantly improve. And I agree also that Dries has been good at keeping a balance of pragmatism in the mix.

I will do my best to keep posting the odd digest of what's going on at a high level with the process for Drupal 8!

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

More information about formatting options