Testing Drupal distributions using Behat, Mink, Drupal Extension, and Travis CI

The problem

Imagine never having to click around your website after a site update, worrying that something may have broken. Imagine never getting a call from a client after a site update, telling you that something in fact has broken.

What if instead you could script all the actions that you would normally do by clicking and have those automatically run each time you push new code to your code repository?

All this is possible using Behat, Mink, Drupal Extension, and Travis CI.

This past week I've spent some time creating a proof of concept test suite for a Drupal distribution. I began with the Drupal 7 standard install profile. This will be a walkthrough of the additions I made to add testing to the distribution.

The code

Follow along with the code in my Classic GitHub repository.

The tools

Behat

Behat is a PHP framework for Behaviour Driven Development (BDD). BDD allows for tests that describe the behaviour of a website to be written in English.

Here's an example test (written in the Gherkin syntax) that Behat understands:

Scenario: Logging into the site
  Given I am logged in as a user with the "authenticated user" role
  And I am on "/"
  Then I should not see "User login"

This test uses three step definitions, Given.., And.., and Then...

Mink

Mink is a PHP framework that abstracts browser simulation. For fast and simple queries we'll use the Goutte PHP web scraper. For testing that our ajax is working correctly we'll use Selenium.

Mink Extension

Mink Extension connects Behat and Mink. It also allows us to write our own step definitions.

Drupal Extension

Drupal Extension connects Behat and Mink to Drupal. It provides a number of step definitions that are useful for working with Drupal sites.

Travis CI

While optional, no testing plan is complete without continuous integration. Travis CI is hosted continuous integration that works with GitHub. It is free for open source projects and recently soft launched their paid private plans. Simply enable testing to your GitHub repository in their UI and include a .travis.yml file in the root of your project and each time your push your project to GitHub, Travis will run your tests and report back.

Tying it all together

Drush make files

First I copied the Drupal 7 standard profile into a new directory and initialised a git repository. Then I created the following drush make files:

  • build-classic.make
  • drupal-org-core.make
  • drupal-org.make

To test ajax I added the dialog contrib module and enabled it in classic.info. At first I tried testing ajax with the core overlay module but Selenium couldn't see the popup. It was able to see the dialog popup just fine.

Install dependencies

Next I created the folder tests/behat/ to store the tests as well as the needed test frameworks. The file composer.json is used by composer to download and install all the dependenices we'll need.

The following commands will install the dependencies (from the directory tests/behat/:

curl -s https://getcomposer.org/installer | php
php composer.phar install

behat.yml

The file behat.yml is used for configuring Behat.

FeatureContext.php

We've defined a custom step definition for testing ajax. It will wait either 2 seconds or until the id #user-login-dialog has been loaded. We defined our context class in behat.yml and it goes in the features/bootstrap/ directory.

Test files

Our test files have the .feature extension and are placed in the bootstrap/ directory.

I've used tags to tell Mink to run one of the tests with the drush driver in Drupal Extension (using the @api tag). I've used another tag to tell Mink to run one of the tests with Selenium to test ajax (using the @javascript tag).

Selenium

The following commands can be used to download and run Selenium. You need to have Selenium running when you run these tests.

wget http://selenium.googlecode.com/files/selenium-server-standalone-2.25.0.jar
java -jar selenium-server-standalone-2.25.0.jar

Running the tests

From the tests/behat/ directory run ./bin/behat. Passing tests will be output to the screen in green. Failing tests will be red.

.travis.yml

The .travis.yml file tells Travis CI how to set up an environment and how to run the tests. The setup includes:

  • creating a MySQL database
  • downloading drush
  • running composer install
  • running drush make to create the Drupal codebase
  • installing the classic distribution
  • placing a drush alias file in the proper place
  • Using xvfb to simulate a window for the browser to run in
  • running the built in PHP web server
  • downloading and running Selenium
  • running the tests

Each time I push new commits to GitHub Travis CI will perform these actions and report back whether the tests passed or failed. Travis CI can be configured to only run tests on certain branches.

Summary

There once was a time when we'd need to click around our sites each time new changes were done or module updates were made. Things break. Sometimes it would take hours, days, or even months before it was obvious that something had broke.

Testing with Behat, Mink, and the other tools described here has made it possible to script exactly what we would do when we were clicking around, and have those actions automatically performed before each deployment. What a time saver!

Comments

Testing like a boss

Cool, thanks for sharing this and creating a repo
First time I saw about BBD using Behat + Mink + Drupal was in drupalwatchdog.com/2/2/behat-mink

advantage is there really one with drupal?

I think this post is good, my experience is that the drupal extension does not really make much difference. What from the drupal extension are you really extracting more benefit than what is already builtin with the other extensions?

for me it was a hindrance as things were not working and the behat context from drupal was weird to configure. But tell me if you used good stuff so i can take advantage too. If there is really meat there, definitely would like to know.

Thanks

I've found Drupal Extension

I've found Drupal Extension to be quite valuable. It has a number of Drupal specific steps (creating users, checking for content in regions) and it is under active development with more steps coming up.

Tests don't work for me

Hi all,

Thanks for such a good profile which demonstrate how behat works with drupal.

But unfortunately I can't run tests with drush to test login by role features. It throws with such error "Failed to log in as user '3j2ARoEy' with role 'authenticated user'". I've tried to apply this patch but it still not working. I also tried to use patch from this issue (this need to update scenario) but it also fails.

Can someone point me to fix this issue and run all these tests from the github repo?

Thanks.

I'm not sure what the problem

I'm not sure what the problem could be. You'll have the best luck in the Drupal Extension issue queue on drupal.org. Good luck!

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

More information about formatting options