Drupal SimpleTest Module Abridged

This post is part of our Abridged series, which aims to explain the basics of some of the more ominous yet awesome Drupal projects in simple and practical terms. We hope these posts will help demystify some of these projects for people who have been hesitant to try them out!

Here, we'll take a look at the SimpleTest module/framework, including a review of its history within the Drupal project, the current state of the module, how to start using it, resources, and a note on how we've been using it ourselves. HUGE thanks to Drupal SimpleTest co-maintainer Rok Zlender for teaching us about using SimpleTest when he was in Vancouver last summer - the code samples are care of Rok's example custom test.

Background

First to tackle some definitions, SimpleTest is a Drupal module based on the SimpleTest unit testing framework, which can be used for any PHP testing.

"SimpleTest is an open source unit test framework for the PHP programming language and was created by Marcus Baker. The test structure is similar to JUnit/PHPUnit. SimpleTest supports mock objects and can be used to automate the regression testing of web applications with a scriptable HTTP Client that can parse HTML pages and simulate things like clicking on links and submitting forms." [Wikipedia]

Now let's break this down into pieces.

Unit testing is a method of automating tests for small individual pieces of code. It is highly specific, and each unit test checks one function or procedure.

Functional testing, on the other hand, performs more complex actions, such as clicking a series of links, or filling out and submitting a form. This allows you to make sure all of your site is functioning properly at any time, without having to click around and test everything by hand. The efficiency this provides is multiplied when you add in having to test for various roles, and other conditions. (This article has more details and a comparison between functional and unit testing.)

A framework is like a nicely packaged up library of code that has been extended to have some default behaviours and be extended via an API.  It lays the groundwork for developers so they don't have to start from scratch every time.

Mock objects (in the context of object-oriented programming) simulate "objects" in your software. In Drupal up to version 6, this would for example, refer to a node (ie. any content type: blog post, story, page, etc.); in version 7 this will include things like users and taxonomy terms. In the context of testing, the mock objects can be manipulated as if they were real objects in your site, so that you can perform tests with them or see how they interact with other objects, without actually affecting your site.

Regression testing is when testing is performed to check for "software regressions", in other words, to check if software has unexpectedly stopped working as it should. Software regressions are often a result of changes made while working on related parts of the software. Previously run tests can be run as a form of regression testing to see whether previously tested, working software has been affected by new work.

The background of the SimpleTest module in short, is that it began as an independent PHP testing framework, which was adapted into a Drupal-specific module that provides a customized testing API.

Current State

In the current version of Drupal 6, the SimpleTest module project is a contributed module. It is largely maintained by three community members who work for NowPublic, Rok Žlender, Károly Négyesi (aka. chx), and Jimmy Berry (aka. boombatower). The newest (recommended) Drupal 6.x-2.x version is a backport of the improved version for Drupal 7, and requires a core patch (documented in the INSTALL.txt file) to be used. SimpleTest 6.x-2.x and up no longer uses the SimpleTest libraries, and instead runs on completely custom Drupal code. For the upcoming version, Drupal 7, SimpleTest is being incorporated into core as the Testing module, and all core modules will now come with a set of tests. 

In Drupal, SimpleTest is used mainly for functional testing. When a test case is run, it creates a completely separate testing environment, ie. a duplicate of your site in the database. Simpletest builds and then "tears down" this testing environment each time you run tests, to be sure that the testing environment is totally isolated and produces reliable results. Because of the way the testing environment is set up and then torn down, it is usually most efficient to bundle tests for a particular module into a single test case that can be run.

How to Use It

There's two basic parts to using SimpleTest: creating the tests and running them.

Writing your own tests

SimpleTest provides some basic tests by default, and you can also write your own tests for custom modules, forms, etc. that you create. Tests use .test files, and should be placed into the directory of the module that is being tested.

Each test case you create must have four basic parts (click here to download the example test in its entirety - it has a .txt rather than .test extension just so it is viewable in your browser if you right click on the link and open it in a new tab/window):

1. A description of the test
2. Set up
3. Tests
4. Tear down

1. The description

class ABTestCase extends DrupalWebTestCase {
  public static function getInfo() {
    return array(
      'name' => 'Testing AB module',
      'description' => 'Ab test.',
      'group' => 'AB'
    );
  } 

This is necessary for the test to work. It says that we are extending DrupalWebTestCase (Drupal's functional testing case) with the custom ABTestCase (for our example module that we're testing), and then provides some basic information about the test case to Drupal.

2. Set up

function setUp() {
  parent::setUp('abtest');
  $web_user = $this->drupalCreateUser(array('access hello form'));
  debug($web_user);
  $this->drupalLogin($web_user);
}

As mentioned above, when you run a test case, an entirely separate duplicate copy of your site is set up, used, and then torn down. The visual of the workflow (from the Drupal.org SimpleTest documentation) shows this process, where when you run test, the test case is accessed, directs the set up of the testing environment, runs the test case, tears down the testing environment, and then reports the results. This process is repeated for each test case that is run.

Because it is a completely separate testing environment, your setup has to perform any necessary configurations such as setting up users and permissions. If we look at the code example above, the setup is enabling the custom module that we're testing, creating a user who has access to some specific pieces of the site, and then logging that user in.

3. Tests

Next, write your tests! I know, this sounds like a daunting task at first, but start small. Test a form field or submit, or whether a page exists. There is a ton of SimpleTest documentation on Drupal.org to walk you through more specifics of test writing, and many more examples. Here are a couple very basic example test functions:

function testLoad() {
  $this->drupalGet('ab/hello');
  $this->assertResponse('200', t('URL is reachable'));
}

This test function is accessing the URL "example.com/ab/hello", and printing "URL is reachable" if the URL is reachable (the '200' here is a an HTTP response status code, like a '404' for page not found).

function testLong() {
  $word = $this->randomName(300);
  $edit['text'] = $word;
  $this->drupalPost(NULL, $edit, 'Save');
  $this->assertNoText($word, t('Text displayed'));
  $this->assertText('text cannot be longer than 128 characters', t('Warning displayed'));
}

This test function is making sure that if someone attempts to enter too many characters in a field, that this is not allowed and an error displays. It takes a random string or word of 300 chars, and placing it in a text field, then saving the page, which should then display the text. However, there is a limit of 128 characters on this field, so the saving the page will fail, and the warning "text cannot be longer than 128 characters" will be displayed.

4. Tear down

Finally, at the end of the test case, the testing environment needs to be dismantled or "torn down", as it is no longer needed.

function tearDown() {
  parent::tearDown();
}

Running tests

This is the easy part! In Drupal6 you just navigate to "example.com/admin/build/testing" (in Drupal7 it's "example.com/admin/config/development/testing"), and click the checkboxes for the test(s) you want to run. Then you can watch as your tests run and pass or fail... And view a detailed report of the results. And even more amazing, if you are debugging, you can select the "Provide verbose information when running tests" checkbox on the SimpleTest settings page. It will actually output "verbose messages" that are listed in the results, which when you click through look like a display of a page in your site plus some debug code.

It may seem like a lot of overhead, but when you start building, modifying, and maintaining a lot of sites, or more complex customizations, having automated testing ready to go at the flip of a switch is a huge benefit. It allows you to find bugs, and assure the quality and functioning of your code.  Plus, as Rok likes to put it, it helps you sleep better at night. Nobody can argue with that!

Resources

• SimpleTest documentation on Drupal.org (http://drupal.org/simpletest)
• SimpleTest module on Drupal.org (http://drupal.org/project/simpletest)
• SimpleTest PHP Library (applies up to version 6.x-1.x) (http://simpletest.sourceforge.net/)
• Lullabot's "A Drupal Module Developer's Guide to SimpleTest" (http://www.lullabot.com/articles/drupal-module-developer-guide-simpletest)
• Lullabot's "An introduction to unit testing" (http://www.lullabot.com/articles/introduction-unit-testing)
• SimpleTest co-maintainer Jimmy Berry's (aka. boombatower) blog (http://blog.boombatower.com/)
• Lullabot Podcast's SimpleTest episode (http://www.lullabot.com/audiocast/podcast-59-simpletest)
• Testing and Quality Assurance group on Drupal.org (http://groups.drupal.org/unit-testing)
• "Intro to Simpletest" session at DrupalCon DC (http://dc2009.drupalcon.org/session/intro-simpletest)
• "SimpleTest: Because clicking on forms is for suckers" at DrupalCon Boston (http://www.archive.org/details/DrupalconBoston2008-Simpletest-BecauseClickingFormsSucks)

How We've Used Simpletest at Affinity Bridge

Despite being sold on the idea of automated testing, we're just learning to implement it at this point. We wanted to share what we learned right off the bat, but will post updates with any interesting lessons learned as we integrate this into our development practices. We'd love to hear about others' experiences with starting to use SimpleTest!