A few months ago, one of the teams I work with went on the hunt for a good continuous integration service for running tests on the code we write.  We jumped on the unit test bandwagon at the beginning of the year and wanted to really amp up the quality of our product by having tests run on every commit.  I was tasked with this job (and anyone who knows me knows I LOVE playing with new things, so it was a task I was looking forward to doing)

Most WordPress users are familiar with travis-ci.org and the internets were full of instructions for getting things plugged in and up with travis.  Unfortunately, our project is inside a private github repo so we couldn’t use the free travis plan to run our tests on and the premium plan was a bit to pricey for our first attempt at this.  So after searching around, I stumbled on circle.  From all appearances, circle looked like it would work very similarly to travis and bonus points were that their plans are much cheaper – so great for getting started with.

All their documentation was great for getting things hooked up to your github repo, and getting started with tests, with the exception of one glaring thing.  I couldn’t find ANYTHING on the internets about how to setup a circleci.yml file for automating WordPress and WordPress plugin tests. Boo.  But actually, probably better in the long run because I ended up having to come up with something through trial and error and in the process learned a lot about the magic behind these scripts.

In this post, I’m not going to highlight how to setup unit testing for your WordPress plugin since there’s already a wealth of knowledge on the internets for that. Instead I just want to give an example circle.yml file that we use for our plugins at EventEspresso.com

So in the interest of passing on what I learned for anyone else searching the internets, I present to you our circle.yml script:

## Customize the test machine
machine:

  timezone:
    America/Denver # Set the timezone

  # Version of php to use
  php:
    version: 5.4.21

  # Add some environment variables
  environment:
    CIRCLE_ENV: test
    WP_MULTISITE: 0
    WP_CORE_DIR: /home/ubuntu/wordpress-develop
    WP_TESTS_DIR: /home/ubuntu/wordpress-develop/tests/phpunit
    plugin_loc: /home/ubuntu/$CIRCLE_PROJECT_REPONAME
    plugin_slug: $CIRCLE_PROJECT_REPONAME
    plugin_dir: /home/ubuntu/wordpress-develop/src/wp-content/plugins/$plugin_slug
    plugin_tests_dir: /home/ubuntu/wordpress-develop/src/wp-content/plugins/$plugin_slug/tests


## Customize dependencies
dependencies:
  pre:
    #enable xdebug.  LINE 1/2 to uncomment if you want to run a code coverage report.
    # - sed -i 's/^;//' ~/.phpenv/versions/$(phpenv global)/etc/conf.d/xdebug.ini
    #setup WP install
    - git clone git://develop.git.wordpress.org/ $WP_CORE_DIR;
    - cd $WP_CORE_DIR && cp wp-tests-config-sample.php wp-tests-config.php && sed -i "s/youremptytestdbnamehere/wordpress_test/" wp-tests-config.php && sed -i "s/yourusernamehere/root/" wp-tests-config.php && sed -i "s/yourpasswordhere//" wp-tests-config.php;
    # move plugin into tests/src
    - mv $plugin_loc $plugin_dir;
    # set up database
    - mysql -e 'CREATE DATABASE wordpress_test;' -uroot;
    # setup phpunit
    - wget https://phar.phpunit.de/phpunit.phar && chmod +x phpunit.phar && mv phpunit.phar /home/ubuntu/.phpenv/shims/phpunit

## tests override
test:
  override:
    # comment out the below line to run a code coverage report.
    - cd $plugin_tests_dir; phpunit
    ## LINE 2/2 to uncomment if you want to run a code coverage report.
    # - cd $plugin_tests_dir; phpunit --coverage-html $CIRCLE_ARTIFACTS

Where does the script go?

The circle.yml should be in the root directory of your WordPress plugin, and I’m assuming that the plugin itself has all its tests in a /tests/ directory off of the root.

What does all that stuff mean?

The circle.yml script itself is based off of the sample file circle provides. I just had to use trial and error to figure out setting up all the components needing setup on the virtual machines circle spins up to run the tests on. Let’s walk through the sections one by one.

machine:
The machine section basically is the group for all the machine variables controlling the vm setup.

timezone:
In this section you can set what the timezone will be setup for the spun up machine. In reality, you don’t have to put anything here, because WordPress runs in UTC. But if you are doing anything else on your machine or are collecting errors etc., setting the timezone could be useful.

php:
Here you set what php version you want the tests to run in. One thing I haven’t been able to figure out on circle yet is whether I can have our tests run consecutively on different php versions (which seems to be REALLY easy with travis). Note, PHP5.2 is NOT an option with circle (which may be an option with travis, not clear on it).Since most hosts dropping support for PHP are jumping right to PHP5.4 (and skipping PHP5.3) that’s the version we’ve decided to run our automated tests on.

environment:
In this section you set all the environment variables you want exposed in bash for your test setup. Note the following variables I’ve setup:

• CIRCLE_ENV: test – currently unused, I just used this to set that we’re in the CIRCLE_ENV testing environment. At some point I may use this in our test scripts so I have a way of knowing we’re running tests on a circle server.
• WP_MULTISITE: 0 – used by WordPress test library, this basically allows us to designate that we are not running our tests on WordPress multisite (of course when we DO want that to happen we change this).
• WP_CORE_DIR – here we set the path where wp core tests suite will be installed. Note that circle vm’s are spun up with a user named ubuntu. So all files are typically found in the ubuntu user directory.
• WP_TESTS_DIR – here we set the path for where the tests directory will be for the WordPress phpunit tests.
• plugin_loc – here is the path set for where the plugin will be installed. Note I use a special environment variable Circle automatically exposes called $CIRCLE_PROJECT_REPONAME. Circle automatically pulls in the project from the repo its connected with and installs it in a path in the home directory. I set this as a variable so later I can instruct the script to move the project from that location to its new home in the WordPress wp-content/plugins folder.
• plugin_slug – just to save typing and be more explicit – this is an alias for $CIRCLE_PROJECT_REPONAME
• plugin_dir – this is the path where the plugin will be moved to after WordPress is installed.
• plugin_tests_dir – this is the path to the tests folder where phpunit will be run for our plugin.

dependencies:
This section of the circle.yml file is where you can setup all the dependencies before the tests are run. In our file, we only use the “pre” index (but there are others you can read about in the circle docs).

pre:
This is where you list instructions for how circle should setup the machine before the tests are run. It appears that any bash commands can be included as separate line items in this list.

• sed -i 's/^;//' ~/.phpenv/versions/$(phpenv global)/etc/conf.d/xdebug.ini – note this line is commented out by default. it basically takes care of enabling xdebug for php. If you want to do a coverage report at the end of your tests then you need to remove the commenting.
• git clone git://develop.git.wordpress.org/ $WP_CORE_DIR; – here we’re pulling in the WordPress development repo that contains all the WordPress tests.
• cd $WP_CORE_DIR && cp wp-tests-config-sample.php wp-tests-config.php && sed -i "s/youremptytestdbnamehere/wordpress_test/" wp-tests-config.php && sed -i "s/yourusernamehere/root/" wp-tests-config.php && sed -i "s/ – here is where we give instructions to cd into the WordPress installation we just installed, and then modify the wp-tests-config.php file to have the db instructions.
• mv $plugin_loc $plugin_dir; – we’re moving the plugin the tests are being run against from the directory it was installed in, into the WordPress plugins folder where it needs to be.
• mysql -e 'CREATE DATABASE wordpress_test;' -uroot; – This just sets up the db to be used by the tests suite.
• wget https://phar.phpunit.de/phpunit.phar && chmod +x phpunit.phar && mv phpunit.phar /home/ubuntu/.phpenv/shims/phpunit – for some reason, the installation of phpunit that comes with circle machine installations is just not setup correctly because it doesn’t work. So this line takes care of installing the latest phpunit so that it does work. This little line is the culmination of hours of work (just because there’s so little usable information, I’m not a server guru, and I had a lot of trial and error).

test:
In this section of the circle.yml file you are able to give specific instructions regarding the tests. The only parameter we use in our file is the “override” one. This just ensures that the default circle test isn’t run but instead what we specify.

override:

• cd $plugin_tests_dir; phpunit – here we cd into the tests directory for our plugin and run phpunit. That’s it! Now if you want to run a code coverage report then you make sure this line is commented out and uncomment the following line…
• cd $plugin_tests_dir; phpunit --coverage-html $CIRCLE_ARTIFACTS – this line is commented out by default. However, if you wish to run a code coverage report then xdebug has to be enabled, and then you remove the comments from this line and comment out the previous line. What happens is when the test is complete, phpunit will create a html code coverage report and install it in the $CIRCLE_ARTIFACTS path. The groovy thing about this is that when complete, the circle artifacts path is visible on the internet so you can click the provided link (or share it with the team) to access the code coverage report. The reason why this is commented out by default is it takes significantly longer to generate a report than just running the tests, so we don’t want a report generated on every commit.

Conclusion

That’s it! I hope this is useful for anyone wanting to get WordPress unit testing setup on the excellent (cheaper) circle service. If any of you reading this are more of a server guru than I am and have suggestions for improvement I welcome that in the comments!