Using circleci.com for automated WordPress plugin testing.

A few months ago, one of the teams I work with went on the hunt for a good continuous integration service for running tests on the code we write.  We jumped on the unit test bandwagon at the beginning of the year and wanted to really amp up the quality of our product by having tests run on every commit.  I was tasked with this job (and anyone who knows me knows I LOVE playing with new things, so it was a task I was looking forward to doing)

Most WordPress users are familiar with travis-ci.org and the internets were full of instructions for getting things plugged in and up with travis.  Unfortunately, our project is inside a private github repo so we couldn’t use the free travis plan to run our tests on and the premium plan was a bit to pricey for our first attempt at this.  So after searching around, I stumbled on circle.  From all appearances, circle looked like it would work very similarly to travis and bonus points were that their plans are much cheaper – so great for getting started with.

All their documentation was great for getting things hooked up to your github repo, and getting started with tests, with the exception of one glaring thing.  I couldn’t find ANYTHING on the internets about how to setup a circleci.yml file for automating WordPress and WordPress plugin tests. Boo.  But actually, probably better in the long run because I ended up having to come up with something through trial and error and in the process learned a lot about the magic behind these scripts.

In this post, I’m not going to highlight how to setup unit testing for your WordPress plugin since there’s already a wealth of knowledge on the internets for that. Instead I just want to give an example circle.yml file that we use for our plugins at EventEspresso.com

So in the interest of passing on what I learned for anyone else searching the internets, I present to you our circle.yml script:

## Customize the test machine
machine:

  timezone:
    America/Denver # Set the timezone

  # Version of php to use
  php:
    version: 5.4.21

  # Add some environment variables
  environment:
    CIRCLE_ENV: test
    WP_MULTISITE: 0
    WP_CORE_DIR: /home/ubuntu/wordpress-develop
    WP_TESTS_DIR: /home/ubuntu/wordpress-develop/tests/phpunit
    plugin_loc: /home/ubuntu/$CIRCLE_PROJECT_REPONAME
    plugin_slug: $CIRCLE_PROJECT_REPONAME
    plugin_dir: /home/ubuntu/wordpress-develop/src/wp-content/plugins/$plugin_slug
    plugin_tests_dir: /home/ubuntu/wordpress-develop/src/wp-content/plugins/$plugin_slug/tests


## Customize dependencies
dependencies:
  pre:
    #enable xdebug.  LINE 1/2 to uncomment if you want to run a code coverage report.
    # - sed -i 's/^;//' ~/.phpenv/versions/$(phpenv global)/etc/conf.d/xdebug.ini
    #setup WP install
    - git clone git://develop.git.wordpress.org/ $WP_CORE_DIR;
    - cd $WP_CORE_DIR && cp wp-tests-config-sample.php wp-tests-config.php && sed -i "s/youremptytestdbnamehere/wordpress_test/" wp-tests-config.php && sed -i "s/yourusernamehere/root/" wp-tests-config.php && sed -i "s/yourpasswordhere//" wp-tests-config.php;
    # move plugin into tests/src
    - mv $plugin_loc $plugin_dir;
    # set up database
    - mysql -e 'CREATE DATABASE wordpress_test;' -uroot;
    # setup phpunit
    - wget https://phar.phpunit.de/phpunit.phar && chmod +x phpunit.phar && mv phpunit.phar /home/ubuntu/.phpenv/shims/phpunit

## tests override
test:
  override:
    # comment out the below line to run a code coverage report.
    - cd $plugin_tests_dir; phpunit
    ## LINE 2/2 to uncomment if you want to run a code coverage report.
    # - cd $plugin_tests_dir; phpunit --coverage-html $CIRCLE_ARTIFACTS

Where does the script go?

The circle.yml should be in the root directory of your WordPress plugin, and I’m assuming that the plugin itself has all its tests in a /tests/ directory off of the root.

What does all that stuff mean?

The circle.yml script itself is based off of the sample file circle provides. I just had to use trial and error to figure out setting up all the components needing setup on the virtual machines circle spins up to run the tests on. Let’s walk through the sections one by one.

machine:
The machine section basically is the group for all the machine variables controlling the vm setup.

timezone:
In this section you can set what the timezone will be setup for the spun up machine. In reality, you don’t have to put anything here, because WordPress runs in UTC. But if you are doing anything else on your machine or are collecting errors etc., setting the timezone could be useful.

php:
Here you set what php version you want the tests to run in. One thing I haven’t been able to figure out on circle yet is whether I can have our tests run consecutively on different php versions (which seems to be REALLY easy with travis). Note, PHP5.2 is NOT an option with circle (which may be an option with travis, not clear on it).Since most hosts dropping support for PHP are jumping right to PHP5.4 (and skipping PHP5.3) that’s the version we’ve decided to run our automated tests on.

environment:
In this section you set all the environment variables you want exposed in bash for your test setup. Note the following variables I’ve setup:

  • CIRCLE_ENV: test – currently unused, I just used this to set that we’re in the CIRCLE_ENV testing environment. At some point I may use this in our test scripts so I have a way of knowing we’re running tests on a circle server.
  • WP_MULTISITE: 0 – used by WordPress test library, this basically allows us to designate that we are not running our tests on WordPress multisite (of course when we DO want that to happen we change this).
  • WP_CORE_DIR – here we set the path where wp core tests suite will be installed. Note that circle vm’s are spun up with a user named ubuntu. So all files are typically found in the ubuntu user directory.
  • WP_TESTS_DIR – here we set the path for where the tests directory will be for the WordPress phpunit tests.
  • plugin_loc – here is the path set for where the plugin will be installed. Note I use a special environment variable Circle automatically exposes called $CIRCLE_PROJECT_REPONAME. Circle automatically pulls in the project from the repo its connected with and installs it in a path in the home directory. I set this as a variable so later I can instruct the script to move the project from that location to its new home in the WordPress wp-content/plugins folder.
  • plugin_slug – just to save typing and be more explicit – this is an alias for $CIRCLE_PROJECT_REPONAME
  • plugin_dir – this is the path where the plugin will be moved to after WordPress is installed.
  • plugin_tests_dir – this is the path to the tests folder where phpunit will be run for our plugin.

dependencies:
This section of the circle.yml file is where you can setup all the dependencies before the tests are run. In our file, we only use the “pre” index (but there are others you can read about in the circle docs).

pre:
This is where you list instructions for how circle should setup the machine before the tests are run. It appears that any bash commands can be included as separate line items in this list.

  • sed -i 's/^;//' ~/.phpenv/versions/$(phpenv global)/etc/conf.d/xdebug.ini – note this line is commented out by default. it basically takes care of enabling xdebug for php. If you want to do a coverage report at the end of your tests then you need to remove the commenting.
  • git clone git://develop.git.wordpress.org/ $WP_CORE_DIR; – here we’re pulling in the WordPress development repo that contains all the WordPress tests.
  • cd $WP_CORE_DIR && cp wp-tests-config-sample.php wp-tests-config.php && sed -i "s/youremptytestdbnamehere/wordpress_test/" wp-tests-config.php && sed -i "s/yourusernamehere/root/" wp-tests-config.php && sed -i "s/ – here is where we give instructions to cd into the WordPress installation we just installed, and then modify the wp-tests-config.php file to have the db instructions.
  • mv $plugin_loc $plugin_dir; – we’re moving the plugin the tests are being run against from the directory it was installed in, into the WordPress plugins folder where it needs to be.
  • mysql -e 'CREATE DATABASE wordpress_test;' -uroot; – This just sets up the db to be used by the tests suite.
  • wget https://phar.phpunit.de/phpunit.phar && chmod +x phpunit.phar && mv phpunit.phar /home/ubuntu/.phpenv/shims/phpunit – for some reason, the installation of phpunit that comes with circle machine installations is just not setup correctly because it doesn’t work. So this line takes care of installing the latest phpunit so that it does work. This little line is the culmination of hours of work (just because there’s so little usable information, I’m not a server guru, and I had a lot of trial and error).

test:
In this section of the circle.yml file you are able to give specific instructions regarding the tests. The only parameter we use in our file is the “override” one. This just ensures that the default circle test isn’t run but instead what we specify.

override:

  • cd $plugin_tests_dir; phpunit – here we cd into the tests directory for our plugin and run phpunit. That’s it! Now if you want to run a code coverage report then you make sure this line is commented out and uncomment the following line…
  • cd $plugin_tests_dir; phpunit --coverage-html $CIRCLE_ARTIFACTS – this line is commented out by default. However, if you wish to run a code coverage report then xdebug has to be enabled, and then you remove the comments from this line and comment out the previous line. What happens is when the test is complete, phpunit will create a html code coverage report and install it in the $CIRCLE_ARTIFACTS path. The groovy thing about this is that when complete, the circle artifacts path is visible on the internet so you can click the provided link (or share it with the team) to access the code coverage report. The reason why this is commented out by default is it takes significantly longer to generate a report than just running the tests, so we don’t want a report generated on every commit.

Conclusion

That’s it! I hope this is useful for anyone wanting to get WordPress unit testing setup on the excellent (cheaper) circle service. If any of you reading this are more of a server guru than I am and have suggestions for improvement I welcome that in the comments!

FireHost and WordPress Multi-site… how well do they play together?

Recently, one of my clients purchased a server with FireHost.com.  We’d been on the search for a new web host for some time now to serve as the infrastructure supporting upcoming web applications we have in the works.  We needed a company who is well recommended, and will help us scale and scale quickly.

You pay a more for a host like this  but it’s part of the investment costs you need to make if you want to be positioned well for solid growth as a business.

Anyways, the purpose of this post is NOT to discredit or gripe about yet another hosting company that fails to live up to expectations. We actually really like the setup we have at FireHost. Although getting things setup were a bit of a pain – their support has been very prompt and generally okay.  No, this post is more of a fyi for folks who are in a similar situation as us.  I couldn’t find any information on this subject on the nets so thought I’d post my own findings.

One of the major components of some projects we are working on this year involves the use of WordPress multisite.  It will provide the backbone of what we are building and is crucial that we have a server environment that supports this.  FireHost does except for one niggling problem.  They have a super awesome “Web Application Protection” firewall that works really well, too well, and prevents normal usage of WordPress multisite.

At issue is that any subsites created on WordPress multisite will fully function as long as those subsites don’t post any images or certain html in their posts.  Cause if they do,  BAM, the firewall sees that as a xss attack and shuts her down.

That’s no good is it?  No.  But wait, FireHost has the solution.  All you have to do, is whenever this happens you just send them the path for the sites that the firewall does its thing on and they’ll add an exception.  Greeatt!  Except that we’re planning on using WordPress multisite to well actually make it easy for people to signup and get started on a new site right away (you know kind of like how people expect things to work? right?).  So yeah, major pain to have to send a block of paths every time the firewall acts up.

I’m not going to tell you the solution we worked out but let’s just say not ideal.

Silly?  Yeah.  I get it. But obviously someone at FireHost needs to do some thinking about how this firewall is setup and put something in place to allow for easier management of WordPress multisite while keeping the firewall working on things that it should work on, or at least be clear about the side affect of the WAP for those using WP multisite, would have saved a lot of back and forth with tech staff.  At the very least, create some sort of API or secure service for automating the firewall exceptions in cases like this where sites are being created dynamically via an application like WordPress multisite.

Anyways, again we do like what we have so far with FireHost except for this firewall experience, but I just wanted to post this up in case anyone else is thinking of using multi-site with FireHost and wondering why its not working as expected.

(I also have another reason for posting…secretly hoping some server guru out there will be able to explain how I’m either an idiot for expecting the firewall to be set up so multi-site works, or how FireHost can do things so multisite will work fine).

Update: December 2014

Just posting an update for anyone who visits this post via search engines (there are a few of you its seems.  Besides the fact that Firehost did contact us within the 30 day window that was put forward by their CEO/founder in the comments to this post, we never did get on any “beta” program.  The solution they are proposing for clients needing their own managed WAF is a $1500+/mo cost solution, which is not startup friendly.  They did offer to help get us up on ModSecurity as an alternative to their WAF solution but in the end my client and I just decided to fire Firehost (yeah pun intended) as our hosting provider and we joined the cool cats over at Digital Ocean.  We figured if we’re going to be doing most of the server setup/managing ourselves then no sense in paying a “managed” hosting provider to do it for us.  We’ve been on Digital Ocean for about 5 months now and absolutely LOVE it.

WP 3.7 drops with an interesting surprise…

I thought I was following the development of WordPress 3.7 fairly closely but something totally missed my notice and only caught my attention when a plugin I develop stopped working with the latest version of WordPress.

The culprit?

do_action( 'save_post', $post_ID, $post, $update );

Notice anything different?  The difference is that this hook used to only have 2 parameters, “$post_ID”, and “$post” but NOW it has a third one, “$update”.  It’s actually a nice addition as it makes it super easy to determine whether the post is being updated or not.  However, due to the way I hooked into this action (with a function that had extra parameters on it), Organize Series broke.  Easy enough fix, but quirky enough that I thought it deserved a post as I haven’t seen anybody mention this little addition!

 

Organize Series 2.1.7 Released

I knew I wanted to get the next version of Organize Series out the door and with the surprise (well maybe not quite a surprise) release of WordPress 2.9 I decided to get my butt in gear and finish off the threads needed.  As far as I know Organize Series works fine with WordPress 2.9 (I’m running it fine on my websites) but if you notice any problems report them to the support forums as usual.

This release is mostly bug fixes.  Please make sure you read through the changelog if you want all the details.  Some of you will be happy with some persistent bugs you’ve been dealing with (I hope!).

What’s next?  Well, I have had a number of different feature requests that have come my way.  Some I may be doing as custom work in the new year (that means getting paid to implement) and others I have quoted on but I think it’s outside of the budget of those wanting them.  I’m strongly considering doing some work on some plugins to Organize Series that will implement some of the most requested features and offer it up for sale on a service like the wpplugins app store for a nominal amount (between $10-20).   Anything I do this way will still be release GPL etc. but it will be a way for me to still contribute new features to Organize Series and help pay my bills as well.   The core Organize Series plugin of course will still always be offered free of charge and I will continue to do maintenance and bug fixes on it going forward.  All I would be doing is offering new features as a paid option.

Why am I posting this?  I want some feedback, what do you think?  Would you pay $10-20 for a plugin that would extend the features of Organize Series?  Let me know in the comments!

Organize Series 2.1.5 has galloped in…

I just finished getting the next version of my super duper series plugin for WordPress launched to the wild.  Organize Series 2.1.5 has got some neat stuff in it:

  • New Feature: A bundled plugin called “Organize Series Publisher”.  You don’t have to activate it if you don’t want but it’s there if you want to try it out!  This new add-on for Organize Series is a take-off of the Issues-Manager plugin created by Jonathan Brinley and was sponsored by Amanda Giles who hired me to port Issues-Manager so it works with Organize Series.  Amanda graciously agreed to have this released to all users of Organize Series!  Basically, this add-on allows you to set a series so any posts assigned to it won’t publish until you explicitly publish the whole series.  Read more about it on the Organize Series Plugin Page
  • New Feature: custom base for series permalinks.  Yeah, now you can change ‘series’ to something like ‘myawesomesequenceofposts’…well, you can..
  • New Feature: Added a new function for you power users who want to be able to fully control the output of series data.  get_series_ordered() has joined the party and you can thank Amanda for it too!  With this function you can get series data from the database and manipulate it however you want when it’s returned.  Do the tango with it if you want! Uh…yeah…
  • MAJOR fixes:  As always, the users of Organize Series have been doing a great job of reporting bugs and stuff that doesn’t work and its kept me hopping spraying the raid code.  I managed to knock off some major roaches in the process and I think some of you will be happy with that (I know I am!).  Wanna know what I fixed?  Head on over to the changelog and you can catch up on it.

There’s some other typos and minor stuff that I managed to polish up for this release as well.  As always, I’ve tested things as much as I can but there may have been some nasties that slipped through.  If you spot any, head on over to the support forums and let me know.

Plugin support provided by WordPress HelpCenter.Oh, and one more thing.  I’ve partnered with a company called wphelpcenter.com which is billed as “Your On-Call WordPress experts”.  It’s a great newish service that has a team of cool folks who can help you with all your WordPress needs.  If you have something that you need done with Organize Series “right away!” you can contact wphelpcenter.com (email: help@wphelpcenter.com or phone: 512-788-9236).  If you give them my affiliate code: 14322, I’ll get commission on the help you buy for Organize Series.  It’s a great way to help support the development of Organize Series and get help when I’m not available.   However, I’ll still entertain any custom work you may have, just get in touch with me.

Well, what are you waiting for?  Go try out the new version of Organize Series already!

Organize Series 2.1.1 released – Major Bug Fix

A pretty major bug slipped through with the release of Organize Series 2.1.  What happens with the bug is when someone posts a comment to a post that is part of a series, that post will lose it’s series information.  Not good.  Organize Series 2.1.1 should fix that.  You’ll want to upgrade as soon as possible.

Organize Series 2.1 Released

At long last, another update to my flagship WordPress Plugin (well, really the only WordPress plugin I’ve developed…)!  This is a significant release which is why I skipped doing a version 2.0.9 and went right to 2.1.  A couple of quick notes about this release.

  1. Organize Series 2.1 will only work with WordPress 2.8 and higher.   The code was starting to get bloated with trying to support all the previous releases and I really don’t have the time to keep supporting earlier versions of WordPress.   If you really need to use Organize Series with an earlier version of WordPress then I suggest you keep using Organize Series 2.0.8.
  2. Because of some of the feature add-ons you really should read the note about upgrading (read this – scroll down a bit for the upgrade notes)  so you don’t have any unexpected stuff happen.
  3. Going forward I’m still going to be offering limited support via my support forums.  What does limited mean? It means I will answer basic questions that don’t take me a lot of time to answer and I will also respond to bug reports (and work on fixing them).  However, I am unable to devote any more time to significant usage related questions or new feature requests.  Feel free to post such requests in the forums and there may be other users willing to help you.   Another option is to contact me and hire me to do custom work for you related to Organize Series.

You can read all the release notes at the plugin listing found on WordPress.org/extend – or you can visit the Organize Series Plugin Page here on my blog.

Donate Friday

If you’ve used Twitter for any length of time you’ve probably noticed the phenomenom called “Follow Friday” where people tweet about people they think are worth following and actively search out new people to follow themselves.   If anything, “Follow Friday” shows us how something can gain traction when it’s simple and consistent.  Probably not too many people heard about it when it first started but because the few that did kept doing it and used a common hashtag – it quickly gained a following (pardon the pun) in the twitter realm.

I’m writing this post because of my love for WordPress and especially all the developers out there who contribute code to both the opensource project and the thousands of plugins and themes that make it such a great tool for all of us.  There has been discussion over the years of ways in which we can express appreciation to the developers who freely offer up their code to the world to use (and all the hours put into developing, maintaining, and supporting that code) but as a plugin developer myself (only one at that),  although I’ve appreciated gratitude sent my way and reviews posted on people’s blogs nothing has stoked me more than seeing that little notification from PayPal that someone has donated a sum of money to me.  The dollar amount doesn’t really matter – it’s more the realization that someone thinks what I’ve invested so much time in worthwhile enough to pay for.

How are the above two paragraphs connected?  Well, I’d like to propose starting something this Friday called “Donate Friday” – each Friday for as long as I can do it – I’m going to pick a plugin that I’m using on any of my websites and donate to them and then tweet about the plugin (including linkies of course) with the hashtag #donatefriday (I checked it and it’s not in use at the time of this post).  If enough of us do this, it will be an incredible way to express our appreciation to the developers in the WordPress community who help make WordPress so awesome.

How can you participate? In keeping with the #followfriday example:

  • pick who you will donate to.  Most plugin or theme authors have a donate link either on their plugin/theme listing at WordPress extend OR on the page they host their plugin/theme at on their own website.  You may also just want to donate to someone who contributes code to the WordPress project.
  • Pick an amount.  It doesn’t have to be a lot,  even as little as $5 is fine!  Some of you may be able to donate more than others – if you can, think of what you would be willing to pay for what you are using if you had to pay for it.  However, remember the whole point is just to lift up that developers day and give them reason to keep on working away at it!  Surely, you can spare around the cost of a gourmet coffee once a week?
  • Tweet about it using the #donatefriday hashtag.  Make sure you include the plugin/theme name,  the developer and link to the plugin page in your tweet.  Do something like, “Just donated to {name of developer} for {name of plugin/theme}.  Makes my blog purr. #donatefriday”
  • MAKE sure you actually donate!

What do you think folks?  A good idea?  I really do hope it gains traction – think of the impact this will make (especially for developers in between jobs) in the WordPress community?

UPDATE: I realize I posted this fairly early in the week, but by doing so I hope the idea gains more steam before this Friday so it maximizes participation.  If you are reading this and want to participate but think you may forget this Friday let me remind you, follow me on Twitter and you’ll catch my #donatefriday this week (or you can use something like Peoplebrowsr.com to save a #donatefriday search).

UPDATE (JULY 31, 2009): I  made my first donation for #donatefriday.  Sure hope more people do this!! Also Dave Moyer, the cohost and producer of WordCast (the #1 WordPress and blogging podcast) has picked up this idea and is helping to promote it.  He got in touch with me and if you tweet your donations with @wordcast and #donatefriday (he is also said #df but that tag is already in use so I think it’d be better if we stuck with #donatefriday) then he’ll read your name on the air.  Thanks Dave!

UPDATE (August 14, 2009): Using “Twitter for WordPress Extended” by Marc Schieferdecker to display #donatefriday tweets in my sidebar on this page.  Wanna give you peeps donating more recognition!