Tag Archives: Docker

How to Run Cucumber Tests with Docker

Full stack web development includes a number of dependencies including web server, database, caching and test runners causing a huge hurdle for a developer new to the project. Setting up of the these dependencies can be a daunting task because the documentation is out of date, development environments differ between OS version and hardware and inevitably half of the team is working with an outdated version of a dependency because they haven’t updated. This all leads to variability in test environments, defeating a team’s ability to confidently deploy well tested code. We’ll show how to use tools such as Docker to help overcome these issues.

laptop

Setting up automated Cucumber acceptance tests is one of the more involved steps for getting started with a new project. Cucumber is primarily a communication tool, but it results in a suite of automated tests that run through a browser by executing scripts written in an English like syntax called Gherkin. To run these tests, you’ll need a number of dependencies including gems, web browsers and a full environment to test against. With all of the different dependencies, creating a consistent and repeatable environment for testing can be very difficult.

This is where Docker comes in. Here’s the pitch from docker.com:

“Docker allows you to package an application with all of its dependencies into a standardized unit for software development.”

In this article we’ll go through the steps of setting up a very simple Rails web app, then running Cucumber tests while utilizing Docker to ensure a consistent and repeatable test environment.

A few things this article is not:

  1. An overview of Docker, or an example of the best way to use Docker for your project
  1. An overview of Cucumber or an example of best practices on how to use Cucumber for your project

* If you’d like to learn more about cucumber, checkout https://cucumber.io
1. An overview of how to make a Ruby on Rails app.
* Learn more about Ruby on Rails at http://rubyonrails.org

Prerequisites before beginning:

If you’d like to skip the Rails and Cucumber setup steps, checkout Base app with Cucumber from the Git repo and jump to the Setup the application to run through Docker section of this article.

Create a new Rails application

Open your terminal and create a new Rails app with:

rails new CucumberInDocker

Change into the new application directory and start up the rails server with:

cd CucumberInDocker
rails s -b 0.0.0.0

NOTE: The binding parameter(-b) is important, especially if you going to run Docker on Mac OS X

rails welcome page

You can now view the running application in the browser of your choice at http://localhost:3000
Stop your rails server with control-c after confirming it started correctly.

Setup the application for Cucumber testing

Add the cucumber-rails, database_cleaner and selenium-webdriver gems to your Gemfile under the development and testing group so it looks something like this:

group :development, :test do
gem 'cucumber-rails', :require => false

# database_cleaner and selenium-webdriver are needed as default dependencies for cucumber-rails
gem 'database_cleaner'
gem 'selenium-webdriver'

# other gems can go here
end

Run bundle install to install the necessary gems locally.

Now add Cucumber to the project with:

rails generate cucumber:install

Important Step:
Run rake db:migrate to setup the development and test databases. Despite the fact this simple app won’t use these databases, Cucumber expects one to be fully setup.

Write Cucumber tests

To make this example simple, the following Cucumber tests will check the title of the welcome page that is provided when you create a new Rails application. It also will use the default tools installed by the cucumber-rails gem. If I were doing this for a long running project I would use page-object by Jeff “Cheezy” Morgan.

Create a new file called welcome_page.feature under the features directory and paste in the code below:

Feature: Newly generated rails application welcomes users
As a Ruby on Rails developer
There should be a default welcome page for a new Rails application
So that I can confirm that my site is working

Scenario: The Welcome Page presents welcome message
Given I am on the Welcome Page
Then I should see the title "Ruby on Rails: Welcome aboard"

Add the corresponding steps to a file called welcome_page_steps.rb into the step_definitions directory under the same features directory:

Given(/^I am on the Welcome Page$/) do
Capybara.current_driver = :selenium
visit('http://localhost:3000')
end

Then(/^I should see the title "(.*?)"$/) do |message|
fail unless page.has_title? message
end

Your files should look something like this:
cucumber file tree

If you have Firefox installed you can try running the cucumber tests locally with:

rake cucumber

docker console
Notice how the test failed to find a page with the title: “Ruby on Rails: Welcome aboard”? That’s because there isn’t a server running at http://localhost:3000.

In another terminal window or tab run:

rails s -b 0.0.0.0

and try running the cucumber tests again. Your tests should now pass.
passing tests console output

Setup the application to run and test inside Docker

Docker prescribes to the idea that no one container should have multiple responsibilites. A typical web application stack would have individual containers for web server, database, caching, test runner, etc. Knowing this, two Docker images are needed – one for the Rails webserver and one for running tests. To get the webserver running in Docker, create a file named Dockerfile in the root directory of the application. With the text editor of your choice, add the following as the contents of the Dockerfile.

# Base the image off of the latest pre-built rails image
FROM rails

# make and switch to the /app directory which will hold our app
RUN mkdir /app
WORKDIR /app

# move over the Gemfile and Gemfile.lock before the rest of the app so that we can cache the installed gems
ADD Gemfile /app/Gemfile
ADD Gemfile.lock /app/Gemfile.lock

# install all gems specified by the Gemfile
RUN bundle install

# copy over the rest of the rails app files
ADD . /app

# start the rails server
# NOTE: The '-b 0.0.0.0' is very important!
# Especially if you are using a Mac.
CMD rails s -b 0.0.0.0

Build the Docker image with:

docker build -t cucumberindocker-webserver .

Run the application through docker with:

docker run --name cucumberindocker-webserver -i -t --rm -p 3000:3000 cucumberindocker-webserver

You can now open your browser and once again see the application running.
NOTE: If you are running Mac OS X, the IP address will match your Docker Machine IP and not localhost. Mostly likely this IP is http://192.168.99.100:3000.

Once you have confirmed that your Rails application ran through Docker, control-c the running Docker container to stop and tear it down.

To get the Cucumber tests to run through Docker, create a new Dockerfile that’s almost identical to the first Dockerfile, but have it run rake cucumber instead of starting up the Rails server. Save this as Dockerfile.cucumber in the root of the project.

# Base the image off of the latest pre-built rails image
FROM rails

# make and switch to the /app directory which will hold our app
RUN mkdir /app
WORKDIR /app

# move over the Gemfile and Gemfile.lock before the rest of the app so that we can cache the installed gems
ADD Gemfile /app/Gemfile
ADD Gemfile.lock /app/Gemfile.lock

# install all gems specified by the Gemfile
RUN bundle install

# copy over the rest of the rails app files
ADD . /app

# run the Cucumber tests
CMD rake cucumber

Did you notice how the two Dockerfiles are identical up until the last line? This allows both images to share Docker’s caching, resulting in much quicker build times. The only difference is that this Dockerfile will execute the Cucumber tests rather than the Rails server. Try building and running this new image and container with the commands below:

docker build -t cucumberindocker-cucumber -f Dockerfile.cucumber .
docker run --name cucumberindocker-cucumber -i -t --rm cucumberindocker-cucumber

The tests failed with a nice warning that Firefox isn’t installed.

firefox not configured console output

This Docker container runs headless without a graphical user interface, but Firefox expects there to be a GUI running so that it can display the browser window. Xvfb (X Virtual Framebuffer) is a great solution to running a virutal Xserver. Docker containers don’t have a typical startup like a full Linux OS, so Xvfb will have to be manually started when launching Firefox. The Dockerfile.cucumber will copy over the shell script xvfb-firefox by Pim Snel to coordinate the starting of Xvfb any time Firefox is run. (See the contents of the script below the Dockerfile.cucumber).

Update your Dockerfile.cucumber to install Firefox and Xvfb so it looks like this:

# Base the image off of the latest pre-built rails image
FROM rails

# make and switch to the /app directory which will hold our app
RUN mkdir /app
WORKDIR /app

# move over the Gemfile and Gemfile.lock before the rest of the app so that we can cache the installed gems
ADD Gemfile /app/Gemfile
ADD Gemfile.lock /app/Gemfile.lock

# install all gems specified by the Gemfile
RUN bundle install

RUN apt-get update && \
apt-get install -y --fix-missing \
iceweasel \
xvfb
# iceweasel is the Debian package name for Firefox
# X Virtual Framebuffer is needed since the image is headless and we need a Xserver to run Firefox on.

# copy over the xvfb-firefox script
ADD xvfb-firefox /usr/bin/xvfb-firefox

# make sure the script is executable by all
RUN chmod a+x /usr/bin/xvfb-firefox

# rm the old firefox script
RUN rm /usr/bin/firefox

# replace the firefox script with a link to the the xvfb-firefox script
RUN ln -s /usr/bin/xvfb-firefox /usr/bin/firefox

# copy over the rest of the rails app files
ADD . /app

# run the cucumber tests
CMD rake cucumber

Save the following script as xvfb-firefox in the root of the project directory.

#!/bin/bash

_kill_procs() {
kill -TERM $firefox
wait $firefox
kill -TERM $xvfb
}

# Setup a trap to catch SIGTERM and relay it to child processes
trap _kill_procs SIGTERM

XVFB_WHD=${XVFB_WHD:-1280x720x16}

# Start Xvfb
Xvfb :99 -ac -screen 0 $XVFB_WHD -nolisten tcp &
xvfb=$!

export DISPLAY=:99

iceweasel $@ &
firefox=$!

wait $firefox
wait $xvfb

Thanks again to Pim Snel for sharing the above script and inspiring much of the this solution. You can find his Docker and Watir setup at https://github.com/mipmip/docker-watir-xvfb.

Try the build and run commands again:

docker build -t cucumberindocker-cucumber -f Dockerfile.cucumber .
docker run --name cucumberindocker-cucumber -i -t --rm cucumberindocker-cucumber

Another error except this one is familiar!
docker console output

This is the same error as before when there wasn’t a webserver running for the Cucumber tests to run against. To keep everything in Docker, run and link the webserver container before executing the tests.

Run the following commands in order:

docker build -t cucumberindocker-webserver .
docker run --name cucumberindocker-webserver --rm -i -t -p 3000:3000 cucumberindocker-webserver

In another terminal window or tab:

docker build -t cucumberindocker-cucumber -f Dockerfile.cucumber .
docker run --link cucumberindocker-webserver:localhost --name cucumberindocker-cucumber -i -t --rm cucumberindocker-cucumber

And viola! The tests are passing while running in a Docker container!

The important part is the --link cucumberindocker-webserver:localhost in the final command. This links the cucumberindocker-webserver container to the cucumberindocker-cucumber container and maps it to localhost allowing Firefox to load http://localhost:3000 and hit the webserver in another container.

Stop your running webserver Docker container with control-c to before you continue.

Make this all repeatable and easy

It’s great that the tests run through Docker, but running 4 lengthy shell commands in order isn’t repeatable and opens the possibility of human error. The ideal situation would be to type in one command and have it do everything. This is a perfect time to use Rake!

Building the Docker images from Rake is a good first step towards one command to rule them all.

Add the following to your Rakefile:

# Use the parent directory to name our project
PROJECT_NAME = File.basename(%x(pwd)).strip.downcase
WEBSERVER_NAME = PROJECT_NAME + "-webserver"
CUCUMBER_NAME = PROJECT_NAME + "-cucumber"

namespace "docker" do
namespace "build" do
desc "Build the primary app Docker image"
task :app do
sh "docker build -t #{WEBSERVER_NAME} ."
end

desc "Build the Docker image for running Cucumber tests"
# build the main app image before building the cucumber image
task :cucumber => 'docker:build:app' do
sh "docker build -t #{CUCUMBER_NAME} -f Dockerfile.cucumber ."
end
end

You can now build the app image with:

rake docker:build:app

And build the image to run your tests with:

rake docker:build:cucumber

Now add a couple tasks to run and stop the webserver container. Add these two tasks under the “server” namespace inside the “docker” Rakefile namespace.

namespace "server" do
desc "Start a persistant default Rails server in a docker container"
# build the main app image before running the server
task :start => 'docker:build:app' do
sh "docker run --name #{WEBSERVER_NAME} -d -p 3000:3000 #{WEBSERVER_NAME}"
end

desc "Stop the Rails server previously started by docker:server:start"
task :stop do
begin
sh "docker stop #{WEBSERVER_NAME}"
sh "docker rm #{WEBSERVER_NAME}"
rescue => e
puts "Task docker:server:stop failed"
puts "#{e.class}: #{e.message}"
end
end
end

The first thing to note, is the running the server now uses the -d flag wich detaches the running container and puts it in the background. Second, the stop task tears down the running container, allowing you to use that port and contianer name in the future. Finally, Since the rake docker:server:start first builds the app image, the number of commands to run the Cucumber tests has now been reduced to these three:

rake docker:server:start
rake docker:build:cucumber
docker run --name cucumberindocker-cucumber -i -t --rm --link cucumberindocker:localhost cucumberindocker-cucumber

Run rake docker:server:stop after your tests have passed.

Finally, add a task to build and run the cucumber tests against the server that runs as a result of the Rake tasks that were just added. Add the following “test” namespace and it’s task inside the “docker” namespace so your final Rakefile should look something like this:

require File.expand_path('../config/application', __FILE__)

Rails.application.load_tasks

# Use the parent directory to name our project
PROJECT_NAME = File.basename(%x(pwd)).strip.downcase
WEBSERVER_NAME = PROJECT_NAME + "-webserver"
CUCUMBER_NAME = PROJECT_NAME + "-cucumber"

namespace "docker" do
namespace "build" do
desc "Build the primary app Docker image"
task :app do
sh "docker build -t #{WEBSERVER_NAME} ."
end

desc "Build the Docker image for running Cucumber tests"
# build the main app image before building the cucumber image
task :cucumber => 'docker:build:app' do
sh "docker build -t #{CUCUMBER_NAME} -f Dockerfile.cucumber ."
end
end

namespace "server" do

desc "Start a persistant default Rails server in a docker container"
# build the main app image before running the server
task :start => 'docker:build:app' do
sh "docker run --name #{WEBSERVER_NAME} -d -p 3000:3000 #{WEBSERVER_NAME}"
end

desc "Stop the Rails server previously started by docker:server:start"
task :stop do
begin
sh "docker stop #{WEBSERVER_NAME}"
sh "docker rm #{WEBSERVER_NAME}"
rescue => e
puts "Task docker:server:stop failed"
puts "#{e.class}: #{e.message}"
end
end
end

namespace "test" do
desc "Build and run the Cucumber tests in the test environment ran by 'docker:environment:test"
task :cucumber => ['docker:build:cucumber','docker:server:start'] do

test_failed = false

begin
sh "docker run -i -t --rm --link #{WEBSERVER_NAME}:localhost #{CUCUMBER_NAME}"
rescue => e
# a failing test throws an exception, but that doesn't mean the task failed. Record the failing test and continue
test_failed = true
puts "Task docker:build:cucumber failed"
puts "#{e.class}: #{e.message}"
end

# Teardown the running server
Rake::Task['docker:server:stop'].invoke

if test_failed
fail "[FAILED]"
else
puts "[OK]"
end
end
end
end

There is now one command to rule them all.

rake docker:test:cucumber

Now one command line builds, runs, tests, and tears down everything needed to perform the automated Cucumber test suite!

Get the final code here: Cucumber in Docker

 

Developing an amazing technology product of your own? Take our 1-Minute self-assessment to make sure you’re project is on-track for a successful launch!  Or, reach out to us at LeanDog.com! We’d love to hear all about it!