Speed up test kitchen

I have been working one some cookbooks using test kitchen. Especially during my early phases of developing a cookbook I had to converge quiet often.

Some of my cookbooks required the installation of software packages which were downloaded from their respective internet repositories. Which depending on internet connection speed and availability, considerably increased t he time each converge took.

So I tried to figure out a way to speed up the converging.

The first thing I did is switch from yum to plain rpm installation were it was possible. Obviously that approach has the drawback of having to manager dependencies myself. But for the time being I was willing to expect that.

The second thing I did, that really got things up to speed, was hosting the rpms on my development machine. This involved the following steps:

1. Create a subdirectory rpms in my cookbook into which I downloaded the rpms to install:
   
   Shell

   drwxr-xr-x 2 tom1299 allusers 4096 Mar 4 13:33 . drwxr-xr-x 13 tom1299 allusers 4096 Mar 4 13:25 .. -rw-r--r-- 1 tom1299 allusers 6 Mar 4 12:03 .gitignore -rw-r--r-- 1 tom1299 allusers 41814431 Mar 3 09:23 artifactory-3.5.2.1.rpm -rw-r--r-- 1 tom1299 allusers 474746577 Mar 3 09:24 chef-server-core-12.0.5-1.el6.x86_64.rpm -rw-r--r-- 1 tom1299 allusers 61704031 Mar 3 09:24 jenkins-1.598-1.1.noarch.rpm -rw-r--r-- 1 tom1299 allusers 170074522 Mar 3 09:24 opscode-manage-1.6.2-1.el6.x86_64.rpm -rw-r--r-- 1 tom1299 allusers 90739876 Mar 3 09:24 sonar-5.0-1.noarch.rpm

   1 2 3 4 5 6 7 8

   drwxr-xr-x  2 tom1299 allusers      4096 Mar  4 13:33 . drwxr-xr-x 13 tom1299 allusers      4096 Mar  4 13:25 .. -rw-r--r--  1 tom1299 allusers         6 Mar  4 12:03 .gitignore -rw-r--r--  1 tom1299 allusers  41814431 Mar  3 09:23 artifactory-3.5.2.1.rpm -rw-r--r--  1 tom1299 allusers 474746577 Mar  3 09:24 chef-server-core-12.0.5-1.el6.x86_64.rpm -rw-r--r--  1 tom1299 allusers  61704031 Mar  3 09:24 jenkins-1.598-1.1.noarch.rpm -rw-r--r--  1 tom1299 allusers 170074522 Mar  3 09:24 opscode-manage-1.6.2-1.el6.x86_64.rpm -rw-r--r--  1 tom1299 allusers  90739876 Mar  3 09:24 sonar-5.0-1.noarch.rpm

   
    
2. Set up a private network in my kitchen yaml file so that the guest was able to access the host. This stackoverflow post describes how to do it and which Ips to use:
   
   Ruby

   network: - ["private_network", {ip: "192.168.50.4"}]

   1 2	network:   - ["private_network", {ip: "192.168.50.4"}]

   
    
3. Start a simple python http server in the rpm directory:
   
   Shell

   python -m SimpleHTTPServer 9999

   1	python -m SimpleHTTPServer 9999

   
    
4. Change the urls for downloading the rpms to point at the host’s http server and add the host’s ip to the no proxy list:
   
   Ruby

   suites: - name: default run_list: - recipe[ci-server::default] attributes: ci-server: all_proxy: <%= ENV['http_proxy'] %> ftp_proxy: <%= ENV['http_proxy'] %> http_proxy: <%= ENV['http_proxy'] %> https_proxy: <%= ENV['https_proxy'] %> no_proxy: <%= ENV['no_proxy'] %>,ci-server,ci-server.jeeatwork.com,192.168.50.1 chef-server: downloadurl: 'http://192.168.50.1:9999/chef-server-core-12.0.5-1.el6.x86_64.rpm' opscode-manage: downloadurl: 'http://192.168.50.1:9999/opscode-manage-1.6.2-1.el6.x86_64.rpm' jenkins: downloadurl: 'http://192.168.50.1:9999/jenkins-1.598-1.1.noarch.rpm' sonar: downloadurl: 'http://192.168.50.1:9999/sonar-5.0-1.noarch.rpm' artifactory: downloadurl: 'http://192.168.50.1:9999/artifactory-3.5.2.1.rpm'

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

   suites: - name: default   run_list:   - recipe[ci-server::default]   attributes:     ci-server:       all_proxy:  <%= ENV['http_proxy'] %>       ftp_proxy:  <%= ENV['http_proxy'] %>       http_proxy:  <%= ENV['http_proxy'] %>       https_proxy:  <%= ENV['https_proxy'] %>       no_proxy:  <%= ENV['no_proxy'] %>,ci-server,ci-server.jeeatwork.com,192.168.50.1     chef-server:       downloadurl: 'http://192.168.50.1:9999/chef-server-core-12.0.5-1.el6.x86_64.rpm'     opscode-manage:       downloadurl: 'http://192.168.50.1:9999/opscode-manage-1.6.2-1.el6.x86_64.rpm'     jenkins:       downloadurl: 'http://192.168.50.1:9999/jenkins-1.598-1.1.noarch.rpm'     sonar:       downloadurl: 'http://192.168.50.1:9999/sonar-5.0-1.noarch.rpm'     artifactory:       downloadurl: 'http://192.168.50.1:9999/artifactory-3.5.2.1.rpm'

   
   After doing so converge runs really went pretty fast. But this approach has its limitation and drawbacks. When installing yum packages this solution obviously does not work. And setting up a local yum repository is fare more work than starting a simple http server.

   Nevertheless do I think that using this approach during development could make life a little bit easier.