Cooking up the nodes: Thuis Cookbook

In my last post I explained the basics of Chef, and in the last week I worked on defining the configuration of each node. I selected several cookbooks from the Supermarket and wrote some myself. Using a series of recipes I defined the software and configuration of two of the nodes of Thuis in the Thuis Cookbook. In this post I’ll show you my choices and give some code samples to let you set up your own Chef config.

Let’s start with some of the cookbooks I’m using from the Supermarket:

[table width=”600″ colwidth=”25%|75%” colalign=”right|left|left”]
Cookbook,Description
apt, Takes care of keeping the apt-get cache up to date
firewall, Install and configure UFW
hostnames, Automatically configures the hostname of each node based on a pattern
mosquitto, Install and configure Mosquitto (MQTT broker)
tar, Download and extract a tar file
sshd, Install and configure the SSH deamon
sudo, Configure which users can sudo
timezone_lwrp, Configure the time zone
users, Add default users with SSH keys
wildfly, Install and configure WildFly and Java
[/table]

Basics: default recipe

Execute on each the device the default recipe provides the basic setup of a node. All it does is installing a few packages and including other recipes:

package ['rpi-update', 'nano']

include_recipe 'apt'
include_recipe 'hostnames'
include_recipe 'timezone_lwrp'
include_recipe 'mosquitto::client'
include_recipe 'thuis::firewall'
include_recipe 'thuis::sshd'
include_recipe 'thuis::users'

Of course the recipes have some configuration, this is provided through the attributes file:

# System
default['set_fqdn'] = '*.internal.thuisapp.com'
default['tz'] = 'Europe/Amsterdam'
default['authorization']['sudo']['passwordless'] = true

# Firewall
default['firewall']['allow_ssh'] = true
default['thuis']['open_ports'] = [80, 8080, 9990] # TODO: finetune

The firewall, sshd and users recipes are part of the thuis cookbook, the first two are fairly straight forward:

# Include the default firewall recipe
include_recipe 'firewall::default'

# Allow incoming on the configures ports
ports = node['thuis']['open_ports']
firewall_rule "open ports #{ports}" do
  port ports
end

# Don't allow logins through SSH using a password
openssh_server node['sshd']['config_file'] do
	PasswordAuthentication 'no'
end

The users recipe is based on this nice blogpost: it’s creating my default user (robin), adding my SSH key and allows it to sudo without using a password.

Specifics per node

For both nodes (more will follow later) I’ve created a specific recipe. These recipes include the default recipe plus take care of the specific needs of that node.

thuis-server-core

As mentioned in my post about the architecture of Thuis the core node will use WildFly, a MQTT broker and Z-Way:

include_recipe 'thuis::default'

include_recipe 'mosquitto'
include_recipe 'thuis::wildfly'
include_recipe 'z-way'

Mosquitto for now uses the default configure of the cookbook, I’ll finetune this later. I did have to update the cookbook a bit as it didn’t have support for Debian/Raspbian Jessie yet, for that I did a PR, which is accepted.

WildFly required a bit more effort as the standalone.xml configuration file wasn’t update with the latest version and the underlying java cookbook doesn’t have support for the ARM packages needed for the Raspberry Pi. The latter was solvable using some custom configuration which selects a different download based on the architecture:

# Java
default['java']['arch'] = kernel['machine'] =~ /x86_64/ ? 'x86_64' : kernel['machine'] =~ /armv/ ? 'armhf' : 'i586'

java_home_arch = 'i386'
if (node['kernel']['machine'] == 'x86_64')
	java_home_arch = 'amd64'
end
if (node['kernel']['machine'] =~ /armv/)
	java_home_arch = 'armhf'
end
force_default['java']['java_home'] = "/usr/lib/jvm/java-#{node['java']['jdk_version']}-#{node['java']['install_flavor']}-#{java_home_arch}"

default['java']['jdk']['8']['armhf']['url'] = 'http://download.oracle.com/otn-pub/java/jdk/8u91-b14/jdk-8u91-linux-arm32-vfp-hflt.tar.gz'
default['java']['jdk']['8']['armhf']['checksum'] = '79dda1dec6ccd7130b5204e75d1a8300e5b02c18f70888697f51764a777e5339'

default['java']['jdk']['8']['x86_64']['url'] = 'http://download.oracle.com/otn-pub/java/jdk/8u91-b14/jdk-8u91-linux-x64.tar.gz'
default['java']['jdk']['8']['x86_64']['checksum'] = '6f9b516addfc22907787896517e400a62f35e0de4a7b4d864b26b61dbe1b7552'

Next is overriding the WildFly standalone.xml configuration:

include_recipe 'wildfly::default'

resources("template[#{::File.join(node['wildfly']['base'], 'standalone', 'configuration', node['wildfly']['sa']['conf'])}]").cookbook 'thuis'

The template file is a copy of the original, but updated using a diff between the original and the version from WildFly 10.0.0.Final. The needed configuration in attributes.rb is:

# WildFly
default['wildfly']['version'] = '10.0.0.Final'
default['wildfly']['url'] = 'http://download.jboss.org/wildfly/10.0.0.Final/wildfly-10.0.0.Final.tar.gz'
default['wildfly']['checksum'] = 'e00c4e4852add7ac09693e7600c91be40fa5f2791d0b232e768c00b2cb20a84b'
default['wildfly']['enforce_config'] = true
default['wildfly']['mysql']['enabled'] = false
default['wildfly']['postgresql']['enabled'] = false
default['wildfly']['jpda']['enabled'] = false
default['wildfly']['java_opts']['other'] = ['-client']

It changes the version to use, disables a few modules and important on the Raspberry Pi it uses -client instead of -server in Java options.

The most difficult to get working was Z-Way. There is no cookbook available yet, so I had to build this one from scratch. I could have taken a relative easy way out by just letting the cookbook execute Z-Way’s install script, but as I want to learn Chef I went for the hard road. On the right you can see the structure of the cookbook. The full cookbook is available on Github at Edubits/chef-z-way.

The cookbook uses a few recipes to install Z-Way to the device, and more importantly to safely upgrade it while retaining the configuration of all devices. It also installs the required services and enables them to automatically start like:

service 'z-way-server' do
	service_name 'z-way-server'
	supports restart: true
	action [:enable, :start]
end

A few things are kept as in the original script in the template file install.sh.erb and executed during as part of the install recipe. As I’m currently testing on an old Raspberry Pi with the Razberry hardware, I could not verify the full installation yet.

thuis-server-tv

The TV node will mostly take care of connecting to the home cinema system using CEC. For this a Java EE application is used, so WildFly is needed here as well. Next to WildFly we need libcec.

include_recipe 'thuis::default'

include_recipe 'thuis::wildfly'
include_recipe 'thuis::libcec'

libcec in Jessie is only version 2 and we need 3, so we’ll grab this package from the Stretch repository:

# Add stretch apt repository
apt_repository 'stretch' do
	uri          'http://archive.raspbian.org/raspbian/'
	distribution 'stretch'
	components   ['main']
end

package ['libcec3', 'cec-utils'] do
	default_release 'stretch'
end

Testing

Chef has a very nice way of testing cookbooks using Kitchen and Vagrant: it spins up a virtual machine from an image and runs the recipes on that. I used this for the general testing, however quite some of my changes and configuration are specifically made for the Raspberry Pi. This means to test those I had to use an actual device. For this I used my good old Raspberry Pi 1B. This required some patience, as one chef-client run without any changes takes about 5 minutes on this device. As there are some differences in architecture between the 1B and the 3B I expect there will be some small changes needed when deploying the cookbooks to the new device.

Bootstrap the nodes

As soon as the kit arrives (somewhere in the coming week) I will install them using raspbian-ua-netinst, assign a static IP in my router and then I can bootstrap the Raspberry Pi’s with just one command each:

knife bootstrap 10.0.0.201 -t raspbian-jessie-chef.erb --ssh-user root --ssh-password 'raspbian' --node-name 'thuis-server-core' --run-list 'recipe[thuis::thuis-server-core]'
knife bootstrap 10.0.0.202 -t raspbian-jessie-chef.erb --ssh-user root --ssh-password 'raspbian' --node-name 'thuis-server-tv' --run-list 'recipe[thuis::thuis-server-tv]'

Now the Pi’s are fully installed and ready to be used and it’s time to actually build & deploy automation software and connect some hardware! Let the fun begin 🙂