February 23, 2016
In practical terms, continuous integration, or CI, is the process of taking completed code (feature, bug fix, or otherwise) and integrating them into the main repository on a regular basis.
The concept isn’t foreign to most experienced developers – once a feature is completed, the code is pushed back into the repository – either the main development branch, or a feature branch. The next step involves work that many haven’t experienced. Upon integrating/merging the code into the repository, a build server pulls the changes and builds the project, testing it for functionality, often through unit tests. Depending on the project, the process of building or compiling on the server may also identify newly introduced issues not otherwise noticed by the developer (in theory testing “works on my machine” syndrome).
This process allows teams to quickly identify problems that might otherwise go unnoticed for some amount of time. Quickly identifying issues means they can be resolved before a system of dependencies become built around them – something that becomes much more difficult to unwind as more code is added.
Because CI is often coupled with proper unit and functional testing, the build server is able to verify that the tested functionality hasn’t changed since the last successful integration. Furthermore, CI can be used to test code in numerous build and runtime environments. Some practical examples could include testing builds against various system libraries and configurations, an array of browsers and versions, or a suite of test phones and tablets spanning sizes and operating systems. Here the developer makes large efficiency gains as repetitive tasks are handled by the server.
While investigating various CI services, the mobile team here at Valtech consider several factors that influenced our final decision. Among them,
We needed a system capable of building for both iOS and Android (implicitly, Mac OS X ).
Ease of use
Is it relatively easy to configure the system and keep it running with minimal intervention?
Can we customize the build process outside of modifying the application project files?
Some services are cloud-based, while others are self-hosted (possibly only available on the local network).
Depending on the company size, number of repositories, or concurrent builds, price may become prohibitive
Based on this list we evaluated the following CI tools and services:
- Jenkins (http://jenkins-ci.org/)
- CircleCI (https://circleci.com/)
- Bitrise (https://www.bitrise.io/)
- Buddybuild (https://buddybuild.com/)
- and many others along the way
Jenkins is an open source CI server that has been one of the go-tos for a few years (more if you count its roots with Hudson). It finds favour in the development community because of its versatility and platform support (Mac, Windows, Linux, BSD). With myriad publicly-contributed plugins, Jenkins is quite customizable.
Let’s see how it stacks up against our needs:
As mentioned above, Jenkins runs on Mac OS X, Windows, Linux, and BSD. This meets our first requirement – Mac OS X.
Ease of Use
Once installed and with jobs successfully running, Jenkins is quite maintainable. Once running, it just keeps going. Coupled with tools like Fastlane, (and soon Android) it makes the build process very easy to configure.
There’s a caveat: Getting Jenkins running in the first place can be difficult. On Macs, at least for the purpose of building iOS projects, there are a few special steps that must be taken in order to allow Xcode to operate properly. Kunstmaan Labs has a good step-by-step guide to installing Jenkins and configuring Fastlane.
Besides setup issues, one important factor is your team’s ability to maintain a server, should anything happen. Some developers may feel completely comfortable caring for a server, while others completely lost. This is an important factor to consider when deciding whether or not a self-hosted solution like Jenkins is right. Generally, our experience with Jenkins running on Macs has been positive, and we have enough expertise to handle administration should an issue arise.
Jenkins is one of the most customizable CI tools available – certainly more so than popular cloud-hosted tools. Out of the box, it provides a simple build template: Provide the repository’s URL, branch(es) to build, frequency the build should run, and a short shell script used to actually build the project. A number of plugins are available for both Xcode and Android projects to ease build configuration. Since we use Fastlane (https://fastlane.tools/), the command line is sufficient, but it doesn’t end there. If you need more functionality, Jenkins can do things like send status emails, create build reports, upload binaries for testing, perform specific unit and functional tests (including software simulators and actual hardware), post notifications, deploy to staging and production, and even turn on a Hue light.
Some of these features are bundled with hosted CI services, but that’s where the extensibility ends with them. Because we value being able to tweak our build configurations and perform extra tasks, Jenkins gets a bit of extra credit while others fail miserably.
This is rather important – are you able to access the build machine when you need to? Self-hosting a Jenkins server could mean it sits behind a firewall, blocking your access while out of the office. The typical work-around involves port forwarding to the Jenkins server, or requiring the team to VPN into the network. Another less common option is leasing a Mac server in a datacenter such as MacMiniColo.net. Because we already have a spare Mac Mini in the office, that will be our Jenkins box, so the colocation idea is out. Deciding on how to connect from the outside is yet to be determined.
Jenkins is free. When you look at the available features, you get a lot more than what many paid services offer. Don’t be fooled that the free price tag means it’s completely free. The hidden cost of maintenance, upgrades, and downtime are extremely important. Though Jenkins is easy to use once running, there are times when an engineer must spend time doing routine work, which comes at a cost of their time. Hosted services don’t incur these costs (or presumably, are baked into the monthly or per-user fees). Given a twelve-month timespan, it may be a wash.
As we’ve seen, Jenkins offers us many benefits and a few drawbacks. On one hand, we have a very powerful, extensible system. On the other, the burden of maintenance is on our team and potentially limited access is a concern.