How to manage your build configuration, from the developer platform to the CI
Before starting this article, I want to thank Mark Vieira, Gradle core developer, and the publishing teams from Gradle and Genymobile for their review and assistance.
TL;DR
The syntax apply from: allows you to inject Gradle scripts from files. You can check the current defined environment variables to know the current build environment (developer platform or continuous integration server) and apply the corresponding files.
All parameters you don’t want to write in clear can be defined outside your Gradle files using Gradle Properties (via gradle.properties files, environment variables or command line arguments) and used as variables into Gradle files. Some of these methods can be controlled directly from the continuous integration management interface.
Introduction
Two years ago, at Genymobile, we were all very enthusiastic about one of Google I/O announcements: Gradle is going to be the future-new-official-but-still-young build system for Android.
A few weeks later, when we measured its potential power, all Android app developers were waiting impatiently for a stable and fully usable duet “Android Studio & Gradle”.
Our first observation after adopting Gradle was that we were living a real professionalization of the tools provided by Google. In short, Gradle was as powerful as the well-known Maven, with a very light boilerplate for the configuration and the customization of the build. It also had this very nice wrapper that really made the adoption enjoyable. Gradle led to a better industrialization of the Android development, with an official support by the Android SDK team.
Now we all implement automated tests, don’t we? We take seriously into account static analysis tools, right? And we run all of them on our continuous integration server, sure!
Thanks to the build.gradle files of our projects, we can now control all tasks we run automatically. But sometimes, some tasks must behave differently whether they are run on our development environment or on a continuous integration server (also named CI).
An example of the problem
At Genymobile, we develop Genymotion, an Android emulator. We worked hard to make it the perfect emulator for Android developers and we recently released a Gradle plugin. This plugin allows us to declare, in the build.gradle script, the Genymotion devices to launch before running the tests.
Let’s imagine that on our computer, we want to run tests on an emulated phone (Nexus 5) and an emulated tablet (Nexus 9). Then we add this block to the build.gradle file:
genymotion {
devices {
Nexus5 {
template “Google Nexus 5 - 5.1.0 - API 22 - 1080x1920”
}
Nexus9 {
template “Google Nexus 9 - 5.1.0 - API 22 - 2048x1536”
}
}
}
On the CI server side, we want to test more devices. So we add a Galaxy S3, an S4, a Nexus 7 and a Nexus 10, with different Android versions:
genymotion {
devices {
Nexus5 {
template “Google Nexus 5 - 5.1.0 - API 22 - 1080x1920”
}
Nexus9 {
template “Google Nexus 9 - 5.1.0 - API 22 - 2048x1536”
}
S4 {
template “Samsung Galaxy S4 - 4.3 - API 18 - 1080x1920”
}
S3 {
template “Samsung Galaxy S3 - 4.2.2 - API 17 - 720x1280”
}
Nexus7 {
template “Google Nexus 7 - 4.1.1 - API 16 - 800x1280”
}
Nexus10 {
template “Google Nexus 10 - 4.4.4 - API 19 - 2560x1600”
}
}
}
Those additional configurations offer a better coverage for the instrumented tests and they will all be run automatically.
Now, we have two pieces of script, supposed to be in the same file at the same time. Let’s see how we can handle this problem.
Apply From a File
The best way to solve this is to put both scripts into separate files:
- genymotion-dev.gradle will be applied when the build will be running on the developer’s computer;
- genymotion-ci.gradle will run on the CI.
They can be both be applied using the apply from: syntax, as shown below:
build.gradle
apply from: “$rootDir/genymotion-dev.gradle”
But how can the build system decide what files to apply?
For this, it needs to know whether the build is running on the CI or on a dev environment, and it is pretty simple.
Each time your continuous integration server launches a job, it opens a worker and it injects some environment variables. These can be used by the running scripts to get information about the context of the build. They are also convenient to know whether the Gradle script is executed on a CI or not.
You can get the list of these default values on each CI documentation. Here are examples for Jenkins and Teamcity.
Some people would prefer to avoid the build system configuration to depend on the CI installed, allowing to switch easily from a server to another. To solve this, you can define an environment variable yourself for all your jobs instead of using the one from your server.
On Jenkins, you can do this using EnvInject plugin. On TeamCity, you can use Build Parameters.
We just need to test that this environment variable is set to determine whether the build is running on the CI and then apply the dedicated Gradle file:
build.gradle
def isCiServer = if(System.getenv().containsKey("IS_CI_JOB"))
if(isCiServer)
apply from: “$rootDir/genymotion-ci.gradle”
else
apply from: “$rootDir/genymotion-dev.gradle”
The path as a URL
As a bonus, Gradle adds a nice feature to the apply from instruction: we can use a URL.
Instead of storing the file in a specific place on the CI server or on the repository, we can put it behind a HTTP server like this:
apply from: “http://ci.mycompagny/genymotion-ci.gradle”
Managing Local Parameters
Now that the build setup is defined, we are almost done for the build system configuration. However, in most cases, you still need to configure settings related to the computer running the build. These settings can be related to a specific local path, credentials, or any other specificity of the computer.
For Example
The Gradle plugin for Genymotion, can now declare parameters to set the local path of Genymotion binaries or account credentials. They can be defined in the following way:
build.gradle
genymotion.config {
genymotionPath = “/home/user/genymotion”
username = “eyal”
password = “mypassword”
}
To Note: Storing credentials in plain text is not a good idea, especially if you place this file into your repository. We usually strongly advise to manage your Genymotion credentials directly from the software (through the UI or the command line).
However, this example is a good occasion to show how to manage your credentials from your Gradle build when you are obliged to.
As we will see different use cases concerning paths and credentials, let’s define the different computers building your app during your production cycle:
- the development platforms, for each developer;
- the continuous integration slaves where the builds and tests will be run (including sometimes the CI server itself).
The development platforms
The simple (but dirty) way
You’ve seen previously the genymotion.config declaration containing local properties. On the development machine, we can paste them into the genymotion-dev.gradle file in order to store them.
Then, before a git commit, we will be careful to avoid adding local parameters (git add -p for the win!).
As stated in the title, this method is simple but dirty. Gradle proposes other simpler and cleaner ways to set this kind of information locally without adding them directly into the project.
The Android way: Local Properties
At the Android project root, we can find a local.properties file where we can set our local Genymotion settings. This file is supposed to be added by Android Studio, but you can create it if needed and add the following lines:
local.properties
genyPath=/home/user/genymotion genyUsername=eyal genyPassword=mypassword
When set into this file, properties are directly accessible into you build script and can be used this way:
genymotion-dev.gradle
genymotion.config {
genymotionPath = genyPath
username = genyUsername
password = genyPassword
}
The Genymotion way
Another solution, very specific to the Gradle plugin for Genymotion, is to add these lines to the file, instead of the one defined before:
local.properties
genymotion.genymotionPath=/home/user/genymotion genymotion.username=eyal genymotion.password=mypassword
This way, you will directly set Genymotion properties for your build, without needing to add content into your Gradle files.
As local.properties must be ignored by your versioning system, you will have less chances to leak information and it will be simpler to use.
To Note: The local.properties file is generated by Android Studio, its header contains a big warning asking to not modify it because “the content can be erased”.
According to (a discussion I had with) the Android SDK team, this file is actually properly accessed by Android Studio and if you add content, it should not be lost in the future. As it is part of the development environment, we can consider local.properties safe enough to use as the content can rarely be removed.
It is natural to set this value on the development environment but it is harder to do it from the continuous integration server, right after a git clone.
Fortunately, Gradle properties are really convenient; there are many ways to set them. Here are some of them which allow to easily set these parameters on the CI server.
The Continuous Integration Slaves
Having CI-specific configurations explicitly declared inside the build.gradle or genymotion-ci.gradle file is not really good practice because most of the time, the script can be run on different computers where local settings cannot be shared (file path or credentials are different).
Also, on public projects, private information could leak (credential access, directories structure, etc.). Finally, the project depends on the infrastructure setup which is not good.
Gradle provides several possibilities to get rid of this dependency.
Properties from HOME
In addition to the Android project’s gradle.properties, Gradle also handles another gradle.properties file: the one located in <USER_HOME>/.gradle/ folder.
This way, you set properties as explained above but on another file:
/home/user/.gradle/gradle.properties
genyPath=/home/user/genymotion genyUsername=eyal genyPassword=mypassword
And add the same genymotion.config declaration for the CI:
genymotion-ci.gradle
genymotion.config {
genymotionPath = genyPath
username = genyUsername
password = genyPassword
}
Properties from command line
Another way to define these settings is to add the property on the fly by adding it to the executed command line. The --project-prop or -P option is designed for this.
On our CI interface, we need to add the property to the Gradle task switches (arguments). Taking our current example, the command line to launch the connectedAndroidTest Gradle task should be:
gradle connectedAndroidTest -PgenyPath=/home/user/genymotion -PgenyUsername=eyal -PgenyPassword=mypassword
As it is impossible to set different values for each CI slave, this method cannot be used in this example. But it is my favorite for setting parameters related to the whole project because we can manage it directly from the CI server interface. And as we must set it directly on the CI job definition, it is quite easy for the team to read and modify this value if needed; and it does not get lost somewhere in the slave’s file system.
Properties from environment variable
Gradle also handles properties from environment variables; we just need to declare them with the ORG_GRADLE_PROJECT_ prefix. In our example, we need to export the following values:
Command Line
export ORG_GRADLE_PROJECT_genyPath=/home/user/genymotion export ORG_GRADLE_PROJECT_genyUsername=eyal export ORG_GRADLE_PROJECT_genyPassword=mypassword
On Jenkins, slaves are called nodes and you can set, for each of them, all the environment variables (called Node Properties) you want on their configuration panel.
On TeamCity, you need to set them on the agent directly by modifying the buildAgent.properties file.
Depending on your setup, this method can be better than using Gradle properties from Home because we can set or at least read these values from the CI interface.
Now we know three ways to configure the file path on the server side. It is your choice to decide which one fits best your needs and constraints.
Avoid Aborting
Finally, in some cases, we might prefer to avoid build failures because of a missing property. We can simply check the presence of the needed properties using the hasProperty() method before applying it.
For Example
genymotion.config.genymotionPath declaration is not needed for build slaves running under OSX or Windows if Genymotion is installed on the default path. In this case, we can avoid defining this parameter for these platforms.
Here is the result:
build.gradle
if (hasProperty("genyPath") {
genymotion.config.genymotionPath = genyPath
} else {
logger.debug(“Missing genyPath property. It will not be set.”)
}
Conclusion
Managing the build.gradle file among different configurations can be sometimes tricky but Gradle provides simple means to handle it.
By applying different Gradle files into your build.gradle and by using Gradle properties, we have everything that we need to tackle any kind of complicated infrastructure that needs specific information injected into the build script. This really makes Gradle a very good system to make all our projects compatible with the continuous integration methodology.
About the author
Eyal is a software engineer at Genymobile, working on Genymotion – an Android emulator. He’s been working with Android since 2009.
You can find his social network links here.
