EarnIn Case Study

EarnIn brings down testing time by scaling their Android application testing on Genymotion SaaS

EarnIn has helped millions of US citizens access their wages before payday to control and spend their earnings better since 2013. The company is also well-known for its community-supported financial platform with a suite of tools that let people take out small loans with no fees. Thus, with more financial peace of mind, community members can pay bills in due time and better face unexpected bills.

EarnIn application has been downloaded over 15M times and has provided access to $15 billion in earnings. Highly trusted by the community, the EarnIn application has been referred 570k times, which sets high expectations for the application’s responsiveness and quality. Hence, Testing as fast as possible to test new application features continuously is critical for the EarnIn team.

The challenge: Bring down CI time, improve stability and testing with a mock server

As EarnIn saw their Android test suite grow, and so did their test execution times, they needed to scale up while maintaining the reliability and control of the device/emulator. 

Their previous CI/CD provider worked fine then, but they noticed that the overall time to run the tests was creeping upward, and the tools provided didn’t suit their needs. They were also planning to integrate a mock server in their automation and needed more control of the running devices.

EarnIn relied on a combination of solutions:

• Firebase Test Lab as a cloud device provider ;
• CircleCI as continuous integration server,
• Flank as parallel test runner for Firebase Test runner to run their tests.

Although it had its advantages, such as simplifying the overall setup of launching a device and automatic sharing, it lacked the flexibility they were looking for. They wanted a more fine grain approach to sharding and test reports. For example, each matrix in Firebase would have its own report and it wasn’t consolidated.

To improve stability and reduce CI time, they decided to switch their CI to Jenkins and incorporate a mock server in their automation. But establishing a connection between the mock server and a virtual device in the cloud required a proxy tunnel. Firebase didn’t provide this feature so they needed an alternative.

The solution: Genymotion SaaS

EarnIn found the solution with Genymotion SaaS and gmsaas CLI tool:

• Its Adb tunnel provides a proxy between the mock server and the device ;
• It allows sending adb commands to the device, passing the necessary flags needed for the mock server to run.

Genymotion SaaS also allowed them to scale the number of automation they can run.

The results

Thanks to Genymotion SaaS, EarnIn can benefit from a cloud Android device solution to:

• test their application that needs access to a local mock server on a private network 
• easily scale their testing by running 60 virtual devices in parallel, something they couldn’t achieve before. 
• pull build artifacts from the devices, such as screenshots saved to the sdcard, thanks to the device full file access

Their testing time has been cut almost in half and they are able to execute twice as many builds on Jenkins.