Testing with CucumberJS: a team endeavour to create code quality: Peerio
November 12, 2018

Testing with CucumberJS: a team endeavour to create code quality

By far the most exciting days at Peerio are Wednesdays. We not only gather to internally test the product update that’s about to ship, but we do it playing trivia.

That’s the essence of our testing approach: an accessible activity where everyone participates.The tools allowing us to do so are the Gherkin language and the Cucumber testing framework.

A language for collaboration

Gherkin enables us developers to communicate with designers, project managers, stakeholders, and users. We are able to understand their goals and set clear expectations together.

It’s easy for non-technical teammates to read existing scenarios and come up with their own. Our Gherkin files and Cucumber tests are open source so our users too can access them and participate in testing.

Sample scenarioSample scenario

When questions about functionality come up everyone refers back to the scenarios.

Composed out of only 8 keywords, Gherkin is still flexible enough to express hundreds of scenarios our users might encounter while using our app. Eventually, we transform the Gherkin scenarios in automated tests by using CucumberJS.

Sample output while running Cucumber JSSample output while running Cucumber JS

Code quality and safety nets

Whenever the occasional bug reaches the user, we capture the faulty behaviour by writing a regression scenario and adding it to our test suite.

Every time developers push code on our central repository, a continuous integration system triggers the run of the test suite and notifies us of the results. We immediately know if something got broken and we can fix it before it reaches the user.

Sample output from the continuous integration systemSample output from the continuous integration system

For SDK testing we focus on end-to-end testing, a methodology that verifies whether the application flow is performing as designed from start to finish. It mimics actual user behaviour and reproduces their actions in a system.

Good tests follow examples of how someone uses the app to perform a real world task. We even wrote our own chatbot called Cucumbot to simulate two users talking to each other - it helped us many times to test complicated behaviour!

For testing the mobile apps we combine CucumberJS and UI testing following the Page Object design pattern. Our mobile tests are cross-platform, meaning that the same testing code powers two platforms (iOS and Android) in parallel.

All code is open source and we maintain quick start guide for testing here and here, if you’re curious about it.

(Mostly) fun and games

As our codebase grows, so does the test suite, which comes with some challenges. We maintain the test suite to reflect changes in production and whenever we add a feature, corresponding Gherkin scenarios and CucumberJS tests are added too.

Though end to end tests are infamously difficult, they offer the best fidelity when it comes to replicating user behaviour. This helps us ship code we’re proud of, reliably, and on time.

Sample output while running a Cucumber JS test

Being our own users

At its core Peerio is a product for teams who collaborate often and effectively. For this reason, we want our testing methodology to be accessible, follow human-centered principles, and solve real people’s problems.