Who put that Selenium in my Cucumber? (Or how to test the Front-end)
On every big project involving web or native apps there’s the need to test functional features from a UI perspective (beyond the scope of Unit Tests). Furthermore, this is not done just once, but within every regression or integration tests too. Tests are key to deliver good quality to the customer, which of course is one of our main focuses. But testing this by hand usually takes too much time, time in which the customer is not benefiting from your feature. Because of this, at eDreams ODIGEO (eDO) we thought of testing all our Services and Apps, and we want to share our experience in this article about the automation process of the web app with Selenium and Cucumber.
As stated before, providing new features to the customer is very important, and the faster the better. However, quality should not suffer because of this “hurry”, because at eDO we care about both. In order to deliver quality, every feature we added had to be tested in many ways (depending on the change). This meant that the more features we added on each release, the more time was needed for integration and regression tests. All the tests already had a good amount of tests plans, thus it seemed natural to move into Cucumber and automate all of them. This change gave us an incredible boost on delivery time.
The test framework
Once we thought of the languages, tools and platform we wanted to use, it was time to start developing! CucumberJS is already more or less a framework that gives a set of hooks and utils to set up the process for each test and start implementing steps and scenarios (test cases).
- Execution command: First thing we do is execute a command that invokes grunt which has several optional arguments such as the browser to use or the features to test. Also we can provide some development mode modifiers, such as skipping the deployment process (so we can use an already deployed version of the app in our local environment).
- Deployment and startup: Right after Cucumber has started, we hook into the “BeforeFeatures” event, which is fired once before starting to execute any feature. We use this hook to start up the selenium standalone server, mocked services and also deploy the app.
- Scenario setup: The hook “Before”, however, is executed every time we test an scenario. It is mainly used to prepare all the mocked services (clean their states if they have them) and restore the initial setup. Then, we can also start the browser with the desired configuration (for example emulation parameters).
- Scenario clean: At this point the execution of a scenario has ended, so the “After” hook is executed. This hook provides the result, either if it was completed successfully or not. If the scenario failed, we need to gather the resources to study the case later:
- A screenshot of the browser at the moment of failure.
- The logs from the app, the selenium server, the browser and the mock servers.
All of the information gathered is then attached to Cucumber report. Lastly, it’s important to close the browser and clean all logs.value
- Shutdown: Finally, the last hook “AfterFeatures” may be the time to do some clean up. Stop mock services undeploy and in our case create a little resume of an average of the time taken per scenario and per feature.
Epilogue of Part 1
Summing it all up, we test the front-end as a completely independent black box, with all of its dependencies (services and so on) mocked, which helps speeding up the test process. In the process of automating we found some other challenges like dealing with page loadings (as our front-end is a SPA), scrolling and other issues, as well as randomization and mocking specifics (analytics, and so on) that will be covered in the next parts of this series of articles.