Execute Bulk Scenario in Parallel in BDD using Cucable Maven Plugin

BDD (Behavior Driven Development) Test Automation Framework is very popular nowadays because it allows us to describe the test cases from the user end perspective in Cucumber Gherkin format. Any stakeholder, whether it’s Developer, QA, Business Analyst, or Project Manager can easily understand what we are testing because the test cases are written in plain simple English. If you have worked on BDD Framework you would know that to execute a test scenario we need a runner class, in which we can define tag, feature file path, and other details as needed. But what if we have to deal with more than 10,000 such scenarios with different tags involved? It’s very difficult to execute all the scenarios using runner files as you would need many of them to create. 

We have faced the same problem in our experience and Maven Cucable Plugin helped a lot to handle such a huge test bed. So, let’s see how it can help. As we know we need to use a Junit runner class to execute a Cucumber Scenario. Typically, a runner class looks like below:

import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;
import org.junit.runner.RunWith;

@RunWith(Cucumber.class)
@CucumberOptions(
        glue = "stepdefinition",
        features = {"src\\test\\resources\\features\\stock_Analysis\\StockAnalysis.feature"}
        plugin = {"json:target/cucumber-report/StockAnalysis.json"}
)

public class debug {
}

In Cucable, we need to first create a sample runner class template. Here is how it should look if you want to generate a runner class as shown above. 

import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;
import org.junit.runner.RunWith;

@RunWith(Cucumber.class)
@CucumberOptions(
        glue = "stepdefinition",
        features = {"target/parallel/features/[CUCABLE:FEATURE].feature"},
        plugin = {"json:target/cucumber-report/[CUCABLE:RUNNER].json"}
)
public class CucableJavaTemplate {
    // [CUCABLE:CUSTOM:comment]
}

As you can observe, the Feature Name and Plugin Name has been replaced with [CUCABLE: FEATURE] and [CUCABLE: RUNNER] keyword respectively. 

Now let’s discuss how Cucable can generate Junit runner classes automatically. We need to pass the feature file tag name in the maven command line. Cucable will scan all the scenarios that match the given tag and generate a feature film and a Junit runner class for each matching scenario and store it in the target folder. This is when the above runner class template comes into the picture. Cucuable will replace the auto-generated feature file name (which was generated for each scenario) in place of the [CUCABLE:FEATURE] keyword and the auto-generated runner file name in place of the [CUCABLE:RUNNER] keyword. Let’s take an example. Suppose we have 2 scenarios as shown below:

sample feature file

And we execute the scenarios using the below maven command:

clean verify -Denv=STOCK -Dtag=@STOCK_ANALYSIS -Dfeature=stock_Analysis -DforkCount=1

As mentioned earlier, Cucable will scan all the feature files and find that there are two scenarios matching the tag “@STOCK_ANALYSIS”. So now Cucable will create two separate feature files and two separate runners on the fly. Here is what it should look like. Below is the example of auto-generated feature file for the first scenario:

# language: en

@STOCK_ANALYSIS
Feature: Stock Analysis

Scenario: Stock Analysis of given stocks
Given we read all the company name from excel "StockName" and sheet name "Set1"
When landed on the google homepage
Then capture all the stock statistics

# Source feature: src/test/resources/features/stock_Analysis/StockAnalysis.feature
# Generated by Cucable

Whereas below is an example of auto-generated runner class file for the first scenario:

import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;
import org.junit.runner.RunWith;

@RunWith(Cucumber.class)
@CucumberOptions(
        glue = "stepdefinition",
        features = {"target/parallel/features/StockAnalysis_scenario001_run001_IT.feature"},
        plugin = {"json:target/cucumber-report/Runner_StockAnalysis_scenario001_run001_IT.json"}
)
public class Runner_StockAnalysis_scenario001_run001_IT {
    // [CUCABLE:CUSTOM:comment]
}

// Generated by Cucable from src/test/java/some/template/CucableJavaTemplate.java

The Target folder will look like the one below. As we can observe, under target/parallel/features all the feature files have been generated dynamically, whereas corresponding runners have been generated in target/parallel/runners.

 

As soon as all the runners and feature files are generated, the maven surefire plugin will pick the runners and execute them in parallel. So, what are the advantages we are getting from Cucable?

• Runner files are generated automatically.
• Each runner file is capable of running one scenario. So, think about the parallel execution that can take place. From my experience, I used to trigger 1000 test cases in one Jenkins build with 80 threads in parallel which means 80 test scenarios in parallel on Selenium Grid resulting in hours of savings in execution. 

Now the question is how to integrate Cucable in pox.xml. Here is an example that you can include under the <build></build> section in pom.xml.

Conclusion

Cucable is a powerful maven plugin to execute a bulk scenario in parallel. I have personally used this plugin in my experience to manage a regression suite of 10,000+ scenarios and it performed well. For more details, you may visit their official GitHub repository.