Why Achieving In-sprint Test Automation is the key to Agile?

A critical factor in any Agile environment is to deliver “working software.” Working software is that which works as per the acceptance criteria—not just for the functionality that gets delivered but also for all the previous functionalities that were delivered in previous sprints. Imagine a team delivering a complex pricing calculation functionality in a sprint but breaking the login functionality! This would hardly provide any value to the end user. “Working software” is something that is tested thoroughly, and the team must be confident that the delivered software contains no bugs and is ready to be used by the end user. 

Automated tests are critical in this process. In my experience, having both automated tests not just for the functionality to be delivered in the sprint and also an automated regression suite to test all functionalities is the only way to truly deliver working software. 

In this article, we’ll take a look at: 

• What in-sprint test automation is 
• Issues with an N-1 or N-X test automation approach 
• Challenges in achieving in-sprint test automation 
• How you can use behavior-driven development (BDD) to achieve in-sprint automation 
• Challenges in achieving BDD in-sprint automation 
• And how you can use NoCodeBDD to achieve BDD in-sprint automation 

What is In-sprint Test Automation?

The first principle in Agile Manifesto is

“Our highest priority is to satisfy the customer
through early and continuous delivery
of valuable software.”

1st Principle in Agile Manifesto

To deliver valuable software continuously, it is critical to automate your test cases. Achieving continuous delivery, in my view, is impossible when the manual testing approach is taken. 

When the team automates the test cases, all functionalities that get delivered should have equivalent test automation scripts. Thus, when the team delivers the software, it can be tested via these automation scripts and moved to the automation regression suite. Then, when the team delivers the next sprint, the regression suite can be run to make sure none of the previously delivered functionalities are impacted. 

Put simply, in-sprint test automation means having adequate automated test scenarios to test all functionalities implemented in the sprint. 

Many teams fail to achieve in-sprint automation due to cost considerations. Other challenges come with the territory as well. Let’s look at these challenges and different options that one can consider to achieve smooth in-sprint automation. 

Challenges in achieving In-sprint Test Automation

Cost

One of the biggest challenges is that of cost. When teams write scripts to automate test scenarios, costs can soar. Testers—responsible for automating the test scenarios—need to test the application and write the required scripts to automate those steps: this can take a considerable amount of time. 

In our experience, achieving in-sprint automation requires that each dev be paired with an automation tester. Let’s say a team has 5 developers: unless the team has 4–5 automation testers to pair them with, it would be extremely difficult to achieve in-sprint automation. 

Per Glassdoor, the cost of an automation test engineer is more than a dev: https://www.glassdoor.co.uk/Salaries/test-automation-engineer-salary-SRCH_KO0,24.htm. You don’t need a mathematics degree to recognize the high cost of achieving in-sprint automation. 

Resources

Even if the client or the business is ready to spend money to achieve in-sprint automation, you might still struggle to do so. Acquiring good automation test engineers is difficult. Because it’s a difficult skill to master, there is a market scarcity of such engineers. As a result, you might struggle to hire the right resources. 

Skills

Even if you have the resources, you might struggle to achieve in-sprint automation depending on how your tech stack evolves. For example, let’s say when you start your project, the tech stack has UI, Mobile, and Relational Database. Based on this stack, you hired resources with skills in these areas to automate tests.  

As your business scales, your tech stack must evolve to support the scaling business. As a result, you might now require automation test engineers to support your evolving stack: you might now work with Kafka and No-SQL databases, thus requiring resources to support that tech. 

Losing to Dev

Due to the relative dearth of resources and skills, many companies spend time and money on training their resources—mainly the manual testers or junior resources—to become automation engineer. In my experience, this doesn’t always work. These personnel often want to transition to a dev role once they get the coding skills. As a result, the team is always short of automation engineers and falls back on manual testing. 

N-1 Test Automation

Many teams take the approach of N-1 automation. N-1 automation means that your test automation is one sprint behind the current sprint. 

When opting for N-1 automation, a common approach is employing two separate test resources: manual testers and automation testers. Manual testers are responsible for testing the functionality in that sprint, and the automation tester’s responsibility is to automate the scenarios tested by manual testers in the subsequent sprint. 

Segregating the team in this way rarely if ever works for two reasons: 

• Automation test engineers always try to catch up with the number of scenarios that manual testers add 
• Automation test engineers write automation scripts without understanding the test cases or the domain, potentially resulting in invalid scripts or may not test the right scenarios 

Invalid Process; Poor Collaboration, Incomplete User Stories, etc…

Even if the team has the right resources and skills, poor user stories and/or lack of collaboration between three amigos can spell danger for achieving in-sprint automation. 

This issue is quite common even in strong agile environments. The product owner’s, dev’s, and QA’s interpretation of user stories can vary, and each is likely to deliver their own version of the user story. Therefore, your automation scripts can be either invalid or fail to reflect the real-world scenarios. 

In the next section, let’s look at how behavior-driven development (BDD) could be used to address some of the above-mentioned issues. 

How NoCodeBDD can help

When a team follows a BDD process, the first step should be to collaborate and derive real-world scenarios. These scenarios form a contract between all team members: the devs use these scenarios to implement solutions, and the testers write automation test scripts for each of the scenarios. 

The following article shows how BDD should be rolled out in a project:

https://blog.nocodebdd.com/bdd-and-user-stories/embed/#?secret=m4qzmAic0n

Using NoCodeBDD, the derived scenarios are directly entered in NoCodeBDD. The dev engineers use these scenarios to implement solutions, and the testers can use NoCodeBDD’s codeless automation modules to automate scenarios with ease. 

Let’s look at how this process eliminates the issues above and facilitate in-sprint automation. 

Reduced Cost

Using NoCodeBDD, you reduce your costs considerably as you don’t need to hire additional automation test engineers. Using NoCodeBDD, your manual testers could automate the scenarios. Some of our clients even get their business analysts to automate scenarios: NoCodeBDD doesn’t require any coding skills. Below is one of the testimonials from our client CodeMantra: 

‘I wanted my team to use BDD to collaborate effectively with cross-functional teams and achieve close to 100% automation in-sprint. After evaluating several tools, we chose NoCodeBDD, as it handles complex business scenarios and different tech stacks like UI, API and relational DB. Using NoCodeBDD, we were able to significantly accelerate BDD automation, achieving in-sprint automation with 600% ROI. I recommend NoCodeBDD to any organization looking to roll out ’Shift-left Testing Automation’.

Murali, VP of Engineering, CodeMantra

Resource Optimisation

Using NoCodeBDD, many of our clients have optimized their resources by using manual testers for the majority of the automation tasks. They assign technical resources with coding skills  to projects that require coding either on the dev side or on the automation side for which NoCodeBDD can’t be used. The following article shows where NoCodeBDD isn’t applicable. 

https://blog.nocodebdd.com/when-not-to-use-nocodebdd-or-in-generate-test-automation-tools/

Supports Varied Tech Stacks

Using NoCodeBDD, your Testers can focus on automating business scenarios rather than spending time on understanding the tech stack—or worse, not automating due to a mismatch of skills. At the time of writing this article, NoCodeBDD supports: 

• UI 
• Web (No Selenium knowledge required) 
• Mobile (No Appium knowledge required) 
• REST API (No API coding knowledge required) 
• Relational Database (Basic SQL is required but all other complexities like setting up DB connection, connection pool, and asserting values from DB are covered by NoCodeBDD) 
• MQ (No JMS MQ coding skills required) 
• No-SQL (Automate NoSQL database without having to write any NoSQL code. Provide the required data or document, and NoCodeBDD takes care of automation) 

We are in the process of integrating the following: 

• Kafka 
• Support for SalesForce 
• Support for AMQP 
• Desktop applications 

Even for complex scenarios that our modules can’t support, you can write custom code and integrate it with any of the above modules. The following video shows how a custom code module is used in conjunction with other modules in NoCodeBDD: 

https://nocodebdd.live/custom-code-purpose

Empowering Resources

Quality should be the team’s responsibility. Period. Not just tester’s responsibility. By using NoCodeBDD and taking the BDD approach, you can empower your team and avoid the blame game. The dev should have thought about the scenario, the tester should have tested a particular scenario, and the product owner should write requirements in detail. 

The tester’s responsibility is to automate the test and spend additional time doing some exploratory testing. The developers should make sure all derived scenarios work as expected before delivering the code. By agreeing on scenarios upfront—and by automating them via NoCodeBDD to make sure the implemented solution works as expected—the entire team becomes responsible for the delivery. Thus, the quality of the product improves significantly via this empowered team. 

Reduce Automation Complexity

NoCodeBDD is built by industry experts with more than 50 years of combined experience in software development. NoCodeBDD is built using this collective experience and feedback from various clients, as well as work on various complex projects. As a result, NoCodeBDD has built-in best practices such as avoiding hardcoded values and handling multiple environments, reusability, easy integration with third parties, and collaboration with Git and Jira through the click of a button, various reports, etc. 

Reduce Maintenance

With NoCodeBDD, there is little or no code to maintain. All you have to do is document your scenarios and provide some information about your application. NoCodeBDD takes care of the automation. 

NoCodeBDD has a Free version, which will always remain free. You can download NoCodeBD from www.nocodebdd.com/download