Categories: Uncategorized

by Jerome Josephraj

Share

A lot of debate surrounds whether behavior-driven development (BDD) is good for UI Automation. In our experience, using BDD only for UI Automation has a few issues: 

In this article, let’s look at these issues in detail and discuss an alternative that works fantastically well for many of our clients—something that you could easily adopt. 

Definition of BDD?

According to Wikipedia, BDD is 

an agile software development process that encourages collaboration among developers, quality assurance testers, and customer representatives in a software project.[1][2][3] It encourages teams to use conversation and concrete examples to formalize a shared understanding of how the application should behave. 

BDD Definition 

The key, operative words in the definition of BDD are collaboration and concrete examples to develop a shared understanding of how the application should behave

This is not only critical for any team rolling out BDD in their project, but also for any team delivering high-quality software. This article covers more about BDD collaboration and an awesome technique called OOPSI. 

Let’s pin the above definition, as it forms the basis of any discussion around automating at the UI layer. Before we look at how automating at the UI layer is not the right approach, let’s look at what it means to automate at the UI layer. 

What is Automating BDD at the UI layer?

Simply put, automating BDD at the UI layer involves BDD scenarios using automated scripts in Selenium, Cypress, Appium (for mobile) or using scriptless tools like NoCodeBDD. The following video shows how this is done using NoCodeBDD in minutes, without having to write any code: 

For privacy reasons YouTube needs your permission to be loaded. For more details, please see our Footer.
I Accept
Automating BDD UI using NoCodeBDD

Issues in Automating BDD at the UI layer

When BDD automation—or any automation, using BDD or not—is done at the UI layer, it could suffer from several, if not all, the following issues: 

  • Brittle 
  • Flaky 
  • Slow 
  • Not scalable 
  • Doesn’t test the behavior of your application 
  • Paves the way to imperative BDD steps, which are similar to manual test scripts and don’t bode well for automation 

Let’s look at these issues in detail in the below sections. 

Brittle

When BDD scenarios are automated for the UI layer, particularly if they are automated using XPath, they are almost certain to fail intermittently. These XPaths can change for various reasons: 

  • Page content has changed (e.g., static text gets included at the top of the page, which would change XPath for an element automation script is referring to) 
  • Each browser might have different XPath values (particularly when it comes to IE) 
  • Each browser resolution might have a different XPath 

Even if you use IDs (i.e., element IDs), your UI layer is likely to change more often than the application system. As and when the UI changes, your BDD will break and will embrittle the entire BDD automation. 

Flaky

Flaky means that the tests fail intermittently. For example, when you run your BDD automation suite, it will show all scenarios have passed. But when you run the same set of tests without any changes to your application or BDD, it might fail. In other words, your BDD automation suite is not reliable: it’ll pass sometimes and fail other times, even if there are no changes. 

This is most certainly the top reason why you should avoid automating BDD—or, for that matter, any automation—at the UI layer level. There could be several reasons why your BDD Automation would be flaky: 

  • Network speed – You might have written a script to check for an element, and it might pass on your machine. But when it is run on a different machine due to slowness, the UI element might take time to load; as a result, your test will fail. There are few techniques to handle this, but this has proven to be one of the contributors to flaky tests. 
  • Asynchronous calls – Your UI might make an Ajax call, and depending on the result from the Ajax call, it might take some action. When you automate for this scenario, your test might end up working intermittently because there is no guarantee of when the Ajax call might return. 
  • Dynamic Elements – These elements’ attributes change when the page is reloaded, even though the page content remains the same. These are difficult to handle and will make your test flakier. 
  • Device configuration – This particularly applies to mobile devices, as there could be many devices with an umpteenth number of configurations. Your UI might render completely differently in each version of the device; as a result, your UI automation might work in one version and not in another. 

Slow

When your BDD is automated for the UI Layer, it is more likely to be slow. Since there could be several layers the UI has to traverse through, it would take that much longer to get the result when your automation scripts interact with UI. Thus, it affects the overall automation running time tremendously. 

Over time, your entire automation suite becomes so slow as to become unusable. Agile is all about getting fast feedback. When your automation suite doesn’t provide this, the team becomes less agile. As a result, your time to market increases. 

Automation Engineer life

Not scalable

As more and more scenarios focused on UI automation get added to your BDD Automation suite, the combination of brittleness, flakiness, and sluggishness makes it less scalable. 

When the team has only a few scenarios, UI automation might seem to work. Yet inevitably, the team will add more scenarios, and slowly all the issues discussed above will start seeping in. This all causes the automation suite to become less reliable and fail more often than not for issues that are not related to functionality. 

Not Testing Behavior

Software products are always delivered to a user. You might write a backend software or a software that sits behind several other software—but at end of the day, your software is always used by a human. Therefore, it’s critical to understand what the user wants and what behavior software should provide to satisfy the user’s need. 

Let’s put it simply: if a team needs to deliver a high-quality product, the first and foremost is they need to understand the requirements well. That’s the starting point. Get that wrong, and it doesn’t matter how swanky your technical architecture is, whether you have 100% automation, or tech superstars—everything would go wrong. They’ll end up delivering a product that works fantastically well—while delivering upon the wrong requirements. 

Requirements mis-management

How does this impact automating at the UI level?

When the team focuses on BDD automation at the UI level, they tend to focus on the website rather than the behavior of the application. In other words, they tend to lose track of the big picture and focus on the web elements. 

Thus, the team may end up writing imperative scenarios, which are explained in the next section, rather than declarative. Declarative scenarios tend to focus more on the behavior and exactly what the customer wants. 

Imperative Scenarios

Imperative scenarios are those that explain every action but not the behavior. An example of an imperative scenario is given below: 

Imperative Scenarios

For an end user, these steps don’t mean much. In other words, when a customer gives a requirement for a registration page, they don’t give something like: 

  • User enters first name 
  • User enters last name 
  • User selects age 
  • User presses submit button 

Customers give requirements from a behavior perspective something like: 

  • As a user, when I enter all mandatory fields on the registration page, I should be registered in the system and a dashboard should be shown. Mandatory fields are First name, last name, and age. 

When the scenarios are defined as imperative, the team loses sight of actual requirements and ends up focusing on the details. As a result, they might end up delivering/testing invalid requirements, as the actual requirement is lost in the detail. 

Conversely, declarative scenarios explain the behavior—not the steps. The following screen shows how the previous written imperative scenario is shown as a declarative scenario: 

Declarative Scenario

Imperative scenarios are also likely to change more often than declarative scenarios, which focus on the actual behavior. For instance, if capturing first name and last name is changed to capture just name, but in the background you still split the name and send it as first name and last name to the server-side, you end up changing your BDD scenarios—even though the behavior of the application hasn’t changed. 

Despite all the aforementioned issues, it is important to have BDD scenarios at the UI layer level. Let’s look at why we might require scenarios at the UI layer level. 

Why BDD Automation is required at the UI Layer level

When your application is quite heavily based on UI, it is important that you have adequate scenarios to cover that. Ignoring the UI layer and moving everything to the API layer might not work, as some of the necessary behavior might not get tested. 

User Registration Form

If your automation script focuses on API, then the automation script would pass all the fields that are required for the API to work and check if it gets stored successfully in the DB. 

However, let’s assume the UI is developed in such a way that, when the user clicks the student or alumni radio button, it passes only the “student” value to API, even if the user selects Alumni; then the DB will always store the alumni type as student. 

Obviously, storing alumni as students should be checked, and it should fail. Yet if the BDD scenario actually tests the UI layer, your script would actually select alumni and your scenario would fail: you’d be checking for alumni, but DB the record would have stored that as student. Because your UI to API mapping is wrong, all records are stored as student. So, checking the UI layer is extremely important in ensuring that all scenarios are covered properly and that your application is behaving as intended. 

So how do you go about testing a UI layer without introducing all the issues mentioned above? The following section shows how you can use a technique that allows you to focus on UI scenarios while at the same time reducing the issues related to UI automation. This technique has worked fantastically for many of my own projects and those of many of our clients. 

Solution; Use a Combination of API and UI

One of the best solutions that I have implemented in many large projects with thousands of scenarios is to effectively use a combination of UI scenarios and API scenarios. Using this technique, you basically split the scenarios for a particular functionality to check it using UI and via API. 

The splitting of scenarios should be done in such a way that the UI scenario focuses up to the point where all the elements are covered; API scenarios should focus much more in-depth on the functionality. 

Example

Let’s look at an example of how this could be applied in a real-world application. The application for which the BDD automation is written has a UI, which looks something like this: 

User Registration Form

The API takes all the following fields and stores it in DB. You could split the scenario as follows: 

The first two scenarios focus on UI. The first scenario checks if all entered information on the UI works as expected. In this case, it checks whether it gets saved in DB successfully. The second scenario checks whether all fields are validated successfully, and whether the right error messages are shown. These two scenarios should take care of all UI-related checks and make sure that the handshake between UI and API is working as expected (i.e., all data from the UI are transferred to the API and are stored in DB successfully; and all error messages are shown correctly at the UI level). 

The next four scenarios handle all API-related scenarios, which is where the core business logic gets checked. Different behaviors and rules of your application can be checked in these scenarios and can be applied at the API level. 

This approach efficiently shifts all heavy work—checking different application behavior and rules—to the API layer and keeps the scenarios to a minimum at the UI layer. 

By combining the UI and API layers, you can achieve the best of both worlds: testing the UI layer and at the same time reducing the issues of automating the UI layer for all the BDD scenarios. 

Using NoCodeBDD, both UI and API automation can be achieved in minutes. Download a free version of NoCodeBDD from https://www.nocodebdd.com/download 

STAY IN THE LOOP

Subscribe to our free newsletter.

Related Posts

View all

  • Continue reading
  • With NoCodeBDD, teams can automate Behavior-Driven Development (BDD) Scenarios for mobile applications with few clicks. To create test automation for mobile scenarios, typically, one would use Appium. Using NoCodeBDD, all you have to do is: Following gif shows how this is done for a sample mobile application: NoCodeBDD Mobile Automation NoCodeBDD […]

    Continue reading
  • if("student".equals(userType)){ Assert.assertEquals("STUDENT REGISTERED SUCCESSFULLY",actualMsg); }else{ Assert.assertEquals("ALUMNI REGISTERED SUCCESSFULLY",actualMsg); } When you have test code written like that given above—or. in other words, the code is written to assert an outcome based on conditional logic—it won’t test the scenario as it should. When writing test automation code for behavior-driven development (BDD) […]

    Continue reading
  • There is a general mis-conception that BDD should be used for testing. Many projects consider using BDD during “Testing phase” and only the QA team uses BDD. When BDD is rolled out this way, the teams don’t see the benefits. In this short video, I have explained how Behavior Driven […]

    Continue reading