TDD vs BDD: Pros & Cons

Within the last 12 years, our company has successfully realized dozens of projects, both big and small. During this time, development process has significantly revolutionized. Five years ago, the workflow looked this way: we would write code first, and then, if we had enough time, we would create a set of unit tests for the existing code. Test coverage for the code used to be less than 50%. As we can see now, such an approach had a number of major deficiencies. Earlier, developers often failed to see the inconsistency of logic, and code that had errors in it was sent to the test server. It is worth mentioning that this low-quality code never made it to production since the lack of computer-aided testing was compensated with the high-quality work of the QA department. Code review was also of much help, as it was executed only by senior developers with the decades of experience in this area. Clients were always happy with the final product they saw on production, but in the process tickets often failed to meet the expectations of the QA department and were sent back to developers. Meanwhile, a new code requirement came into power. It stated that a high percentage of code has to be covered by unit tests. Taking everything mentioned above into account, several years ago our company started implementing a number of computer-aided testing techniques and utilities like Selenium. Also, we use continuous integration tools like Jenkins for almost all projects. In case the committed code did not come through the unit tests, it won’t be deployed on the test server, and all the developers will receive a corresponding report via email. So, let’s look closely into different methods of computer-aided testing, their pros and cons.

Unit testing and TDD Pros & Cons

Unit testing is a method used for testing separate code blocks (classes, components) logics which they base on when functioning. In reality, unit tests are run automatically and represent a small block of code for testing expected output accuracy from a single or a set of components. These components are tested separately from their dependencies in an isolation test: DB, storages, filing systems, networks, etc. Test Driven Development (TDD) is a development methodology based on writing small computer-aided tests for code (unit tests). As a result of using this methodology, we get a full set of unit tests that can be run at any time whenever we need to check whether the application code works correctly. TDD is used almost universally by companies that use Agile development methods. Note that in TDD we write the test first (and not the realization in code!) After the first launch, this test fails to meet all the technical requirements. Then a developer has to implement the minimum functionality in the code to ensure that the test is coming through successfully. After this, refactoring and code improvements take place.

The short motto of the Test Driven Development can sound as “Red-Green-Refactor”:

• «Red»: create a unit test and run it to see it fail
• «Green»: implement the logic in a code to complete the test
• «Refactor»: improve the code, to avoid duplications, improve the architecture in order for the test to be completed successfully

Together with tests written before the actual logic execution in code, TDD ensures the highest quality of a product and helps the whole dev team focus on creating a simple and understandable code. But since unit tests are focused on the inside concept in the application code, outside developers will find it hard to understand the concept behind the application. Moreover, new difficulties emerge with evaluation of the concept and code coverage as well as the quality of unit tests before integration testing. That is why, the entire responsibility for unit tests is not on QA dept, but on developers because unit tests handle low-level code blocks and require the knowledge of application software architecture. Besides, testers should not work with unit tests given the iterations between creating a test be a developer and implementation for its successful passing.

TDD cons

• some developers still view tests as a complete waste of time
• the necessity to create additional code in tests increases the time needed for development
• tests can be easily implemented in a wrong way, they will check the work of specific classes and their methods, but not the system in general

TDD pros

• a set of unit tests ensures constant feedback about the functioning of each and every system element
• unit tests are a part of a project and cannot get outdated unlike specific documentation that will be long-forgotten sooner or later
• TDD requires clear understanding of code functioning logic, since without a clear understanding of the expected results one cannot run the test
• project code quality grows, because a developer can refactor the code at any time and check the accuracy of its performance
• the number of tickets returned by the QA back to developers decreases, because a part of errors in code are checked by unit tests
• a set of tests functions as a safety net because during bugfix developers create tests to check will problems repeat or not. This way, the probability of emergence of similar bugs decreases

Finally, we cannot but mention two core practical methods that are applied in integration unit tests code – mocking and stubbing. If you look up these words in a dictionary, you will see that the noun «mock» means «made for imitation». Mocking is mostly used in unit tests because the object often has dependencies in the form of other complex objects. To isolate the behavior of a tested object, you can replace its dependencies with mocks that simulate the behavior of real dependencies. It is a useful technique since it is impractical to include the major part of real objects into the unit test. To put it short, mocking is a process of creation of objects that simulate the behavior of real object dependencies. Sometimes, we can view mocking as the opposite of stubbing. Stub is a stopper. Or in other words a “minimum” or empty simulated object, often having no logic or behavior. This means that stub is created to help the test run successfully. This allows developers to check whether the tested objects interact with the mock correctly or uses it.

Let’s look at an example:

We often need to create fake DB in unit tests to test an object that has DAO in dependencies. By creating a stub, we easily imitate DB since we create a structure of data to store data. Now we can easily create DAO in the form of a dependency to a tested object of the service that will store and remove entries from a fake in-memory database stub. This will allow us to run a test successfully. But before that, we cannot test the general behavior of a tested object, in case there are only 3 entries with a specific set of field values in DB. For these purposes, a regular stub is not enough, so we need to create a mock and add specific data into it. After this we will specify the assertions in a unit test to check service’s behavior in a test case with specific data collections. We should also point out that the fact that all the unit tests come through successfully does not mean that the app works well in general. To check the general work of application you need tests of a higher level (or integrational tests). Functional (or integrational) tests view the project’s logic as a single functional thread. They represent a comprehensive interaction of internal and external objects (components) aiming to achieve the expected behavior of a tested application. Functional tests are high-level tests, and if the code goes through them successfully, this means that an app functions well. Failure, in its turn, means that the code does not provide the asserted functionality.

Behavior Driven Development Pros & Cons

Behavior-Driven Development (BDD) is based on TDD, but TDD is focused on the internal processes of software and precision of code performance (unit tests), while BDD puts requirements and Business Value of software at the top of software priorities (acceptance tests). Meanwhile, acceptance tests are often modeled according to the User Stories and acceptance criteria. These tests are normally described in simple words so people from the outside of the IT industry (like shareholders, business analytics, QA engineers and Project Managers) understand them better. BDD makes sure that in the process of product development tests were created in the first place. These first tests must describe the expected functionality of a product and software behavior. Then, product functionality realization is executed in terms of these tests. It is very convenient, so we use BDD for acceptance testing. From this, we can assume that BDD and TDD complement each other, as they represent different approach to solve similar problems. As development management is accomplished through a test, and in the process each component goes “from red to green”, meaning that it fails at first (no functionality) but then comes through successfully (its functionality complies with a specification). Thanks to putting User Stories at the top of priorities while composing tests in BDD, the final result meets client’s expectations, for it is better to write down user’s behavior model, his actions and features he might need before starting the development. Our team uses Python Cucumber and Gherkin to write and execute our User Stories. Gherkin is a Business Readable, Domain Specific language. It can be used to describe user’s behavior pattern loud and clear by splitting it into many various scripts. Each script represents a separate user story. Each line in such a script is a requirement to the software that is mapped to the function that runs in the Python language. Thanks to such integration type, we can utilize BDD to check functional aspects of user’s experience with a browser application. For instance, we use Selenium Browser Automation Tool along with Python Bindings to execute our acceptance test for UI from Gherkin. Each User Story is checked in the latest version of an app inside the browser, and Python sends instructions to a browser for different user activity emulation (e.g., clicks), with further result checking of this action according to a set of criteria (Successful execution message, validation error, etc.) As we already mentioned above, BDD requires the creation of user’s actions script in the first place. That is why, first of all we create a script in which we describe examples of situations for each of our components. Creation of the software is carried out last of all.

Here’s how our development cycle looks like:

1. Write a script on Gherkin
2. Run the script and realize that nothing works properly

   2.1 Identify situations when this script must work

   2.2 Start checking these situations and realize that nothing functions well

   2.3 Identify and implement minimal functionality necessary for all the examples to come through the test

3. Execute everything one more time. In case of new errors, go back to point 2.1
4. The script works! Start creating a new script covering the major part of requirements

Dan Nort was the first one to spell the BDD approach claiming that this method is here to eliminate issues with TDD

BDD cons:

• requires a deep understanding of a larger number of concepts, that does not allow to recommend BDD to a junior developer before he completely understands TDD concept
• since it is a concept, turning it into a technical practice or connecting it to one set of tools means ruining it

BDD Pros

• business team and developers demonstrate real teamwork and let the right people discuss the right things at the right time
• makes the business justify the functionality priority as it shows its real value
• allows developers teams to focus on features prioritized by the business thanks to a better understanding
• developers will rarely fight with the business team to write some features before the other ones
• thanks to the language utilizing a common knowledge base, the business team and developers will work on one project and stay on the same page about it
• it helps you to focus on the user’s needs and expected behavior instead of diving into all the implementation details right away
• can help teams focus specifically on details of functionality and test things that are important instead of simply creating tests for the whole code
• requires constant growth in understanding the product requirements, it makes the development of constantly changing apps easier
• makes people work closely together especially when it comes to developers and members of business teams, that allows normalizing the level of problem area understanding and implementation accuracy
• does not let you get lost in the piles of outdated documentation and code
• teams are more confident in their work and tend to foresee its development

Conclusion

To wrap it up, we only wanted to add that the most important thing is to understand why and how to apply different methods and tools of computer-aided testing without writing tests for nothing. Proper implementation of computer-aided methods of testing and development based on running tests has a significant influence on the process of development in general and the quality of the final product.