Search by labels

Sunday, January 5, 2020

The Testing Pyramid

Ever worked in a company where developers wouldn't write unit tests for the code they implemented, where there was little to no automated tests for the back-end part of the application, where the majority of automated tests were focused on the front-end part and there was a huge volume of manual repetitive testing?
If your answer is yes, then you worked with an ice-cream cone testing anti-pattern:
This is very inefficient (d'oh!?😃) and unusable in the CI/CD world of today.
What can be done? Keep the order and change the proportions, shape it into a pyramid:
This pyramid testing model is most efficient because it saves time and money, the more you go to the top of the pyramid - the more time consuming and expensive it is to perform and maintain the tests, while the more you go to the bottom of the pyramid - the more faster to execute the tests and the cheaper to maintain them it is.
Basically, the test categories in the pyramid can be grouped into two major sections - Business Facing Tests and Technology Facing Tests:
Why is it important to have all types of testing?
  • Automated Unit Tests → these should be implemented by developers and can discover problems very fast and early in the development process. This interesting article from Martin Fowler's website about Apple's famous "goto fail" bug and OpenSSL’s “Heartbleed” bug makes more than a compelling case about their importance.
"Hand washing alone does not a doctor make, nor a patient save, but we would not trust a doctor who didn’t wash his or her hands. As software developers, we should assume a similar duty of care on behalf of our users. No one should trust software developed without unit tests."
  • Automated Back-end Tests → if the application's API is sufficiently documented, QA engineers can completely handle this point. From my experience, SwaggerUI is the bee's knees and makes an exceptional job in exposing and documenting APIs. Two reliable and efficient solutions can be used for Back-end test automation: Java with the RestAssured library or Apache JMeter, with the second solution bringing the advantages of having an easily understandable user interface and the ability to switch from functional to performance testing on hundreds of threads simultaneously with the flick of a button.
  • Automated Front-end/End-to-end Tests → these should cover the main use cases of the application with the smaller edge cases already being covered by the two above points. The standard and widely accepted solution to use here is Selenium WebDriver with Java and Serenity BDD for easily understandable test definition and reporting (Gherkin's Given-When-Then).
  • Manual Exploratory Testing → because Quality Assurance is much more than just test automation, it also involves critical thinking, observation and experimentation - human skills that technology cannot yet bring to the table (AI is a hot topic also in this field so in the upcoming decade things might change). I recommend this article from Bas Dijkstra which in my opinion debunks the "silver bullet" myth of test automation.
"If you're a tester and all you do is execute and check off scenarios that were defined by someone else and yell 'pass' or 'fail' at the end of each scenario, then you might indeed be in trouble. But as we all know, that's not what proper testing looks like. Testing encompasses many other activities, and most of them cannot be automated. Risk analysis, communication and mitigation. Determining whether the product provides value to people who matter (the definition of 'quality' as it is given by James Bach and Michael Bolton, based on an earlier definition by Jerry Weinberg). Critical thinking. And the list goes on. All part of testing, all inautomatable."

No comments:

Post a Comment