Bug Chasing - How To Improve QA
Some Tips on How to Improve the Project's QA Phase
Hey, fellow Leader 🚀,
I am Artur and welcome to my weekly newsletter. I am focusing on topics like Project Management, Innovation, Leadership, and a bit of Entrepreneurship. I am always open to suggestions for new topics. Feel free to reach me on Substack and share my newsletter if it helps you in any way.
This article will go through some tips on how to improve the project’s QA practices. This is always a popular topic, especially on critical projects and critical application maintenance. The funny part about the tips below is there is not much on the testing phase itself, but rather a series of proposals that strengthen the software quality that reaches the QA Engineers. However, the tips below take into assumption the project has already some sort of UAT (User Acceptance Tests) or testing scheduled on the life-cycle. The “Let’s Do It Live” mentality is for another article.
Typically there is a big focus and pressure on QA teams to avoid those bugs, and typically a series of Test Books are created in parallel with the Software’s Development phase. Both Waterfall and Agile practices require dedicated testing periods and Project Managers tend to lean into QA teams to ensure the Software is on Scope and in the desired levels of Quality. Especially to avoid that bug to see the light of day in Production.
Unfortunately, more often than not, the bug is discovered in Product which can trigger a series of impacts. To note, bugs will always be found on Production. Is inevitable. The Manager’s job is to mitigate and risk of the occurrence of those bugs, setting in place a series of methodologies and practices.
Start with DevOps
Yes, Quality Assurance starts with DevOps. In today’s day and age is important to invest in process automation. Long ago, deployments were done manually and errors were brought into systems which lost so much Engineers’ time and effort. Not only does it mitigate the risks of human error, but also the philosophy of DevOps encompasses a series of systems and metrics that can mitigate the occurrence of bugs.
Modern DevOps strategy involves Code Reviews and Analysis which are the first line of defense. Code Reviews can share knowledge among peers, as well ensuring the team’s best practices are in place. Is important to note that code reviews will not test the features, but are a comprehensive 4-eyes-check by another developer and contribute to detecting some practices that might trigger future errors and ease maintenance.
In conjunction with Code Reviews, there are multiple software which can analyze code and detect code smells, vulnerabilities, and other important practices that might be skipped by the human eye.
For these reasons, DevOps is not only the automatization of a deployment procedure but also a set of measures that contribute to the project’s overall quality.
Automate Those Tests
It is astonishing the number of projects which don’t have one automatic test. Not investing in automatic tests (Unit or UI testing), will come at the expense of QA effort and errors in Production.
The creation of automatic tests allows the coverage and run of multiple tests in a matter of a few minutes, which otherwise could take hours, if not days, to cover them manually. Having a comprehensive Testing Strategy is highly important, for understanding how the features will be tested, not only through their “Happy paths” but trying to actively detect bugs or inconsistencies. Building automatic tests that cover a set of multiple scenarios ensures that every new feature will be tested with those test steps, which would consume too much time and effort to do it manually.
These tests can also provide important metrics in combination with DevOps, especially the percentage of code coverage, which can provide an idea of how much testing is already done on the software.
However, be mindful that there is a sunk and maintenance costs related to automatic tests. The level of investment in these tests is directly connected with the level of the project’s concern and allocated budget for testing. Good-designed tests are less expensive, however, there is always a trade-off to be made: The tests are all done manually and cover less scope with a higher risk of Production errors, or there is an investment in automatic tests which raise the level of confidence and overall quality.
Quality Gates
The problem of improving the QA phase is that a series of steps are needed even before the testing phase e is launched. Until now, all the tips described in the article involve automation and little classic QA measures. This is due to Quality being interconnected between the project’s technical and human assets.
To implement Quality Gates in a profitable manner, the project needs to have a comprehensive strategy involving DevOps (automated deploys, code reviews, code analysis) with Automatic Testing.
Before a piece of code is deployed in any environment, is important to determine which are the project’s quality gates. These are the metrics and thresholds that the project considers the code has enough quality to proceed to the next stage. For example, the most notable practices are:
Defining a percentage of code that needs to be run by automatic testing;
Ensuring that one peer analyzed the new code and gave its approval, enforcing the respect of the team’s practices and standards.
No increase of code smells or other findings, etc.
Is up to the project to decide which of these metrics should be applied and safeguard that all the code deployed is aligned with the standards.
Lessons Learned Tests
In an old project, we detected a faulty code that have passed both Development and QA teams. A bug that only occurred in a particular situation, in a particular environment, but with high consequences for the organization. A particular combination of hell, that made the application offline for days in Production. The main cause was a security patch done in the application that was tested by the Developer and the QA team, but unfortunately, one critical step was not covered during those tests. The reason why this bug was not detected before pushing into Production, is another article on its own, but was related to the testing environment setup and the QA colleague didn’t execute a set of tests in a specific context. The main message I wanted to take from here is that, ever since we fixed that problem, we added new tests in our Test Base which covered the steps to detect and avoid that exact incident.
The more the application is used in Production, the more bugs arrive to the Development team for fixing. For that reason, to mitigate future occurrences of old bugs, is important to create “Lessons Learned Tests” as part of the regression tests pack. This provides added security and confidence for the people in the project, more robustness to the QA phase, and previous important errors are continuously checked before pushing the code into Production. Of course, extra points if those tests are automated.
Continuous Feedback
In any project is important to make retrospectives of how things were done and implemented in the past. Providing new insights for the future. There is great value in organizing post-mortem or retrospective meetings to understand how the team could have prevented a bug from appearing in QA or Production, which were the missing parts, etc.
A popular feedback I see from the Developers front is the usage of Test Driven Development. I might create an article about this topic, however, the adherence to TDD is more on the Developer side than actually the project. If the project offers DevOps with automated unit tests, is up to the developer to create the tests beforehand. I am highlighting this fact because is a popular topic brought by technical teams which is entirely on their scope of action, but it seems to be below expectation in terms of execution.
One final note is to avoid creating tests just for the sake of creating tests. This is a consequence of chasing that 1% rise in the code coverage. Any test should be aligned with a Testing Strategy. If a feature is developed in an already existing code base, the team needs to address the change in the proper test. The goal is to avoid having a multitude of tests and doing so many things, and the developer is lost as to what is the test’s objective in the first place. The tests should have a focus on quality, not quantity.
That’s it. If you find this post useful please share it with your friends or colleagues who might be interested in this topic. If you would like to see a different angle, suggest in the comments or send me a message on Substack.
Cheers,
Artur