5 Ways To Improve Regression Testing

 How often does your team run regression cycles?  Is it every week?  Perhaps at the end of every sprint?  At one time, we were testing every other sprint.  The cadence for testing cycles is best defined by the business needs, and there is not one general rule for when it should be.  However, testing regression cycles can become mundane and prone to the Pesticide Paradox.  The best way to test effectively is to keep tests dynamic, keep testers interested and keep testing in new and different ways.  Here are 5 ways to keep your regression cycles strong and ensure high quality testing and high quality bug identification.

    1. Integration

There are multiple levels of testing including component, integration, system and acceptance testing.  When running Regression Cycles its extremely important get the tests as close to production as possible.  This means when you can test with integrations, do so.  Perhaps your product is Fintech and has integrations with several acquirers.  Can you use the production level acquirers when testing?  If so, do so.  If not, how close can you simulate those integrations?

Perhaps your product is cyber, and you are testing which integrations create suspicious events.  Add those integrations in and make them trigger.

Or perhaps you are finally putting components together in a system environment and now you are seeing how the gears function with each other.  

Whatever the case may be, integrating the parts together in the machine, will significantly strengthen the outcomes of your tests and find many more significant bugs.

    2. Hardenings

A sprint hardening is defined by the Scrum Dictionary (yes, there is such a thing!) as a specialized sprint dedicated to stabilizing the code base so that it is robust enough for release.  Or in otherwards, what I like to think of as "Everyone Freeze, and fix/clean your code".  It can be used as an opportunity to clean up old bugs, to clean up legacy code, or refactor code that's been sitting around for a long time, waiting for an opportunity to meet the modern world.  In my experience, hardenings are rare, because it means less new features.  But I have also learned that hardenings are extremely valuable for improving quality and bringing code up to par.  Having hardenings, either as their own cycle or part of a regression cycle, helps weed out the old and legacy bugs, and keeps your product fresh and strong.  

   3. Proper Automation Cycles

How do you use your automation suites?  I've seen automation suites be written, and run only during regression.  I've seen elegant test suites written, and lose to the fight of flaky tests and low pass rates.  I've also seen robust and high coverage automation suites created, but no one uses them.  There are several ways to use your automation suites, but the most important aspect of them is .... Use Them!  Have Dev run suites when they integrate code.  Incorporate the suites in gated check ins.  Make sure you have CI/CD automation suites running.  Make them nightly test schedules with a high pass rate and low flaky test rate.  Each business needs to decide what works best for them, but not using them properly, is just a waste of a valuable resource that can improve your testing dramatically.

  4. Metrics 

Preparing and reporting with proper metrics is one of the best ways to prove qualitative improvements to the product.  Showing trends over recent periods allows us to look back and retrospect to see if what we are doing is working.  In addition it provides a visibility to Management that they may not have otherwise.  Management need to push business along, thats where the money is.  But if they are not aware of their quality status, they may not know how or where to place more emphasis.  Keep management in the know, and provide proper metrics so you can mold and improve your tests to cover the correct business based goals.

   5. MBF Bugs

MBF, or Must Be Fixed Bugs is a list of bugs that are currently in the system, with a top level severity or priority.  That definition is a little esoteric, so lets take it down a notch.  MBF bugs are bugs that you need to fix.  Go to your tracker, and pull a list of the bugs that have a CRITICAL or BLOCKER or HIGH severity and make sure these bugs are prioritized to be fixed.  One way to do this is call a BUG TRIAGE with relevant teams, and go through each bug.  Another is to notify each of the relevant teams and put the bugs as a priority for them.  Whichever way your company works, the need to prioritize MBF bugs is a crucial method to improving the regression cycles.  You can't release a version, even if it has a high test pass rate and high test coverage rate if you have open critical bugs.  It goes hand in hand.  

In conclusion, a proper respect and set exit criteria of the Regression Cycles help make it a strong and successful cycle.  The goal is always the same: high quality release.  The question is how?  Follow these steps and you are much closer to ensuring your release comes with a higher standard of quality.