The Magic of Exploratory Testing

 

Definition

Wikipedia has a definition of Exploratory testing that I like.  It says: 

    "A style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the quality of his/her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project."

Well, that's a mouthful.  See if you can say that ten times fast!  In simpler words, exploratory testing is running tests without the limitations of test steps or a test case, but rather the freedom of the tester to "explore" and to "play around".  

One of the questions I like to ask on interviews is about the mystery box.  Say I gave you a box.  I tell you it has electrical components but I dont explain what the box does.  What do you do with it?  Those who are natural testers, and those who enjoy testing will give many different examples of playing with the box, and opening it and testing to discover what it will do.  They may try to open it, or break it.  They may try to plug it in, or attach electricity to it.  They will essentially work with the box to "explore it" and try to figure out not just what its meant to do, but also what it can do.  Exploring the box allows the user to move beyond the limits of what they see, and try to discover what more it can do.  

Take a shoebox, for example.  You can put shoes in it.  You can use it for other storage.  And we can write test cases that will test all different kinds of storage options.  Maximum volume, and maximum weight.  Shoe limitations and how many pairs of shoes and what kinds.  Fine.  We can write the world's best test case for storage.  That's the feature and that's what product defined it as.  

Now lets explore:

Can we turn the box into a background for a play dough adventure?  Can we use the box as a boat on water?  Can we turn the box into a radio or a piggy bank?  Exploratory tests allow us to "think outside the box" (pun intended).

Why Its So Magical

The magic of exploratory is that it opens up a new layer, or (if I may be so bold) a new world of possibilities that perhaps no one considered, exactly because the tester is not limited to the expectations of product or business definitions. 

The point of exploratory testing stretches the borders of testing, and allows the user to consider different possibilities.  Like the shoe box above.  Or even for a web page.  Did you consider how the user can access this?  What happens if he clicks here, or does a right click there?  Can he save the image?  Do we want them to be able to save images?  There is a world of possibilities that can we can discover by just clicking around and seeing what can be done on the page in front of us.  Furthermore, by poking around, we can ask business logic questions like: "do we want the customer to be able to get here" or "how did I get to this page?  That's not supposed to happen!"

When Should We Use It?

But let's not fool ourselves.  Exploratory doesn't take the place of proper progression (new feature) testing or regression testing.  The best way to test is by using both test cases and exploratory to cover all your bases.  I usually ask my teams to run exploratory after the progression is completed.  This way we know the business logic is met, and now we can see what else can be done with it.  Priorities for fixing bugs that result in non-business testing features are often lower.  But explatory can also reveal important or critical errors that we didn't consider.  For example, perhaps the login feature is working perfectly, but when we use a different language or when we right click on Submit, the page crashes.  That's an example of something that perhaps could be discovered while playing around without specific steps, as opposed to sticking to the strict guidelines of a test case.

Acceptance tests is another area where I like to use these tests.  Checking to make sure everything is working properly in an efficient and concise testing manner.   We can also keep an eye out for any high priority issues.  At the same time, running exploratory on a newly released build or feature will help prove the quality of the drop.  If the drop has any major or critical bugs, a few random tests will likely pop them quickly.  I often do some exploring on a new build before I even let my team begin testing, so as not to waste anyone's time.

In conclusion, I highly recommend using exploratory testing as a means to strengthen the quality of your tests and bring out a higher level of expected results and user experience.  It can also be used to discover hidden features or hidden bugs, as well as testing the strength of a new build or release.  Its truly, a magical testing tool!  In Bill Watterson's immortal words, as he published his last Calvin and Hobbes strip.... LETS GO EXPLORING!