Automation vs Manual Testing: The Great War

There is a great battle currently being fought on the QA battle fields.  In a popular whatsapp group of QA managers and team leaders there was a discussion about automation and manual.  My favorite part of the discussion went something like this: "All the big hi tech companies are claiming to be using only automation developers in left-shift development to write tests, and don't use manual at all, but I used to work at some of them, and I know for a fact that they contract manual testers as well - they just aren't on the same payroll."
I love it.  Let's say we are doing automation only testing, but still we'll bring in some manual testers through the back door so no one sees.

How much automation should we be doing?

Manual testing is expensive, and short term, we should be investing in automation.

Developers can write tests just as well as QA.

I've heard it all, and seen it all.  Recently I met with someone who was looking for a QA manager, and his test for interviews involved complicated programming questions.  Nowhere during the meeting was there a discussion about test strategy, test planning, or what a test case should include.  So many people have become so involved in the need to develop automation, they are forgetting to ask the question: What is a test case?  There was such a strong focus on the coding aspect of writing automation, that they have already lost the focus to provide "testing".

Paul Grizzaffi over at Responsible Automation gave an interesting session at Test Bash SF discussing the importance of viewing your automated test cases as proper production code.  Its not just a test case to verify a feature, but in fact needs to be a robust, scalable code that can be molded, improved and expanded.  It should be built using basic and good programming principles like architecture, maintainability and code review.  Treat your automation the same way you treat your production code.

At the same time, I wish to add to his suggestions.  It also needs to be a proper test case.  One of the issues with Shift-left development, where developers are writing automated test cases is that they don't plan out the test case.  They think in terms of unit tests.  When writing test cases, you need to consider the whole picture of the item being testing.

- what is the goal of this feature?

- where else does this feature effect?

- is the design of the feature correct?

- what is the test data needed to test this feature correctly?

Automation is extremely important, and its critical in our fast moving world.  Automation management has proven we can keep our product strong and verify changes don't break what we've already created.  However, manual testing reaches corners we never can.  As I pointed out in my last post, a manual tester will always look at a test case slightly different, something a machine code never can.  In my opinion, each new feature should be tested by a manual tester and then passed to automation.  

I know, I know, I hear some of you screaming at me.  "Its not necessary" "its a waste of time" "you are adding expensive resources when developers can create the same tests"

In the end, the economic decisions are what's appropriate for each company.  But I stand by my recommendation: manual testers who write test cases, and run the tests manually, think differently than an automation engineer.  They consider the whole picture of the product and look at it from a bird's eye view.  They can change the data easily in each execution, and shift into exploratory testing at no cost.

tl;dr: Automation vs Manual is a constant battle of resources vs finances vs needs.  But don't forget that your tests - automated or manual, need to have the right focus and design.  Additionally, you will always maintain a higher level of quality if you maintain manual testers.