For a very long time now, AGILE software development teams have been used to the concept of test-driven development (TDD). Admittedly, it's a custom often more honoured in the breach than strictly adhered to. That said, TDD - namely the writing of the tests before or at least in tandem with the application code - together with its more business oriented cousin Behaviour Driven Development (BDD), are at least now almost universally acknowledged as practises to be aspired to in order to write better shipped product, less buggy application code and create good engineering design.
The major problem with adopting TDD is always that it is utterly counter-intuitive - the concept of trying to test something before you build it sounds at first ridiculous. To the human brain, the older Waterfall method of develop-then-test sounds far more sensible.
This natural bias, I would argue, still actually affects the productivity of development teams in another profound but rarely understood way. Namely that whatever, albeit patchy, progress we have made concerning the adoption of TDD practises, we still have a waterfall mindset when it comes to team building and recruitment. After all, there is no point in getting testers on board until there is something to test, right?
Wrong. Very wrong.
In every single AGILE team I have ever belonged to (just in case any of my ex-colleagues are reading this, I am not singling out any of you specifically - it really is a universal issue), my fellow testers and I have been late to the onboarding party. The developers/engineers, scrum masters and product owners are already well embedded, with usually an overworked DevOps engineer from a central team at least theoretically on call. (WebOps culture is another AGILE practise paid lip-service rather than adhered to, but that is a discussion for another day.)
Testers, however, are usually brought on board when the initial development stories have already shipped or are, if he or she is lucky, in flight. By then things are already far too late.
To understand this, consider the list of tasks and functions a senior QA will be expected to do as part of their job description.
- Write tests.
- Create a test strategy.
- Build automation frameworks.
- Help DevOps construct and maintain CICD pipelines running automated test regression suites.
- Create test and build reports.
- Backfill test coverage on pre-existing parts of the application.
- Help with AGILE transformation of the team towards TDD/BDD practises.
Enter the Janus-Faced QA
Of these, I would argue that AGILE transformation is easily the most value-added task a good QA can bring to the team. The task of embedding quality assurance into every part of an AGILE team's processes takes time and a great deal of intense effort. Writing tests is only part of the story: the QA must sit down and spend considerable time liasing with business analysts, product owners and engineers as part of story analysis (the '3-amigos' process) and actively pairing with engineers during, not after, application code development within sprints. Very importantly, the QA must win the respect of the engineers if he is going to bring them along with him on the BDD/TDD journey. Even the most open-minded engineers tend to find TDD an onerous and frustrating practise when they first encounter it, and the QA will need to spend considerable time charming, cajoling and very often firmly enforcing good behaviour. Added to that is the fact that the ratio of QAs to developers is almost always less than 1 and in most teams no better than 1:3, meaning the QA is already stretched on this function alone.
Now look again at the list of other tasks that a QA must do, in particular test and build automation. Test automation is pretty much de rigeur these days and usually requires the set up of a good deal of bespoke framework code, helper code classes and of course the tests themselves. The framework and the tests they run must then be deployable too, runnable in and reportable on a number of environments within the CICD stack as application code makes its way to Production.
In short, the test framework is an application of itself, and in modern development teams it will be required to have been written to application engineering standards.
So our hapless QA enters the scene, and is immediately tasked with testing the application stories already in flight and in many cases being expected to backfill automated tests for parts of the application written before he came on board. Only problem is that no automation or deployment framework for writing and deploying the tests exists! His only course of action is to manually test the stories in flight while he frantically tries to set up the test framework, therefore splitting his time between what the team would like him to be working on - sprint tickets - and what he needs to do to get automation and CICD up and running.
The QA is therefore now Janus-faced (after the two-faced Roman god pictured above), forced to continually context switch between very different tasks. Context switching is in itself inefficient and tends to result in doing two jobs badly, and even in the best-case scenario the QA simply does not have the bandwidth to devote to testing sprint tickets with the depth required.
Red Builds in the Morning....Red with Anger Developers Warning
Quality of course suffers and bugs start appearing in shipped code, much to understandable annoyance of the wider team, particularly embarrassed developers, who want to know why the QA isn't doing their job and code quality remains poor. Furthermore, with every sprint that passes a backlog of tests to be automated builds up, meaning that even after the automation framework is deployable and ready to roll - and this usually takes a good few sprints of work - their focus is still split on test sprint tickets and backfilling automation on previously manually tested tickets.
Moreover, with all this catchup work still to do, lack of time to maintain CICD and repair failing tests in the now automated build meaning the build status in the pipeline is often more red than green. Consequently, build status starts to be routinely ignored and flaky tests sometimes even commented out by engineers. In the worst case scenario, I have even known engineers demand scrapping of the automated tests entirely.
The end result is that most AGILE teams and in particular their program managers are deeply disappointed in the lack of value test automation seems to bring. In practise automation is still very much worth doing, but it takes a long time to get the value promised from this expensive undertaking.
AGILE Transformation (lack of)
The problems of bringing on testers too late in the day doesn't end with test automation: the impact on what I have argued is the primary value-added function of a good QA - AGILE transformation - is potentially catastrophic. Having concentrated setting up automation frameworks and build pipelines for their first few sprints, QAs have now spent a lot of their time siloed from the rest of the team, unable to devote their full attention to the intensive task of AGILE Transformation. In the meantime, developers have got used to a sub-optimal ways of working with a lapse back to Waterfall a very common outcome. Moreover, by now QAs may have even lost the respect of engineers, meaning they are more reluctant than before to change their now established working patterns.
Solutions:
1. The Case For Test Driven Recruitment
After depressing you, dear reader, with this tale of woe that to a greater or lesser degree I imagine you are all too familiar with, let me put forward some easy solutions that I think will greatly enhance the happiness, productivity and most importantly of all shipped product quality of your application team.
As I have argued, all of these issues are brought about by bringing on testers too late in day. The answer is to bring in testers at the very least in parallel with - maybe in some cases before - engineers and other members of the team. As this blog has shown, there is a large number of tasks that need doing before any application code is shipped that QAs should be heavily involved in - namely setting up test framework code, liasing with DevOps on the construction of CICD pipelines and so on. Remember also that Quality Assurance is ultimately a business-oriented discipline: before the application starts to be developed, QAs should also be forming relationships stakeholders, business analysts and product owners and helping to analyse and understand the business goals that the application you are building is meant to bring about.
2. Treat Your Test Framework as Application Code.
The test framework should be treated as an integral part of, not separate from, the application itself, and built to the same engineering standards. Remember that a good QA, should they really wish to transform the team towards being quality-centric in everything they do, will look to creating a sense of ownership around the tests within the engineering team. An obvious way of doing so from the outset is sharing the construction of the test framework with the engineers. More usually, this is left to the sole responsibility of the QA which means there is often a jarring disconnect between how the test code and application is constructed as well as the standards of build. Unsurprisingly, this means developers are more reluctant to take ownership of writing and maintaining test code later on. In the absolute worst case scenario, application and test code are written in entirely different languages, which is a thing never to do, as it means developers looking to write or maintain tests will have to constantly switch context between languages. Going down this path immediately guarantees defeat for any serious AGILE transformation process, as developers will simply refuse to maintain the test code base as time goes on.
Instead QAs brought on at a sensible early juncture can bond with engineers by building the test framework as a shared endeavour in the early days of the project, thereby creating both a sense of co-ownership and releasing the QAs to concentrate on in-sprint testing and AGILE transformation once the rubber hits the road and application code starts to be shipped (hows that for a mixed transportation metaphor?)
3. Roll On Early, Roll Off Early
With everything they have to set up and do, the job of a QA is actually front-loaded, with workloads getting progressively lighter over time: a good AGILE Transformation QA will have the same goal as a good AGILE Transformation Scrum Master, namely to make themselves progressively redundant from the team. Whereas a Scrum Master task is to enforce good process and leave the team a self-organising unit, the role of the AGILE Transformation QA is to create a culture of quality-centrism, with ultimately the developers responsible for owning, building and maintaining the tests themselves.
Therefore, although there is of course an additional upfront cost to bringing on QAs earlier in the team-building process (though as I have argued it is worth it alone in terms of better Software Development Life Cycle outcomes) remember that you should be looking to roll off your QAs prior to the end of the project, so hire QAs who take their role as AGILE transformers very seriously and watch their progress towards this ultimate goal.