There have been a lot of blogs and commentary lately about the role (or not) of the Test Manager in Agile projects. This paper provides some ideas of how this role may look in the future within Agile.
“Why is a Test Manager required on an Agile project?” if there is a Scrum Master, Product Owner and Project Manager. Are there too many managers and not enough practitioners?
In a word YES there is still a role for the Test Manager (TM) but that role is different to the one that they may have on a traditional project. The good old ‘DEPENDS” affects how this role may look for individuals.
There are now really two roles (which can be combined into one or alternated between different individuals if the organisation has a number of TMs). Each of these roles requires different skills and competencies and some which overlap.
Within the Team
The first of these is as a member of an Agile team. This is a “hands-on” role where the TM immerses themselves within the team and contributes as a full time member. If this is the case there are a number of activities where their skills are invaluable.
Although an Agile project will have a number of testing activities, within an Agile Team the emphasis should be on the shared quality ownership. A number of articles recently stated that quality/testing activities are not necessarily automatically assigned to the “tester” in the team. However in reality and certainly in the early adoption of Agile, I am finding that the majority of the quality ownership still seems to be undertaken by test services resources.
Strategy and Planning
It is the team’s responsibility to set the strategy at the Release Planning meeting; however the TM within the team certainly would be leading the testing part. The most important part of building a strategy are the conversations and key decisions that are made. The Test Strategy need only be a couple of slides in a presentation or a one page Approach document summarising the discussion. As the whole team have all been involved there is no need to capture items such as roles and responsibilities, the defect process etc. but rather the focus should be on WHAT we are going to test.
The risk workshop is a key input to this and the TM is good facilitator of this activity. This is normally not a separate activity but as part of Release Planning at the feature level and can be further discussed at Iteration Planning as more detail is discovered when elaborating the story. This activity comprises risk identification, analysis and mitigation suggestions (often testing). One of the reasons that we are testing is to mitigate product risk and provide confidence in the system under test.
Once we have identified the risks and decided on the strategy we need to come up with a plan of how we are going to achieve this. Again it is the conversations about this that are important, led by the TM. Only key decisions need to be captured within the Test Plan. At the Release level the Test Plan will be very high level and may not be a separate document but as part of the team plan. It needs to be flexible, as detail is added at each Iteration Planning meeting.
Both the strategy and plans need to be living documents that are continuously reviewed. The Test Plan could be updated daily in line with comments within the daily standup. Although this seems like a large overhead when we are talking about being Agile, that does not necessarily mean documented. As long as the team has the same understanding then that is sufficient and only major changes in direction should be recorded. After all, adjustments are welcomed, as we learn more about the product.
Once we have decided on the plan we need to start iteration delivery by taking the testing tasks. The first of course is analysis of the requirements. Within an Agile project the team should have worked together to refine these (and this is done continuously) so the tester (in this case the TM as one of the resources identified as tester in this section) has a much clearer understanding of what the story (requirement) represents.
The tester should have already worked with the BA or Product Owner (PO) to ensure that the story is testable and in fact conforms to INVEST. They may also have helped and coached the PO to evolve the acceptance criteria.
Next would be to design the tests. Often within Agile projects session based testing is used, complimented with exploratory testing. The use of charters and test ideas rather than the more formal writing of test cases with test steps and associated expected outcomes, do require a greater understanding of the application under test and the proposed solution, as there will not be detailed documentation.
The TM may have to “dust off” their testing techniques knowledge as they become increasingly important in an Agile project, writing fewer tests which have maximum coverage. The TM needs to have a high understanding and propose the use of testing techniques during analysis and design of tests, to ensure acceptance and that the Definition of Done is reached for the iteration. This includes all parties that test (most of the team) so that the TM can coach in the correct use of these.
The important values for a TM working within the team are that they need to adapt to being a team member and not a manager. They need to refocus their mindset so that they do not continue with a “command and control” mentality. They need to be a leader and not a manager which is often difficult. Although they will have an excellent skillset in order to lead the testing and instil quality values within the whole team. They need to assume the role of advisor and not to give orders to the team or even the other testers.
The TM will have experience in estimation and be able to confidently articulate why they have broken down the stories into their constituent testing tasks and therefore arrived at the number of story points for poker planning. This does mean that the TM needs to drop into the detail rather than look at the big picture level that they may have been working at in traditional projects.
This activity also requires assessing the technical difficulty of implementation of user stories during the planning/estimation period as the estimate is no longer ones of separate ones for each discipline. The team does the estimation together and includes all activities when they give these.
Encouragement and providing enablement for the team to hit tasks within an iteration are also areas that the TM will have the skills to perform. This involves understanding the quality attributes from discussion with the Product Owner and review of the acceptance criteria. Within Agile teams it is important to ensure the correct coverage of tests, as you only have the iteration timebox in which to complete them, confirming that the working software is fit for purpose.
The TM is the ideal person to support the Scrum Master (SM) in the removal of roadblocks, particularly those related to or impacting on testing. Plus with their experience in testing and quality assurance they can support the whole team in building quality in. This would also include encouraging and enabling staff to hit tasks within an iteration. The TM could also take on the role of the Scrum Master given they have the in depth knowledge of Scrum, facilitation skills and those others required by a SM.
My recommendation would be that it is critical if it is a large project or programme of work that these experienced TM are included as part of the team. This is also true if you have a high risk project. These types of project are more likely to contain more risk that needs to be considered and the associated planning will therefore likely to be more complex:
Supporting the Team (Outside the team)
There are still a number of activities that need to be conducted in support of the team and the wider organisation. The team’s focus should quite rightly be on the delivery of working software. Certainly for bigger enterprises with large numbers of people, if the TM within the team were to do these, then it would be too disruptive for the team and jeopardise completing stories which the team had committed to.
Formulating an Agile Team at the beginning of a project is critical, identifying relevant skills and tools. It is very important to get the right resources identified for a project as an Agile team will usually stay the same throughout the project.
The TM is important in the overall testing resource management activity. As new teams are required or new skillsets within a team identified there needs to be a skills gap analysis performed. Once this has been performed then the right person needs to be identified or in fact hired. The hiring of staff can take a considerable amount of time for the TM even if they are supported by the HR department. Therefore this activity needs to be supported from outside the team.
If it is identified that there are existing staff that can fill the role(s) as part of the skills analysis it may be that they need additional training, then this will need to be organised by the TM. This training may take the form of Agile training focussed on learning the Agile methodology itself, if they have not previously worked in an Agile environment, or in specific skills.
Maybe the team member, coming from a traditional team, has not had much exposure to test planning or test strategy, they will need an uplift in these skills to be effective in an Agile team. This may take the form of attendance on a Certified Agile Tester or Certified Agile Essentials course for example, ensuring skills development of Agile team members so that they can hit the ground running on the project.
Training is only one of the ways that staff can gain a certain level of competency, it may be better to have some coaching or mentoring from the TM. This is a good way to facilitate some knowledge transfer. Sometimes it is necessary to provide coaching on a particular situation that has arisen within a project. Mentoring the team, advising on Agile process, as well as ensuring they understand prioritisation so they can reach optimal velocity within an iteration.
Within an Agile project the whole performance appraisal process may need to be rethought. Certainly the reward process needs to be separated from the personal development aspects. The TM will be involved in individual personal development which may be through the two previous methods of training or coaching or may be others, such as self-learning. It is important that staff still have access to career development advice.
The TM can still be involved in aiding the removal of roadblocks as they would within the team. In fact if road blocks are taking more than a few hours to get resolution it is probably advantageous to hand this to some outside the team to complete or this could have a significant impact on team velocity. In particular issues with environment stability can be time consuming to fix and may involve considerable expense which may need to be escalated and managed to completion.
For a critical or large project – there are a number of tasks that would be carried out by a Test Manager role, particularly around staff management, stakeholder and vendor management and quality control.
In this instance the TM will play a more strategic role within the organisation. They will be working closely across teams to anticipate likely demand for staff as new projects are initiated or as projects come to an end, to ensure that the right resources are available as required by the teams. They will also be looking at the Test Policy and overall Organisation Test Strategy to ensure that testing is aligned to the chosen organisation Agile methodology(ies).
For example this may be looking at overall Tools, Automation and Test Data Strategies. They will be looking at an overall uplift in skillsets across the testing team particularly as teams initially transition from traditional methods to Agile. Typically this is in the increase of technical awareness.
As teams implement practices such as code reviews, pairing and support of TDD/BDD, the testers may need to get/increase their skills in these areas. In order to have productive conversations at iteration planning meetings the testers need to understand the underlying architecture and how the developers are going to code the solution, as this will have significant impact on how and what the testers need to test. They also need to understand the unit testing is conducted by the developers to ensure that there is no overlapping of testing, as there is not enough time for this within the iteration.
With the increase in use of automation in Agile projects these are important skills for the tester to have, at least for a proportion of the testers within a project. At a minimum all testers should understand how to run automated scripts (executable specifications), how to design scripts that generate return on investment and how to analyse automation results, as well as basic maintenance of scripts. If the testers do not have these minimum skills then the TM will need to organise additional training or pair testers with those that have the skills (this may impact the team’s velocity).
The TM will certainly be an ambassador of Agile testing within the organisation. They may be responsible for increasing the awareness of not only Agile testing but Agile in general across the organisation. One of the advantages of Agile is the increased levels of quality within the product delivered.
It is the collective team that has responsibility for quality and the testers are not the sole gatekeepers of quality. The team should work together to code quality in rather than test it out. The TM can have a great deal of influence here particularly working with developers through coaching. The TM could take on these coaching tasks which would normally fall to the team to carry out thereby minimising the impact of team productivity.
Finally the TM may be involved in the reporting capacity to those stakeholders and senior management outside of the team. It may be that these parties require a different level of reporting than that produced by the team as part of their internal delivery of metrics in order for them to be self-managing.
Rather than create the overhead within the team, which would then not be contributing to the delivery of the product, the TM could produce this reporting. Although, you would expect as the level of maturity in Agile within the organisation increases you would, through appropriate education, see the demand for this additional reporting decrease. After all, as the Agile manifesto attests, it is working software that is important. The only metric that really needs to be measured is that of accepted product.
Within both roles there are a number of facilitation services that the TM can perform. One of these is around specialist services.
This service may include automation (although it is mainstream to have these skills within the team nowadays) or non functional (NF) testing. If there are stories which require NF testing within the iteration then the TM may organise this. It is becoming increasingly common that in fact there is some NF testing in most iterations.
With the increase in online products, performance, stress and load testing plus security testing are becoming commonplace. With the increase in mobile technologies usability testing is increasing disproportionally. Accessibility standards and their associated demands on testing are increasing particularly driven by government drives for compliance.
This role will need to support the Scrum Master and Project Manager in the overall delivery. Theoretically the Agile Team will need to be self-managing but initially, at least through the transformation phases, the TM can help shift people’s mindset which is critical in Agile success before teams reach a level of maturity becoming completely self-managing.
The TM will become more of an Agile mentor to enable Testers to develop self-management skills. It is common for TM to transfer their knowledge particularly in the areas of strategy, planning and estimation where they have more varied experiences to draw from.
The TM will also facilitate along with the Scrum Master – the management and removal of roadblocks. A Test Manager’s experience/skills may anticipate a roadblock before it happens and remove it before the team identifies it.
The traditional role of the TM as the person that directs and manages the testing effort certainly does not exist within Agile however there definitely is a position for a person with their level of experience. In fact, there are even greater opportunities for TMs within Agile.
As described within this paper, there are in fact two distinct roles- those within the Agile team itself and one external to the team or a combination of the two. Whilst there are differing skills and competencies needed for both there is certainly some crossover and room for career growth.
Agile tries to move away from role descriptions but embrace an individual’s skill set and competencies in order that the Team is capable of multi-functioning.
There are still a number of tasks/activities that a TM type of role would do but in the future world of Agile Projects – they may be called something else such as Test Lead or Agile Coach? I think we may just need to rebrand this role, but essentially still feel there is a place for it within the organisation.