QA Process Management - Significance of QA Process

Procedurally, the entire QA cycle consists of three stages:

• Test Case Generation: This stage starts quite early in the SDLC, just after the Design Phase. After the feature requirements are frozen and the design agreed upon, Test Cases are generated. These Test Cases are used for feature verification at the end of Development Phase. However this initial draft may undergo several modifications based on tester and end user feedback. Succinctly put, the test cases must test for expected functionality while also ensuring functioning of software as per expectations. The former
  depends on the captured requirement specifications while the latter is the end users’ expectations from the software. Thus maturity of Test Cases is a long drawn process with need for continuous review, updating, and tracking of updating.
• Test Cycle Planning: This stage involves drawing of schedules and resource allocation charts for testing and their approval by supervisors before execution. It gains significance because effectiveness of the Test Cycle depends on balanced deployment of manpower and testing resources like servers, peripherals etc based on criticality of tested features. A timely release requires planning an efficient test schedule.
• Test Cycle Execution: The actual testing takes place in this stage. Activities may include logging of defect(s)/suggestions encountered in testing, test case modifications, analysis of test logs by leads and generation of QA reports based on findings. The test reports are crucial to the actual release of the software.

Significance of QA Process
QA Phase vindicates the engineering effort and ensures completeness of the generated software before it is released for general availability. It forms a very critical node of the Software Development Lifecycle as can be deduced from the activities enumerated above.

• As its chief objective, the QA Phase brings out the lacunae in requirement gathering, design and development of the software through reporting of defects and suggestion of improvements.
• Beyond its main purpose of catching and reporting bugs in the software, QA ensures compliance of functionality with pre-set expectations.
• It also ensures reliability of software over and above the expected functionality.
• It guarantees scalability, concurrent accessibility and satisfactory performance of software under different usage conditions.
• Specialized testing such as Beta Testing validates that the software not only functions as per the defined specifications but also meets customer expectations and market needs. Receiving and incorporating feedback from end users belonging to different market segments brings in competitiveness in the features also.

The actual testing of an application may be of different types viz Black Box Testing, White Box Testing, Graphics Interface Testing etc depending on the type of application. It can also be carried out either manually by a team of Test Engineers or through the use of Test Automation Tools with minimal manual effort.

Irrespective of these specifics, the QA Phase poses several management challenges as is illustrated through an example (check out in the next posts).

QA Process Management - QA Process Activities

The activities and implications of the QA Phase can vary based on the various software development paradigms in the industry. These are depicted in the table below:

1. Software Product Development - A software product release is preceded by an internal QA cycle wherein Test Engineers from the product development group verify the product functionality against the feature specifications gathered from market sources/sales partners. This is followed by Beta Testing of the product by external users - customers, partners, patrons or volunteering testers.
2. Software Application Development or Software Servicing - In this case three levels of testing may be carried out. The party involved in software servicing or application development carries out the first level of testing at the end of the development phase. At this level the developed application’s functionality is verified strictly against client’s specific requirements. This is typically followed by System Integration and testing of the service or application at the client site, either by test engineers from the servicing project or by test engineers hired by the client. If the client is servicing a customer in turn, testing may be done at the end-customer site also by test engineers hired at the customer end.
3. Software Testing Services - This is a category of software services in which the development partner outsources the complete testing of its product, application or service to a third party. Dedicated teams of Test Engineers at the third party site solely perform testing of the developed software.

The example elucidated in this paper falls in the third category but the challenges that are faced and the solution that has been presented are applicable to the other two categories also.

A checklist of model requirements

These requirements are grouped according to testing activity.

A test model should:

1. force a testing reaction to every code handoff in the project.
2. require the test planner to take explicit, accountable action in response to dropped handoffs, new handoffs, and changes to the contents of handoffs.
3. explicitly encourage the use of sources of information other than project
   documentation during test design.
4. allow the test effort to be degraded by poor or late project documentation, but prevent it from being blocked entirely.
5. allow individual tests to be designed using information combined from various sources.
6. allow tests to be redesigned as new sources of information appear.
7. include feedback loops so that test design takes into account what’s learned by running tests.
8. allow testers to consider the possible savings of deferring test execution.
9. allow tests of a component to be executed before the component is fully assembled.

Summary

The V model is fatally flawed, as is any model that:

1. Ignores the fact that software is developed in a series of handoffs, where each handoff changes the behavior of the previous handoff,
2. Relies on the existence, accuracy, completeness, and timeliness of development documentation,
3. Asserts a test is designed from a single document, without being modified by later or earlier documents, or
4. Asserts that tests derived from a single document are all executed together.

I have sketched – but not elaborated – a replacement model. It organizes the testing effort around code handoffs or milestones. It takes explicit account of the economics of testing: that the goal of test design is to discover inputs that will find bugs, and that the goal of test implementation is to deliver those inputs in any way that minimizes lifecycle costs.

The model assumes imperfect and changing information about the product. Testing a product is a learning process. In the past, I haven’t thought much about models. I ostensibly used the V model. I built my plans according to it, but seemed to spend a lot of my time wrestling with issues that the model didn’t address. For other issues, the model got in my way, so I worked around it.

I hope that thinking explicitly about requirements will be as useful for developing a testing model as it is when developing a product. I hope that I can elaborate on the model presented in this paper to the point that it provides as much explicit guidance as the V model seems to.

A different model

Let’s step back for a second. What is our job?

There are times when some person or group of people hands some code to other peopleand says, “Hope you like it.” That happens when the whole project puts bits on a CD andgives them to customers. It also happens within a project:·

• One development team says to other teams, “We’ve finished the XML enhancementsto the COMM library. The source is now in the master repository; the executablelibrary is now in the build environment. The XARG team should now be unblocked –go for it!”·
• One programmer checks in a bug fix and sends out email saying, “I fixed the bug inallocAttList. Sorry about that.” The other programmers who earlier stumbled overthat code can now proceed.

In all cases, we have people handing code to other people, possibly causing themdamage. Testers intervene in this process. Before the handoff, testers execute the code,find bugs (the damage), and ask the question, “Do you really want to hand this off?” Inresponse, the handoff may be deferred until bugs are fixed.

This act is fundamental to testing, regardless of the other things you may do. If you don’texecute the code to uncover possible damage, you’re not a tester.Our test models should be built around the essential fact of our lives: code handoffs.Therefore, a test model should force a testing reaction to every code handoff in theproject. I’ll use the XML-enhanced COMM library as an example. That’s a handoff from oneteam to the rest of the project. Who could be damaged?

• It might immediately damage the XARG team, who will be using those XMLenhancements in their code.
• It might later damage the marketing people, who will be giving a demonstration ofthe “partner release” version of the product at a trade show. XML support is animportant part of their sales pitch.
• Still later, it might damage a partner who adopts our product.

We immediately have some interesting test planning questions. The simple thing to do would be to test the XML enhancements completely at the time of handoff. (By “completely,” I mean “design as many tests for them as you ever will.”) But maybe some XML features aren’t required by the XARG team, so it makes sense to test them through
the integrated partner release system. That means moving some of the XML-inspired testing to a later handoff. Or we might move it later for less satisfying reasons, such as that other testing tasks must take precedence in the near term. The XARG team will have to resign itself to stumbling over XML bugs for a while.

Our testing plan might be represented by a testing schedule that annotates the
development timeline.