QA Metrics - Testing Progress Metrics

Quality Progress Metrics

Start tracking before Test Execution


User Stories, Features or Requirements Coverage Progress
  • Are test cases being authored for all required stories, features or requirements? Are test cases for important features or requirements being targetted first? Are number of test cases (for stories, features or requirements) proportional to effort required for their development? Are there any features, stories or requirements being left over? Is team on target in covering all required features, stories or requirements?

Test Case Readiness Progress
  • When will all the test cases be ready to run? Will all the test cases be ready to run by the end of the iteration? How many test cases must the team still write and review? How many test cases are ready to run?

Test Automation Progress
  • How many test cases has been automated? How many test cases being automated on regular basis? Is the team on track in automating the test cases?

Start tracking during Test Execution


Test Plan Progress
  • How much testing has the team completed? Is the team likely to finish the testing on time? How many tests are left to be run? How many tests are passing, failing or blocked?

User Stories / Features / Requirements Testing Progress
  • Which user stories have a low overall count of test cases? Which user stories have a high overall count of test cases that are blocked or have never been run? Does the test coverage for each user story meet expectations? Which user stories have high rate of test failures? What is the average number of test cases that are defined per user story? Is the test team running test cases for each user story? If test cases are blocked or not being run, does the team understand the blocking issues, and is the team addressing them?

Manual Test Activity Progress
  • Is the trend of manual tests that the team is running consistently increasing? Do you identify any spikes in the test activity that you cannot account for?


Bug Progress
  • How quickly is the team resolving and closing bugs? Is the team fixing bugs quickly enough to finish on time? How many bugs is the team reporting, resolving and closing per day?

References:


Links to missing sections (automation etc),More sample graphs for displaying progress,Discuss on further analysis after verifying the regular progress

Comments

Post a Comment

Popular posts from this blog

Test Case Template for Project Using Agile Methodology

User Story One template that can be used while writing user stories could be – “As a (role) I want (functionality) so that (benefit).” For example – As a ‘Bank Customer’, wants to ‘Transfer Funds’ to one of their friends / acquaintances online, which is fast and auditable. Or, the user story could be described as a single line item (2-5 words) as well. For example – Fund Transfer Test Cases Few of the test cases (summary) for above user story could be as follows.  These test cases should be developed with the help of Business analyst. Transfer fund when fund is in range of acceptable limit and existing balance is sufficient. Or, Fund transfer amount less than existing balance Transfer fund when fund is in range of acceptable limit but existing balance is not sufficient. Or, Fund transfer amount more than existing balance Transfer fund when fund amount is beyond acceptable limit. Or, Fund transfer amount more t...
Read more

Measurement & Metrics

"You cannot improve what you cannot measure" A lot had been talked and spoken already about ‘Measurement & Metrics” in Software Testing.  In this article I am not saying that something new will be written or discussed, but my idea for this article is to summarize on this topic from various different sources. Measuring, Monitoring and Acting on it are key roles of any manager in any industry and Software Testing is no different.  A lot of times it has been seen that KPIs are being measured and monitored but not being acted.  In that case, this whole exercise becomes worthless.  Sometimes, the measurements themselves are not collected and reported correctly, which is more dangerous than not doing this activity at all. Other times, it had been observed that there are numerous KPIs are being monitored without having any goals in mind.  As per me, the number of KPIs that should be measured and monitored should be minimal and any addition to new KPIs sho...
Read more

Performance Test Run Report Template

You have gathered requirements from clients, identified key critical business scenarios, analyzed the workload distributions, found out performance goals and requirements, identified  tool for generating load, setup the test environment, designed the scripts, executed the performance run, collected the raw data, analyzed it, what next?  Send information to stakeholders? No!!  Wait!! It needs to formatted, customized for different stakeholder(s) and reported accordingly. This article suggests one of the ways to report performance run results to client especially if the performance testing activity has been outsourced to yours company.  It discusses the performance report template and importantly the reasons for including the items in the report. The performance report template contains information by data collected from client side only and doesn’t discuss / talk about data from server side like processor utilization etc. Broadly the performance report has been divi...
Read more