Measurement & Metrics
"You
cannot improve what you cannot measure"
A
lot had been talked and spoken already about ‘Measurement & Metrics” in
Software Testing. In this article I am
not saying that something new will be written or discussed, but my idea for
this article is to summarize on this topic from various different sources.
Measuring,
Monitoring and Acting on it are key roles of any manager in any industry and
Software Testing is no different. A lot
of times it has been seen that KPIs are being measured and monitored but not
being acted. In that case, this whole
exercise becomes worthless. Sometimes,
the measurements themselves are not collected and reported correctly, which is
more dangerous than not doing this activity at all. Other times, it had been
observed that there are numerous KPIs are being monitored without having any
goals in mind. As per me, the number of
KPIs that should be measured and monitored should be minimal and any addition
to new KPIs should be discussed with all stakeholders and then only added to
the group. I feel, we should initially
start with only 2 KPIs and both should be at the extreme end, for example, in
testing service company, if we are measuring client satisfaction index at one
hand then the counterpart should be the attrition rate in the company. It is very much possible that company’s clients
might be satisfied, but how about the satisfaction of your most valuable
resource i.e. employees. Similarly, if
we need to measure quality of products, two metrics could be defect density and
code coverage. Anyway, which metrics to
measure and report is not an easy task and should be planned out well in
advance.
Why
should we have metrics? How should we develop metrics? What is the process of
deriving metrics? What is considered as good metric?
What
are some key metrics to monitor testing Process? How do we measure Rework inefficiency? What
is our defect closure period? Defect effectiveness? Process metrics can help us
understand the quality capability of the software engineering process as well
as the testing capability of the software testing process.
What
are some Project metrics like bugs opened and closed trends? How does the test
case completion trend looks like? How many test hours are spent on test
execution of a project? Project metrics can help us understand our status in
terms of the progress of testing and quality on a project
What
are some metrics to evaluate the quality of the product? How well are we reducing risk of product
quality? Product metrics help us to find it out the quality status of the system
under test any point of time.
What
are some of the key metrics to track progress of automation?
What
are some key metrics to manage testing in agile projects?
Asking
the right question should be the first step for expecting the right answers. To
identify the right set of metrics the first thing should be what do we need to
measure and why. For example - Which
areas have the most defects? How long does it take to repair defects? Which
areas have the highest re-work rates? After the bug is fixed how long are we
taking to verify?
Some
nice case studies on metrics.
If
we are looking for a laundry list of all testing metrics for reference, it can
be found from below links
Comments
Post a Comment