PEF/REV: A Multi-dimensional bug tracking metric

Traditional one dimensional bug tracking suffers from a lack of directive capability. If each bug is rated "ReallyBad", "Bad", or "Trivial", no clear direction exists for which ones to fix first. If you concentrate on all the "ReallyBad" bugs, you may fix bugs that only 2% of your users encounter 1% of the time and it breaks other code. You may not get to "Bad" bugs that 90% of the users encounter 90% of the time.

Each bug is rated with a PEF and a REV number.

1. User's View: PEF - the sum of a rating for Pain, Effort, and Frequency.
1. Pain

How painfully is this bug to a user. 1 - low pain, 3 - fairly painful, 9 - very painful

2. Effort

How much effort does it take a user to work around the bug. A rating of 1 requires little effort, 3 means a fair amount of effort, and 9 implies a major amount of work.

3. Frequency

How often does a user encountered this bug. 1 - infrequently, 3 - fairly often, 9 - frequently

The PEF represents the user's experience with the bug. A PEF of 27 means its really painful, is very hard to work around, and happens all the time. A PEF of just 3 means its not very painful, doesn't require much effort, and happens infrequently. All the really high PEF should be candidates to be fixed soon.

2. Programmer's View: REV - the sum of a rating for Risk, Effort, and Verifiability
1. Risk

How risky is this fix to the system. 1 - little risk of collateral damage from the fix, 3 - some risk to other components, 9 - this fix touches lots of code and its very risky to the entire system.

2. Effort

How much effort does it take a programmer to fix this bug. A rating of 1 requires little effort, 3 means a fair amount of effort, and 9 implies a major amount of work.

3. Verifiability

If we fix this bug, can we easily verify that its fixed? 1 - its easy to verify, 3 - we can verify with some amount of work, 9 - its really hard to verify this bug is fixed.

Each bug also gets a REV number. A REV of 27 means its a very risky fix which may endanger other code, it will take lots of programmer effort to fix it, and its going to be difficult to verify the fix. High REV number bugs are bad.

The ideal bug (from a programmer's view point) is a PEF of 27 and a REV of 3. The worst case scenario is a PEF of 27 and a REV of 27.

Using the PEF/REV system you can gain more control over releases by specifying a target goal. This goal might be that we are going to fix all bugs with a PEF over 21 and REV under 21. (This goal should not be immutable. Certain bugs will need to get fixed that are not in this category, and others we may not fix that fall into this criteria. The team should decide on deviations from the policy as a conscious decision.)