What Should I Measure?

Flow Metrics offer their deepest insights when you measure the entire delivery system rather than just a subset of it. So what do we mean by the term delivery system. Well, a delivery system is essentially the following:

1. A value stream or sequence of activities necessary to deliver a product or service to an internal or external customer - Gartner.

2. All of the work items that are currently in flow within the value stream or that have passed through the stream for the period of time we wish to review. Put another way, all of the demands that have been placed on the delivery system.

3. All of the people involved in performing the work that must occur within the value stream. These people represent the capacity of the delivery system.

You may have one or many delivery systems in your organisation. A delivery system may have one or many teams participating in it. You could start to measure work when a request is received from a customer, or perhaps you reduce the scope of your measurements and start once development begins. Regardless of your decisions, there are some factors you should consider.

Include all of the work in the system

Deciding whether to initiate measurements at the first or fifth step of your value stream hinges on the objectives of your measurements and your team's comfort level with flow concepts. Ultimately, the decision rests with you. The same principle applies when determining whether to analyze a month's worth of data or six months' worth. The choice is yours.

However, there is one crucial requirement that must be met to maximise your results and insights: for the period under review, ensure that you capture all the work that has entered the value stream, rather than just a subset of the work. To illustrate the importance of this, consider the following real-world example.

This engineering group consists of three teams, each comprising seven engineers. All teams are dedicated to developing features for a single product. In addition to feature development, they also regularly address bug fixes, security concerns, and technical debt. To effectively track this workload, the group has established specific issue types in Jira to document their efforts. The image below illustrates a detailed breakdown of the group's activities over a span of seven weeks.

Open

^{Figure 1. Time in Flow by Issue Type from Delivery Flow}

In product development organisations, stakeholders often focus heavily on feature development. This can lead to a situation where story issue types are the primary focus when reporting on work. In the example above, if our query focused on feature development work alone, we would be creating a very incomplete picture, as feature development accounted for merely 45% of the items delivered by this group over a seven-week period. Making insights or improvement decisions based on just 45% of the work processed through the delivery system would be precarious at best. The table below highlights some of the key differences in the data.

Neglecting to capture the complete scope of work performed by the team obscures our understanding of a critical factor that affects the performance of a delivery system: the relationship between demand and capacity. Given that the work items mentioned have an average cycle time of 11 days, a demand of 3 items per engineer per week exceeds the available capacity. Any system facing demand that surpasses its capacity will inevitably encounter delays, waste, bottlenecks, and inefficient delivery. While focusing solely on feature development in this scenario reveals issues in the data, it is impossible to pinpoint the root causes of these problems without a comprehensive view of the bigger picture.

Don’t double count

By default, Jira includes issue types that serve as parent issues for other issue types. For instance, an Epic typically represents a substantial body of work that is divided into Stories or Tasks. An Epic will have both a start date and an end date; however, it does not directly represent work itself. Instead, it acts as a placeholder for its child issues, which provide a more accurate depiction of the actual work involved.

When your query retrieves an Epic along with its children, there is a risk of double counting work, depending on how these issue types are utilized within your organization. This challenge also applies to issue types such as Sub-task. A sub-task represents a smaller task that is part of a larger issue.

If your delivery system utilises problematic issue types, the easiest way to avoid this problem is to instruct Delivery Flow to avoid the unwanted issue type via your JQL query. In Running for the First Time, we shared logic for an example flow metric friendly query:

If double counting is a concern, then we need to enhance this logic:

Here is an example JQL query with the suggested issue type amendment: