If you choose to slice user stories by technical function or skill set, you will put your delivery of timely business value at risk.
Originally published April 3, 2014 in TDWI Flashpoint: Tips, Techniques & Critical Research in BI and DW. The full article can be downloaded here.
Sometimes “by the book” guidance for agile teams doesn’t easily apply to data warehousing (DW) teams. For instance, slicing user stories can cause a conundrum when we face two options: (1) slice user stories by technical function/skill set or (2) slice stories into thinner, architecturally complete features that encompass all of the work necessary, from source to report, to meet the organization’s needs. If you choose to slice user stories by technical function or skill set, you will put your delivery of timely business value at risk. However, if you slice user stories into architecturally complete functionality, you keep your team focused on–and honor the agile commitment to–frequent and consistent business value.
When we say that we need to develop “architecturally complete” user stories, we find we can easily do that for the business intelligence (BI) front end of our BI/DW system–if the data already exists in the data warehouse. However, once we find ourselves facing a user story that requires data that has not yet been brought into the data warehouse, then we face a challenge: How will we get all that work done in just a few weeks? This challenge makes it tempting to fall back to our old ways of separating DW and BI work due to (among other reasons) differing skill sets needed for each.
If you choose to slice a user story by technical function or skill set, you would write a user story for the DW work (the “back end”) and one for the BI work (the user-facing “front end”). The business value, which lies in the results of the front-end BI work, is now separated from the heavy lifting that happens in the data warehouse. This may make the work clear for the delivery team(s), but it puts the data warehouse efforts at risk of losing focus on delivering business value . It also separates the delivery team members with DW skills from interacting with the business.
Let’s look at an example to illustrate the point:
User story: “As a call center manager, I would like to understand the percentage of total calls that each customer care rep in the company handles each month, as well as month-to-date for the current month, so that I can properly reward the top performers with special appreciations and provide the lower performers with additional training and support.”
Situation: In this case, the aggregations of calls by month are not yet in the data warehouse. The team members believe they cannot properly develop and test all of this in a single iteration, so they work with the business to slice this work into smaller chunks to fit into various iterations. They can see two options for slicing this relatively simple request. The first option slices the work by technical function:
- Story 1: Create the aggregations and percentage calculation in the data warehouse, which focuses on the modeling and ETL skills needed for this piece.
- Story 2: Create a report reflecting these calculations, which focuses on the metadata and report-building skills needed for the front end.
In this option, the DW work has to be done first before anything of value can be demonstrated to the business stakeholder(s). This means the DW team will spend several weeks working on something that doesn’t yet provide value to the business. It also means that by the time the BI work is completed in a future iteration and the stakeholders provide feedback on the whole effort, the DW folks have moved on to something else. This feedback would then be an interruption to their work.
The second option slices the work into three architecturally complete, if embryonic, features by doing the DW and BI work in the same story:
- Story 1: Monthly (past) aggregations
- Story 2: Month-to-date aggregations
- Story 3: Percentage calculations for each customer care rep
In this option, the business stakeholder(s) will see real, working results in the BI layer when the first story is completed. In addition, the biggest risk to the whole requirement is in getting the monthly aggregate calculations correct, considering source system anomalies around what is considered a “call.” The team and the business would like to focus on that first and prove it’s working before moving on to the relatively straightforward tasks of applying those business rules to the month-to-date aggregation as well as calculating the percentage of total calls by rep.
[…] I wrote another well received article about building user stories and slicing them for Data Warehousing. Check out, Why Build Architecturally Complete BI User Stories? […]