Lynn Winterboer

Always learning; always growing

How to Create Acceptance Criteria

by

Acceptance Criteria for Data-Focused User Stories: Part 3

In parts 1 and 2 of this blog series, I explained what acceptance criteria are/how they benefit data teams and what happens when the product owner does not accept a story. Now, I am going to walk you through the creation of acceptance criteria.

Creating acceptance criteria is started and led by the product owner, but it involves the entire team to ensure complete understanding and clear guidelines. The team and product owner refine the acceptance criteria until everyone understands exactly what the goals are. More experienced product owners will come to these conversations having anticipated many of the team’s questions and will have them already addressed in their initial acceptance criteria.

Sometimes this can manifest in the opposite manner, with too much detail in the acceptance criteria. Only the most important points should be included as acceptance criteria and the others should be included as notes. It is also important to be able to slice some of those details out and put them into follow-on stories. The idea behind this is that you make it work, and then you make it better later.

Below is an example of exactly what that process should look like. We will use a business intelligence story for this example.

First, the product owner brings the story to the team:
As a call center manager, I want to understand the percentage of total calls that each customer care rep in the company handles so that I can mentor those who are not handling enough.

Next, the product owner provides their initial acceptance criteria:
Results match a manual calculation, based on raw data from both call center applications for the last 12 complete calendar months.

The team now must ask themselves if this is enough information to deliver a working feature. At this point, conversation ensues:
ETL Developer: I wasn’t aware there were two call center applications. I know about the one the customer service representatives use here at HQ – where is the other one?
PO: It’s an old one we don’t use very often, but when customers call in for service-related issues (versus most of our calls are about our products), the representatives log the call in the second application.
Tester: With two sources, it might take longer to load the data each day. What do you want to see during the data load? Do you want to see yesterday’s data until today’s is done loading or show 0% for all representatives until the data is done loading?
PO: I really only need this on a monthly basis for now. Though, I could see a daily view being useful in the future – I’ll make that another user story. I also think I would rather see 0% until the monthly aggregate is completely ready at the end of the month. Let’s add that as new acceptance criteria.

As a result of the above conversation, new acceptance criteria are set:
If the data warehouse does not have a complete month of data, this metric should reflect 0% for all representatives until complete data can be used on the calculation.

The team analyzes these new acceptance criteria and ask themselves again: Is this enough information to deliver a working feature? The conversation could continue like this:
BI Developer: How do you want to see the percentages displayed? A pie chart? A table?
PO: A pie chart with the percentages of each ‘slice’ would be great.
They could then create a lo-fi diagram on the whiteboard:

BI Developer: Don’t we have more than 5 call center reps? Seems like that pie chart would get really busy.
PO: Good point. Could we show the percentages first by teams under each call manager, then drill down to individual representatives? 
They could then edit the lo-fi diagram with the added details:

BI Developer: Sure. That’s not hard. Also, when you say you want to know which reps are ‘handling enough’ calls, how do you know what each rep should be handling?
PO: It’s a gut feel for now. I’ll make a new story when the call center managers have found the time to quantify this.

This continues on until the team decides they have enough information to deliver a working feature. These conversations will be different depending on the members, their experience, and how they tend to work. Some need more detail, and others need less.

This team would end with these acceptance criteria:

  • Monthly calculation, displaying 0% until the monthly aggregation is complete.
  • Results match a manual calculation, based on raw data from both the product and the service call center applications, for the last 12 complete calendar months.
  • Pie chart showing the percentages first by the teams who are under each of the call center managers, then drill down to individual representatives.

This looks much different than the original acceptance criteria offered by the product owner. This is completely normal, as the creation of acceptance criteria is a process that involves the entire team.

Creating acceptance criteria is all about gaining a common understanding. It requires the team to think about the proposed acceptance criteria, and then bring their questions and thoughts to the table with the product owner.

[bctt tweet=”Creating acceptance criteria is all about gaining a common understanding. It requires the team to think about the proposed acceptance criteria, and then bring their questions and thoughts to the table with the product owner.” via=”no”]

What Happens When A Story is Not Accepted

by

Acceptance Criteria for Data-Focused User Stories: Part 2

In the first part of this blog series, I defined what acceptance criteria are and why data teams need them. Now, I am going to address a possible outcome related to acceptance criteria when it comes to a user story:

What happens when the product owner does not accept a story?

The answer is not a simple one-size-fits all, as there is no one way to handle a story that is not accepted. The product owner actually has four different options depending on the situation:

Choice #1

Make the story a top priority for the next sprint. In this scenario, the product owner decides that this work is important enough to take precedence over other work. They bump the next items in the sprint to continue work on the current story until it’s ready to be accepted.

Choice #2

Slice the story thinner and prioritize the slices. This happens when the product owner decides that certain pieces need to take priority, while others do not. This also happens when some pieces are right and others aren’t. So they slice it to distinguish them apart and rework the pieces that need it.

Choice #3

Delay it until a later time. Sometimes, stories take longer than expected. It is up to the product owner to determine whether or not effort is being put in the right place. Other stories may bring value faster, and so that story may be put back into the backlog to make time for others.

Choice #4

Abandon it. In some cases, the story may be abandoned. This is a healthy response of a mature product owner, and it typically happens when the data does not provide the information the product owner was hoping to get. They see that what the team can realistically create based on what’s in the system is not what they or the stakeholders had envisioned.

There are several ways to handle a story that is not accepted by the product owner, and the correct response must be decided on a case-by-case basis. It is important that the data team stays focused on creating value, and that requires regular reviews and adaptations. The sooner the team knows there is an issue, the better. This is why it is imperative to do reviews iteratively and as soon as possible. This way, there are no surprises or time-consuming fixes at the end.

Read Acceptance Criteria for Data-Focused User Stories Part Three- How to Create Acceptance Criteria

What Acceptance Criteria Are; Why Data Teams Need Them

by

Acceptance Criteria for Data Focused User Stories: Part 1

Acceptance criteria are a game-changing tool for data teams. They directly lead to more efficient team members who understand the impact their work has on the organization. Unfortunately, acceptance criteria are not always used or implemented well. It is important to know what they are (and what they aren’t) in order to get the best results.

The Benefits Acceptance Criteria Have on Your Team

There are several factors that create a fulfilled and productive data team:

Yet I’ve noticed that agile software teams are happier and more fulfilled than data teams. This stems from the acceptance criteria portion of that equation. That’s because we as humans like to see tangible progress, and that’s easy to see with software products. With data, not so much.

For example: Agile web development projects. The team always knows when they are finished because they have a tangible product, they feel a sense of accomplishment and move on to the next part. This contrasts to the experience of most data teams.

So how can data teams create this knowing of goal accomplishment? Acceptance criteria are the product owner’s way of ensuring their team knows when the job has been completed.

What Acceptance Criteria Are – And What They Aren’t

[bctt tweet=”Acceptance Criteria are often confused with the Definition of Done.” username=”agilelynn”]In order to achieve that sense of goal accomplishment, it is important for data teams to understand the difference so that they can use acceptance criteria in the correct way.

Acceptance Criteria

Acceptance criteria applies to individual stories. They come from the product owner and are used by the stakeholders to assess whether or not the purpose of a story has been accomplished. Here’s an example:

  • Recognize the revenue from the sale of a product when:
    • Software product has shipped
    • Hardware product has been received by the customer
    • Installed product has been installed at the customer site for 30 days
    • Services are complete
  • Test against 16 specific orders to confirm combinations of products on a single order.
  • Test against Q3 last year data to confirm revenue recognition aligns with prior finance system results (this scopes it to a certain time frame).

These are great acceptance criteria because the team knows what the goal is and they also know what’s not in scope (e.g. data outside of Q3).

Definition of Done

The definition of done will differ from team to team (so don’t worry if yours does!). What follows is an example from one team I worked with. “Done” applies to all stories and it comes from the team. It’s their agreement of how they are going to do work. Here’s an example:

  • Unit and acceptance tests are written before development
  • All unit, acceptance, and regression tests pass
  • Each story has been reviewed by a peer to confirm the development and performance standards have been met
    • Informatica Standards v 1.3 (2/5/17)
    • Tableau Standards v2.5 (11/15/16)
  • Documentation updated:
    • Support guides for Tiers 1, 2 and 3
    • User training – onboarding and updates
    • Enterprise data dictionary

Happy data teams know what their end users want and how their work benefits the organization. They feel accomplished and have clear goals. And you get there by creating good acceptance criteria.

Read Acceptance Criteria for Data-Focused User Stories Part Two- What Happens When a Story is Not Accepted

Ten Mistakes to Avoid When Building a Sustainable Agile BI Practice

by

A few favorites from TDWI’s recent article on Agile BI.

TDWI (The Data Warehouse Institute) recently released a new article in their very useful “Ten Mistakes to Avoid” series.  Author Mark Giesbrecht from the Canadian National Railway outlines ten suggestions for a sustainable agile BI practice.  I am particularly fond of the following ideas presented in the article, and thought they would be useful to you as well:

PRODUCT OWNERSHIP: Sponsors often find it more difficult to free the Product Owner from a “day job” to focus on the project, than to get funding for the project!  A healthy agile BI team will, as Giesbrecht puts it,  “delay project start until the appropriate product owner commitment has been obtained.”

DESIGN: Giesbrecht provides a great analogy for just enough up front design: Watch out for urban sprawl where “project teams gravitate to the suburbs” (i.e. building new warehouse artifacts) over “gentrifying the existing neighborhoods and rehabilitating brown fields” (such as refactoring an existing fact or dimension to meet the needs of the existing and new development).

DEFINITION OF DONE: Be sure to include the support team’s perspective in the project team’s definition of done: Standards for naming conventions, as-built documentation, job flows, traceability, and restartability, to name a few, can greatly lower a solution’s TCO for little additional development cost.

AUTOMATION: Tools and automation can be very helpful to agile BI teams, including those that help with data profiling, data modeling, code version management, and testing automation. However, Giesbrecht makes it clear that “the choice of vendors is often less important than the process used to drive tool adoption.”

I believe these “Ten Mistakes” articles are only available for TDWI Premium Members, which I have been since 1999. If you are interested in these valuable resources and aren’t yet a TDWI member – I encourage you to join and gain the benefits of this very useful and established data-focused industry organization.

I will be teaching Agile Analytics: Road Mapping, Chartering, and Release Planning on Monday, September 22, 2014 at the upcoming TDWI World Conference in San Diego – Hope to see you there!

Also, check out Dr. Ken Collier’s Ten Mistakes in an Agile BI Transformation.

Agile Documentation

by

How To Manage Regulated Documentation in Agile Projects

FOUR FREQUENTLY ASKED QUESTIONS

I recently had the honor of presenting a series of webinars on “Writing User Stories and Slicing Epics for DW/BI Teams” for Cutter Consortium, where I am a senior consultant. After each webinar, I addressed questions from the audience. Last week, I answered the first question, QUESTION #1: HOW CAN A USER STORY SPAN ALL LAYERS — IS THAT NOT AN EPIC?

Here’s another popular question and my answer.

QUESTION #2: In regulated industries, such as medical devices, how are the regulatory requirements taken into account? Releases are difficult to coordinate when there has to be a definitive implementation to draw a line in the sand from a documentation standpoint.

LW: That’s a really good question. And a common one. In regulated industries, such as medical healthcare and financial industries, there are a lot of questions that come up. I believe you really need to try and include that information in your definition of done. If you can fill out your documentation a little bit at a time, it makes it not such a big task to fill it out in the end. Including your regulatory requirements in your definition of done is one way to manage it.

Another way that I’ve seen teams do it is to have what they call a hardening sprint, which is useful for Data Warehousing teams because DW teams might do pre-production testing, or production-level testing, or they might do load and stress testing during that time — things that may be difficult to do every couple of weeks given your data volumes and maybe your technical infrastructure can’t support testing on huge data volumes every couple of weeks, but you do have a test environment where you can load lots and lots of data and really pound it. A hardening sprint would be right before your release and during that hardening sprint is when I have seen financial institutions, in particular, take that time to really meet all of their regulatory requirements.

You have to step back and think creatively and look at values and principles. You have to say that your goal is to deliver value to the business and that value is defined by the business, not by the IT team, and you want to deliver it frequently and consistently. So if you do a release every 3 months, and you need to take a hardening sprint to work on meeting regulatory requirements, typically your business doesn’t mind because they understand. They’re part of that business and they respect that the regulatory requirements are there for a reason and you can’t get out of them. So you have to do them.

COMING UP ON LYNN’S BLOG:

LYNN ANSWERS THESE POPULAR QUESTIONS:

  • Q3: What’s an example of good acceptance criteria — outside of that the report should match the mockup/data should be accurate?
  • Q4: What is your recommended sprint length for a data warehouse team given the fact that it takes longer to deliver a Data warehouse solution?

Questions or comments about Done? About managing releases in agile? Chime in!