Lynn Winterboer

Always learning; always growing

BI Product Owners Love an Agile PMO!

by

The Role of an Agile PMO

[Note: This was first published by Cutter Consortium in an Executive Update. I am a senior consultant for Cutter Consortium]

by Lynn Winterboer, Senior Consultant, Cutter Consortium

Organizations seeking to respond quickly in an ever-changing marketplace are asking more and more from their data warehousing (DW) and business intelligence (BI) teams. Fortunately, many BI teams are finding they can meet top-priority needs in short time frames when they adopt Agile practices. Within an Agile team, the product owner (PO) guides the team’s priorities to meet the organization’s goals.

Product owners for Agile BI teams often face unique challenges compared to their peers on other functionally specific delivery teams. An Agile project management office (PMO),1 as we explore in this Executive Update, helps mitigate some of the key additional challenges BI teams face, thereby allowing the BI product owner to more effectively drive high-quality, high-priority results for the organization.

The role of product owner on an Agile team carries substantial responsibilities. The PO must do the following:

  1. Identify the business needs.
  2. Write and refine good user stories that accurately reflect the drivers and details of each need.
  3. Prioritize the user stories into a backlog.

But these substantial responsibilities are only a portion of what a BI product owner must do. The PO for a single BI team that supports multiple business functions, such as sales, finance, service, and HR, has a bigger challenge. His or her challenges, beyond what other product owners face, include:

  • Understanding the drivers and values of each business function in order to groom strong user stories.
  • Prioritizing stories across these multiple business functions to come up with a ranked backlog that represents the organization’s overall priorities.
  • Aligning the unique business intelligence value chain with Agile practices

Although a BI team can certainly find ways to mitigate these challenges without a PMO, it’s awfully nice to work in an organization that has a strong Agile PMO, which can help the BI product owner tremendously by:

  1. Facilitating cross-functional communication and understanding
  2. Driving and enforcing corporate priorities across teams
  3. Providing coaching to newly Agile teams

FACILITATING CROSS-FUNCTIONAL COMMUNICATION AND UNDERSTANDING

An Agile PMO facilitates cross-functional collaboration for the business as a whole. A key component for Agile enterprises is the Scrum of Scrums, which is a regular gathering of representatives from all the Agile delivery teams where they communicate what they have accomplished, what they are working on next, and where they need help from another team or otherwise have roadblocks to success. An organization can certainly hold a Scrum of Scrums without a PMO to coordinate and facilitate it, but the best PMOs I’ve seen do a great job of this, as well as helping to remove roadblocks that arise.

The information that comes out of the Scrum of Scrums is incredibly useful to the DW/BI team that needs to understand multiple business domains and their source systems (that the other teams support). For example, since our product is often dependent on data from these other teams’ systems, we need to know if those systems are changing so we can react appropriately. We also benefit from being involved early on in source system enhancements so we can extend the value chain of the new functionality by eliciting user stories for the key reporting and analytics the business will need in order to fully leverage the new functionality.

BI teams benefit from having this help because it allows them to focus on BI-specific activities while the PMO focuses on cross-functional facilitation and communication tasks. It is much easier to participate in a cross-functional meeting than to define, schedule, facilitate, and participate in one! Without the Scrum of Scrums, a BI product owner would have to find ways to gather this information elsewhere.

BI product owners are also often required to be (or become) subject matter experts on several business domains at the same time. Ideally, the product owner would have deep experience working in each business domain that he or she represents. A product owner for a sales application, for instance, would benefit from having worked as a salesperson in the field. Similarly, a finance analyst would make a fine PO for a finance application. However, there aren’t many people out there who have worked as both a salesperson and a finance analyst, which would be ideal for a BI product owner who supports both sales and finance BI needs.

Therefore, the PMO, which also must satisfy the needs of multiple stakeholders, has insights into the priorities and challenges of all business domains and can be helpful to a BI product owner who is stronger in some and less knowledgeable in others. I have found the PMO to be particularly good at pointing out synergies and conflicts between business domains, so the BI product owner doesn’t have to discover these issues on his or her own.

DRIVING AND ENFORCING CORPORATE PRIORITIES ACROSS TEAMS

Corporate-wide priorities are critical to BI teams because we need to understand which of our multiple stakeholder groups’ requests are the most important to the organization’s overall goals and therefore work on those first. Many of us in the data world have often experienced situations where the reporting and analytics considerations of large projects are not thoroughly considered until after the source system and business process reengineering are well under way. This mistake can be avoided. For instance, one of my clients holds a monthly steering committee, facilitated by a very effective PMO, which brings the product owner and technical lead of each delivery team together with the company executives to review and prioritize key features and risks. This approach ensures the BI team is working in line with corporate priorities, and puts the BI-related efforts to support key system initiatives on par with those initiatives.

A good Agile PMO will also ensure delivery teams actively support corporate priorities. The PMO therefore provides a natural “backup” to the BI product owner when one department’s project is a higher priority for the BI team than another’s. For a BI product owner who serves multiple stakeholder groups, it can be hard to have to “push back” on several of them while the BI team focuses on the top priority. Having a strong Agile PMO partner can be really helpful in taking the emotion out of the situation by reminding stakeholders of the priorities the company’s executives have set and maintained in the regular steering committee decisions.

PROVIDING COACHING TO NEWLY AGILE TEAMS

The strongest Agile PMOs I’ve seen are very good at helping teams that are new to Agile “hold the Agile process” and not succumb to the temptation to start making exceptions. A BI team that has multiple stakeholder groups, specialized tools (and related skill sets), and large volumes of data certainly has some additional challenges than other IT teams.

In addition, while many IT teams support a single system, BI teams often support a process with layers of systems: from source to staging to integration to data warehouse to cubes to BI and analytic applications. This unique aspect of the BI value chain has, over the years, been a driver for BI teams behaving differently than their IT peers.

It is therefore tempting to say, “We’re different, and therefore we can’t adhere to ____ Agile value.” A good Agile coach can help the team think through its approach and find ways to adhere to the core values and principles of an Agile approach.

For example, exceptions I regularly see when coaching teams include:

  • Separating DW and BI user stories, which leads to an exception to the Agile tenet, “Satisfy the customer through early and continuous delivery of valuable software.”2 I don’t know many business stakeholders who are satisfied with a key metric being available in a table in the data warehouse. They often want it in a report or on a dashboard before they find value in it.
  • Developing in a “mini waterfall” within each iteration, leaving testing of all stories to the end. This practice shortchanges the testing activities (just as it does in a traditional waterfall process), and goes against the grain of continuous attention to technical excellence.3 It’s very difficult to deliver technically excellent code when it hasn’t been well tested.
  • Avoiding test automation because the most popular tools out there are GUI-focused and don’t necessarily test “data.” Any Agile team can quickly get overwhelmed with the amount of regression testing required with iterative development; test automation is incredibly helpful here. BI teams, which are used to specialized tools for almost every aspect of what we develop, are often hesitant to take extra time and effort to figure out how to automate their tests.

An Agile coach is in a position to challenge these practices and talk the team through an assessment of the pros and cons of each to determine whether they will contribute to the organization’s Agile goals overall, or end up causing more problems than they are worth for the near-term convenience.

CONCLUSION

With a strong Agile PMO, the BI product owner can now focus on BI-specific activities while the PMO facilitates cross-functional prioritization and communication, enforces company priorities, and provides Agile coaching when needed.

ENDNOTES

1Well-known Agile author and trainer Mike Cohn writes more comprehensively on the role of the PMO in an Agile organization at: Cohn, Mike. “The Roles of the Project Management Office in Scrum.” Mountaingoat Software, 1 September 2010

2“Principles Behind the Agile Manifesto”

3“Principles Behind the Agile Manifesto” (see 2).

Agile DW/BI Testing

by

Agile DW/BI Testing — Just Get Started!

by Ken Collier and Lynn Winterboer

This article was first published in TDWI BI This Week on July 9, 2013

[TDWI Editor’s note: Lynn Winterboer and Ken Collier are leading a session at the TDWI World Conference in San Diego (August 18-23, 2013), Agile BI: Test Automation and Test-Driven Database Development. We asked them to share a realistic scenario of how agile testing works in the real world.] [Read more…]

Agile Testing – Software Delivery

by

Agile Testing Challenges Faced by Agile Teams

Agile Testing-Software Delivery: Common Testing Challenges

I attend a lot of webinars and sessions at conferences, and like to highlight the ones I find particularly useful. I recently sat in on an HP presentation that was clear, concise, and yet deep enough to be interesting and offer value: “Agile Software Delivery — Get Speed Without Sacrificing Quality.”

The presenters, Michael Cooper (@QACooper) and Jon Bailey, walked through common testing challenges faced by agile teams, which in general is useful for DW/BI teams to understand they aren’t alone in dealing with: [Read more…]

Do Less by Committing to More

by

By Ken Collier and Lynn Winterboer (first published in TDWI BI This Week on July 30, 2013)

Agile leaders focus more on the continuous delivery of high-priority value to business stakeholders and less on making sure each worker is always busy. We explain how your BI program will benefit by limiting developer multi-tasking.

Ben and Susan are each a DW/BI director at their respective companies; each leads a department of roughly the same size. Both have the staffing necessary to organize three dedicated, cross-functional teams. Ben’s department is organized into dedicated teams while Susan has functional skills “pools” from which project teams are staffed. Both directors are planning for the coming year and have a similar set of 12 projects to schedule.

Through collaboration with business stakeholders, product owners, project managers, and the program management office (PMO), Ben has a prioritized backlog of funded project requests to schedule into the calendar. With help from technical leaders, he has completed some high-level project sizing and assigned durations of two, three, or four months per project. Projects longer than four months are divided into multiple phases, each designed to fit into one of these smaller time periods. This ensures that long-running projects do not block other projects by dominating the department’s capacity.

Ben assigns a dedicated team to each of the first three projects in the backlog. He then estimates when each team might be free to start its next project, continuing in this manner until he has a reasonable program plan for the coming year:

Because Ben continuously pulls from a program backlog, the whole process takes less than an hour of Ben’s time, and he is comfortable sharing this project road map with stakeholders and the PMO. This company’s agile culture includes an agreement that once a project is started, the team will stay focused on that project until the project time frame is finished. Adaptation to project reprioritization does not affect projects that are already underway. Because business and management stakeholders are involved in the prioritization process, there are no surprises about the program schedule. Everyone also understands that the program backlog is always evolving based on the changing needs of the business.

Meanwhile, Susan is also doing resource planning for her department. She has met with each of the BI team’s key stakeholders to get their input on priorities for the year. She’s heard passionate explanations from each about why their top priority projects are important to the company, such as “I need sales pipeline analytics ASAP so I can ensure the marketing team is ready to support our key industries!” (from the VP of sales and marketing) and “I need net profit margin per product so I can report accurately to our investors!” (from the VP of finance).

Susan understands the importance of each of these projects, and wants to get them started as quickly as possible. Moreover, her CIO is focused on maximizing personnel utilization, desiring that every team member be utilized 100 percent on project work. Susan believes if she gets team members started on more projects sooner, and has them spread across multiple projects, her CIO and stakeholders will be confident she’s doing everything she can to focus her teams appropriately. Therefore, she structures each project to have a dedicated core team of two developers with the part-time support of product owners, project managers, data modelers, testers, and business analysts. Several of these projects are expected to take 6-12 months to complete, so Susan needs to run at least six projects concurrently in order to meet the urgent needs of the business.

As Susan’s delivery strategy gets underway project team members plan in two-week iterations for the multiple projects to which they are assigned. This means that each team member is attending at least two, and sometimes three, planning meetings every two weeks. They are also attending multiple daily stand-up meetings, iteration showcases, and retrospectives. Project managers and product owners are only able to devote cursory time to assist on each project, and they often feel disconnected from what’s actually happening. The core developers dedicated to each project are frustrated because they cannot count on having the support they need from data modelers, testers, business analysts, and others. This is further complicated by the fact that there doesn’t seem to be any common time when the entire project team can work together.

Every team seems to over-commit on each iteration, failing to finish stories as planned. In part this is caused by the significant overhead incurred as team members switch from one project context to another (approximately 20 percent per additional project, per Weinberg, 1991).

Susan failed to account for this waste in her resource plan. This triggers two side effects. It is evident that the projects will not finish on time, and teams’ (and Susan’s) ability to accurately predict when each will finish is nebulous. Each team is cutting testing corners under the duress of over commitments, which will cause increased maintenance/support issues.

As much as Susan tries to create a great culture and work atmosphere, people in her department are unhappy. They are stressed by the job pressures and often feel like hamsters on a wheel, running fast and getting nowhere. There is a general lack of collegiality in the BI group and people often blame their colleagues when things go badly. Many staff members have remarked on the difficulty of trying to do too many things at once, but Susan doesn’t think they understand the pressure she is under to meet the needs of the business.

Consider the differences in the directors’ situations.

  Ben  Susan
Project focus Finishing projects as soon as possible Starting as many projects as possible
Staffing philosophy Prioritize project delivery over staff utilization Prioritize staff utilization over project delivery
Corporate prioritization PMO facilitates cross-functional prioritization for the BI team so the BI director is not left with making these important decisions him/herself BI director makes cross-functional prioritization decisions
Corporate culture Supports a team staying focused on a single project until it is complete Team splits focus across multiple projects at a time
Stakeholder expectations Stakeholders understand the relatively short timeframe when they need to be available for each project to be successful Stakeholders must be available for long periods of time if they want their projects to be successful
Project completion Project focus is constrained; each successive project starts later but is completed earlier Project focus is stretched; each project starts as early as possible and is completed much later
BI director’s confidence Confident enough to get started on project work, knowing he’s done the right amount of planning at this point. Glad it only took him an hour! Expects several easy re-planning sessions throughout the year due to changing corporate priorities. Worn out after an entire afternoon spent wrestling with a spreadsheet and resource allocations. Hopes she didn’t miss anything in the details. Uncomfortable with the message she will take to the CIO and stakeholders, not to mention the impacts of stress on her team. Expects several painful re-planning sessions throughout the year due to changing corporate priorities.

Big Themes

It is often counter intuitive that dedicating a team to one project at a time will better satisfy stakeholders, yet agile teams that focus exclusively on one thing at a time are able to deliver value more quickly to their organizations because:

  • Starting projects earlier does not ensure they will be finished sooner
  • Project multi-tasking has an unseen penalty cost that will further delay finish dates
  • Focusing on maximizing utilization over project completion has unintended consequences

Resources

Brown, R. (2010, June 29). Multitasking Gets You There Later. Retrieved June 20, 2013, from InfoQ
Cohn, M. (2009). Succeeding With Agile. Boston: Addison-Wesley Professional.
Weinberg, G. M. (1991). Quality Software Management Volume 1: Systems Thinking. Dorset House.

What is Agile

by

Defining Agile

A Brief Look at Agile Resources for the Beginner

There are a number of books and websites that will help anyone interested in, or new to, this incredible subject learn the basics. Here, I share a few websites worth reviewing.

First, every one must become familiar with its genesis and subsequent manifesto. Agile Manifesto

For Newbies

An explanation by Mike Cohn, an authority on the subject.

What is Agile Development?

An interview with Ken Collier, author of “Agile Analytics: A Value-Driven Approach to Business Intelligence and Data Warehousing”

Also, check out my blog post on more recommendations. I ran into someone at a conference and after I replied to them, I created this post about more books and articles I recommend for the new practitioner.

Agile Development Methodologies & Book Recommendations

by

Where to Start…


A friend, who I met at a data geek conference last year, recently emailed me for advice:

“Hi Lynn,

Do you have a recommendation for a good book (or two) about agile development methodologies? I’m trying to get more formal development processes in place here, and I’ve read through most of the Wikipedia articles. I’m looking for something with a little more meat, and maybe some real-world examples. A lot of the focus seems to be on scrum. Is that the predominant agile method that people find most effective? I also saw literature about XP and Kanban. Do you have any opinions on the good and the bad about these other ones? Thanks for your advice, I really appreciate getting your perspective.” [Read more…]