Peter J Herr

Delivery

Case Study: Delivering Efficiency at a Travel Startup

By

1. Executive Summary

The most innovative thing I’ve done is to create a process of continuous improvement at Plusgrade, a startup in the travel industry.

When I arrived at Plusgrade in January 2018 the company was faced with consistently missed customer deadlines, weekly production issues, and inefficient processes and communication. By applying methods from several disciplines including Agile (Kanban), Continuous Improvement (Kaizen), Change Management, and traditional Project Management, I shaped an organizational culture that upholds customer commitments, is focused and efficient, and flexibly organizes around the most important company priorities.

This effort led to a reduction of defects, improved quality, and increased Product and Engineering efficiency by 40% from Q1 to Q3 2018. Along with the entire team’s efforts and contributions, this helped Plusgrade secure $150 million in investment in November 2018, valuing the company at $450 million.

2. About Plusgrade

Plusgrade is a startup that provides upgrade solutions for customers in the travel industry. Their flagship product allows travelers to purchase an upgrade to premium seats through an auction platform, which in turn, allows airlines and cruise lines to maximize the use of excess inventory.  

Example: An airline customer, Scandinavian Airlines (SAS), uses Plusgrade’s white label platform to reach passengers traveling from Boston to Stockholm by email. Passengers submit bids on upgrades to First Class using the auction platform, and if selected, prior to departure their seat assignments are manually or automatically upgraded (based on airline preferences) through backend reservation systems.

3. Understanding the Problem

As Director of Product at Plusgrade, I oversaw two areas:

  1. Delivery (or execution) of development work for 6 teams (7 Product Managers)
  2. Oversaw the PMO, including 2 Project Managers managing 10+ simultaneous projects.  I was responsible for ensuring Plusgrade kept our commitments to customers from a delivery perspective.

When arriving at Plusgrade in Jan-2018, we consistently did not meet customer commitments. The company faced the following problems:

  • Missed customer deadlines by months causing customers to threaten to switch to a competitor
  • Critical production issues every week (i.e., site availability, reduced customer conversion, execution of seat upgrade)
  • Poor communication within and across teams (i.e., unnecessary and wasteful meetings)
  • Inability to keep up with the workload load to initiatives that never finished or cancelled
  • High stress and low morale across the organization

4. Correcting the Problem

4.1 External Customer Commitments

I began by addressing what I believed to be the biggest bottleneck (concept: theory of constraints). The biggest problem that we faced was the fact that we did not meet our customer commitments: we did not actually do what we said we would. In order to increase trust and good favor with customers, I worked with the team on the following:

  • PMO Process Improvement: I established critical processes for the PMO which did not exist:
    • I developed a project health report to communicate project status, risk and mitigation strategies
    • We developed a project/Gantt timeline template  
    • We defined best practices for managing projects
  • Estimation: Some said we were not ‘supposed’ to do traditional bottoms-up estimation in Kanban. Yet our largest & most complex projects were consistently way over budget. If we want to meet customer committed deadlines, we had to estimate. And so, I developed and matured the estimation process with Product Managers and Engineering Team Leads to obtain estimates for all projects 1+ month in duration. In doing so, the team gained the ability to estimate with varying degrees of speed & granularity, identify risk, and understand critical areas where technical approach should clearly defined.

Result: Our large projects and most critical initiatives for our Customers were now consistently completed on time. This led to increased confidence with our Account Managers, and more importantly improved satisfaction and trust with customers.

4.2 Kanban Process Improvement

At Plusgrade, we were constantly being directed to work on 300% of what our available capacity actually showed we could complete. However, right-sizing this or “doing less” wasn’t a simple argument that the CEO would accept; he expected us to figure it out.

I gained credibility and buy-in from key stakeholders across the organization to embark on numerous process improvement and continuous improvement initiatives. Key examples include:

 

  • Track Expedites: We had too many fires. We established Expedite-level ticket prioritization, whereby anyone in the company could create a critical issue alerting PagerDuty and dropping messages in Slack, triggering the action and awareness needed to resolve the problem. Result: We tracked and measured Expedite issues, and fixed technical debt. This improved Engineering team morale as they had fewer stressful situations and late nights.
  • Define “Ways We Work” Policy: Work was not being prioritized and pulled by Product Managers in an efficient manner. This democratically-defined process document established baseline commitments that everyone agreed to:

 

Before “Ways We Work” Policy After “Ways We Work” Policy
  • Tickets with no descriptions
  • Tickets worked in incorrect status
  • Developers pulling own tasks
  • PMs pulling incorrect priority tasks
  • Poor communication and status
  • Tasks clearly defined/detailed
  • Tickets always up to date
  • Only PMs pulling tasks with clear understanding of priority
  • WIP limits defined
  • Team metrics/goals defined
  • Fewer meetings; better communication

Result: The company objectives/KPIs were now much more closely aligned with development work performed by the 6 Product & Engineering teams. We reduced inefficiency and waste.

  • Prioritization Meeting: With 65+ customers and ever-changing priorities for 6 engineering teams, Product Managers didn’t always have the latest information from Leadership, Sales or Account Management about what’s most important to the customer. We established the Prioritization Meeting to review incoming tasks just before development would start to ensure it was always next highest in priority and tickets were clean/ready. Result: This provided much needed accountability so that engineering capacity was always applied to the most important company priorities.

4.3 Product and Capacity Planning

Using a revenue share model with customers, when a new contract was signed Plusgrade began to get paid once the product went live and was processing Upgrade payments.  What I learned and expressed clearly was that when a new customer was signed, the implementation of their product immediately became the highest priority project for the company, to be started immediately as it represented the highest revenue opportunity for the company. This was effectively like adding an additional 10-15% company-wide workload to the top of the priority list for the next 8-12 weeks onto a team who’s capacity was already committed to other work. These signings were relatively unpredictable, coming in fast from Sales.

Plusgrade planned upcoming work in 3-month increments, and was continuing to underestimate and overcommit to the amount of work that could be completed. Without key data points and insights, the Leadership team lacked the appropriate details to collectively understand why Product and Engineering was having project delays and missed milestones.  

Several Operational metrics I put in place include:

 

  • System and Customer Lead Time
  • Work in Progress (WIP)
  • Escape Defects (bugs allowed into production)
  • # of trips from testing back to development
  • Ticket type (i.e., bug vs improvement)
  • Type of Request

 

Result: These metrics allowed us to quantify and report on where effort was being spent and how Product and Engineering teams were improving.

  • Development teams now had more information on their own performance which made their retrospectives more effective.
  • Product and Engineering more clearly communicated progress on key initiatives, where capacity was being spent, steps being taken to improve efficiency, and ultimately was able justify additional staff to support Product efforts for FY 2019.

5 Summary

Plusgrade’s Product and Engineering organizations had poor communication, lacked efficiency and did not meet customer commitments. Through initiatives I led, the team was able to correct these issues resulting in the following:

  • Process improvement efforts increased customer satisfaction, reduced wasted effort, and increased efficiency by 40% from Q1 to Q3 2018.
  • The Leadership team had a better understanding of 1) current Product and Engineering initiative status and health and 2) where capacity was being spent, which improved organizational alignment.
  • The Product and Engineering team justified additional resources to expand the 40+ person team by 30 people for FY 2019.
  • These efforts supported Plusgrade’s ability to secure $150 million in investment in November 2018, valuing the company at $450 million (https://bit.ly/2IgUicm).

Creating a Culture of Continuous Improvement at Work

By

Some companies and start-ups struggle to deliver new products and features on time.

Some take too long to innovate on their products. Their MVPs are poorly defined, and they simply don’t know how to focus on the things that will make their business successful.

Some overburden their teams. They are overworked and stressed which leads to poor quality and cutting corners.

Many organizations waste a good portion of their development capacity on inefficient delivery or initiatives that won’t move the needle. 

  1. Do you complete everything you set out to each sprint, month, or quarter?
  2. Are you confident your currently defined MVP is the true MVP?
  3. Are you certain your delivery team is building the right things given all your competing priorities?
  4. Are product development or engineering decisions made with transparency to the business?
  5. Do you find time to prioritize maintenance, technical debt, or product innovation?
  6. Does your organization try to do too much in too little time, yet fail to deliver much of anything on time or with quality?

Start-ups and technology companies should be constantly improving their product development and delivery practices. If you haven’t already, the organization and team should collaboratively establish best practices, standards, and processes for how work gets done.  Not bureaucracy, but collaboration with everyone having a voice.

Next, we should measure our team’s performance and use feedback loops to maintain and improve these policies.  Continuous improvement- be that of yourself, your team, or your company- is something that requires continued attention in order to get to the next level.

Overburdened teams often feel stuck and that improvement is impossible.

If you’ve been in this situation, it can be nerve-wracking, stressful, and you find you make mistakes because you’re overwhelmed. Once in a situation like this, it can be difficult to know how to stabilize it. Two things to consider:

  1. Is your team working efficiently? Have you made continuous improvements to your team’s standards and best practices to improve? If not, should you be spending that additional 10% of your 110% effort to make your team more efficient?
  2. Can you make a strong case as to why your team needs more resources?

      For each 10% efficiency gain you achieve  

X    For every 10 people on your team

=    Increased capacity equivalent to +1 person

Remember, it’s not about resources, it’s about resourcefulness.

The Role of the Delivery Manager

By

The job title of the Delivery Manager has become more and more prevalent over the past years. Delivery Managers oversee the development and execution of new technology products and services. They are responsible for working with product management and engineering teams to ensure these efforts are successfully executed.

In many companies today, the responsibilities of the Delivery Manager tend to be part of other job titles. Team members who do this job (whether they specifically acknowledge it or not) include the ScrumMaster, Project Manager, Product Manager, Technical Lead, or Program Manager. There’s no problem with this, as long as the responsibilities are clear and the work is done well.

Delivery Managers’ responsibilities tend to include the following:

  • Directly oversees the work processes and tasks being performed by the teams
  • Enables visibility of project progress and maintains strong communication with appropriate stakeholders
  • Ensures the team is operating efficiently, working smoothly, resolving bottlenecks and ensuring best practices are followed
  • Works collaboratively to define and improve policies, definitions of done, and way the team works
  • Responsible for ensuring work is completed accurately and on time
  • Encourages and motivates the team as an expert facilitator
  • Creates and enables an environment for continuous improvement
  • Ensures the right work gets pulled into progress based on business prioritization
  • Coordinates team management duties including planning, time tracking, scheduling of tasks, estimation, etc

A competent and strong Delivery Manager focuses on managing flow to keep their team working efficiently and unblocked. This is the difference between being proactive versus being reactive.

As companies continue to adopt agile practices beyond Scrum, including lean, kanban, XP, and devops principles, they will steadily move in the direction of becoming ‘service delivery organizations’, becoming ever more fit for purpose and aligning their operations and engineering organizations directly in line with the needs of the business.