Question Page

How do you define and set SLAs with engineers?

I'm currently struggling to define checkout error rates for our e-commerce platform. We're currently at 1.5%. Personally, I think it's too high. However, I have nothing to substantiate my opinion.
2 Answers
Virgilia Kaur Pruthi (she/her)
Virgilia Kaur Pruthi (she/her)
Expedia Group Senior Director of Product, Head of Trust and SafetyJanuary 31

I would recommend first building a relationship with your technical lead/engineering counterpart. Have them show you how your e-commerce platform (or product area) works end to end, from the backend perspective. Make sure that you first understand the end to end flow and specifically the systems design (which is critical in any e-comm platform). Once you understand how the customer's journey equates to the systems design, then start looking into each customer interaction with the site and make sure your team is tracking those metrics. You will end up at the checkout rates. If you have a good pulse on SQL or pulling and analyzing data, you could probably do the error rate comparison on your own. If you don't feel comfortable, work with your engineering lead (or data analyst) to dig into those numbers. Build out a report or dashboard that you can look at a regular basis. This will give you the background to ask or share opinions.

680 Views
Mani Fazeli
Mani Fazeli
Shopify Director of ProductDecember 14

Service Level Agreements (SLA) are driven by three factors: (1) industry standard expectations by customers, (2) differentiating your product when marketing, (3) direct correlation with improving KPIs.

For checkout, you'll have uptime as an industry standard, but it's insufficient because subsystems of a checkout can malfunction without the checkout process outright failling. You could consider latency or throughput as market differentiators and would need instrumentation on APIs and client response. With payment failures or shipping calculation failures, you would directly impact conversion rates and trust erosion (hurting repeat buying), which are likely KPIs you care about. So your SLAs need to be a combination of measures that account for all of the above, and your engineering counterparts have to see the evidence that these matter in conjunction.

Of the three types, the one that's most difficult to compare objectively is the third. In your question, you mention 1.5% error rates. You could go on a hunt to find evidence that convinces your engineering counterparts that these are elevated vs. competition, or that they're hurting the business. What's more likely to succeed is running A/B tests that attempt to improve error rates and demonstrating a direct correlation with improving a KPI you care about. That's a more timeboxed exercise, and with evidence, you can change hearts and minds. That's what can lead to more rigorous setting of SLAs and investment in rituals to uphold them.

2167 Views
Top Product Management Mentors
Poorvi Shrivastav
Poorvi Shrivastav
Meta Senior Director of Product Management
James Heimbuck
James Heimbuck
Doppler Principal Product Manager
Paresh Vakhariya
Paresh Vakhariya
Atlassian Director of Product Management (Confluence)
Natalia Baryshnikova
Natalia Baryshnikova
Atlassian Head of Product, Enterprise Agility
Zeeshan Qamruddin
Zeeshan Qamruddin
HubSpot Senior Director of Product Management, Flywheel
Clare Hawthorne
Clare Hawthorne
Oscar Health Senior Director, Product Operations
Orit Golowinski
Orit Golowinski
Jit.io VP of Product Management
Anton Kravchenko
Anton Kravchenko
Carta Sr. Director of Product Management
Tom Alterman
Tom Alterman
Notable Head of Product
Rishabh Dave
Rishabh Dave
Stripe Product Lead, Financial Infrastructure