Halyard Retail · Commerce platform
Found the p95 regression that six dashboards missed
Slicing latency by consumer rather than by service revealed that one SDK version was responsible for 71% of the platform's slow requests.
- −42%Platform p95 after one SDK deprecation
- 71%Of slow requests traced to one client version
- 1 dayFrom first look to root cause
The problem
Halyard's p95 had been climbing for two quarters. Every service dashboard looked healthy in isolation, and the platform team had rebuilt its tracing twice looking for the culprit.
The problem was not a service. It was a client: an old SDK release that made three sequential calls where the current one made a single batched request — and it was still shipping inside two large partners' apps.
What they did
Latency by integration
The consumer view broke p95 down by integration and SDK version. The offending release was visible in the first screen.
Deprecation tracking
Halyard marked the endpoint pattern deprecated and watched the count of calls from affected versions fall week by week.
Targeted outreach
Instead of a platform-wide announcement, two partners got a specific message with their own numbers attached.
What changed
- Platform p95 fell 42% within six weeks of the two partners upgrading.
- Halyard now reviews consumer-level latency in the same meeting as service latency.
- The deprecation playbook is documented and has been run four more times.
“Seeing our p95 split by integration told us in an afternoon what six months of service dashboards had not.”
Their stack
- Node origin services
- Fastly edge
- Flintwake sidecar collector
- Grafana
- Linear
Halyard Retail is a fictional company created for this template. Every figure on this page is demonstration data.
More stories
Two others worth reading.
- Payments infrastructure
Four days to under an hour on API support
A payments API with 340 integrators replaced screenshot-driven support with wake IDs, and cut median ticket resolution from four days to fifty-one minutes.
Read - Freight and logistics
Caught a settlement regression with eleven thousand replays
Before every release, Cedarlift replays a slice of real production traffic against the candidate build and diffs the responses.
Read
Run the same experiment on one endpoint.
Four minutes to the first captured request, and no change to your application code.