Testing on Production – deep backend edition
Embrace Production as a first-class testing environment to decrease costs and improve quality.
Thorough testing before merging to master is great, but it doesn’t reveal the unknowns. Staging on shared environments tends to be slow, unreliable and costly to support. Why not just learn from the only true environment by conducting safe and efficient experiments?
This talk is based on my experience of increasing the delivery rate within the context of back-end systems of bol.com (one of the biggest online retailers of the Netherlands; logistics and purchasing domains), where correctness is often a bigger concern than performance, and recovery might require a bit more than users hitting the refresh button of their browser.
Testing on production is often associated with A/B testing or canary releases, but those aren’t always the best - or even applicable - techniques. We’ll look instead at shadow and dry runs, controlled experiments, survival of the fittest; how to apply these techniques and what to be aware of.
Related art
What do you know about testing in production? https://youtu.be/z-ATZTUgaAo Michael Bryzek QCon San Francisco 2016
GTAC 2007: Ed Keyes - Sufficiently Advanced Monitoring is Indistinguishable from Testing https://www.youtube.com/watch?v=uSo8i1N18oc
On Uber’s leverage of Multi-tenancy for progressive delivery https://eng.uber.com/multitenancy-microservice-architecture/
Past
Nov. 26 2019 Utrecht, The Netherlands - DomCode MeetUp slides
Nov. 7 2019, Potstam, Germany – Agile Testing Days slides
Nov 1-2 2019, Kyiv, Ukraine - Devoxx Ukraine slides video
Oct 28-29 2019, Gdańsk, Poland – Agile & Automation Days slides
June 2019 Test Leadership Congress 2019
April 2019 code.talks commerce – video
February 2019 meetup of Continuous Delivery Amsterdam