As part of our data-science and machine learning effort to always provide more relevant features and results, one of our many experiments unfortunately had an unexpected negative impact on our search cluster. We have identified the action that caused the outage and we have made sure to optimize it in a way that keeps this from happening again.