Alerts: You need a budget!
Alert fatigue is a spectre haunting DevOps teams. We all know that the spectre is there. In our post-mortems, we all acknowledge its impact on detection times and burnout. Yet, as an industry, we have no structural solution, and we rely on our burnt-out colleagues to tune, improve, ignore, and delete alerts when we feel the effects of alert fatigue set in. If you have ever felt that your alerts can be divided into categories of "Alerts I can ignore" (warning) and "Alerts I cannot ignore" (critical), then you are well aware that your sanity is finite. You need a budget!