6th November 2020

Hook server Involuntary DDoS

We identified an involuntary (hopefully!) DDoS-like traffic, which caused an overload in the hook server (which is used to accept new notification requests).

After identifying the cause, traffic has been restored successfully.

This outage lasted approx. between 04:30 and 06:30 UTC

Some metrics:

  • We experienced about 32mln processed requests before being able to understand and mitigate the issue (~90min).
  • After the mitigation was applied via load balancer filtering, we experienced other ~750mln requests for the next 6hrs.