RIPE Atlas probe (re)connection issues

Incident Report for RIPE NCC

Resolved

This incident has been resolved.

The root cause of the issue was an overzealous maintenance process that removed "old" container images from our repository even though they were still actively used. Already running images (probe handlers) were not affected, but new ones could not be started on demand.
Posted Nov 13, 2024 - 12:50 CET

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Nov 13, 2024 - 11:07 CET

Identified

The issue has been identified and a fix is being implemented
Posted Nov 13, 2024 - 10:39 CET

Investigating

We're experiencing a problem with accepting new and reconnecting probes in the infrastructure. Probe that are currently connected, as well as new and ongoing measurements and data delivery are not affected.
Posted Nov 13, 2024 - 09:24 CET
This incident affected: Non-Critical Services (RIPE Atlas).