One process on our collectors has a slow memory leak that gradually consumes system resources. When configuration management applied changes, this exhausted system resources. In this case, the system became completely unavailable when resources were exhausted.
We use PyPy instead of Python 3 for this process. We recently evaluated Python 3, and confirmed it reduces memory usage for our processes. While we had already planned a migration to Python3 to reduce memory consumption, we had only deployed the fix to the most memory constrained collectors.
We will move forward this transition to Python 3 on all RRCs.