Internal Operations Issues Observed
Postmortem

on July 7, between 19:03 and 20:37 UTC our monitors in Seoul and Sydney received intermittent “Bad Gateway” 502 responses from a system health check of the helix-run-query service, caused by an error from returned by Googles Identity and Access Management (403).

Considering that the 403 recovered without any manual intervention, as well as the fact that helix-run-query is used for internal logging, it is not likely that any interruption has been observed from this incident.

Posted Jul 07, 2020 - 22:09 UTC

Resolved
This incident has been resolved.
Posted Jul 07, 2020 - 20:18 UTC
Investigating
We are observing issues that are affecting page delivery for Project Helix customers. The issue is under active investigation and we are working with full effort to reach a speedy resolution.
Posted Jul 07, 2020 - 20:12 UTC