this incident has been resolved

isolated the culprit, systems are running now, we are monitoring this for a couple hours

identified a slow query adding load on the db server

We are investigating an intermittent issue where sermonstudio,net and sermon.net (media centers) pages time out when loading. Live streaming and restreaming is not affected

this issue has been fixed

We are experiencing issues with the sermon.net Roku channel. This is NOT impacting any other platforms

the direct impact on us from these third party issues have been resolved

[01:58 PM PST] We can confirm significant recovery for the DNS resolution errors for Cloudfront distributions.

[01:46 PM PST] We are seeing early signs of recovery, and continue to work toward full recovery.

From our provider: [01:40 PM PST] We can confirm errors for DNS resolution for some Cloudfront distributions. During this time, customers may receive an NXDOMAIN response. Additionally, customers may also experience delayed propagation for changes to CloudFront distributions. We have identified the root cause of the issue and are actively working on multiple paths to resolving the errors. We have verified that our initial mitigation effort on a portion of the affected subsystem was successful, and we are actively working toward performing that mitigation across the fleet. We recommend customers continue to retry any failed requests while we work toward mitigation. A few services that use CloudFront distributions for delivering content may also be affected at this time.

Our service provider is currently experiencing an incident that might cause  some of our web services to fail loading

Details from our provider (AWS): [01:15 PM PST] We are investigating DNS resolution failures for some specific Cloudfront distributions. We are actively investigating and will provide additional information in the next 30-60 minutes.

no new issues were experienced in the last days

We have deployed a new DB fix and are currently monitoring

unfortunately yesterdays issues are reappearing now intermittently. We are investigating this to get to the bottom of it

This incident has been resolved

our internal tests confirmed everything is working now, we will monitor for some time before closing this incident

This incident has been resolved

We are currently experiencing an issue where for some of our customers the direct upload of an episode to YouTube fails. We are working on a fix for this. In the meantime, the workaround is to manually download the video file and upload directly to your YouTube channel.

This does not affect YouTube simulcasting

this incident has been resolved

the new API was deployed and tested, we are monitoring until Oct 30.

We’re currently upgrading our Facebook API integration for simulcasting. During this time, new Facebook connections cannot be established. Existing connections are unaffected. We expect this to be resolved by Wednesday, October 29.

As a temporary workaround, follow this article to manually add and rtmp connection (see note about custom rtmp at the bottom).

Thank you for your patience.