Search this Site

Site Features

See this page for more about this site including its features.

The Cobourg internet Server was down for 13 ½ hours - from 1:35pm on April 16 to 2:53am April 17. And that's a really bad outage.  The cause was a connection problem in the data centre in Provo, Utah.  As usual, little detail is provided but it was not the actual server so no data will be lost.  Emails would queue waiting to get in and out so no emails would be lost.

The first support comment was:

We have received an update from our Provo Datacenter. It appears that there is a networking issue at this time. Our network engineers are continuing to work through these issues.

Later they said:

We continue working through the issues impacting performance at one of our data centers. Our entire team is fully engaged to restore services quickly. We apologize for any disruptions you have experienced and genuinely appreciate your understanding in this matter.

This despite the fact that many, including me, do not "understand".  It's more than "issues", it's a large number of servers down.

The problem is likely to be hardware ( e.g. a router) and since it is in the Provo location, it originally belonged to Blue Host which was bought out by Endurance International. 

I know that the original owner of Blue Host was always trying to squeeze the most out of each of his pieces of hardware (to maximize profit) so it's possible that much of the hardware was low price and needs upgrading.  Reminds me of the problems Cogeco had in Cobourg when they bought out Northumberland Cable.  Most of the hardware had to be replaced before reliability became acceptable.  But eventually it did become acceptable.  Let's hope the same happens in this case.
When a similar thing happened last August and Corporate headquarters (Endurance International) got involved, they set up a Corporate web site to supposedly communicate better.  After the problem had continued for 2 ½ hours, I sent an email via that site as follows;

My dedicated server at Hostgator has been down for 2 ½ hours; they say it will be fixed soon!  On your Endurance site, I cannot access Blue Host and Hostmonster - but then they are no doubt hosted at Provo which is the data centre affected.  I am disappointed that after the flurry of "we will do better next time in keeping everyone informed" that there is no mention of this here:   At Hostgator, the updates are not frequent although the ticket I submitted got a swift answer saying, in effect "we are working on it".  I'd rate Hostgator with an A, the Provo Data center with a C - it needs to also be A.  I'd guess you'll now lose some more accounts.

I did get a response that said:

Hello John,
Thank you for contacting Endurance International.  I would first like to apologize for this experience, and am grateful for your honest feedback.   I wish I could tell you more than what has already been told to you, however I do not have anything different to say.  Our entire team is engaged, and focused on determining a permanent resolution for this. I encourage you to continue to monitor the Hostgator forum, as well as following our social media feeds as we feel this is where we are able to communicate with our customer base in the quickest and most efficient manner.
Kind Regards,
Raynelle Sibel
Executive Response Manager
Endurance International Group, Inc.

Not much help - but better than ignoring me.

At 7:30, their Facebook page and Forum posting said:

We have restored service for most impacted customers. We understand some customers are still experiencing issues and we assure you that work is continuing non-stop towards a full resolution.

At 8:30 the Forum posting said:

We continue to work towards a full resolution. There remains a percentage of servers still experiencing issues. The NetOps team is working tirelessly towards restoration of all services.

At 10:38 they said:

Service update: The NetOps Team is reporting that they're making steady progress toward restoring connectivity for all customers. While we are not yet able to provide an ETA, the team is working to resolve an issue with our network configuration -- and that it's now isolated to a fairly small group of customers. Please know that everyone at HostGator, from our CEO to our support agents, is engaged in either resolving the issue or supporting our customers.

Unfortunately, Cobourg Internet's server was one of those.

Service was restored at 2:53 am but at  3:51 am , there was this report:

Just a quick update on the work we've done to restore our network. Our Networks Operation team uncovered a bug in firmware utilized in our vendor's hardware that caused the network issues, which impacted some of our customers. We worked closely with the vendor to establish and implement a bug fix. That fix is now being executed and propagating across the network. During this rollout, you may find some performance inconsistencies, which will resolve fairly quickly. Our entire team remains engaged and committing to fully restoring services and performance for all impacted customers.

My apologies for this poor performance.  Reports on Facebook suggest that many customers were severely affected.  Here's one:

Laparoscopy Hospital (in India) Our website is a big hospital website and due to this hostgator many of the patients life is in danger as all their blood report and MRI report of the patient we cannot receive from the different laboratories. These company just to meke money use cheaper datacentre. Even our email down since last 14 hours.

Note that Cobourg Internet uses Hostgator as the provider of hosting since such a service cannot be properly provided except on a large scale.  A good host service needs a secure room, backup power, air conditioning, people to fix it 365/24 and redundant high speed internet.

The question remaining: Will Hostgator have problems like this again and is there any benefit to moving to another host company?  I think that despite the downtime, support was good - I question the quality of the equipment.

Google Ad