downtime

[j0lle]

ff in verband met de downtime de laatste dagen:

First of all we would like to apologize for the inconveniences caused further to this night power outage, the second in less than 1 week. We would like to inform you on the chain of events, the actions
undertaken so far and the future plans:
Last Saturday, 25/04/2009 at 20:55h, the Scarlet datacenter in Vilvoorde experienced a power-outage. Commercial power to the Scarlet datacenter was interrupted as a result of a traffic accident.
Since the Scarlet datacenter is equipped with a backup-power infrastructure, automatic switch-over to this backup-power was initiated. However, because of a failure in the logic controlling this, switch-over did not occur as designed and the power to the server rooms was interrupted. All Scarlet- and customer-systems housed in this infrastructure became unavailable.
A specialized engineering-team was sent on-site and backup-power was switched on manually after 12 minutes. Major part of the Scarlet-services were restored after approximately 15 minutes.
Some services however experienced problems till Sunday afternoon. Briefly after the incident, the supplier of the backup-power systems has gone on site to investigate the problem in depth. Only a couple of weeks earlier, they had performed a satisfactory routine-maintenance on the system. The Scarlet datacenter was running on backup-power while the supplier was planning a complete overhaul on the primary backup-system.
To anticipate on this overhaul, Scarlet was planning to secure the primary backup-system by a secondary backup-system. Unfortunately, this night at 19h20, the primary backup-power failed before the secondary backup-system was operational. Currently, the datacenter is powered by commercial power, protected by the secondary backup-power which has been taken into service this evening.
Scarlet will keep you closely informed about the progress to completely restore the normal operational modus.

en ivm vandaag:

Het Scarlet datacenter ervaart opnieuw stroomproblemen. Details zijn er nog niet, maar van zodra de informatie binnenkomt, wordt het via dit kanaal aangekondigd. U kan onstijdens de kantooruren steeds bereiken op +32.70250225 en na te kantooruren op support@easyhost.be

The scarlet datacenter is currently experiencing a power outage. Few details are available at this time, but all information will be posted here in real time.
You can contact us by phone on +32.70250225 during business hours and you can also send us email after business hours at support@easyhost.be

(update 17:10) Scarlet bevestigt dat het stroomprobleem dezelfde oorzaak heeft als het probleem van 29 april. Meer updates komen er straks nog aan
(update 17:15) Er is opnieuw stroom, maar om hardwareproblemen ten gevolge van plotse stroomtoevoer te vermijden, zullen de zekeringen systematisch heractiveerd worden. Alle systemen zullen op erg korte termijn terug online komen

voila, nu weet ge't

[Math Wolf]

Bedankt voor die post, ik zat juist die dagen zonder internet, nu weet ik ineens hoe het kwam.

[j0lle]

update

Dear Customer,
Despite all the necessary precautions taken last Wednesday May 5th, we were confronted with another unplanned outage between 16:22h and 22:48h.
Although the power was initially quickly restored, Scarlet decided to shutdown the datacenter in a controlled way again at 20:13h.
Between 20:13 and 22:48h, engineering-teams on-site made the necessary repairs to the datacenter power-supply infrastructure.
The outage was caused by a failure of the main electrical switch making the connection between the commercial power, protected by MUPS (Mobile UPS), and the datacenter computer rooms.
Together with our supplier, we decided to bypass this electrical switch to minimize the outage period and to rule out any re-occurrence of switch-failure (the replacement of such a high-load electrical switches would have taken too long).
The current status of the Scarlet datacenter power-infrastructure is as follows:
Scarlet datacenter is currently powered, as designed, by commercial power, secured by a Mobile UPS.
The failed electrical switch has been by-passed.
The supplier had already started work to improve the robustness of the power-supply to the datacenter. These works will be accelerated.
Meanwhile, all elements that are not used, and are off-line due to the bypass to MUPS will be overhauled and tested.
As soon as the timeline and the exact details of these works are clear, we'll communicate this plan clearly towards you.
For any further information or questions you can contact the Scarlet NMC or your Scarlet account-manager.

dus, alles komt goed

[j0lle]

http://tweakers.net/nieuws/60106/tv-vla ... ingen.html