Server Downtime and Data Loss

Archival storage for Announcements. Peruse old Announcements here!

Moderators: Global Moderators, Community Team

Forum rules
Please read the Community Guidelines before posting.

Server Downtime and Data Loss

Postby bigWham on Fri Oct 04, 2013 8:14 am

In the late evening of Oct 3 (CC Time) one of our core system data tables suffered data loss and could not be recovered.

The only efficient solution was to roll back our entire database to the most recent backup, which happened to be approximately 24 hours before.

Unfortunately all data from the period between the backup and the event has been lost, including moves made, games started, chat, private messages and forum posts.

Everything is back and running now and 24 hours have been added to all game timers.

We apologize for the inconvenience and will be crediting Premium Members with a 2 days of extension to their Membership, and Freemiums with 4 free speed games in recognition.

Sincerely,
A weary Admin crew.
Brigadier bigWham
Webmaster
Webmaster
 
Posts: 2361
Joined: Mon Aug 26, 2013 12:08 pm
Medals: 25
Standard Achievement (2) Doubles Achievement (1) Assassin Achievement (1) Manual Troops Achievement (1) Freestyle Achievement (1)
Polymorphic Achievement (1) Nuclear Spoils Achievement (1) Fog of War Achievement (2) Trench Warfare Achievement (1) Speed Achievement (2)
Random Map Achievement (1) Cross-Map Achievement (1) Beta Map Achievement (1) Tournament Achievement (5) General Achievement (4)

Re: Server Downtime and Data Loss

Postby wi1bert on Fri Oct 04, 2013 8:23 am

Sounds like a horrible 24 hours for you guys. Well done and thank you.
Captain wi1bert
 
Posts: 75
Joined: Wed Oct 19, 2011 4:19 pm
Medals: 41
Standard Achievement (2) Doubles Achievement (2) Triples Achievement (2) Quadruples Achievement (2) Terminator Achievement (1)
Assassin Achievement (1) Manual Troops Achievement (1) Freestyle Achievement (1) Polymorphic Achievement (1) Nuclear Spoils Achievement (1)
Fog of War Achievement (3) Trench Warfare Achievement (2) Speed Achievement (1) Teammate Achievement (1) Random Map Achievement (2)
Cross-Map Achievement (3) Ratings Achievement (3) Tournament Achievement (2) General Achievement (1) Clan Achievement (9)

Re: Server Downtime and Data Loss

Postby timrs2001 on Fri Oct 04, 2013 8:24 am

Well that's a drag but it's certainly better than the alternative. Thanks for the quick work.
"I have come here to chew bubblegum and kick ass...and I'm all out of bubblegum"
User avatar
Lieutenant timrs2001
 
Posts: 29
Joined: Fri Sep 12, 2008 2:17 pm
Medals: 15
Standard Achievement (1) Doubles Achievement (3) Triples Achievement (1) Quadruples Achievement (1) Manual Troops Achievement (1)
Freestyle Achievement (1) Nuclear Spoils Achievement (1) Fog of War Achievement (1) Teammate Achievement (1) Ratings Achievement (4)

Re: Server Downtime and Data Loss

Postby EminenceGrise on Fri Oct 04, 2013 8:25 am

Well done team, those of us in the backup and restore of data salute you.
User avatar
Colonel EminenceGrise
 
Posts: 29
Joined: Wed Jun 23, 2010 10:25 pm
Location: Canberra, Australia
Medals: 87
Monthly Leader Bronze (1) Standard Achievement (4) Doubles Achievement (2) Triples Achievement (2) Quadruples Achievement (2)
Terminator Achievement (2) Assassin Achievement (2) Manual Troops Achievement (2) Freestyle Achievement (1) Polymorphic Achievement (1)
Nuclear Spoils Achievement (1) Fog of War Achievement (4) Trench Warfare Achievement (1) Speed Achievement (1) Teammate Achievement (1)
Random Map Achievement (1) Cross-Map Achievement (3) Battle Royale Achievement (1) Bot Achievement (1) Ratings Achievement (4)
Tournament Achievement (1) General Achievement (1) Clan Achievement (48)

Re: Server Downtime and Data Loss

Postby Dustin45 on Fri Oct 04, 2013 8:26 am

A simple sorry would be better then offering something that doesnt help with the problem that 24 hours of data loss created. I understand that things happen . So good job that it wasnt worse than 24 hours. CAuse im sure if it was more than 24 hours of lost data there would not be many people that would have faith and continue playing on a site than cant do a back up.
Last edited by Dustin45 on Sat Oct 05, 2013 9:28 am, edited 1 time in total.
User avatar
Major Dustin45
 
Posts: 6
Joined: Wed Nov 29, 2006 10:17 pm
Location: Hell
Medals: 6
Standard Achievement (2) Manual Troops Achievement (1) Fog of War Achievement (1) Speed Achievement (1) Ratings Achievement (1)

Re: Server Downtime and Data Loss

Postby hyposquasher on Fri Oct 04, 2013 8:26 am

Thanks much for your efforts in the restore. The IT/Infrastructure side of things is exactly what I do for a living. These kind of high-tension outages suck. At least you had backups. Nicely done.
This image cannot be clicked.
Image

High Score: 4086 - 11-27-2014
User avatar
General hyposquasher
 
Posts: 233
Joined: Thu Aug 16, 2007 1:19 pm
Location: Pennsylvania, USA
Medals: 50
Standard Achievement (2) Doubles Achievement (3) Triples Achievement (3) Quadruples Achievement (3) Terminator Achievement (1)
Manual Troops Achievement (1) Freestyle Achievement (1) Polymorphic Achievement (1) Nuclear Spoils Achievement (1) Fog of War Achievement (3)
Trench Warfare Achievement (2) Speed Achievement (1) Teammate Achievement (1) Random Map Achievement (2) Cross-Map Achievement (3)
Beta Map Achievement (1) Ratings Achievement (4) Tournament Achievement (1) General Achievement (2) Clan Achievement (14)

Re: Server Downtime and Data Loss

Postby curiousone1 on Fri Oct 04, 2013 8:29 am

These things happen. Pleased all is back up and appr :D eciate the membership extension
Image
Lieutenant curiousone1
 
Posts: 20
Joined: Wed Sep 07, 2011 11:15 am
Medals: 45
Standard Achievement (4) Doubles Achievement (2) Triples Achievement (2) Quadruples Achievement (2) Terminator Achievement (2)
Assassin Achievement (1) Manual Troops Achievement (2) Freestyle Achievement (1) Polymorphic Achievement (1) Nuclear Spoils Achievement (2)
Fog of War Achievement (3) Trench Warfare Achievement (2) Speed Achievement (2) Teammate Achievement (2) Random Map Achievement (1)
Cross-Map Achievement (3) Beta Map Achievement (2) Bot Achievement (1) Ratings Achievement (4) Clan Achievement (4)
Challenge Achievement (2)

Re: Server Downtime and Data Loss

Postby Artimis on Fri Oct 04, 2013 8:29 am

That explains why all my games vanished. :D
==================================================
This post was sponsored by Far-Q Industries.

Far-Q Industries: Telling you where to go since 2008.
User avatar
Captain Artimis
 
Posts: 809
Joined: Sat Oct 11, 2008 9:09 am
Location: Right behind ya!!! >:D
Medals: 34
Standard Achievement (4) Doubles Achievement (2) Triples Achievement (2) Quadruples Achievement (2) Terminator Achievement (3)
Assassin Achievement (1) Manual Troops Achievement (1) Freestyle Achievement (3) Nuclear Spoils Achievement (1) Fog of War Achievement (3)
Trench Warfare Achievement (1) Speed Achievement (2) Teammate Achievement (1) Cross-Map Achievement (3) Beta Map Achievement (1)
Ratings Achievement (4)

Re: Server Downtime and Data Loss

Postby L M S on Fri Oct 04, 2013 8:30 am

Dang. Sounds like a mess.
Glad I picked yesterday to work on the bot medal, about 50 times :(
“One of God's own prototypes.....never even considered for mass production.
Too weird to live, and too rare to die.”
User avatar
Lieutenant L M S
SoC Training Adviser
 
Posts: 2074
Joined: Fri Dec 01, 2006 5:00 pm
Location: Denver, Colorado USA
Medals: 161
Standard Achievement (4) Doubles Achievement (4) Triples Achievement (4) Quadruples Achievement (4) Terminator Achievement (3)
Assassin Achievement (3) Manual Troops Achievement (4) Freestyle Achievement (4) Polymorphic Achievement (1) Nuclear Spoils Achievement (4)
Fog of War Achievement (4) Trench Warfare Achievement (3) Speed Achievement (3) Teammate Achievement (3) Random Map Achievement (3)
Cross-Map Achievement (4) Beta Map Achievement (1) Battle Royale Achievement (1) Bot Achievement (3) Ratings Achievement (4)
Tournament Achievement (30) General Achievement (13) Clan Achievement (18) Training Achievement (1) Challenge Achievement (1)

Re: Server Downtime and Data Loss

Postby sarphinius on Fri Oct 04, 2013 8:30 am

Nice works, folks. Get some sleep, you earned it!
Lieutenant sarphinius
 
Posts: 12
Joined: Sun Aug 07, 2011 8:39 pm
Medals: 38
Standard Achievement (4) Doubles Achievement (2) Triples Achievement (1) Quadruples Achievement (2) Terminator Achievement (1)
Manual Troops Achievement (2) Freestyle Achievement (1) Polymorphic Achievement (1) Nuclear Spoils Achievement (1) Fog of War Achievement (4)
Trench Warfare Achievement (2) Speed Achievement (3) Teammate Achievement (1) Random Map Achievement (1) Cross-Map Achievement (1)
Ratings Achievement (1) Tournament Achievement (1) General Achievement (1) Clan Achievement (8)

Re: Server Downtime and Data Loss

Postby betiko on Fri Oct 04, 2013 8:31 am

had some turns that really went well and that went bad for the opponent.. I guess dice don't keep this kind of stuff into consideration? lol..
Image
User avatar
General betiko
 
Posts: 6119
Joined: Fri Feb 25, 2011 3:05 pm
Location: J'vais vous montrer qui c'est Raoul
Medals: 162
Conquer Cup Gold Achievement (1) Standard Achievement (4) Doubles Achievement (3) Triples Achievement (3) Quadruples Achievement (3)
Terminator Achievement (2) Assassin Achievement (3) Manual Troops Achievement (3) Freestyle Achievement (3) Polymorphic Achievement (2)
Nuclear Spoils Achievement (3) Fog of War Achievement (4) Trench Warfare Achievement (3) Speed Achievement (4) Teammate Achievement (2)
Random Map Achievement (3) Cross-Map Achievement (4) Beta Map Achievement (2) Battle Royale Achievement (2) Bot Achievement (4)
Ratings Achievement (4) Tournament Achievement (17) General Achievement (31) Clan Achievement (12) Training Achievement (2)

Re: Server Downtime and Data Loss

Postby benga on Fri Oct 04, 2013 8:33 am

Can you slow down a bit in your updates and new things, there have been a lot of setbacks lately.
User avatar
Private 1st Class benga
 
Posts: 4497
Joined: Tue Nov 17, 2009 4:15 pm
Medals: 173
Standard Achievement (4) Doubles Achievement (4) Triples Achievement (3) Quadruples Achievement (3) Terminator Achievement (3)
Assassin Achievement (3) Manual Troops Achievement (3) Freestyle Achievement (2) Polymorphic Achievement (2) Nuclear Spoils Achievement (3)
Fog of War Achievement (4) Trench Warfare Achievement (2) Speed Achievement (2) Teammate Achievement (2) Random Map Achievement (3)
Cross-Map Achievement (4) Beta Map Achievement (1) Battle Royale Achievement (2) Tournament Achievement (31) General Achievement (16)
Clan Achievement (38) Challenge Achievement (6) Tournament Contribution (31) General Contribution (1)

Re: Server Downtime and Data Loss

Postby generalmighty on Fri Oct 04, 2013 8:35 am

That is great news. I lost a 20 v 4 roll yesterday afternoon.
Captain generalmighty
 
Posts: 25
Joined: Wed Jul 14, 2010 10:27 am
Medals: 35
Standard Achievement (3) Doubles Achievement (2) Triples Achievement (2) Quadruples Achievement (2) Terminator Achievement (1)
Assassin Achievement (1) Manual Troops Achievement (2) Freestyle Achievement (1) Nuclear Spoils Achievement (1) Fog of War Achievement (4)
Trench Warfare Achievement (2) Teammate Achievement (1) Random Map Achievement (2) Cross-Map Achievement (3) Ratings Achievement (3)
Clan Achievement (5)

Re: Server Downtime and Data Loss

Postby tarareinhart on Fri Oct 04, 2013 8:35 am

Terrible
Private 1st Class tarareinhart
 
Posts: 1
Joined: Wed Dec 26, 2007 10:25 pm
Medals: 11
Standard Achievement (3) Manual Troops Achievement (1) Nuclear Spoils Achievement (1) Fog of War Achievement (2) Trench Warfare Achievement (2)
Random Map Achievement (1) Cross-Map Achievement (1)

Re: Server Downtime and Data Loss

Postby robchask on Fri Oct 04, 2013 8:36 am

These things happen no problem :)
Corporal 1st Class robchask
 
Posts: 4
Joined: Mon Aug 03, 2009 11:58 am
Medals: 44
Standard Achievement (4) Doubles Achievement (3) Triples Achievement (2) Quadruples Achievement (2) Terminator Achievement (2)
Assassin Achievement (2) Manual Troops Achievement (4) Freestyle Achievement (3) Nuclear Spoils Achievement (1) Fog of War Achievement (4)
Trench Warfare Achievement (3) Speed Achievement (2) Teammate Achievement (3) Random Map Achievement (1) Cross-Map Achievement (3)
Beta Map Achievement (1) Ratings Achievement (3) Tournament Achievement (1)

Re: Server Downtime and Data Loss

Postby ausprime on Fri Oct 04, 2013 8:37 am

It's Ok. I'm fine with taking my turn again xD
Lieutenant ausprime
 
Posts: 1
Joined: Tue Nov 02, 2010 9:20 am
Medals: 6
Standard Achievement (2) Manual Troops Achievement (1) Freestyle Achievement (1) Fog of War Achievement (2)

Re: Server Downtime and Data Loss

Postby iAmCaffeine on Fri Oct 04, 2013 8:37 am

We've experienced this in work before and it's a real pain. Still, this is incredibly frustrating for our current clan war as moves that had already been played, whether as we wanted or not, now have to be re-made and have so far all gone terribly.

benga wrote:Can you slow down a bit in your updates and new things, there have been a lot of setbacks lately.

+1
Image
User avatar
Major iAmCaffeine
 
Posts: 3821
Joined: Mon Apr 01, 2013 5:38 pm
Medals: 78
Standard Achievement (3) Doubles Achievement (3) Triples Achievement (3) Quadruples Achievement (3) Terminator Achievement (2)
Assassin Achievement (2) Manual Troops Achievement (2) Freestyle Achievement (2) Polymorphic Achievement (2) Nuclear Spoils Achievement (2)
Fog of War Achievement (4) Trench Warfare Achievement (3) Speed Achievement (2) Teammate Achievement (2) Random Map Achievement (3)
Cross-Map Achievement (4) Beta Map Achievement (2) Battle Royale Achievement (3) Ratings Achievement (4) Tournament Achievement (8)
General Achievement (6) Clan Achievement (5) Challenge Achievement (2) Tournament Contribution (1) General Contribution (5)

Re: Server Downtime and Data Loss

Postby lutherxyz on Fri Oct 04, 2013 8:38 am

just one the Best game ever played here!!! 12 player and went to capt.. now i wont win unless i get lucky and repeats.. dont think that will happen!! Go Cardinals!!!!!
Captain lutherxyz
 
Posts: 5
Joined: Wed Sep 08, 2010 7:54 am
Location: st louis missouri
Medals: 10
Standard Achievement (3) Manual Troops Achievement (1) Freestyle Achievement (1) Speed Achievement (2) Ratings Achievement (3)

Re: Server Downtime and Data Loss

Postby Evil Semp on Fri Oct 04, 2013 8:39 am

betiko wrote:had some turns that really went well and that went bad for the opponent.. I guess dice don't keep this kind of stuff into consideration? lol..


I had some turns that went bad for me a good for the other guy.
Image
User avatar
Sergeant Evil Semp
 
Posts: 7177
Joined: Sun Jul 02, 2006 8:50 pm
Medals: 81
Standard Achievement (4) Doubles Achievement (3) Triples Achievement (2) Quadruples Achievement (2) Terminator Achievement (3)
Assassin Achievement (2) Manual Troops Achievement (1) Freestyle Achievement (2) Polymorphic Achievement (1) Nuclear Spoils Achievement (2)
Fog of War Achievement (3) Trench Warfare Achievement (2) Speed Achievement (2) Teammate Achievement (2) Random Map Achievement (2)
Cross-Map Achievement (3) Beta Map Achievement (1) Ratings Achievement (2) Tournament Achievement (21) General Achievement (7)
Clan Achievement (2) General Contribution (12)

Re: Server Downtime and Data Loss

Postby Metalmarsh89 on Fri Oct 04, 2013 8:39 am

Ah, that's why my games are missing. Thanks for the message, and sorry to hear that. :(
User avatar
Cook Metalmarsh89
 
Posts: 21
Joined: Wed Sep 18, 2013 10:06 am
Medals: 20
Standard Achievement (2) Doubles Achievement (1) Triples Achievement (1) Assassin Achievement (1) Manual Troops Achievement (1)
Freestyle Achievement (1) Fog of War Achievement (2) Trench Warfare Achievement (1) Speed Achievement (2) Random Map Achievement (2)
Cross-Map Achievement (1) Beta Map Achievement (1) Ratings Achievement (2) Tournament Achievement (1) General Achievement (1)

Re: Server Downtime and Data Loss

Postby timrs2001 on Fri Oct 04, 2013 8:40 am

Dustin45 wrote:well the four freemium games doesnt help when i have 7 games going already. Makes me happy I dont pay for this anymore

How do you have 7 games going when you don't pay to be on here? I can only get 4.
"I have come here to chew bubblegum and kick ass...and I'm all out of bubblegum"
User avatar
Lieutenant timrs2001
 
Posts: 29
Joined: Fri Sep 12, 2008 2:17 pm
Medals: 15
Standard Achievement (1) Doubles Achievement (3) Triples Achievement (1) Quadruples Achievement (1) Manual Troops Achievement (1)
Freestyle Achievement (1) Nuclear Spoils Achievement (1) Fog of War Achievement (1) Teammate Achievement (1) Ratings Achievement (4)

Re: Server Downtime and Data Loss

Postby blakebowling on Fri Oct 04, 2013 8:40 am

benga wrote:Can you slow down a bit in your updates and new things, there have been a lot of setbacks lately.

Just to note: This wasn't caused by an update. Database corruption is something that happens occasionally. (To put it in perspective, the last time CC had data loss like this was in 2010).
12:39:59 AM rdsrds2120: sorry, I had a lot of lasagna tonight
12:40:06 AM rdsrds2120: I'm pretty out of it. CRAAZZYYY NIGHT
12:31:04 * Metsfanmax quit (kicked from Social by Metsfanmax - reason: haha gotcha now)
12:59:32 * #1_stunna gropes blakebowling
Private blakebowling
Administrator
Administrator
 
Posts: 5055
Joined: Wed Jan 23, 2008 12:09 pm
Location: 127.0.0.1
Medals: 41
Standard Achievement (3) Doubles Achievement (1) Triples Achievement (1) Quadruples Achievement (1) Terminator Achievement (1)
Assassin Achievement (2) Manual Troops Achievement (1) Freestyle Achievement (3) Fog of War Achievement (3) Speed Achievement (3)
Teammate Achievement (1) Random Map Achievement (1) Ratings Achievement (3) Tournament Achievement (1) General Achievement (3)
Tournament Contribution (3) General Contribution (10)

Re: Server Downtime and Data Loss

Postby Armandolas on Fri Oct 04, 2013 8:43 am

Well, it happens.
1 day reverted is not so bad, and i can easily forget the annoyance of repeating some turns again(some in benefit, some not so much)
Please add a comment on my new map:
Image
User avatar
Brigadier Armandolas
 
Posts: 1704
Joined: Fri Jun 06, 2008 6:32 am
Location: Lisbon
Medals: 99
Standard Achievement (3) Doubles Achievement (2) Triples Achievement (2) Quadruples Achievement (3) Terminator Achievement (2)
Assassin Achievement (1) Manual Troops Achievement (2) Freestyle Achievement (2) Polymorphic Achievement (1) Nuclear Spoils Achievement (2)
Fog of War Achievement (3) Trench Warfare Achievement (3) Speed Achievement (2) Teammate Achievement (2) Random Map Achievement (2)
Cross-Map Achievement (4) Beta Map Achievement (2) Bot Achievement (1) Ratings Achievement (4) Tournament Achievement (19)
General Achievement (4) Clan Achievement (5) Challenge Achievement (1) Tournament Contribution (27)

Re: Server Downtime and Data Loss

Postby fishydance on Fri Oct 04, 2013 8:45 am

For some reason this morning I have this weird sense of déjà vu, like maybe I repeated the past day or something..... :lol:
Image
User avatar
Colonel fishydance
 
Posts: 437
Joined: Wed Aug 15, 2007 8:28 am
Medals: 38
Standard Achievement (1) Doubles Achievement (2) Triples Achievement (4) Quadruples Achievement (3) Manual Troops Achievement (1)
Nuclear Spoils Achievement (1) Fog of War Achievement (2) Trench Warfare Achievement (1) Speed Achievement (1) Teammate Achievement (2)
Cross-Map Achievement (3) Beta Map Achievement (1) Bot Achievement (1) Ratings Achievement (4) Tournament Achievement (3)
General Achievement (1) Clan Achievement (7)

Next

Return to Announcement Archives

Who is online

Users browsing this forum: No registered users

Login