Status Update: DDoS Resolved - Infrastructure Migration Finishing

[update: As of 7pm Friday Jul 20 systems are humming along faster than ever.  Systems have been stable since Sunday the 15th.  There was an issue with jobsco.re shortened links starting Thurs the 19th but those should be resolved now.]

Status Update:

This is a followup to the Distrubted Denial of Service (DDoS) attack that began in the afternoon of Wednesday July 11th.  As we mentioned in our previous post, we have migrated infrastructure providers and Jobscore is now hardened against future DDoS attacks.

As a result of the migration, Jobscore has had to re-distribute its back-end infrastructure.  We worked overnight and unfortunately there was some down time during the day today.  

Issues Resolved:

  • OpenID/Google Account Logins were temporarily down due to the infrastructure switch:  OpenID/Google Account Logins we’re restored as of 7am, Fri Jul 13
  • System Slowdown on Fri Jul 13, 2am-7am.  We experienced some slowdowns and email delivery delays due to infrastructure changes made to handle the DDoS.
  • Down for 20mins from 1pm to 1:24pm while deploying queue management changes to match the new infrastructure.

Jobscore has has less downtime in the last five years than it has in the last 5 days, and we’re committed to making sure this doesn’t happen again. 

We hope that’s it for now.  

Planned Downtime: Our last planned downtime due to the migration will occur on Saturday July 14th at 4pm for approximately 10 minutes.

Thank you for your patience and support during the DDoS attack. 

[updated] JobScore availability

JobScore is under a DDoS attack (distributed denial of service) by someone outside of JobScore who is flooding our servers with requests.

Your data is safe and things are okay. We are doing everything in our power to keep the service available and are putting in place measures to avoid this in the future.  Here’s the play-by-play update:

  • Since ~3:30 PM Pacific time on July 11th 2012 JobScore has been under continuous DDoS attack.
  • We were not returning requests from 3:30PM to 6:30PM Pacific time on July 11th - when we were able to restore service.
  • The attacks continued and we’ve experienced two separate periods of roughly 15 minutes when our ability to return requests was impaired at 10:30AM and 2:00PM Pacific time on July, 12th.  
  • Due to a security flaw related to Wordpress we have disabled the Jobscore blog and will be migrating to a new hosted blog provider.
  • We have engaged with a specialist security service provider and will have permanent DDoS counter-measures in place shortly.

We apologize for the intermittent downtime and appreciate your patience as we  fend off this malicious attack.

Thanks again for your patience and support through this trial,

Dan Arkind
Founder & CEO
JobScore

Loading ...