FS#39777 - [OpenSSL bug] Archlinux.org server update, SSL certificate should be revoked and regenerated

Attached to Project: Arch Linux
Opened by Thomas (thomasbk) - Tuesday, 08 April 2014, 04:05 GMT
Last edited by Allan McRae (Allan) - Thursday, 01 May 2014, 08:10 GMT
Task Type Bug Report
Category Web Sites
Status Closed
Assigned To Pierre Schmitz (Pierre)
Aaron Griffin (phrakture)
Thomas Bächler (brain0)
Dan McGee (toofishes)
Allan McRae (Allan)
Ionut Biru (wonder)
Florian Pritz (bluewind)
Jan Alexander Steffens (heftig)
Architecture All
Severity Critical
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 8
Private No

Details

Due to bug https://bugs.archlinux.org/task/39775, the server running archlinux.org is vulnerable as well to a compromise of server memory, including the private key for SSL. This could be confirmed with http://filippo.io/Heartbleed/#archlinux.org/.

The server should be updated, and then the SSL certificate revoked and renewed.
This task depends upon

Closed by  Allan McRae (Allan)
Thursday, 01 May 2014, 08:10 GMT
Reason for closing:  Fixed
Comment by Thomas Bächler (brain0) - Tuesday, 08 April 2014, 09:28 GMT
Servers updated. Getting new SSL certificates takes time and we have no idea if something actually got compromised.
Comment by Pierre Schmitz (Pierre) - Tuesday, 08 April 2014, 16:00 GMT
Note that revocation and reissue of a StartSSL certs does not seem to be free of charge (I guess like 25$ + 50$).

I'd say it's not very likely that someone got our private key during that short period our server was unpatched (and the issue was publicly known).
Comment by Thomas Bächler (brain0) - Tuesday, 08 April 2014, 16:03 GMT
I already asked Aaron to get us a new cert, just to be safe. I pointed him to this bug report so he considers your comment.

If the issue was publicly known for a day, it is still possible that it was known to certain people for a lot longer. I wouldn't take any chances.
Comment by Aaron Griffin (phrakture) - Tuesday, 08 April 2014, 16:16 GMT
I need to get revalidated by StartCom, which is about $60 USD and takes about a day. I do not recall a fee for revocation last time.

I will update the ticket when this is complete
Comment by Pierre Schmitz (Pierre) - Tuesday, 08 April 2014, 16:24 GMT
It's 25$ on top of that; see http://www.startssl.com/#72
Comment by Thomas (thomasbk) - Tuesday, 08 April 2014, 22:16 GMT
Consider taking advantage of https://www.globalsign.com/ssl/ssl-open-source/
Comment by Pierre Schmitz (Pierre) - Wednesday, 09 April 2014, 17:43 GMT
That does not help as only StatSSL can revoke the certs. Anyway, I see no way around this; I just dislike this policy.
Comment by Daniel Micay (thestinger) - Wednesday, 09 April 2014, 17:57 GMT
You're not the only one who dislikes the policy. They might consider changing it if they get enough bad publicity. I doubt that most people using the free tier are going to bother revoking it...

* https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=744027
* https://bugzilla.mozilla.org/show_bug.cgi?id=994033
Comment by arch user (archuser474747) - Wednesday, 09 April 2014, 22:42 GMT
Ars is reporting that a bot has been exploting heartbleed in the wild since at least nov 2013:

http://arstechnica.com/security/2014/04/heartbleed-vulnerability-may-have-been-exploited-months-before-patch/
Comment by Thomas Bächler (brain0) - Friday, 11 April 2014, 11:30 GMT
Aaron, any news on the new certificate?
Comment by Pierre Schmitz (Pierre) - Friday, 11 April 2014, 15:14 GMT
At least for me the process is stuck. While my personal cert was revoked yesterday immediately, the one for archlinux.de is still not revoked and the system does not let me create new ones. About 12 hours ago I was told that "It will take more few hours. We'll revoke it ASAP."
Comment by Thomas Bächler (brain0) - Friday, 11 April 2014, 15:18 GMT
We'll need a new cert first anyway, before revoking the old one.
Comment by Pierre Schmitz (Pierre) - Friday, 11 April 2014, 15:24 GMT
Their COO/CTO told me that this wont be possible. The cert needs to be revoked first. Apparently you have a few hours between the revocation request is accepted and it's actually rolled out. I am not that happy how they handle this; but we are kind of trapped here.
Comment by Pierre Schmitz (Pierre) - Friday, 11 April 2014, 19:32 GMT
@Aaron: After another email exchange I finally got the revoke accepted and could replace the certs. So one might need to push them a little.
Comment by Aaron Griffin (phrakture) - Saturday, 12 April 2014, 03:24 GMT
Sorry, I *just* got personal verification completed some hours ago (they needed to call me on the phone).

I can do the new wildcard cert tomorrow. Sorry for the delay.
Comment by Aaron Griffin (phrakture) - Saturday, 12 April 2014, 22:08 GMT
Turns out some of the site is down for the weekend or something. Sigh. Might need to happen on Monday.
Comment by Aaron Griffin (phrakture) - Monday, 14 April 2014, 15:39 GMT
Just submitted the revocation request, assuming Pierre is right that it takes a few hours. This is a weird process. Maybe we should look into someone else for these certs...

I will generate the new cert as soon as possible and stick it in my home dir on gerolde like I did last time. Will report back when this is complete
Comment by Aaron Griffin (phrakture) - Monday, 14 April 2014, 16:09 GMT
Done. bluewind is going to do the actual server updates, so I'll let him update/close this thing out
Comment by Florian Pritz (bluewind) - Monday, 14 April 2014, 16:55 GMT
Updated the cert on alderaan, someone else please take care of gudrun.

Any other boxes that need updates? celestia and nymeria don't seem to support ssl.
Comment by Florian Pritz (bluewind) - Monday, 14 April 2014, 17:18 GMT
Ok updated gudrun and gerolde too
Comment by Thomas Bächler (brain0) - Monday, 14 April 2014, 21:27 GMT
I updated alberich/releng.

Is there anything left using the old certificate? Aaron, I deleted the newcerts.tar.gz file from your $HOME.
Comment by beta990 (beta990) - Thursday, 24 April 2014, 19:57 GMT
"All good, archlinux.org seems fixed or unaffected!"

Loading...