FS#54233 - [wpa_supplicant] TLS: Failed to load private key

Attached to Project: Arch Linux
Opened by Foudil (foudfou) - Tuesday, 30 May 2017, 15:35 GMT
Last edited by Bartłomiej Piotrowski (Barthalion) - Wednesday, 07 June 2017, 09:24 GMT
Task Type Bug Report
Category Packages: Core
Status Closed
Assigned To Bartłomiej Piotrowski (Barthalion)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 6
Private No

Details

Upgrading to wpa_supplicant-1:2.6-5 breaks my EAP-TLS client setup.

wpa_supplicant is actually driven by connman-1.34-1, with the following config:

[service_someorg_wifi]
Type = wifi
Name = ring
EAP = tls
CACertFile = /etc/ca-certificates/trust-source/anchors/someCA.crt
ClientCertFile = /home/me/.certs/some_bare.pem
PrivateKeyFile = /home/me/.certs/some_key.pem
PrivateKeyPassphrase = s3cret
Identity = me@some.org

Reading my TLS encrypted private key in PEM from the command-line with openssl 1.1.0f works fine.

All I know is that downgrading to wpa_supplicant-1:2.6-4 lets me access the network again.

Here is a log excerpt from wpa_supplicant -dd:

mai 29 13:28:23 mypc wpa_supplicant[3208]: OpenSSL: SSL_use_certificate_file (PEM) --> OK
mai 29 13:28:23 mypc wpa_supplicant[3208]: Enter PEM pass phrase:
mai 29 13:28:23 mypc wpa_supplicant[3208]: OpenSSL: tls_read_pkcs12 - Failed to use PKCS#12 file error:0D0680A8:asn1 encoding routines:asn1_check_tlen:wrong tag
mai 29 13:28:23 mypc wpa_supplicant[3208]: OpenSSL: pending error: error:0D07803A:asn1 encoding routines:asn1_item_embed_d2i:nested asn1 error
mai 29 13:28:23 mypc wpa_supplicant[3208]: OpenSSL: tls_connection_private_key - Failed to load private key error:00000000:lib(0):func(0):reason(0)
mai 29 13:28:23 mypc wpa_supplicant[3208]: TLS: Failed to load private key '/home/me/.certs/some_key.pem'
mai 29 13:28:23 mypc wpa_supplicant[3208]: TLS: Failed to set TLS connection parameters
mai 29 13:28:23 mypc wpa_supplicant[3208]: ENGINE: engine deinit
mai 29 13:28:23 mypc wpa_supplicant[3208]: EAP-TLS: Failed to initialize SSL.

I'm not sure anyone can work anything out of this, but I'll be happy to provide further info.
This task depends upon

Closed by  Bartłomiej Piotrowski (Barthalion)
Wednesday, 07 June 2017, 09:24 GMT
Reason for closing:  Fixed
Comment by Lucian Lepadatu (lucian.lepadatu) - Wednesday, 31 May 2017, 09:01 GMT
Hi,

I can confirm that wpa_supplicant-1:2.6-5 + openssl 1.1.0.f-1 breaks EAP-TLS for me too. I get the exact same errors from wpa_supplicant as in the bug report.
The problem occurred after upgrading openssl 1.1.0.e-1 to 1.1.0.f-1.

The workaround that I use is removing the password for the private key.
I also tried encrypting the private key with different algorithms (3des, aes128, aes256) and although openssl can decrypt them without problems the wpa_supplicant still gives the same errors.

If more info is needed for debug I'll be glad to help.
Comment by Thomas Millward Wright (ttrmw) - Wednesday, 31 May 2017, 13:51 GMT
Yeah, same problem + versions + error messages for me, resulting from upgrading openssl to the .f-1 release today.

A possibly better work around is just to rollback openssl to 1.1.0.e-1? Not really sure of the ramifications of each approach, but it feels like removing the passphrase is not ideal.
Comment by Tim Weippert (weiti) - Wednesday, 31 May 2017, 17:40 GMT
I can confirm, downgrade openssl to 1.1.0.e-1 works. So, this is not an wpa_supplicant bug?

Has someone try to rebuild wpa_supplicant with 1.1.0.f-1? I currently had no access to an Build Environment ...

Just my thoughts.
Comment by Lukasz Siudut (lsiudut) - Wednesday, 31 May 2017, 17:44 GMT
Me and few of my colleagues were also hit by this bug. In my case converting the key and certificate to pkcs12 format helped (connecting via network manager).

Edit: from my perspective also seems that upgrade of openssl triggered the issue.
Comment by Lukasz Siudut (lsiudut) - Wednesday, 31 May 2017, 17:54 GMT
It seems that wpa_supplicant-1:2.6-5 was compiled against openssl-1.1: https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/wpa_supplicant&id=5ee77e5059e6accf8f100ca690ae5010de1e86fa . That would explain why downgrading wpa_supplicant helps. So it seems that for now there are three reasonable workarounds:

- removing password from private key
- downgrading wpa_supplicant
- downgrading openssl
Comment by Lukasz Siudut (lsiudut) - Wednesday, 31 May 2017, 18:23 GMT
I have a suspicious what might have break wpa_supplicant, asked the question in pull request: https://github.com/openssl/openssl/pull/2870
Comment by Mauro Santos (R00KIE) - Wednesday, 31 May 2017, 18:51 GMT
For me if wpa_supplicant is linked against openssl-1.1 it breaks EAP-PEAP, see [1].
I have modified the PKGBUILD (patch is attached) to link against openssl-1.0 but I haven't had the chance to confirm that the recompiled package works. From my previous tests it should work given that openssl-1.0 has not received any updates but I can't confirm until Friday.

[1] http://lists.infradead.org/pipermail/hostap/2017-May/037633.html
Comment by Bartłomiej Piotrowski (Barthalion) - Wednesday, 31 May 2017, 21:14 GMT
Tested it today with EAP-PEAP at my work. It's definitely broken with this as well, but I didn't had time to investigate that. Mauro, I will see if building against openssl 1.0 helps tomorrow morning, but I won't have time for anything more till Monday.
Comment by Lukasz Siudut (lsiudut) - Wednesday, 31 May 2017, 21:56 GMT
I can confirm that I'm correct. I've rebuild openssl with provided patch and wpa_supplicant is connecting successfully. I don't understand why did they change this behavior, maybe there's a good reason for it.

I'll submit a bug...

Edit: https://github.com/openssl/openssl/issues/3594
Comment by Mauro Santos (R00KIE) - Wednesday, 31 May 2017, 22:53 GMT
@Lukasz
That commit might fix things for openssl 1.1 but I'm afraid it might not help with EAP-PEAP. The current openssl 1.0 release already includes that patch and it doesn't break EAP-PEAP. I suspect that this is something a bit more tricky than just what one patch can fix.

@Bartłomiej
I'll also give the recompiled wpa_supplicant version a try this Friday once I have access to a network protected by EAP-PEAP. If I have time I might also test the patch posted by Lukasz but I suspect that it will not help, judging from what I've been told in the mailing list.
Comment by Lukasz Siudut (lsiudut) - Wednesday, 31 May 2017, 23:03 GMT
@ROOKIE it surely won't - it's strictly addressing problems with encrypted PEM certificate loading. It's about the original bug report.
Comment by Lukasz Siudut (lsiudut) - Friday, 02 June 2017, 08:43 GMT
It seems that long-lasting bug in OpenSSL was fixed recently and this is why wpa_supplicant stopped to work. See discussion on github: https://github.com/openssl/openssl/issues/3594 . I'll try to patch wpa_supplicant later and publish it here.

I mailed hostpad mailing list about the issue.
Comment by Lukasz Siudut (lsiudut) - Friday, 02 June 2017, 10:24 GMT
This patch fixes this is semi-proper way that should be compatible with both - older versions of OpenSSH and newer one. Basically I'm doing what @mattcaswell suggested here: https://github.com/openssl/openssl/issues/3594#issuecomment-305493300 - calling both SSL_CTX_() functions and the SSL_() functions. Works for me, would be nice to get feedback from bigger group of testers.
Comment by Mauro Santos (R00KIE) - Friday, 02 June 2017, 14:49 GMT
I have tried the wpa_supplicant linked with openssl 1.0 and no extra patched and it does work as expected for EAP-PEAP.

If I use the patch Lukasz has attached then I get two outcomes:

Linking with openssl 1.1, wpa_supplicant compiles fine but EAP-PEAP is broken.

Linking with openssl 1.0, wpa_supplicant fails to compile and complains:
../src/crypto/tls_openssl.o: In function `tls_connection_set_params':
tls_openssl.c:(.text+0x4909): undefined reference to `SSL_set_default_passwd_cb'
tls_openssl.c:(.text+0x4917): undefined reference to `SSL_set_default_passwd_cb_userdata'
tls_openssl.c:(.text+0x49fe): undefined reference to `SSL_set_default_passwd_cb'

It seems both EAP-PEAP and EAP-TLS problems need to be looked at and fixed by upstream because so far we can't fix both problems at the same time and shipping known broken packages does not seem the right thing to do.

@Lukasz
Have you tried linking wpa_supplicant against openssl 1.0 without adding any extra patches and check if it works (or is also broken)?

@Bartłomiej
We are dealing with two separate bugs here, both due to openssl in some way. Do you want me to open a separate bug report for the problem with EAP-PEAP or will you make this a 2 in 1 fix since both fixes are to be coordinated?
Comment by Lukasz Siudut (lsiudut) - Friday, 02 June 2017, 14:59 GMT
@Mauro this is exclusively for openssl-1.1f from the topic. I'd create separated task for your problem to maintain clarity.
Comment by Tim Weippert (weiti) - Saturday, 03 June 2017, 10:05 GMT
I had rebuild the package on my system with the provided patch from Lukasz and
can confirm that it correctly loads the key file and authenticates correct on the wifi.

Looks good.

[weiti@weiti-p772 ~]$ wpa_supplicant -v
wpa_supplicant v2.6
Copyright (c) 2003-2016, Jouni Malinen <j@w1.fi> and contributors

[weiti@weiti-p772 ~]$ openssl version
OpenSSL 1.1.0f 25 May 2017
Comment by Bartłomiej Piotrowski (Barthalion) - Sunday, 04 June 2017, 21:44 GMT
I will apply Łukasz' patch tomorrow morning; thanks for nailing it down!
Comment by Bartłomiej Piotrowski (Barthalion) - Tuesday, 06 June 2017, 07:35 GMT
Pushed 1:2.6-6 to [testing]. If you could sign it off here, I will move it to [core] sooner.
Comment by Matthias Lisin (matthias.lisin) - Tuesday, 06 June 2017, 08:41 GMT
testing/wpa_supplicant 2.6-6 works here.
thanks!
Comment by Mark Ignacio (marki) - Tuesday, 06 June 2017, 20:22 GMT
testing/wpa_supplicant 2.6-6 + openssl-1.1.0.f-2 successfully decrypts password-protected PEM keys for me too. Thanks!
Comment by Bartłomiej Piotrowski (Barthalion) - Wednesday, 07 June 2017, 09:24 GMT
Moved to stable. Thanks folks.

Loading...