FS#22897 - [openssh] SSH is falling to connect to some server Read from socket failed: Connection reset by peer
Attached to Project:
Arch Linux
Opened by Ronan Arraes Jardim Chagas (Ronis_BR) - Monday, 14 February 2011, 22:22 GMT
Last edited by Gaetan Bisson (vesath) - Saturday, 09 April 2011, 10:46 GMT
Opened by Ronan Arraes Jardim Chagas (Ronis_BR) - Monday, 14 February 2011, 22:22 GMT
Last edited by Gaetan Bisson (vesath) - Saturday, 09 April 2011, 10:46 GMT
|
Details
Description:
It seems that Archlinux is having problem to connect to some servers using ssh. I have at my internal network three devices: 1) A bleeding edge Arch Linux; 2) An one year old Arch Linux; 3) An android cellphone. When I try to connect to my remote server, 1) hangs and 2) and 3) connects perfectly. Btw, I can connect 1) to a local ssh server. The server log isn't showing anything about the access and the client debug log is: OpenSSH_5.8p1, OpenSSL 1.0.0d 8 Feb 2011 debug1: Reading configuration data /etc/ssh/ssh_config debug1: Applying options for * debug2: ssh_connect: needpriv 0 debug1: Connecting to ronan-ita [161.24.19.187] port 22. debug1: Connection established. debug1: identity file /home/ronan/.ssh/id_rsa type -1 debug1: identity file /home/ronan/.ssh/id_rsa-cert type -1 debug3: Incorrect RSA1 identifier debug3: Could not load "/home/ronan/.ssh/id_dsa" as a RSA1 public key debug2: key_type_from_name: unknown key type '-----BEGIN' debug3: key_read: missing keytype debug2: key_type_from_name: unknown key type 'Proc-Type:' debug3: key_read: missing keytype debug2: key_type_from_name: unknown key type 'DEK-Info:' debug3: key_read: missing keytype debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug3: key_read: missing whitespace debug2: key_type_from_name: unknown key type '-----END' debug3: key_read: missing keytype debug1: identity file /home/ronan/.ssh/id_dsa type 2 debug1: identity file /home/ronan/.ssh/id_dsa-cert type -1 debug1: identity file /home/ronan/.ssh/id_ecdsa type -1 debug1: identity file /home/ronan/.ssh/id_ecdsa-cert type -1 debug1: Remote protocol version 2.0, remote software version OpenSSH_5.6 debug1: match: OpenSSH_5.6 pat OpenSSH* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_5.8 debug2: fd 3 setting O_NONBLOCK debug3: load_hostkeys: loading entries for host "ronan-ita" from file "/home/ronan/.ssh/known_hosts" debug3: load_hostkeys: loaded 0 keys debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug2: kex_parse_kexinit: ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1 debug2: kex_parse_kexinit: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-dss-cert-v01@openssh.com,ssh-rsa-cert-v00@openssh.com,ssh-dss-cert-v00@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-rsa,ssh-dss debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: zlib@openssh.com,zlib,none debug2: kex_parse_kexinit: zlib@openssh.com,zlib,none debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: first_kex_follows 0 debug2: kex_parse_kexinit: reserved 0 debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1 debug2: kex_parse_kexinit: ssh-rsa,ssh-dss debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 debug2: kex_parse_kexinit: none,zlib@openssh.com debug2: kex_parse_kexinit: none,zlib@openssh.com debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: debug2: kex_parse_kexinit: first_kex_follows 0 debug2: kex_parse_kexinit: reserved 0 debug2: mac_setup: found hmac-md5 debug1: kex: server->client aes128-ctr hmac-md5 zlib@openssh.com debug2: mac_setup: found hmac-md5 debug1: kex: client->server aes128-ctr hmac-md5 zlib@openssh.com debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP Read from socket failed: Connection reset by peer Additional info: * OpenSSH_5.8p1, OpenSSL 1.0.0d 8 Feb 2011 * Kernel 2.6.37-ARCH * At #archlinux channel on freenode, bencahill and fsckd verified that the connection to my server works on Ubuntu and Debian. Also, bencahill and sledgehammer verified that bleeding edge Archlinux hangs when try to connect to my server, as I saw here. * soupcan is also having the same problem: https://bbs.archlinux.org/viewtopic.php?pid=892227#p892227 Steps to reproduce: * Update Archlinux * Try to connect to an external ssh server (not sure if the problem is with all ssh servers) |
This task depends upon
Closed by Gaetan Bisson (vesath)
Saturday, 09 April 2011, 10:46 GMT
Reason for closing: Upstream
Additional comments about closing: See last comment.
Saturday, 09 April 2011, 10:46 GMT
Reason for closing: Upstream
Additional comments about closing: See last comment.
Here, for instance, I have no problem connecting to RHEL OpenSSH_4.3p2, FreeBSD OpenSSH_4.5p1, Fedora OpenSSH_4.7p1, Debian OpenSSH_5.1p1, etc.
On the client side, does it solve your problem to downgrade openssh to 5.7p1? 5.6p1?
It would really help to isolate the problem if we knew whether *on the same machine* 5.8p1 fails while another version works.
Are you using the default /etc/ssh_config? Do you have a ~/.ssh/config?
Maybe also try downgrading openssl.
With OpenSSH 5.7 I also see the problem, but it is gone with OpenSSH 5.6p1-2
By now, I'll use OpenSSH 5.6p1 :) Thanks for the advice!
Can you be more specific about the SSH server? What is its distribution and package version?
Is the server's /etc/sshd_config the default one? Can you do a "ls -la /etc/ssh/" on the server so that I can see which host keys it has?
moduli ssh_host_dsa_key ssh_host_key.pub
ssh_config ssh_host_dsa_key.pub ssh_host_rsa_key
sshd_config ssh_host_key ssh_host_rsa_key.pub
and the sshd_config is the default one
At the client, I have already tried with default ssh_config.
export CVSROOT=anoncvs@anoncvs.mindrot.org:/cvs
export CVS_RSH=/usr/bin/ssh
cvs get -D 2010-09-01 openssh
produces a client which cannot connect to your server (the client initiates key-exchange and gets no response from the server), while replacing 2010-09-01 by 2010-08-31 does.
Now this could be an upstream bug but there is certainly something weird about your server (since I can connect to other 5.6p1 sshd alright).
- Can you try restarting sshd?
- Can you try (re)installing openssh-5.6p1-2 and then restarting sshd?
- Can you try (re)installing openssh-5.6p1-1 and then restarting sshd?
Now, I'll try to update the server to bleeding edge ArchLinux to see if my 5.8 client can connect to 5.8 server.
Let me know if your issue disappears with 5.8p1; if it does and the other person who had a server with this issue does not give us insight by then, I will close this bug since we won't have any way of getting a diagnosis.
I have upgraded server ssh to 5.8p1 (default configuration) and the problem still exists.
I can not log with ssh client 5.8p1 but I can with ssh 5.6p1 client.
Hi,
I have a problem connecting to HP iLO with current version: openssh 5.8p1.
I have compiled from source 5.6p1 5.7p1 and 5.8p1 with no any options passed to ./configure but prefix. The result is attached
What we now need to find out is **what makes these servers special** since most servers out there obviously do not have this issue.
(Roman's server is affected, he upgraded sshd from 5.6p1 to 5.8p1 and the issue remains, so this is most likely related to the configuration of sshd or its interaction with another package.)
Can someone with access to an affected server do what Leonid suggested ("sshd -d" to get debug output from the server)?
Yes I can, but just on Monday, when I'll have physical access to the computer.
Thanks!
I found a solution and the reason is new default HostKeyAlgorithms, introduced in 5.7. When I remove elliptic curve cryptography modes from the list, I'm able to connect again
in my ~/.ssh/config :
HostKeyAlgorithms ssh-rsa-cert-v01@openssh.com,ssh-dss-cert-v01@openssh.com,ssh-rsa-cert-v00@openssh.com,ssh-dss-cert-v00@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-rsa,ssh-dss
You can see the dafault order in ssh_config manual page
However, lots of us use the default HostKeyAlgorithms of 5.8p1 and can connect perfectly well to various kinds of SSH servers, so we need to dig deeper if we want to find the underlying cause to this problem.
Edit: For other workarounds and debugging information, see: http://lists.mindrot.org/pipermail/openssh-unix-dev/2011-February/thread.html#29361
OpenSSH 5.8 is working here with your suggestion!
I'll use it as a workaround until we figured out why this problem is affecting just some servers.
I have a interesting thing here:
If I connect by VNC on server and bind that ports:
ssh -R 2222:localhost:22 <USER>@<MY IP> -p <MY LAPTOP SSH SERVER PORT>
than I'm able to connect to my server from my laptop by:
ssh localhost -p 2222
using OpenSSH 5.8p1.
Can this bring new information?
> cryptography modes from the list, I'm able to connect again
Are you saying that the relevant files are /etc/ssh/{ssh_host_ecdsa_key,ssh_host_ecdsa_key.pub}? Can you post the output of ls -lt /etc/ssh?
EDIT: and when ssh 5.7 was installed from pacman.log...
What I would do, is to remove openssh on the server. Then, remove /etc/ssh/keys*, then install openssh back and regenerate all the host keys... Otherwise, this is some black magic :)
Hi Leonid,
Yes, those files were generated when openssh is upgraded to 5.7p1:
from pacman.log:
[2011-01-31 07:00] upgraded openssh (5.6p1-2 -> 5.7p1-2)
ls -l /etc/ssh/{ssh_host_ecdsa_key,ssh_host_ecdsa_key.pub} :
-rw------- 1 root root 227 Jan 31 07:03 /etc/ssh/ssh_host_ecdsa_key
-rw-r--r-- 1 root root 176 Jan 31 07:03 /etc/ssh/ssh_host_ecdsa_key.pub
My logs:
client: http://pastebin.com/T1tYGbDL
server: http://pastebin.com/nUXDJfSj
http://lists.mindrot.org/pipermail/openssh-unix-dev/2011-February/029361.html
Debian: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=612607
Ubuntu: https://bugs.launchpad.net/ubuntu/+source/openssh/+bug/708493
A better way of fixing this seems to be simply adding "-c aes128-ctr" to the ssh command line.
Shortening the cipher list (see above) is a workaround, but a cleaner fix requires more debugging.
Any relevant info should be posted in this thread: http://lists.mindrot.org/pipermail/openssh-unix-dev/2011-March/029456.html