Jailkit stopped working after update Jessie (8) to Stretch (9)

Hi,
I've updated Jessie to Stretch and everything worked flawlessly but Jailkit.
It stopped working and I can't ssh to accounts without deactivating Jailkit first for that ssh-account.

I receive:
Sep 18 16:54:25 server jk_chrootsh[25503]: ERROR: failed to execute shell /bin/bash for user ssh-7-xxxx (5039), check the permissions and libraries of /var/www/clients/client7/web190//

I already reinstalled Jailkit but this did not fix the problem. Any ideas how to get rid of this problem?

 

I double checked the permissions and they look fine to me.
I also used the "resync" Feature of ISPConfig to rsync everything but sadly, it didn't work.

Sep 19 13:06:15 server jk_chrootsh[22235]: now entering jail /var/www/clients/client7/web190 for user ssh-7-xxxx (5039) with arguments -c /usr/lib/openssh/sftp-server
Sep 19 13:06:15 server jk_chrootsh[22235]: ERROR: failed to execute shell /bin/bash for user ssh-7-xxxx (5039), check the permissions and libraries of /var/www/clients/client7/web190//
Sep 19 13:06:16 server sshd[22234]: Received disconnect from xxxx port 59811:11: cleanup
Sep 19 13:06:16 server sshd[22234]: Disconnected from xxxx port 59811
Sep 19 13:06:16 server sshd[22225]: pam_unix(sshd:session): session closed for user ssh-7-xxxx
Sep 19 13:06:16 server systemd-logind[670]: Removed session 32169.
-- Subject: Session 32169 has been terminated
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- Documentation: http://www.freedesktop.org/wiki/Software/systemd/multiseat
--
-- A session with the ID 32169 has been terminated.
Sep 19 13:06:16 server systemd[1]: Stopping User Manager for UID 5039...
-- Subject: Unit [email protected] has begun shutting down
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- Unit [email protected] has begun shutting down.
Sep 19 13:06:16 server systemd[22227]: Failed to enqueue exit.target job: Access denied
Sep 19 13:06:16 server systemd[22228]: pam_unix(systemd-user:session): session closed for user web190
Sep 19 13:06:16 server systemd[1]: Stopped User Manager for UID 5039.
-- Subject: Unit [email protected] has finished shutting down
-- Defined-By: systemd
-- Support: https://www.debian.org/support

 

@Jesse Norell you helped me a lot with Jailkit last time, you seem to be an expert for this. Do you have any idea, how to fix this problem?

 

I already did that. (also have a cronjob for this)
It does not work :/

 

@till this is a pretty critical issue for me and I am running out of time. Would you mind supporting me? Even paid if necessary

 

I have run the same issue after upgrade the system.
serverfault.com/a/162368 This post solved this issue.
Symlink for {jail}/lib/64/ld-linux-x86-64.so.2 was linked to [uninstalled version] /lib/x86_64-linux-gnu/ld-2.19.so
I just remove old wrong symlink and added correctly to /lib/x86_64-linux-gnu/ld-2.23.so (for me).
Check this on your machine - probably it may help you.

 

@cgi thank you very much. May I ask you for the list of commands you've used for it?

 

@Hbod

Code:

// find installed version
ls -la {jail}/lib/x86_64-linux-gnu/ | grep ld
-rwxr-xr-x 1 root root  162632 Jun 16 20:57 ld-2.23.so
lrwxrwxrwx 1 root root      10 Sep 19 20:34 ld-linux-x86-64.so.2 -> ld-2.23.so

cd {jail}/lib64
ln -s /lib/x86_64-linux-gnu/ld-2.23.so ld-linux-x86-64.so.2

Something like that

 

Code:

ls -la /var/www/.../lib/x86_64-linux-gnu/ | grep ld
-rwxr-xr-x 1 root root  153288 Jun 15 21:17 ld-2.24.so
lrwxrwxrwx 1 root root  10 Sep 19 13:36 ld-linux-x86-64.so.2 -> ld-2.24.so
ln -s /lib/x86_64-linux-gnu/ld-2.24.so ld-linux-x86-64.so.2
cd /lib64
ls
ld-linux-x86-64.so.2
Sie haben neue Post in /var/mail/root.
ldd ld-linux-x86-64.so.2
statically linked

I dont see the /bin/bash error anymore, but "Transmit" still answering with "Command failed"..
Sep 20 00:20:14 server jk_chrootsh[22024]: now entering jail /var/www/clients/client7/web190 for user ssh-7-xxxx (5039) with arguments -c /usr/lib/openssh/sftp-server
Sep 20 00:20:15 server sshd[22023]: Received disconnect from xxx port 59082:11: cleanup
Sep 20 00:20:15 server sshd[22023]: Disconnected from xxxx port 59082
Sep 20 00:20:15 server sshd[22014]: pam_unix(sshd:session): session closed for user ssh-7-fabianb
Sep 20 00:20:15 server systemd-logind[670]: Removed session 34975.
-- Subject: Session 34975 has

 

Try to run second sshd daemon with debug mode enabled and different port and connect to it.

 

Well I'm not experienced enough to achive what you suggest, as I don't know what you are basically talking about

 

https://serverfault.com/a/395373

Code:

/usr/sbin/sshd -p 2222 -d (or -dd or -ddd, increasing debug info)
sftp -P 2222 user@remotehost

And see logs again.

 

Theres a bunch of stuff inside the logs. But basically, it's looking positive... @cgi

Code:

Server listening on :: port 2222.
debug1: Server will not fork when running in debugging mode.
debug1: rexec start in 5 out 5 newsock 5 pipe -1 sock 8
debug1: inetd sockets after dupping: 3, 3
Connection from XX.XXX.XXX.XX port 59492 on XXXXX port 2222
debug1: Client protocol version 2.0; client software version OpenSSH_7.4
debug1: match: OpenSSH_7.4 pat OpenSSH* compat 0x04000000
debug1: Local version string SSH-2.0-OpenSSH_7.4p1 Debian-10+deb9u1
debug1: Enabling compatibility mode for protocol 2.0
debug1: permanently_set_uid: 104/65534 [preauth]
debug1: list_hostkey_types: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519 [preauth]
debug1: SSH2_MSG_KEXINIT sent [preauth]
debug1: SSH2_MSG_KEXINIT received [preauth]
debug1: kex: algorithm: ecdh-sha2-nistp256 [preauth]
debug1: kex: host key algorithm: ecdsa-sha2-nistp256 [preauth]
debug1: kex: client->server cipher: aes128-ctr MAC: [email protected] compression: [email protected] [preauth]
debug1: kex: server->client cipher: aes128-ctr MAC: [email protected] compression: [email protected] [preauth]
debug1: expecting SSH2_MSG_KEX_ECDH_INIT [preauth]
debug1: rekey after 4294967296 blocks [preauth]
debug1: SSH2_MSG_NEWKEYS sent [preauth]
debug1: expecting SSH2_MSG_NEWKEYS [preauth]
debug1: SSH2_MSG_NEWKEYS received [preauth]
debug1: rekey after 4294967296 blocks [preauth]
debug1: KEX done [preauth]
debug1: userauth-request for user ssh-7-huhu service ssh-connection method none [preauth]
debug1: attempt 0 failures 0 [preauth]
debug1: PAM: initializing for "ssh-7-huhu"
debug1: PAM: setting PAM_RHOST to "XX.XXX.XXX.XX"
debug1: PAM: setting PAM_TTY to "ssh"
debug1: userauth-request for user ssh-7-huhu service ssh-connection method publickey [preauth]
debug1: attempt 1 failures 0 [preauth]
debug1: userauth_pubkey: test whether pkalg/pkblob are acceptable for RSA SHA256:XXXXXXXXXXX [preauth]
debug1: temporarily_use_uid: 5039/5011 (e=0/0)
debug1: trying public key file /var/www/clients/client7/web190/./home/ssh-7-huhu/.ssh/authorized_keys
debug1: fd 4 clearing O_NONBLOCK
debug1: matching key found: file /var/www/clients/client7/web190/./home/ssh-7-huhu/.ssh/authorized_keys, line 4 RSA SHA256:XXXXXXXXXXX
debug1: restore_uid: 0/0
Postponed publickey for ssh-7-huhu from XX.XXX.XXX.XX port 59492 ssh2 [preauth]
debug1: userauth-request for user ssh-7-huhu service ssh-connection method publickey [preauth]
debug1: attempt 2 failures 0 [preauth]
debug1: temporarily_use_uid: 5039/5011 (e=0/0)
debug1: trying public key file /var/www/clients/client7/web190/./home/ssh-7-huhu/.ssh/authorized_keys
debug1: fd 4 clearing O_NONBLOCK
debug1: matching key found: file /var/www/clients/client7/web190/./home/ssh-7-huhu/.ssh/authorized_keys, line 4 RSA SHA256:XXXXXXXXXXX
debug1: restore_uid: 0/0
debug1: do_pam_account: called
Accepted publickey for ssh-7-huhu from XX.XXX.XXX.XX port 59492 ssh2: RSA SHA256:XXXXXXXXXXX
debug1: monitor_child_preauth: ssh-7-huhu has been authenticated by privileged process
debug1: Enabling compression at level 6. [preauth]
debug1: monitor_read_log: child log fd closed
debug1: PAM: establishing credentials
User child is on pid 8087
debug1: SELinux support disabled
debug1: PAM: establishing credentials
debug1: permanently_set_uid: 5039/5011
debug1: rekey after 4294967296 blocks
debug1: rekey after 4294967296 blocks
debug1: ssh_packet_set_postauth: called
debug1: Enabling compression at level 6.
debug1: Entering interactive session for SSH2.
debug1: server_init_dispatch
debug1: server_input_channel_open: ctype session rchan 0 win 2097152 max 32768
debug1: input_session_request
debug1: channel 0: new [server-session]
debug1: session_new: session 0
debug1: session_open: channel 0
debug1: session_open: session 0: link with channel 0
debug1: server_input_channel_open: confirm session
debug1: server_input_channel_req: channel 0 request subsystem reply 1
debug1: session_by_channel: session 0 channel 0
debug1: session_input_channel_req: session 0 req subsystem
debug1: subsystem: exec() /usr/lib/openssh/sftp-server
Starting session: subsystem 'sftp' for ssh-7-huhu from XX.XXX.XXX.XX port 59492 id 0
debug1: Received SIGCHLD.
debug1: session_by_pid: pid 8088
debug1: session_exit_message: session 0 channel 0 pid 8088
debug1: session_exit_message: release channel 0
debug1: Got 100/10 for keepalive
Received disconnect from XX.XXX.XXX.XX port 59492:11: cleanup
Disconnected from XX.XXX.XXX.XX port 59492
debug1: do_cleanup
debug1: do_cleanup
debug1: PAM: cleanup
debug1: PAM: closing session
debug1: PAM: deleting credentials
debug1: audit_event: unhandled event 12

 

Code:

Starting session: subsystem 'sftp' for ssh-7-huhu from XX.XXX.XXX.XX port 59492 id 0
debug1: Received SIGCHLD.

Killed once it started.. Idk, try to google similar issues
http://www.unix.com/linux/161311-users-cant-sftp-into-my-server.html
Try to re init sftp in jail, run daemon with -ddd and client -vvv

 

@cgi digging more deeply into it, I found following: (see -A)
ssh [email protected] -p 2222 -A

/bin/bash: error while loading shared libraries: libdl.so.2: cannot open shared object file: No such file or directory

So this is the reason for the SIGCHLD. So my questions are:
1) How to fix this? Any ideas? Maybe @till ?
2) How to fix this issues for all Jails
3) Will newly created jails work out of the box?

 

You can try to update the jail or look at post 7

 

@HSorgYves don't you think I've tried like 100 times? I googled my fingers bloody but I can't get it work. I already tried everything... I can't fix this.

 

I did everything and the first error
failed to execute shell /bin/bash for user ssh-7-xxxx (5039), check the permissions and libraries of /var/www/clients/client7/web190//

is gone. But now I have:
/bin/bash: error while loading shared libraries: libdl.so.2: cannot open shared object file: No such file or directory

Something is missing... I don't know what...

 

What is the output of:
1) ls -al /lib/x86_64-linux-gnu/ | grep libdl
2) ls -la {jail}/lib/x86_64-linux-gnu/ | grep libdl