Logging into CentOS 7.5 server with Chroot root set to Jailkit is getting the login failure above for both FileZilla and WinSCP. I can login fine with Putty and the same parameters, I have also tested manual password and regenerating the password. A workaround at present for this user is to set Chroot set to None. I have one user the is working correctly with the Jailkit set however this user was configured prior to upgrading ISPConfig (maybe 3.1.7p1) and also upgrading CentOS from 7.3. Comparing the /home/<user_name> directories shows the working one has a empty .pki dir and empty .profile, so possibly they are additions through some other utility.
The chroot does not contain the programs to connect with SCP or SFTP by default, that's why the chroot works for SSH but not for SSH file transfers with scp / sftp. If you want to use SFTP instead of FTPS for file transfers, then you'll have to install the SFTP applications in the jail. If I remember correctly, there is a thread on that topic in the forum.
