Get this message in auth.log: Code: Apr 3 16:24:20 systemd: pam_unix(systemd-user:session): session opened for user web9 by (uid=0) Apr 3 16:24:20 jk_chrootsh[6710]: now entering jail /var/www/clients/client1/web9 for user c1_rosental (5006) with arguments -c /usr/lib/openssh/sftp-server Apr 3 16:24:20 jk_chrootsh[6710]: path /bin/bash is not owned by user 0 Apr 3 16:24:20 jk_chrootsh[6710]: path /bin/bash is not owned by group 0 when i make Code: ls -la /var/www/clients/client1/web9/bin/bash i get Code: -rwxr-xr-x 4 5007 client1 1029624 Nov 5 2016 /var/www/clients/client1/web9/bin/bash user 5007 is unknown
Found it, run it, but users cant still login. After changing config for shelluser without jailkit, login works. changed back, user cant login. in auth.log found this: Code: Apr 7 16:51:37 vcawws01 sshd[2712]: Accepted password for c1_calidris from 37.24.124.86 port 51378 ssh2 Apr 7 16:51:37 vcawws01 sshd[2712]: pam_unix(sshd:session): session opened for user c1_calidris by (uid=0) Apr 7 16:51:37 vcawws01 systemd-logind[574]: New session 35503 of user web9. Apr 7 16:51:37 vcawws01 jk_chrootsh[2778]: now entering jail /var/www/clients/client1/web9 for user c1_calidris (5006) with arguments -c /usr/lib/openssh/sftp-server Apr 7 16:51:37 vcawws01 sshd[2710]: Received disconnect from 122.51.213.140 port 60042:11: Bye Bye [preauth] Apr 7 16:51:37 vcawws01 sshd[2710]: Disconnected from 122.51.213.140 port 60042 [preauth] Apr 7 16:51:37 vcawws01 sshd[2712]: pam_unix(sshd:session): session closed for user c1_calidris Apr 7 16:51:37 vcawws01 systemd-logind[574]: Removed session 35503. In Syslog found this one: Code: Apr 7 16:51:37 vcawws01 kernel: [371512.752371] bash[2778]: segfault at 0 ip (null) sp 00007ffe84d5b698 error 14 in bash[400000+100000]
Do you get a bash segfault every time you try to login? Are you loging in with an ssh client or sftp? (If sftp, did you install the sftp server in your jail?)
we are using in 90% winscp, error came every time. error is new since the last 3 or 4 days. running this configuration for over a year. installed ispconfig with perfect server tutorial. There is enough disk and ram space and cpu is max @ 40%.
Curious. Are there any system updates? Update, reboot and hope it's fixed is worth a shot. If that doesn't work, can you login with ssh at all? Does a new jail work, and only old ones fail? Can you run anything chroot in that directory (eg. chroot /var/www/clients/client1/web9 /bin/ls -l / )? Is /bin/sh still bash, and didn't inadvertently get changed to dash?
Seems like your jail is broken. A segfault usually happens when a process tries to access a memory position it is not allowed / deos not exist. In a jail, however, this can occur when critical shared libs, config file or /dev/ entries are missing. To debug: a) Connect to jail, but do not login (do not enter password) b) Find the process id: "ps axu|grep sshd|grep jailuser" c) Debug the process: strace -p PROCESS-ID -ff -o /tmp/debug d) complete login, then read /tmp/debug Edit: Btw, does "ls -la /var/www/clients/client1/web9/bin/bash" now show the right permissions?
ls -la /var/www/clients/client1/web9/bin/bash: Code: -rwxr-xr-x 1 root root 1099016 Mai 15 2017 /var/www/clients/client1/web9/bin/bash After Connect 2 Jail, i got 2 processes Code: root 1087 0.0 0.0 95068 6352 ? Ss 06:18 0:00 sshd: c1_demo [priv] sshd 1088 0.0 0.0 69952 3212 ? S 06:18 0:00 sshd: c1_demo [net] Entering Password => terminal window closed Debug Data attached as file @Jesse Norell Updated to current jailkit and reboot, no changes. MayBe error occurs after jk_updater_ispc.
Based on file size, the last one is "no", but the others? I'll take a quick look at the strace output.
There should be some more debug files in /tmp, eg. debug.8574 and debug.8614, and maybe others. Be sure to change your password or delete your demo account when done here.
New Jails and Old Jails fails the same, demo account is a new one. I will debug a new login after easter holiday, thanks.
Well, I don't gather much more from the strace other than bash was killed with a SEGV. From your output (eg. debug_1.8615) this happens very early in bash execution, right after reading libc and doing some memory mapping: Code: open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\4\2\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=1689360, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f14eb833000 mmap(NULL, 3795296, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f14eae4c000 mprotect(0x7f14eafe1000, 2097152, PROT_NONE) = 0 mmap(0x7f14eb1e1000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x195000) = 0x7f14eb1e1000 mmap(0x7f14eb1e7000, 14688, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f14eb1e7000 close(3) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f14eb832000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f14eb831000 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f14eb830000 arch_prctl(ARCH_SET_FS, 0x7f14eb831700) = 0 --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=NULL} --- +++ killed by SIGSEGV +++ I traced the same way on a machine here, and right at the point yours crashed I see additional memory protection calls happening: Code: openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260A\2\0\0\0\0\0"..., 832) = 832 fstat(3, {st_mode=S_IFREG|0755, st_size=1824496, ...}) = 0 mmap(NULL, 1837056, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f1caf56e000 mprotect(0x7f1caf590000, 1658880, PROT_NONE) = 0 mmap(0x7f1caf590000, 1343488, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x22000) = 0x7f1caf590000 mmap(0x7f1caf6d8000, 311296, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x16a000) = 0x7f1caf6d8000 mmap(0x7f1caf725000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1b6000) = 0x7f1caf725000 mmap(0x7f1caf72b000, 14336, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f1caf72b000 close(3) = 0 mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f1caf56b000 arch_prctl(ARCH_SET_FS, 0x7f1caf56b740) = 0 mprotect(0x7f1caf725000, 16384, PROT_READ) = 0 mprotect(0x7f1caf732000, 4096, PROT_READ) = 0 mprotect(0x7f1caf75d000, 16384, PROT_READ) = 0 mprotect(0x55e03d000000, 12288, PROT_READ) = 0 mprotect(0x7f1caf78e000, 4096, PROT_READ) = 0 munmap(0x7f1caf764000, 8734) = 0 openat(AT_FDCWD, "/dev/tty", O_RDWR|O_NONBLOCK) = 3 close(3) = 0 ... At this point you can go one of 2 routes, 1) try to figure out what changed in your system/environment that started this (package updates, in libc or the kernel? container/memory settings? security related changes? etc.), or 2) try to debug bash further to see more specifically what/where it's failing, to identify what changed. For 1), check command history, package and file timestamps, any records or container/vps type settings, etc. For 2) try installing ltrace and run similar to strace (eg. in your example command output above, you'd run 'ltrace -f -p 1087 -p 1088 -o /tmp/ltrace.out' at the point you ran strace, that will show more info on what's happening. And the next step would be to setup bash to create a core file when it crashes, I've not looked into how to do that specifically, but I'd try to run 'ulimit -c unlimited' ahead of starting bash from jk_chrootsh or so.
Cant find any significant changes. So i do 2 step, attachment is to big to attach, so here is the link: https://we.tl/t-c1ELCel5Y1
Of that, process 7668 is the interesting one, and just before it exits (last few lines of the file): Code: 7668 syslog(3, "abort, home directory %s differs"..., "/home/c1_demo", "", "c1_demo", 5006, "/var/www/clients/client1/web9") = <void> Sounds like a problem in the passwd files (I think the one inside the chroot). What do you get from "grep -E 'c1|client1/web9' /etc/passwd" and "cat /var/www/clients/client1/web9/etc/passwd" ? That jk_updater_ispc script doesn't touch/try to validate/recreate the passwd file, maybe I should look at adding that.
