ISPConfig autoinstall Lightsail not accessible

Discussion in 'ISPConfig 3 Priority Support' started by tlove, Jun 14, 2024.

  1. tlove

    tlove Member HowtoForge Supporter

    ISPConfig autoinstall on Lightsail via Putty appears to complete successfully. However, cannot get to ISPConfig UI and after reboot cannot access server either through Putty or Lightsail SSH console and can not ping through to server. Host file set up using 127.0.1.1 and hostname and hostname -f give correct server and FQDN.
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    Have you configured the AWS firewall to allow access to ISPConfig GUI and access to other required ports like port 8080 and 8081 and also port 80 to get the Let's encrypt SSL cert during installation?

    And which exact command-line options did you use in the installer? Maybe you used options to reconfigure SSH? Also, you should check your AWS VM settings. Maybe you do not have enough RAM to start the VM.

    And just a side note, Amazon is quite expensive regarding performance to price ratio. Plus you must know AWS quite well as its complicated to set up. There are way better alternatives to run hosting servers, in my opinion.
     
  3. tlove

    tlove Member HowtoForge Supporter

    The AWS firewall has TCP ports 22, 80, 443 and 8080 open (just opened 8081 now but no change).
    The installation was on Lightsail instance with 2CPUs 1Gb Ram and 40Gb storage.
    I'll try with next size up
     
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    1GB RAM is likely too small for a full server that runs all services, at least when your system has no SWAP space.
     
  5. tlove

    tlove Member HowtoForge Supporter

    Thanks Till. I changed to a larger instance. A sample ISPConfig is now at the moment up and running (with no sites or emails etc) on 2Gb 2CPU 60G storage on Lightsail. There is a problem. Some time after a reboot, the ISPconfig instance on Lightsail hits an AWS 'instance status failure' which stops access to the server but doesn't show as the server being stopped. It can only be seen by checking the instance metrics. A post elsewhere suggests this may be due a cron that uses up all the CPU resources?
     
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    I've never heard of such an issue; it must be something specific to AWS. There are no cronjobs in ISPConfig that require many resources. And ISPConfig works well on systems with 2GB RAM on other Cloud providers, some of my own cloud systems just have 2GB RAM as well and they run fine for years.

    You can check the system to see if it runs out of memory e.g. by letting top command run on the shell. Or create an additional 2GB swap drive if AWS supports that. Or try to use a bigger instance to see if AWS fails in the same way then.

    At which exact time does it happen?

    However, you should consider using a more stable cloud provider if you plan to use these systems in production. I was not aware that AWS has so many system failures.
     
  7. tlove

    tlove Member HowtoForge Supporter

    Thank you Till, I checked as you suggested. Server ran for just over an hour and failed again. After installation of ISPConfig server Putty and AWS ssh console remained connected until reboot after which the the server was not online and could not be connected by Putty or AWS's SSH terminal and couldn't ping it . But AWS console showed the server was running. Checking with du and free and top and the AWS Metrics tab showed the server resource use was well within capacity (at peak around 50% CPU and 25% of disk) and network Good. AWS Metrics tab showed 'status check failure', 'instance status failure' and 'system status check failure' (no details). Looking at the history today shows the server stayed online for an hour then had a status failure at which point it is disconnected. Reboot didn't stop the status failure. Stopping the server and restarting it onto new hardware brought it back online with new hardware. Top figures for memory look as happy as CPU and storage figures. Question is what is triggering crash of server after an hour or so of running with no input?
    top - 06:02:56 up 56 min, 2 users, load average: 0.00, 0.08, 5.90
    Tasks: 163 total, 1 running, 162 sleeping, 0 stopped, 0 zombie
    %Cpu(s): 0.2 us, 0.2 sy, 0.0 ni, 99.5 id, 0.0 wa, 0.0 hi, 0.0 si, 0.2 st
    MiB Mem : 1921.0 total, 713.2 free, 483.7 used, 724.1 buff/cache
    MiB Swap: 0.0 total, 0.0 free, 0.0 used. 1223.0 avail Mem
    A relevant AWS doc is https://repost.aws/knowledge-center/lightsail-instance-failed-status-check but the fact that the CPU, memory and storage figures are so much within limits suggests something else?
     
  8. till

    till Super Moderator Staff Member ISPConfig Developer

    Most cloud providers have a console where you can get the boot messages. These should show why Linux fails to boot or bring up all services. Maybe you reconfigured something in the network or something that made the system fail to connect the network again.

    I do not use AWS, so I can't help you that much with that specific provider. Also, I've never seen a freshly installed ISPConfig system stop responding on its own, so I'm sure your issue is not ISPConfig specific, especially as ISPConfig only uses stock Debian and Ubuntu packages only. But I know some people use AWS and do not seem to have any issues. Maybe the user @nhybgtvfr can give you some hints as he appears to be using ISPConfig on AWS.

    Or maybe you just get locked out from SSH by Fail2ban.

    You should consider asking AWS support why their VMs fail on standard Debian and Ubuntu systems and how to get detailed error messages to see why this is the case.
     
  9. till

    till Super Moderator Staff Member ISPConfig Developer

  10. nhybgtvfr

    nhybgtvfr Well-Known Member HowtoForge Supporter

    i've not used lightsail.

    having lots of memory free after a reboot isn't an indicator that the amount of memory is ok..
    the figures you get from top are ok for that instant.. but what are the figures when the problem happens?
    look at the metrics charts in aws for the problem periods... that might give you an idea of what's happening.

    if you have 2gb of ram, with no swap but all ispconfig services installed, it will still be quite tight for ram in general... and once clamav starts a scan it will absolutely eat through the whole 2gb and more..
    i would suggest creating a 2 - 4 gb swap file on there and see how it goes then.

    also create some alarms so that aws warns you on the status/system/instance failures.. and forces a reboot when there's 2-3 consecutive failures.. not ideal, you'd still want to find the cause.. but it will at least minimize downtime, especially if the underlying hardware experiences a problem.... it doesn't happen often, but it does happen..
     
    till likes this.
  11. tlove

    tlove Member HowtoForge Supporter

    @ Till - many thanks for your support and useful advice which I will follow.The I'm happy to change from AWS and welcome suggestions as to alternative small server hosting for IPConfig - max about 10 small research/ngo websites.
    @nhybgtvfr Thank you for your clear advice. It may be a Clamav scan causing the problem. I'll set up a swap file as suggested. Was thinking of using alarms as you suggested but getting the server going again seems to require stopping the server completely rather than reboot.
    I'll report back when/if I find a solution.
     
  12. nhybgtvfr

    nhybgtvfr Well-Known Member HowtoForge Supporter

    often when you get an unresponsive instance in aws, and you try a reboot, even from the aws console, it doesn't appear to do anything.. if you try to reboot from the aws console again, it'll give you the option of doing a 'force reboot', that will work.. although it can still take several minutes.

    having set the aws alarms to reboot an instance if an alarm fails for 2 consecutive cycles (set to 1 minute), so far it's always manage to restart it quickly, no more waiting for 5+ minutes trying to reboot and then force reboot/stop from the aws console.
     

Share This Page