Ispconfig 3.1.14p2 web serv er config missing

Discussion in 'Installation/Configuration' started by neutrinohu, Aug 5, 2019.

Tags:
Page 1 of 2
  1. neutrinohu

    neutrinohu New Member

    Hi all,

    I've faced with the below problem:

    Until last friday my ispconfig setup was running fine. Tomorrow i've discovered all of the config is missing from "System -> Server config -> Web". How can i restore it (at least to the factory default)? I have no backup from this database.

    System test output:
    cat htf_report.txt | more

    ##### SERVER #####
    IP-address (as per hostname): ***.***.***.***
    [WARN] could not determine server's ip address by ifconfig
    [INFO] ISPConfig is installed.

    ##### ISPCONFIG #####
    ISPConfig version is 3.1.14p2


    ##### VERSION CHECK #####

    [INFO] php (cli) version is 7.3.7-2+ubuntu18.04.1+deb.sury.org+1

    ##### PORT CHECK #####

    [WARN] Port 22 (SSH server) seems NOT to be listening

    ##### MAIL SERVER CHECK #####


    ##### RUNNING SERVER PROCESSES #####

    [INFO] I found the following web server(s):
    Apache 2 (PID 1506)
    [INFO] I found the following mail server(s):
    Postfix (PID 1575)
    [INFO] I found the following pop3 server(s):
    Dovecot (PID 1120)
    [INFO] I found the following imap server(s):
    Dovecot (PID 1120)
    [INFO] I found the following ftp server(s):
    PureFTP (PID 1701)

    ##### LISTENING PORTS #####
    (only ()
    Local (Address)
    [anywhere]:42222 (1397/sshd)
    [anywhere]:110 (1120/dovecot)
    [anywhere]:143 (1120/dovecot)
    [anywhere]:80 (1506/apache2)
    [anywhere]:465 (1575/master)
    [anywhere]:21 (1701/pure-ftpd)
    ***.***.***.***:53 (773/systemd-resolve)
    [anywhere]:25 (1575/master)
    [anywhere]:993 (1120/dovecot)
    [anywhere]:995 (1120/dovecot)
    [localhost]:10023 (1475/postgrey)
    [localhost]:10024 (1739/amavisd-new)
    [localhost]:10025 (1575/master)
    [localhost]:10026 (1739/amavisd-new)
    [localhost]:10027 (1575/master)
    [anywhere]:587 (1575/master)
    [localhost]:11211 (947/memcached)
    *:*:*:*::*:42222 (1397/sshd)
    [localhost]10 (1120/dovecot)
    [localhost]43 (1120/dovecot)
    *:*:*:*::*:8080 (1506/apache2)
    *:*:*:*::*:8081 (1506/apache2)
    *:*:*:*::*:21 (1701/pure-ftpd)
    *:*:*:*::*:443 (1506/apache2)
    *:*:*:*::*:993 (1120/dovecot)
    *:*:*:*::*:995 (1120/dovecot)
    *:*:*:*::*:10023 (1475/postgrey)
    *:*:*:*::*:10024 (1739/amavisd-new)
    *:*:*:*::*:10026 (1739/amavisd-new)
    *:*:*:*::*:3306 (1473/mysqld)




    ##### IPTABLES #####
    Chain INPUT (policy ACCEPT)
    target prot opt source destination

    Chain FORWARD (policy ACCEPT)
    target prot opt source destination

    Chain OUTPUT (policy ACCEPT)
    target prot opt source destination

    System: Ubuntu 18.04 LTS
    Web server: Apache 2.4
    Mysql 5.7

    I've checked the DB and there is no "[web]" section in the "server" table's "config" field.
    [global]
    webserver=apache
    mailserver=postfix
    dnsserver=mydns

    [server]
    auto_network_configuration=n
    ip_address=192.168.1.100
    netmask=255.255.255.0
    v6_prefix=
    gateway=0.0.0.0
    firewall=ufw
    hostname=myhostname
    nameservers=8.8.8.8,8.8.4.4
    loglevel=1
    admin_notify_events=1
    backup_dir=/www/backup
    backup_tmp=/tmp
    backup_dir_is_mount=n
    backup_mode=rootgz
    backup_time=0:00
    backup_delete=n
    monit_url=
    monit_user=
    monit_password=
    munin_url=
    munin_user=
    munin_password=
    monitor_system_updates=y
    log_retention=10
    migration_mode=n

    [mail]
    module=postfix_mysql
    maildir_path=/home/vmail/[domain]/[localpart]
    maildir_format=maildir
    homedir_path=/home/vmail
    dkim_path=/var/lib/amavis/dkim
    dkim_strength=2048
    relayhost_password=
    pop3_imap_daemon=dovecot
    mail_filter_syntax=maildrop
    mailuser_uid=5000
    mailuser_gid=5000
    mailuser_name=vmail
    mailuser_group=vmail
    mailbox_virtual_uidgid_maps=n
    relayhost=
    relayhost_user=
    reject_sender_login_mismatch=n
    mailbox_size_limit=0
    message_size_limit=0
    mailbox_quota_stats=y
    realtime_blackhole_list=zen.spamhaus.org
    overquota_notify_admin=n
    overquota_notify_client=n
    overquota_notify_freq=7
    overquota_notify_onok=n

    [getmail]
    getmail_config_dir=/etc/getmail

    [dns]
    bind_user=root
    bind_group=bind
    bind_zonefiles_dir=/etc/bind
    named_conf_path=/etc/bind/named.conf
    named_conf_local_path=/etc/bind/named.conf.local
    disable_bind_log=n

    [fastcgi]
    fastcgi_starter_path=/var/www/php-fcgi-scripts/[system_user]/
    fastcgi_starter_script=.php-fcgi-starter
    fastcgi_alias=/php/
    fastcgi_phpini_path=/etc/php/7.2/cgi/
    fastcgi_children=8
    fastcgi_max_requests=5000
    fastcgi_bin=/usr/bin/php-cgi
    fastcgi_config_syntax=2

    [jailkit]
    jailkit_chroot_home=/home/[username]
    jailkit_chroot_app_sections=basicshell editors extendedshell netutils ssh sftp scp groups jk_lsh
    jailkit_chroot_app_programs=/usr/bin/groups /usr/bin/id /usr/bin/dircolors /usr/bin/lesspipe /usr/bin/basename /usr/bin/dirname /usr/bin/nano /usr/bin/pico /usr/bin/mysql /usr/bin/mysqldump /usr/bin/git /usr/bin/git-receive-pack /usr/bin/git-upload-pack /usr/bin/unzip /usr/bin/zip /bin/tar /bin/rm /usr/bin/patch /etc/localtime
    jailkit_chroot_cron_programs=/usr/bin/php /usr/bin/perl /usr/share/perl /usr/share/php

    [vlogger]
    config_dir=/etc

    [cron]
    init_script=cron
    crontab_dir=/etc/cron.d
    wget=/usr/bin/wget

    [rescue]
    try_rescue=n
    do_not_try_rescue_httpd=n
    do_not_try_rescue_mongodb=n
    do_not_try_rescue_mysql=n
    do_not_try_rescue_mail=n


    Any help is appreciated.

    Thanks
     
    neutrinohu, Aug 5, 2019
    #1
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    Use this web section:

    Code:
    [web]
server_type=apache
website_basedir=/var/www
website_path=/var/www/clients/client[client_id]/web[website_id]
website_symlinks=/var/www/[website_domain]/:/var/www/clients/client[client_id]/[website_domain]/
website_symlinks_rel=n
network_filesystem=n
vhost_conf_dir=/etc/apache2/sites-available
vhost_conf_enabled_dir=/etc/apache2/sites-enabled
nginx_vhost_conf_dir=/etc/nginx/sites-available
nginx_vhost_conf_enabled_dir=/etc/nginx/sites-enabled
security_level=20
user=www-data
group=www-data
nginx_user=www-data
nginx_group=www-data
apps_vhost_enabled=y
apps_vhost_port=8081
apps_vhost_ip=_default_
apps_vhost_servername=
php_open_basedir=[website_path]/web:[website_path]/private:[website_path]/tmp:/var/www/[website_domain]/web:/srv/www/[website_domain]/web:/usr/share/php5:/usr/share/php:/tmp:/usr/share/phpmyadmin:/etc/phpmyadmin:/var/lib/phpmyadmin:/dev/random:/dev/urandom
htaccess_allow_override=All
awstats_conf_dir=/etc/awstats
awstats_data_dir=/var/lib/awstats
awstats_pl=/usr/lib/cgi-bin/awstats.pl
awstats_buildstaticpages_pl=/usr/share/awstats/tools/awstats_buildstaticpages.pl
php_ini_path_apache=/etc/php/7.2/apache2/php.ini
php_ini_path_cgi=/etc/php/7.2/cgi/php.ini
check_apache_config=y
enable_sni=y
enable_spdy=n
skip_le_check=n
enable_ip_wildcard=y
overtraffic_notify_admin=y
overtraffic_notify_client=y
nginx_cgi_socket=/var/run/fcgiwrap.socket
php_fpm_init_script=php7.2-fpm
php_fpm_ini_path=/etc/php/7.2/fpm/php.ini
php_fpm_pool_dir=/etc/php/7.2/fpm/pool.d
php_fpm_start_port=9010
php_fpm_socket_dir=/var/lib/php7.2-fpm
php_default_name=Default
set_folder_permissions_on_update=n
add_web_users_to_sshusers_group=y
connect_userid_to_webid=n
connect_userid_to_webid_start=10000
web_folder_protection=y
php_ini_check_minutes=1
overquota_notify_admin=y
overquota_notify_client=y
overquota_notify_freq=7
overquota_db_notify_admin=y
overquota_db_notify_client=y
overquota_notify_onok=n
logging=yes
php_fpm_reload_mode=reload
website_autoalias=
vhost_rewrite_v6=n
nginx_enable_pagespeed=n
CA_path=
CA_pass=
php_handler=no
php_fpm_incron_reload=n
     
    till, Aug 5, 2019
    #2
  3. neutrinohu

    neutrinohu New Member


    Thanks, its working now :)
     
    neutrinohu, Aug 5, 2019
    #3
  4. nhybgtvfr

    nhybgtvfr Well-Known Member HowtoForge Supporter

    and at the very least, start taking regular backups of the master dbispconfig database.
    it contains all your server configs, customers details, website details, database details, user logins, dns settings, etc.
    it's the database you least want to lose...…
     
    nhybgtvfr, Aug 5, 2019
    #4
  5. neutrinohu

    neutrinohu New Member


    Yeah yeah. After this case i know this product is not stable enought to just "let it run". I think its unacceptable in version 3 to loose data randomly from the internal database.... We are not in 2005!
     
    neutrinohu, Aug 6, 2019
    #5
  6. nhybgtvfr

    nhybgtvfr Well-Known Member HowtoForge Supporter

    that's harsh. I've been running it for years, as have countless others, without any such issue, it's proven to be very stable, and does just run. even to the point that 'member' servers will still continue with their existing config and data even with the loss of the master server.
    something I don't see Plesk, cpanel et al doing.
    have you fully investigated the reason for the missing table data? you could have been hacked, a script run in the wrong location, an admin accidentally truncating the wrong table. to just blame randomness and unstable software is lazy and wrong.

    and I would say (and I hope at least 99% of sys admins would also say) it's unacceptable to not have multiple (confirmed recoverable) backups of all customer details, website/database/email/ftp user details/configs, server configs etc. especially when they're all in one easily backed up database.

    to assume your server will never crash, or can never be hacked, and it's data compromised, so there's no need for backups, is idiocy. we're not in 1982. ;)
     
    nhybgtvfr, Aug 6, 2019
    #6
  7. neutrinohu

    neutrinohu New Member

    My server is a test machine without direct internet access. I've reach it from internal network. At friday when i left the work every config was in place, but at monday morning i've found it without webserver config. Nobody logged into the machine, nobody has access to it (except me).
     
    neutrinohu, Aug 6, 2019
    #7
  8. till

    till Super Moderator Staff Member ISPConfig Developer

    I have no Idea what you did and how you managed to wipe out parts of your server config. I develop and use ISPConfig since its beginning and never lost my server config. I know ISP's that run several thousand ISPConfig servers for their customers, they don't lose the server config and there are hundred thousands of ISPConfig systems out there which work perfectly fine.
     
    till, Aug 6, 2019
    #8
  9. Croydon

    Croydon ISPConfig Developer ISPConfig Developer

    Really … That is nonsense. If nobody logged in and nobody changed anything and the server was not altered in any other way there is no way the ISPConfig database can change so it looses server config data. It's not really part of good manners – although sadly often done – to blame software for human faults.
    The only case where this might (!) happen would be to install the master development branch from git, but I guess (and hope) you wouldn't do such a thing.
     
    Croydon, Aug 6, 2019
    #9
  10. Jesse Norell

    Jesse Norell ISPConfig Developer Staff Member ISPConfig Developer

    I've considered commenting a couple times recently how there sure has been an unusually high number of people losing their server config of late. I don't doubt it may be the 'fault' of the user, but I also wonder if either a different design for storing server config, or at minimum some automatic back-up and config syntax check would be worth considering to make things more robust.

    I remember I lost my server config one time by simply using phpmyadmin to examine that column to read the settings - iirc I didn't make any changes, but phpmyadmin changed the newlines so that ISPConfig could not read/parse the field, and of course random things start failing after that. My fault? Sure, I caused it. But it's a pretty reasonable scenario to use the included (in standard install) database editor to examine the database, and it certainly felt fragile, with room for improvement.

    I had never considered it before, but I wonder if there's any way to configure phpmyadmin to not allow inline editing of that column, and make that part of the perfect server guides. For that matter, maybe it's a bug in phpmyadmin that should be fixed.

    But does anyone have thoughts on ways the current design could be made more robust? Json encoding that field with a fallback to legacy ini parsing might be quite simple and more forgiving of errors, or at least of newline changes.
     
    Jesse Norell, Aug 6, 2019
    #10
  11. Croydon

    Croydon ISPConfig Developer ISPConfig Developer

    That is at least worth thinking about. But in fact ini is a lot more user-friendly to edit in plain text than json is. And I doubt it would help in a situation like neutrinohu ran into. Personally, as a developer of course, I like json more than ini, but for the "normal user" I think ini is the better and more readable form of storing the data.
    Editing with phpmyadmin should not mess up things. I did that myself a lot of times without losing new line or other information. I have no idea why that could have happened in your case.
     
    Croydon, Aug 6, 2019
    #11
  12. Jesse Norell

    Jesse Norell ISPConfig Developer Staff Member ISPConfig Developer

    It's been a couple years ago, but I believe you can successfully use phpmyadmin if you click the link to edit the entire row in the servers table, but if you (double?) click an individual value it opens an inline editor, which is where the problem occurred.
     
    Jesse Norell, Aug 6, 2019
    #12
  13. Croydon

    Croydon ISPConfig Developer ISPConfig Developer

    Ah okay, never tried that. I'm a bit sceptic if that can successfully work for non-numeric (or alpha-numeric) values.
     
    Croydon, Aug 6, 2019
    #13
  14. Jesse Norell

    Jesse Norell ISPConfig Developer Staff Member ISPConfig Developer

    How about when reading this field, before ini parsing, another pass is made to normalize newlines to the expected format. Are there any fields within the (parsed) ini which include embedded newlines?

    This is of course assuming my memory is correct in what happened. :) I'll see if I can create a simple, reproducable failure scenario.
     
    Jesse Norell, Aug 6, 2019
    #14
    Croydon likes this.
  15. Croydon

    Croydon ISPConfig Developer ISPConfig Developer

    Not sure if that would work. The parser already replaces \r\n by \n and then splits by \n.
    So the failure could only occur on systems where \r\n or \n are replaces by \r (which is quite unusual). So it might be more probable that the newlines got lost completely and that could not be fixed by the parser.
     
    Croydon, Aug 6, 2019
    #15
  16. till

    till Super Moderator Staff Member ISPConfig Developer

    We should consider migrating the server config data to the sys_ini table which is basically a key/value store where each record is stored in its own row and write an emulator to return the data in the same array format as we get it from the current storage field for server config and system config data.

    @Jesse Norell : Indeed, I remember the phpmyadmin inline edit issue as you mention it now, totally forgot that. was a real problem at that time and possibly it's still one of the sources of these problems.
     
    till, Aug 6, 2019
    #16
    Jesse Norell likes this.
  17. Croydon

    Croydon ISPConfig Developer ISPConfig Developer

    Agreed, we'll get that ready for one of the next releases :)
     
    Croydon, Aug 6, 2019
    #17
    neutrinohu and Jesse Norell like this.
  18. jakrla

    jakrla New Member

    Hi everybody,

    I landed to this thread, as I just had exactly the same issue. Having a problem with installed packages (installation failed) I checked whether there is some data remaining in the database that could cause the error. Used php-myadmin for browsing (aps_instances) but didn't find anything unusual. I definitely didn't alter nor delete anything in the database. However, after that I found that [web] section of the server config was empty. Tried to input the values that Till posted into this thread earlier, but in my form there is no fields for Nginx-related data on [web] section and lack of that data causes errors and thus I cannot save the configuration.

    Nginx Vhost config dir is empty.
    Invalid nginx config directory.
    Nginx Vhost config enabled dir is empty.
    Invalid nginx conf enabled directory.
    Nginx user is empty.
    Invalid nginx user.
    Nginx group is empty.
    Invalid nginx group.
    Nginx CGI Socket is empty.
    nginx_cgi_socket_error_regex

    I'm running ISPConfig 3.1.15p2 on Debian 10

    Screen capture of empty form looks like this:
    Selection_087.png
    I would appreciate any help that could help me out of this.

    ... And thanks developers for great product. I've been running ispconfig couple of years now, and never had any major issues before this.
     
    jakrla, Jan 8, 2020
    #18
  19. nhybgtvfr

    nhybgtvfr Well-Known Member HowtoForge Supporter

    website basedir /var/www
    website path /var/www/clients/client[client_id]/web[website_id]
    website symlinks /var/www/[website_domain]/:/var/www/clients/client[client_id]/[website_domain]/
    vhost config dir /etc/apache2/sites-available
    vhost config enabled dir /etc/apache2/sites-enabled
    apache user www-data
    apache group www-data
    htaccess AllowOverride All
    enable ip wildcard(*) checked

    any other fields you can enable/disable according to your own preferences/requirements
     
    nhybgtvfr, Jan 8, 2020
    #19
  20. jakrla

    jakrla New Member

    Thanks for prompt reply, unfortunately, however, that input results following:
    Selection_087.png
    ...and as earlier, doesn't allow me to save the configuration.
     
    jakrla, Jan 8, 2020
    #20
Page 1 of 2

Share This Page