Just another "stuck on pending updates" - ISPConfig 3.2.9p1

Discussion in 'General' started by dfumagalli, Apr 5, 2023.

  1. dfumagalli

    dfumagalli Member

    Hello,
    I usually manage to fix these issues by myself, but this time I don't know what to do without risking breaking a live multi-server cluster.

    Yesterday - I don't know why - the 000-default Apache config file got broken (a simple AllowOverrides put in the wrong spot by who knows which script) during an OS auto-update.

    Nothing warned me about that, I checked the web server 1 settings on ISPConfig without touching anything. Web server 1 is also where the ISPConfig panel is installed.
    As sometimes happens, even not changing anything causes ISPConfig to queue a server settings change. In this case, the "new" configuration is exactly identical but (looking in the ISPConfig database) with one ".ini" entry shifted to another row of the database text field.
    Then I changed PHP-FPM pm.max_requests on a hosted website (on web server 1 too).

    One of the two changes above triggered an Apache reload and, due to the error above, it failed to restart.

    After a couple of minutes, a customer called me because of his website stopped working. I quickly found out about the Apache config file modification, I fixed it and now everything works again... except for the pending updates on ISPConfig.

    It's stuck on 2 pending updates: "Update server settings: 2".
    I've enabled debug mode for the web server. Here's what happens when I manually run the ISPConfig script:
    Code:
    php -v
    PHP 7.4.33 (cli) (built: Feb 14 2023 18:31:54) ( NTS )
    Code:
    /usr/local/ispconfig/server/server.sh
    05.04.2023-09:34 - DEBUG [plugins.inc:155] - Calling function 'check_phpini_changes' from plugin 'webserver_plugin' raised by action 'server_plugins_loaded'.
    05.04.2023-09:34 - DEBUG [server:217] - Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock
    finished server.php.
    The last entry on sys_datalog is this:

    2023-04-05_001.png

    and the data contents (I removed irrelevant stuff and anonymized it):

    Code:
    a:2:{s:3:"old";a:21:{s:9:"server_id";s:1:"1";s:10:"sys_userid";s:1:"1";s:11:"sys_groupid";s:1:"1";s:13:"sys_perm_user";s:4:"riud";s:14:"sys_perm_group";s:4:"riud";s:14:"sys_perm_other";s:1:"r";s:11:"server_name";s:35:"<censored>";s:11:"mail_server";s:1:"0";s:10:"web_server";s:1:"1";s:10:"dns_server";s:1:"1";s:11:"file_server";s:1:"1";s:9:"db_server";s:1:"0";s:14:"vserver_server";s:1:"0";s:12:"proxy_server";s:1:"0";s:15:"firewall_server";s:1:"1";s:11:"xmpp_server";s:1:"0";s:6:"config";s:5630:"[global]
    webserver=apache
    mailserver=postfix
    dnsserver=mydns
    
    [server]
    auto_network_configuration=n
    ip_address=<censored>
    netmask=255.255.255.0
    v6_prefix=
    gateway=0.0.0.0
    firewall=ufw
    hostname=<censored>
    nameservers=<censored>
    loglevel=0
    admin_notify_events=1
    backup_dir=/var/backup
    backup_tmp=/tmp
    
    ...
    
    ";s:7:"updated";s:3:"147";s:16:"mirror_server_id";s:1:"0";s:9:"dbversion";s:2:"97";s:6:"active";s:1:"1";}}
    
    I tried bumping the dbispconfig->server->updated field (which is set to 149 like datalog_id above) to 150 but nothing changed, so I restored it back to 149.

    Oddly enough, I can still work on the panel, the new updates just "add" on top of the lingering 2 old notifications, get processed and then the indicator is back to "2".

    Anything I can do to fix this?

    Thank you.
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    A datalog_id of 149 seems to be very low, or is this a new setup? Which value is in the 'updated' column of the server table on the master and the slave for this affected slave server?
     
  3. dfumagalli

    dfumagalli Member

    Hello Till,
    The 'updated' field of the server table is set to 149 too. This is the master server, which is also the ISPConfig server and the setting is being updated on the same server as well.
    The multi-servers system as a whole is about 9 months old but it's 3 websites on 2 servers (one website has very heavy traffic), 2 DB servers, 1 mail server.
     
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    Ok, so the number of records in datalog can be correct in this caye. And you ran the /usr/local/ispconfig/server/server.sh script on the system where the updates are pending for and not the master, right? Because according to the output you posted, there are no pending updates for the system where you ran server.sh on. Maybe the pending updates are for a different system?
     
  5. dfumagalli

    dfumagalli Member

    Well, how do I know which system the changes belong to? Because I just entered the servers config, (only) looked around and closed the popup. It happens some times, I do nothing yet ISPConfig wants to update something.
     
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    That's fine and not an issue.

    Then the changes are opening for the system you checked the config. If you don't remember which server it was, run server.sh on each server to see where you get an error when running it.
     
  7. dfumagalli

    dfumagalli Member

    Hello Till,
    I found out one of the server is showing the following error when I run server.sh:

    Code:
    Falsche Anfrage / Wrong Query SQL-Query = SET character_set_results = 'utf8', character_set_client = 'utf8', character_set_connection = 'utf8', character_set_database = 'utf8', character_set_server = 'utf8'PHP Warning:  Undefined variable $state_out in /usr/local/ispconfig/server/lib/classes/plugins.inc.php on line 173
    finished server.php.
    If I enable debug, it complains about an undefined variable returned in the last statement of the file.

    Cli PHP on this server is 7.4.33.
     
    Last edited: Apr 12, 2023
  8. dfumagalli

    dfumagalli Member

    I fixed the topic issue. On two servers, I had to rerun the update and "Reconfigure Permissions in master database? yes".
     

Share This Page