datalog changes are not processed

Discussion in 'General' started by nvl-it-solutions, Apr 9, 2024.

  1. nvl-it-solutions

    nvl-it-solutions New Member

    Since a short while I notice that specific changes are not being processed correctly on the master ispconfig server.
    If I for example create a remote user, the user is being created but the notifications regarding datalog_status_... don't disappear.
    I check the sys_datalog table and found out that server_id is set to 0.
    According to one of the topics I found this means it is a general setting (not sure if I understood correctly).
    My master server has a server_id of 1, so after changing the 0 to 1 in the sys_datalog table for that specific change, the notification disappeared.

    Any idea why I suddenly am having this issue?
    I already tried updating ISPConfig to the latest version, restarted the server, ...
    I'm running PHP8.1 on Ubuntu 22.04 with apache.
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

  3. nvl-it-solutions

    nvl-it-solutions New Member

    Hi Till,

    I checked that topic already, but can't figure out wat is going wrong.
    Below you can find all the output as mentioned in the faq.
    As far as I can see, all changes are being processed by the script, but it doesn´t reflect it on the frontend.

    I can see the following notifications after I went to the client section and after I created a remote user that I need to use the API of ISPConfig.
    • datalog_status_u_client: 2
    • datalog_status_u_remote_user: 2
    When I go to Monitor -> Datalog History I can see that the changes that are not being processed are missing the server name. All other entries do have a server name.

    Code:
    ##### SERVER #####
    IP-address (as per hostname): ***.***.***.***
    [WARN] could not determine server's ip address by ifconfig
    [INFO] OS version is Ubuntu 22.04.4 LTS
     
    [INFO] uptime:  21:00:33 up 1 day, 8 min,  2 users,  load average: 0.25, 0.11, 0.03
     
    [INFO] memory:
                   total        used        free      shared  buff/cache   available
    Mem:           1.9Gi       432Mi       322Mi        10Mi       1.2Gi       1.3Gi
    Swap:          2.0Gi          0B       2.0Gi
     
    [INFO] systemd failed services status:
      UNIT             LOAD   ACTIVE SUB    DESCRIPTION
    ● apt-news.service loaded failed failed Update APT News
    
    LOAD   = Reflects whether the unit definition was properly loaded.
    ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
    SUB    = The low-level unit activation state, values depend on unit type.
    1 loaded units listed.
    
    [INFO] ISPConfig is installed.
    
    ##### ISPCONFIG #####
    ISPConfig version is 3.2.11p2
    
    
    ##### VERSION CHECK #####
    
    [INFO] php (cli) version is 8.1.27
    [INFO] php-cgi (used for cgi php in default vhost!) is version 8.1.27
    
    ##### PORT CHECK #####
    
    [WARN] Port 143 (IMAP server) seems NOT to be listening
    [WARN] Port 993 (IMAP server SSL) seems NOT to be listening
    [WARN] Port 110 (POP3 server) seems NOT to be listening
    [WARN] Port 995 (POP3 server SSL) seems NOT to be listening
    [WARN] Port 465 (SMTP server SSL) seems NOT to be listening
    [WARN] Port 21 (FTP server) seems NOT to be listening
    
    ##### MAIL SERVER CHECK #####
    
    [WARN] I found no "submission" entry in your postfix master.cf
    [INFO] this is not critical, but if you want to offer port 587 for smtp connections you have to enable this.
    [WARN] I found no "smtps" entry in your postfix master.cf
    [INFO] this is not critical, but if you want to offer SSL for smtp (not TLS) connections you have to enable this.
    
    ##### RUNNING SERVER PROCESSES #####
    
    [INFO] I found the following web server(s):
        Apache 2 (PID 228094)
    [INFO] I found the following mail server(s):
        Postfix (PID 1454)
    [WARN] I could not determine which pop3 server is running.
    [WARN] I could not determine which imap server is running.
    [WARN] I could not determine which ftp server is running.
    
    ##### LISTENING PORTS #####
    (only        ()
    Local        (Address)
    ***.***.***.***:53        (585/systemd-resolve)
    [anywhere]:3306        (7114/mariadbd)
    [anywhere]:25        (1454/master)
    [localhost]:11211        (597/memcached)
    ***.***.***.***:22        (886/sshd:)
    *:*:*:*::*:443        (228094/apache2)
    *:*:*:*::*:3306        (7114/mariadbd)
    *:*:*:*::*:80        (228094/apache2)
    *:*:*:*::*:25        (1454/master)
    *:*:*:*::*:8080        (228094/apache2)
    *:*:*:*::*:8081        (228094/apache2)
    
    
    
    
    ##### IPTABLES #####
    Chain INPUT (policy ACCEPT)
    target     prot opt source               destination       
    
    Chain FORWARD (policy ACCEPT)
    target     prot opt source               destination       
    
    Chain OUTPUT (policy ACCEPT)
    target     prot opt source               destination       
    
    
    
    
    ##### LET'S ENCRYPT #####
    Certbot is installed in /usr/bin/letsencrypt
    
    Code:
    10.04.2024-20:51 - DEBUG [plugins.inc:155] - Calling function 'check_phpini_changes' from plugin 'webserver_plugin' raised by action 'server_plugins_loaded'.
    10.04.2024-20:51 - DEBUG [server:177] - Found 2 changes, starting update process.
    10.04.2024-20:51 - DEBUG [plugins.inc:118] - Calling function 'server_ip' from plugin 'apache2_plugin' raised by event 'server_update'.
    10.04.2024-20:51 - DEBUG [system.inc:2430] - safe_exec cmd: which 'apache2ctl' 2> /dev/null - return code: 0
    10.04.2024-20:51 - DEBUG [apache2 plugin.inc:2485] - Writing the conf file: /etc/apache2/sites-available/ispconfig.conf
    10.04.2024-20:51 - DEBUG [plugins.inc:118] - Calling function 'update' from plugin 'apps_vhost_plugin' raised by event 'server_update'.
    10.04.2024-20:51 - DEBUG [system.inc:2430] - safe_exec cmd: which 'apache2ctl' 2> /dev/null - return code: 0
    10.04.2024-20:51 - DEBUG [plugins.inc:118] - Calling function 'update' from plugin 'network_settings_plugin' raised by event 'server_update'.
    10.04.2024-20:51 - DEBUG [network settings plugin.inc:249] - Network configuration disabled in server settings.
    10.04.2024-20:51 - DEBUG [plugins.inc:118] - Calling function 'server_update' from plugin 'rspamd_plugin' raised by event 'server_update'.
    10.04.2024-20:51 - DEBUG [plugins.inc:118] - Calling function 'update' from plugin 'server_services_plugin' raised by event 'server_update'.
    10.04.2024-20:51 - DEBUG [plugins.inc:118] - Calling function 'server_update' from plugin 'webserver_plugin' raised by event 'server_update'.
    10.04.2024-20:51 - DEBUG [modules.inc:240] - Processed datalog_id 12984
    10.04.2024-20:51 - DEBUG [plugins.inc:118] - Calling function 'server_ip' from plugin 'apache2_plugin' raised by event 'server_update'.
    10.04.2024-20:51 - DEBUG [system.inc:2430] - safe_exec cmd: which 'apache2ctl' 2> /dev/null - return code: 0
    10.04.2024-20:51 - DEBUG [apache2 plugin.inc:2485] - Writing the conf file: /etc/apache2/sites-available/ispconfig.conf
    10.04.2024-20:51 - DEBUG [plugins.inc:118] - Calling function 'update' from plugin 'apps_vhost_plugin' raised by event 'server_update'.
    10.04.2024-20:51 - DEBUG [system.inc:2430] - safe_exec cmd: which 'apache2ctl' 2> /dev/null - return code: 0
    10.04.2024-20:51 - DEBUG [plugins.inc:118] - Calling function 'update' from plugin 'network_settings_plugin' raised by event 'server_update'.
    10.04.2024-20:51 - DEBUG [network settings plugin.inc:249] - Network configuration disabled in server settings.
    10.04.2024-20:51 - DEBUG [plugins.inc:118] - Calling function 'server_update' from plugin 'rspamd_plugin' raised by event 'server_update'.
    10.04.2024-20:51 - DEBUG [plugins.inc:118] - Calling function 'update' from plugin 'server_services_plugin' raised by event 'server_update'.
    10.04.2024-20:51 - DEBUG [plugins.inc:118] - Calling function 'server_update' from plugin 'webserver_plugin' raised by event 'server_update'.
    10.04.2024-20:51 - DEBUG [modules.inc:240] - Processed datalog_id 12985
    10.04.2024-20:51 - DEBUG [services.inc:56] - Calling function 'restartHttpd' from module 'web_module'.
    10.04.2024-20:51 - DEBUG [system.inc:2083] - Trying to use Systemd to restart service
    10.04.2024-20:51 - DEBUG [system.inc:2430] - safe_exec cmd: systemctl is-enabled 'apache2' 2>&1 - return code: 0
    10.04.2024-20:51 - DEBUG [web module.inc:246] - Restarting httpd: systemctl restart apache2.service
    10.04.2024-20:51 - DEBUG [server:217] - Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock
    finished server.php.
    
    Code:
    0.04.2024-20:52 - DEBUG [plugins.inc:155] - Calling function 'check_phpini_changes' from plugin 'webserver_plugin' raised by action 'server_plugins_loaded'.
    10.04.2024-20:52 - DEBUG [server:217] - Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock
    
     
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    According to the server.sh output, the issue is not on the server where you ran server.sh. Did you ran this on all servers to see on which one it fails?
     
  5. nvl-it-solutions

    nvl-it-solutions New Member

    No I didn't as I assumed creating a remote user and editing clients is being done on the master ISPConfig server.
    I will try to run server.sh on my other servers later today and will let you know the outcome.
     
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    Your problem is that one of the slave servers stopped pulling changes from the master; that's why you have these pending changes there. So the problem is not the master, it's one of the slave nodes, and therefore, you must run the server.sh there to see which of the slaves fails. or have you recently disabled a slave node but did not remove it?
     
  7. nvl-it-solutions

    nvl-it-solutions New Member

    Thanks for the info.
    I haven't disabled a slave node, but there were some network changes which could prevent the slave node to reach the master node resulting in this issue. I think I'll need to check each slave node as you mentioned to determine where the issue happens.
     
  8. till

    till Super Moderator Staff Member ISPConfig Developer

    nvl-it-solutions likes this.
  9. nvl-it-solutions

    nvl-it-solutions New Member

    I tried the sql statement you're referring to.
    It shows me the result I was already aware of, but I can't pinpoint which server has an id of 0 since my master node has an id of 1. All the slave nodes have higher ids.

    Code:
    +-----+-----------+
    | cnt | server_id |
    +-----+-----------+
    |   4 |         0 |
    +-----+-----------+
    
    Here a list of my servers:
    Code:
    +-----------+
    | server_id |
    +-----------+
    |         1 |
    |        59 |
    |        61 |
    |        63 |
    |        65 |
    |        70 |
    |        73 |
    |        74 |
    |        75 |
    |        76 |
    |        77 |
    |        79 |
    |        80 |
    |        81 |
    |        84 |
    |        85 |
    |        87 |
    +-----------+
    
     
  10. till

    till Super Moderator Staff Member ISPConfig Developer

    ID 0 means it's a broadcast to all servers. What you can try to look at is that you check the updated column in the server table to see which of the servers is behind (lower ID in the updated column) compared to the max datalog_id in the sys_datalog table.
     
    nvl-it-solutions likes this.
  11. nvl-it-solutions

    nvl-it-solutions New Member

    I checked one of my slave nodes and it seems that it can't reach my master node anymore after migrating the server to another ip range.
    I did a quick rollback to see if the notifications would disappear and that seems to be the case.
    So for now it is fixed and I just need to finish my migration completely (arrange proper connectivity) in order to have everything working correctly.
    Thanks for your feedback Till.
     
    till likes this.

Share This Page