Upgraded my master from 3.2.9p1 to 3.2.11 and have 1621 items that need updating

Discussion in 'ISPConfig 3 Priority Support' started by webguyz, Oct 21, 2023.

Page 1 of 2
  1. webguyz

    webguyz Active Member HowtoForge Supporter

    I think its an issue ater I physically moved my entire IPSconfig multiserver network to another subnet. I used ispconfig_update.sh --force and did a reconfigure database and everything was fine until I tried to install upgrade on the master.

    All slave servers are working OK and say "Finished server.php' when I run /usr/local/ispconfig/server/server.sh
    If I add an email client it adds 2 tasks to the task list but then quickly removes those two when server connects.
    Id it possible reconfiguring Permissions somehow screwed something up?
    Thanks!
     
    webguyz, Oct 21, 2023
    #1
  2. webguyz

    webguyz Active Member HowtoForge Supporter

    Checked the dates on the entries and the oldest was 2016 and newest was from earlier today. Various servers, mail and dns entries.
    Is it possible to dump that datalog table? How does some recover from this?
     
    webguyz, Oct 21, 2023
    #2
  3. till

    till Super Moderator Staff Member ISPConfig Developer

    Do not empty the sys_datalog table. Enable debug mode and run server.sh on master and slave node and post the result.
     
    till, Oct 21, 2023
    #3
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    till, Oct 21, 2023
    #4
  5. webguyz

    webguyz Active Member HowtoForge Supporter

    Till,
    Checked my config.inc.php and its OK and already had the "" for the 2 options.

    After more looking it appears there is some kind of issue between my master and one slave. Put both in debug and here are the server.sh results:

    Master:
    root@dotcp:~# /usr/local/ispconfig/server/server.sh
    22.10.2023-11:59 - DEBUG [plugins.inc:155] - Calling function 'check_phpini_changes' from plugin 'webserver_plugin' raised by action 'server_plugins_loaded'.
    22.10.2023-11:59 - DEBUG [server:217] - Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock
    finished server.php.

    Slave:
    root@proaudio:~# /usr/local/ispconfig/server/server.sh
    22.10.2023-12:01 - DEBUG [plugins.inc:155] - Calling function 'check_phpini_changes' from plugin 'webserver_plugin' raised by action 'server_plugins_loaded'.
    22.10.2023-12:01 - DEBUG [server:217] - Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock
    finished server.php.
     
    webguyz, Oct 22, 2023
    #5
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    Please try emptying all dbmaster fields on master server to see if it fixes the issue.

    Both are fine. Do you maybe have more slave nodes listed in ISPConfig under System > server services?
     
    till, Oct 22, 2023
    #6
  7. webguyz

    webguyz Active Member HowtoForge Supporter

    Have 24 slaves. I randomly looked at a few and all said finished server.php. Will check the rest of them. Not uderstanding why the datalog errors go back to 2016. Will turn debug on all the other slaves and checked the cp.
     
    webguyz, Oct 22, 2023
    #7
  8. till

    till Super Moderator Staff Member ISPConfig Developer

    This is fine, ISPConfig deletes server specific records after 30 days, but global records that are required to sync new nodes in the future are kept. So there are only a few records kept and these must go back until you set up the master node.
     
    till, Oct 22, 2023
    #8
  9. webguyz

    webguyz Active Member HowtoForge Supporter

    There must be a field that says changes were updated. The 1621 entries in the red ball at the top include changes that have already been made. New updates work fine, they increase the 1621 a few digits and as they servers phone home and accept changes it decriments the notification balloon back to 1621. ISPConfig system seems to work ok accept for the confusion of these 1621 items to update.
     
    webguyz, Oct 22, 2023
    #9
  10. till

    till Super Moderator Staff Member ISPConfig Developer

    One of your slave nodes currently does not fetch updates from master, and that is why you have this number of pending updates. You can take a look in the server table on master database, each server has a column named 'updated', the number there must be the max datalog_id from sys_datalog table. At least one of the nodes must have a lower number and that's the node that does not fetch updates.
     
    till, Oct 23, 2023
    #10
  11. webguyz

    webguyz Active Member HowtoForge Supporter

    Thanks for the info. Will have to write a script to compare the updated filed with info in the sys_datalog. Is it possible a slave server that was having issues was deleted and recreated using the same server name and now ISPConfig is confused?

    Is there a document that explains what the different codes in the data field in datalog mean
     
    webguyz, Oct 23, 2023
    #11
  12. webguyz

    webguyz Active Member HowtoForge Supporter

    • Till, here are the 1621 updates that are waiting to complete. It seems to cover a number of servers including dns and email slaves. Does anything stand out to you like the datalog_status_u_client with 496 updates or datalog_status_u_sys_ini: 170?
      Code:
      datalog_status_d_client: 11
datalog_status_i_client: 40
datalog_status_u_client: 496
datalog_status_i_directive_snippets: 1
datalog_status_u_directive_snippets: 7
Delete DNS record: 54
Create DNS record: 7
Update DNS record: 10
Delete DNS zone: 7
Update DNS zone: 15
datalog_status_i_dns_template: 2
datalog_status_u_dns_template: 10
datalog_status_d_domain: 13
Delete email domain: 5
Update email domain: 1
Delete email user: 3
Create email user: 1
datalog_status_u_mail_user_filter: 2
datalog_status_i_remote_user: 5
datalog_status_u_remote_user: 12
datalog_status_i_spamfilter_policy: 2
datalog_status_u_spamfilter_policy: 35
datalog_status_i_sys_group: 40
datalog_status_u_sys_ini: 170
Update database: 2
Delete database user: 17
Create database user for database: 134
Update database user: 23
Delete folder protection user: 4
Update folder protection user: 4
datalog_status_i_wizard_template: 1
datalog_status_u_wizard_template: 2
     
    webguyz, Oct 23, 2023
    #12
  13. till

    till Super Moderator Staff Member ISPConfig Developer

    You must run server.sh on the affected servers to see why they can't fetch records anymore, the most likely reason for that is that they are unable to connect to mysql on the master server anymore. Just as info, the pending changes counter is calculated by this query:

    Code:
    SELECT COUNT( * ) AS cnt, sys_datalog.action, sys_datalog.dbtable
                FROM sys_datalog, server
                WHERE (server.server_id = sys_datalog.server_id or sys_datalog.server_id = 0) AND sys_datalog.user = ? AND sys_datalog.datalog_id > server.updated
                GROUP BY sys_datalog.dbtable, sys_datalog.action
    So you do not need a script to compute it and you can see which servers are causing this directly in the database. Open sys_datalog table and take a note which the highest datalog_id is. The look into the table "server" and check which servers have a value in the column 'updated' that is lower then the datalog_id you noted down.

    If you need help with this by remote login, you might want to contact @Th0m from ISPConfig business support: https://www.ispconfig.org/get-support/?type=ispconfig

    No, ISPConfig does not care about server names, servers are referenced by their internal ID only.

    The data is the serialized form of the old and new data record (array) that was changed in this config transaction. But this does not matter for this issue.
     
    till, Oct 23, 2023
    #13
  14. webguyz

    webguyz Active Member HowtoForge Supporter

    Thanks Til. will play with this some more but will consider the Thom option. For the most part the system is working and updates are happening except for these 1621 update.
     
    webguyz, Oct 23, 2023
    #14
  15. webguyz

    webguyz Active Member HowtoForge Supporter

    Till,
    What did you mean by the line below from your earlier response. I missed it but rereading it today I am not sure what you meant
     
    webguyz, Oct 31, 2023
    #15
  16. till

    till Super Moderator Staff Member ISPConfig Developer

    The server config.inc.php file contains a config section for the login to the master database. The master server does not connect to a master database, it connects only to its own database on localhost, so its worth that you try to empty these fields like this:

    Code:
    //** Database settings for the master DB. This setting is only used in multiserver setups
$conf['dbmaster_type']            = 'mysql';
$conf['dbmaster_host']            = '';
$conf['dbmaster_port']            = '';
$conf['dbmaster_database']        = '';
$conf['dbmaster_user']            = '';
$conf['dbmaster_password']        = '';
$conf['dbmaster_new_link']         = false;
$conf['dbmaster_client_flags']     = 0;
    But only on master, on the slaves, these fields must contain valid data to login to master mysql server.
     
    till, Oct 31, 2023
    #16
  17. webguyz

    webguyz Active Member HowtoForge Supporter

    I tried that just now but still no change to my datalog outstanding number. Actually its went up from 1621 to 1625. Went through the datalog history one at a time and any slaves that were involved I logged into them to run server.sh and all finished with no problems.
     
    webguyz, Oct 31, 2023
    #17
  18. webguyz

    webguyz Active Member HowtoForge Supporter

    Till,
    In the sql script you had written earlier what is the ? in sys_datalog.user as below?

    SELECT COUNT( * ) AS cnt, sys_datalog.action, sys_datalog.dbtable FROM sys_datalog, server WHERE (server.server_id = sys_datalog.server_id or sys_datalog.server_id = 0) AND sys_datalog.user = ? AND sys_datalog.datalog_id > server.updated GROUP BY sys_datalog.dbtable, sys_datalog.action

    Thanks
     
    webguyz, Oct 31, 2023
    #18
  19. till

    till Super Moderator Staff Member ISPConfig Developer

    Try running this sql query on the master dbispconfig database:

    Code:
    SELECT sys_datalog.datalog_id, sys_datalog.server_id
                FROM sys_datalog, server
                WHERE (server.server_id = sys_datalog.server_id or sys_datalog.server_id = 0) AND sys_datalog.datalog_id > server.updated
    This should give you the list of changes that could not be applied and also which server they belong to.
     
    till, Oct 31, 2023
    #19
  20. webguyz

    webguyz Active Member HowtoForge Supporter

    Ran it and it gave me 2895 rows. I scrolled thru the whole list and had server_id = 0 for every entry.
    Not sure If I had told it to reconfigure permissions at Master yes whenI did the update. I had to do that to all the slaves with the --force option and I might have did it for the Master but I can't remember. Not sure if that makes a difference.
     
    webguyz, Oct 31, 2023
    #20
Page 1 of 2

Share This Page