While updating ISPConfig, I'm going through all servers and comparing install/server templates to conf-custom files; I don't have a tremendous number to do, but enough I'm thinking again about how to make that process easier. If anyone wants to toss out ideas, maybe we could come up with a couple that are worth implementing? My first issue to address is how to identify A) when upstream conf file which I used as my baseline has changed, and B) specifically what those changes are. Right I'm manually comparing file diff's of the new download vs. my conf-custom files, and trying to remember which changes I made vs which are new in upstream (because in retrospect, I did not clearly indicate my changes in comments ; I'd like to automate this process a little more. Part A) above could be handled by adding tags to the conf and conf-custom files, etc. so conf file has a comment with 'ISPC Conf Version: 20180605.18' or similar at the top, something that lets you tell when a file is "newer" would be nice, though even something that lets you know it's "different" (like a commit id) could be used. Then the corresponding conf-custom file has a comment with 'ISPC Conf Base: 20180503.24' or such. If new version > base version in conf-custom file, the updater prints a warning that updates are needed. Maybe even stick a note in system logs. Part B) would require keeping the original conf file that the conf-custom was based on (which is already there for 'server' conf files, but not 'install' conf files, easy enough to keep those around), and running a diff against that vs. the file found in the ispconfig distribution during update. This could be displayed to the admin running the update, and/or emailed to to the admin, and/or have a cli tool that finds those and prints the diff whenever it's needed. Those two could of course be used together, eg. only run a diff against files when the version number has changed in the ispconfig distribution. You could even get fancy with the installer (or a second cli tool) and pull a diff of the current vs. new upstream versions, then try applying that as a patch against the conf-custom file, and if it applies cleanly, prompt the admin to update the conf-custom automatically. Depending on what was implemented, it might be necessary to keep the base conf file version separate from the current conf file version (eg. a diff could be performed once during an update, but if a diff/patch tool were going to work at a later point, ie. after the update has completed, it would need both the base and current versions; alternately the diff/patch could be a tool available as part of a pre-update, but not available once the update has actually been performed). My next issue/discussion is on how to improve the overall design of conf-custom files. Often the changes I make to conf-custom files are a simple addition of something, though of course they are complete rewrite of existing things at times. Perhaps there could be a way to mark up config files with a "section" name, eg. add a comment like 'ISPC Conf Section: aribitrary-name' and specific "include" points, eg. 'ISPC Conf Include: mail/some-name'. Then when a config file is built, instead of the current "use conf-custom file if it exists, otherwise use conf file" logic, the actual config file is built by processing each section in the conf file looking for conf-custom snippets to include before (eg. 'conf-custom/arbitrary-name.pre'), instead of ('conf-custom/arbitrary-name.replace') or after ('conf-custom/arbitrary-name.post') the conf file, while any "include" names simply include a conf-custom file snippet (eg. 'conf-custom/mail/some-name.include') at that point. That gives you a handle to insert config at any arbitrary point it might be needed, and to replace any sections when that's appropriate. The goal with this would be to allow setting up some config tweaks that don't have to be manually integrated / checked for every update. That would make life easier for anyone doing ISPConfig updates, and of course help cut down on one of the recurring (though low volume) support issues here in the forums.
