conf-custom and post-install/update changes

Discussion in 'Installation/Configuration' started by TonyG, Jan 8, 2021.

  1. TonyG

    TonyG Active Member

    I've configured files in conf-custom/install, including : debian_postfix.conf.master and debian_dovecot2.conf.master

    To get those settings into production files in /etc, I copy/pasted the .master files to /etc, then manually modified the {placeholder} values with values from the previously generated configs. That's working but some fields were still not right.

    Is this all correct?
    1) For Postfix the only time the .conf.master template file is used is on install or update.
    2) For Postfix, ISPConfig only takes the fields that we can maintain from the UI, and feeds them through postconf. The template is not used to regenerate a new application /etc file.
    3) For Dovecot, the only time the .conf.master template file is used is on install or update.
    4) For Dovecot, ISPConfig does not yet have any settings from the UI that update the /etc/files.

    This has been asked before but I want to be sure for the current ISPConfig version:

    Is it safe and reasonable to simply run update.php to regenerate configs from the templates, and conf-custom?

    That's an important and fearful action. I want to be sure that running the update one or more times will not Unconfigure or Overconfigure and just create more breakage.

    I know there was a recent enhancement to backup the files in conf-custom/install. That update seems incomplete to me because if we answer Yes to backup/overwrite conf-custom, then we don't get our mods, but if we answer No then we don't get the new tpl changes. Either way we lose. I think this is what @ahrasis is working on now, and each of us who need to this needs to come up with a similar approach to reconcile updates with local custom configs. Is my understanding of this accurate?

    Are there any other suggestions for how to handle this given the resources (code, templates) that are currently available? I just want to be sure I'm not missing anything. I don't mind dealing with the situation but I need to be sure I understand the situation correctly.

    Thanks.
     
    ahrasis likes this.
  2. Th0m

    Th0m ISPConfig Developer Staff Member ISPConfig Developer

    All files that are in install/tpl and not in server/conf are only used when installing/updating.

    The implementation is to warn users as conf-custom templates often break working systems when updating. When updating, you have to review the changes manually and convert them to your custom template(s).
    There is a request to show the difference: https://git.ispconfig.org/ispconfig/ispconfig3/-/issues/4751
     
  3. Jesse Norell

    Jesse Norell ISPConfig Developer Staff Member ISPConfig Developer

    For postfix, that's correct - also read through the postfix server plugin to see what commands are run (eg. when changing amavis to rspamd or vice-versa).
    Yep, if your changes are in conf-custom, running repeatedly shouldn't be a problem. Note the postfix server plugin commands though, as it sets some things to "known" values, and won't respect your templates if you change them - you probably won't run into a problem, but be aware of that.


    For the various discussions of tracking changes templates, git could work well for that. Eg. pull down the upstream changes into a repo, and make your own template changes in a branch, so you have easy "git diff ... " commands to see what has changed, and all the merge/rebase commands to easily apply your local changes to the upstream templates. Undoubtedly conflicts will arise on occasion, so you can't automate all of it, but much of the time a few git commands could keep your templates current. (I haven't set that up yet, but that's what I'd probably do if I were trying to automate things.)
     
    ahrasis and Th0m like this.
  4. ahrasis

    ahrasis Well-Known Member

    It is a matter of preference on how to do things where some time or most of the time, we simply put forward an idea, or revisit it, just to see whether it is better if we can do things, in some other ways or differently, if compared to the current convention.

    No problem of sticking to what we already have and doing things the way we normally do like the conf-custom folder and install folder under it.

    A git is an approach already suggested by @TonyG in my thread and I already agreed that it is a good idea but again it will become a matter of preference, seeing things from different angles and doing them in a way and in our view we calculated as better and more efficient.

    Whether it can be fully automated or otherwise not, is very subjective, I'd say, as LE SSL scripts for ISPConfig various types of server was looked as very hard things to implement, if not looked as impossible two three years back, but now, we have made it possible with automatic renewal of ispserver.pem and the certs various extension to many other services, if not all.

    I may be dreaming things for now, and I stand corrected on all my views as well, but together, though my meaning is with tolerances, diversities and differences, we can always make this piece of software better and more user friendly for all.
     
  5. TonyG

    TonyG Active Member

    As always, thanks for your feedback.

    To be clear, and there was no indication otherwise, I trust the settings provided in the platform. You guys obviously have a lot more experience in this area than I do, and that experience is baked-in to the product. My intent is not to monkey/play with the configs. There are a lot of questions in this forum that result in "use the conf-config mechanisms". For the average user I think it's really difficult to do that because of how cryptic the mechanisms are until one looks at code and scans a lot of forum posts.

    My intent is to try to clarify in one place how things actually work so that I and others are fully aware. Then we can establish personal patterns for working with the environment and participate effectively in discussions about how to adapt to the environment or change it. More information in one place will also help to transition responses here, from a frustrated "use the conf-config mechanism", which people may not understand how to follow, to a "refer to the documentation on how to use the conf-config mechanism", which is much more concrete.

    So... Can I get feedback on Dovecot, statements #3 and 4 above? It seems like the ISPConfig install/update sets Dovecot with details that work with ISPConfig, and as of now there are no other changes made to the config file(s). Till said something about this a couple years ago in a forum post but I don't know if it is still true. I needed to get into this for at least the following reasons.
    • On initial configuration I needed control over logging so that I could find out exactly what was happening in the environment. That included looking into sieve_trace, for which I found is broken with a Dovecot bug. As issues were resolved I tuned-down the logging.
    • I reconfigured to use LMTP. The ISPConfig default seems to be LDA.
    • On adding Roundcube I needed to understand how Sieve and Managesieve has been integrated, where filtering occurs, etc. It was necessary to understand if and how to add 'sieve' to the protocol list and mail_plugins settings for some protocols, not all.
    • I needed to understand the managesieve_sieve_capability setting, to understand what extensions are supported, what can/should be added, and what (separately) is supported by Roundcube Managesieve.
    Some of that is not directly related to ISPConfig. The challenge is that ISPConfig sets values with a specific intent for use, but as an admin I don't know what that intent is, and cannot manipulate those values from ISPConfig. Until we can maintain those kinds of details from the software, we need to gain understanding and control through the conf-custom mechanism - or to understand where conf-custom isn't going to apply.

    Someone using ISPConfig needs to understand that the config files are not static files, they are templates, and there is a full templating system in the software that replaces {placeholder} text with data. We need to understand when the templating system is invoked, at install/update time and/or from the UI/API so that we know which files to use and where they should be located. Details about this are actively being discussed elsewhere now, by people who are intimate with the process, and I've gathered an understanding from forum, tracker, and code. But it's important to document how this works for people who are coming at this purely from a user/admin perspective.

    As I look at this, I had another idea about maintenance.
     
  6. Jesse Norell

    Jesse Norell ISPConfig Developer Staff Member ISPConfig Developer

    I believe#3 and #4 are correct.

    You shouldn't need any changes in dovecot for lmtp to work, it should always be setup there and postfix is configured to use it if found (in 3.1 that was not the case).
     

Share This Page