Allinstall - In Development - Comments Welcome

Discussion in 'Plugins/Modules/Addons' started by TonyG, Sep 24, 2020.

Tags:
?

What do you think about this so far? :)

  1. Um... I don't get it.

    0 vote(s)
    0.0%
  2. Stupid! Go Home!

    0 vote(s)
    0.0%
  3. Meh. Whatever.

    50.0%
  4. Cool. Let's see how it goes.

    50.0%
  5. Wow. Anxious to see the FOSS.

    0 vote(s)
    0.0%
  6. Dang! I'll pay for that NOW!

    0 vote(s)
    0.0%
  1. TonyG

    TonyG Active Member

    I am working on a private project : Install a new Ubuntu system, download Allinstall, run it to start installing the system with everything we talk about related to ISPConfig. I'm posting info about it here because it seems like this fits as being sort of an ISPConfig addon.

    All of the features described below exist and I'm testing/debugging. It will soon also include the new RoundCube info written by @Th0m ... which I still need to try. This testing process will continue for at least several weeks as I use this in my own network. When it's working for my own purposes I'll open it to the public.

    For now I'm asking for comments and suggestions, questions, support, and/or gentle discouragement. :) Here are questions that I'm asking myself... Does this seem worthwhile? Am I duplicating something that exists? Is the scope of this too broad to be useful? Would anyone else actually use this? Am I missing something obvious? Am I nuts? :confused:

    I've been inspired by the team here collaborating on ISPConfig. I want to use ISPConfig more, and use it effectively. I would not have started this if you guys were not so helpful. THANKS!

    Here is the current README.md
    -------------------------------------------------------------------------------------
    Allinstall
    Internet Server Installation with Applications
    Overview
    Two of the main sources of documentation for ISPConfig include the "Perfect" installation guide and the Multi-Server Installation Tutorial. There are many versions of these documents, for different Linux distros and many versions of each.

    "Allinstall" is based on the step-by-step details found in those guides. Allinstall is a collection of Bash scripts that modularize and implement each component installation detailed in the guides.

    The Base setup scripts the installation of a number of core components required for all systems in a network. For example, all systems get common aliases, they get their hostnames set, they get rootkit protection, PHP, and a dataabase. And of course all packages are updated before a system restart.

    The Application setup installs Fail2Ban, Let's Encrypt Certbot, Apach, PureFTPd, and other components, ending with the download (and later installation) of ISPConfig. Again, of course, followed by another update and restart.

    All systems should share those same components. CLI options allow for toggling most of the defaults. From there systems can be specialized. One system can serve as the ISPController system, database host, DNS, mail server, and primary web server ... or these components can be added individually to different sysetems.

    So for example, in the Multi-server tutorial there are five systems. The first is the ISPConfig controller and web server - that system gets the Base plus Applications setup. Other systems are installed to host the specialty components, starting with Base plus Applications, and then getting one or more other setups to become a mail server, and/or DNS, and/or database server, and/or web server.
    Modularity is Paramount
    Because all functionality is encapsulated in small functions within (an almost ridiculous number of) files, this package can be forked, with branches created to support personalization. In this scenario, branches do not need to merge with master, each branch becomes its own Allinstall "distro" in exactly the way Linux distros are organized. In this Allinstall "Core" package, PHP7.4 is installed in all systems, Postfix is installed with rkhunter, and MariaDB is the default. In your personalized branch, you may prefer another rootkit detection package instead of or in addition to rkhunter. You may prefer MySQLv8. You may prefer nullmailer or esmtp on secondary systems with Postfix only on the controller. And you may not wish to use other default Allinstall packages like phpMyAdmin, Quota, SpamAssassin, or ClamAV.
    Author
    Allinstall is being written as a time saving tool to quickly spin up a number of servers in a private cloud network. Tony G, author, is skilled with many technologies and languages - Bash is not one of them. The code here sucks. It's noobish. Anyone with a clue can easily point out technical and stylistic errors, archaic structures, and downright "what the hell is this?" abuse of Bash and other tools. Please forgive, and provide constructive feedback. The code will get better, for now functionality is key. :)
    Getting Started
    See the docs folder for detailed pages about how Allinstall works and how to use it.
    ----------------------------------------------------------------------
     
  2. nhybgtvfr

    nhybgtvfr Well-Known Member HowtoForge Supporter

    there is already an auto-installer script for ispconfig, i assume it's all bash commands, i've not used it myself, there's a lengthy thread about it somewhere on these forums.
    there's also this one i ran across: https://github.com/servisys/ispconfig_setup, again, i've never actually tried it, so i have no idea if it's related in any way to the one with the thread on here.

    i have created one myself in ansible to install a complete server, which worked well, at least until ansible changes the commands and syntax, which it seems to do far too frequently.
    it's a lot more complicated to do one for a multi-server install though, it's fine if the server/vps instances already exist and you have ssh access. i never quite got the hang of ansible and aws dynamic inventories to check what already exists, and re-run the scripts on existing servers, and also create new vps's, add them to the inventory and install to them. and never had the time to spend any longer trying to learn that part of it. but for a complete new install of several servers (static ansible inventory) with roles already defined for each server, it was useful. creating and thoroughly testing the install script takes a hell of a lot longer than just installing everything manually though, and this was fairly simple, with predefined roles/capabilities on one OS.

    i wanted it as an option so that i didn't need to recreate multiple copies of servers in different locations in case an aws zone went out, i could rebuild the whole server environment in minutes and then just restore the data.

    i would recommend you go with something like ansible / puppet etc, though, and build in lots of checks (ansible already includes many) so that the finished script is idempotent, so it can be re-run multiple times against an existing ispconfig server without breaking anything.

    good luck. you're adding a lot more variability for each server than i did with ansible, and, i assume, looking at getting it to work for multiple linux distro's and release versions. if you succeed it should definitely be worth a look.
     
  3. till

    till Super Moderator Staff Member ISPConfig Developer

    The official auto installer is here on ISPConfig GIT server, it is written in PHP:

    https://git.ispconfig.org/ispconfig/ispconfig-autoinstaller

    It is currently in beta and will get released as final after ISPConfig 3.2 is out.
     
    Th0m and ahrasis like this.
  4. TonyG

    TonyG Active Member

    This is the first time I've seen either of those fine projects. I've achieved high functionality with days of coding but I appreciate the more formalized approaches used in the others, and that they are much more developed.

    @nhybgtvfr - We see this similarly. I used to use Ansible more but in my limited usage I've found it takes much more time to get Ansible to do things right than it takes to just do it all manually. I started doing Allinstall with Bash so that it would be more FOSS-friendly - more people can read and contribute to a project that uses shell commands than Ansible. And since I use Ansible so infrequently, when I want to make edits later I've forgotten the details of the tooling, which just slows down the whole process. So, Ansible is a great tool for this job, just not for immediate purposes.

    @till - That project doesn't look like it's near Beta quality, but it's definitely doing exactly what I've been creating.

    Rather than re-inventing this, based on this new information (to me anyway) I'm going to I'll continue with manual installation now just to get through 3.2 and to build my first network. Then I'll come back to use/contribute to the "official" inpconfig-autoinstaller later, and may only continue on Allinstall as a hobby and for those who simply would like an alternative. For now I simply don't have time to duplicate effort. I was doing this to save time and help others, but if I'm doing neither then it doesn't make sense to keep this going now.

    Thanks!!!
     
    till, ahrasis and Jesse Norell like this.
  5. till

    till Super Moderator Staff Member ISPConfig Developer

    Download the latest version from git and run it instead of using the curl method, the curl download has not been updated with latest changes yet as the installer has not been released yet. The installer download from git is working flawlessly here, we use it to install the test systems for ISPConfig 3.2.
     
    ahrasis likes this.
  6. TonyG

    TonyG Active Member

    I couldn't resist using Allinstall for testing with 3.2, and it's all working great. This has been an exercise in getting more intimate with bash scripting, as much as creating helpful tools for my own personal use. I also see it as a way to ensure that I understand how everything is installing - you learn best by teaching, and writing code to "teach" it to install a system is a great way to internalize how it's all working.

    I still don't want to compete with other better options, and really am more comfortable with PHP anyway. But as I think about it, there's nothing wrong with alternative approaches to a problem, for those who might have a different style, preferences, or skills. Most of this code is for installing a base Ubuntu system - the ISPConfig part is just one of the last steps. So I think there's some value to this afterall, and will publish when it's ready.

    All that said, I will still consider the "official" installer projects as being the primary/authoritative tools, and look forward to using and contributing there.
     

Share This Page