XMPP with ISPConfig

At the moment I am writing on a modification of the code in ispconfig master branch to enable ISPConfig to manage XMPP domains and users.
@till: can you estimate a planned release date for v3.1? Maybe I can place my first merge request in time for 3.1.

Details:
Server Software:
The XMPP server managed by my modification is Metronome, a lightwight, easy to configure and extend XMPP service forked from prosody im. It is also the preferred XMPP daemon for use with Jappix Webchat or Movim.
All required steps to get metronome up and running on a debian based host will be included in my merge request.

ISPConfig:
My Modification adds new client- and reseller-limits for XMPP domain and user count.
It adds a new menu section to "E-Mail" Module: "XMPP" with the two menu links "XMPP Domains" and "XMPP Accounts". (Maybe renaming to "Communication" or "Mail & Chat"?)
Of course it also adds required steps to the install routine of ISPConfig and server configuration options.

Release v1. (merge request placed):
Supported Domain Features (per Domain configuration):
- List of Domain Admins (Jabber IDs)
- Pubsub (Microblogging via XMPP)
- Bytestream Proxy (used to bypass Firewalls for Firetransfer and Voice-/Videochat)
- Status Host per Domain (Status of Server Components in JSON/XML format and 'Webpresence' for integration of users chat status on websites)
- Anonymous Host per Domain (for WebChat with guest access)
- Global BOSH support
- MUC support (Multiuser Chatrooms) with pastebin (autoshorten or manuel triggerd link to server-stored paste instead of long text) and http_archive (Chatroom history)
- Creation of SSL certificates and CSRs for entire server and every single hosted domain
- Auto-Setup of required DNS records

Supported Account Features:
- active/inactive
- server module for SQL authentication to identify users by their password from ispconfig database

Release v1.1:
New XMPP domain option: Account management method (Normal / By Mail Domain)
Option "By Mail Domain" requires the corresponding mail domain to be created before the xmpp domain.
OnInsert and OnUpdate this option will sync all mail accounts to xmpp accounts of this domain. It will not be possible to create/delete XMPP accounts for this domain or to change their passwords. They will be created, updated and deleted by management of corresponding mail accounts.
Furthermore the mail domain cannot be deleted as long as there is a linked xmpp domain. First the XMPP must be deleted or switched back to management "normal".

Further Ideas:
Support of password changes and public registration
Only if I can manage to create a plugin to handle this using ISPC Remote Api.
Maybe, if this modification gets included to ISPC, I can get some support by the developer of Metronome to implement this.

 

The problem is not ISPConfig. But there is no server module for the xmpp server yet to handle password changes and registrations with external data sources.

 

I'm trying another solution. I'm using PROSODY:
- Adding a new field, in table "mail_user", called disablexmpp
- Enabling XMPP authentication through dovecot
- Adding a new checkbox to module mail, on mailbox section, to achieve XMPP login
Can we join our solutions?

 

By adding only a field to the mail accounts you cannot control things like MUC or domain admins.
Also you cannot decouple XMPP accounts from mail accounts. Perhaps some customers want to have seperate accounts for both services.
Does your solution require to run dovecot and prosody on the same machine? I think this would break the multiserver concept to be able to run different servers on different machines.

Furthermore, it is not possible to merge back from metronome to prosody but vice versa.
Metronome has a more efficient storage structure for saving the account/chat data on disk and automatically converts from "old" prosody data structure.

 

I placed merge request #170 with release v1.

 

Hello @theWeird
First I want to thank you for your wonderful work!
I'm really interested in this new feature for ISPConfig, actually I need to implement a communication system and this is what I been searching for, but after 2 days playing with metronome I have found some problems/bugs with the ISPConfig templates.

Server environment:
- Debian Jessie
- ISPConfig 3.1.1p1
- Metronome v3.7.9

1. In metronome_conf_host.master, Anonymous login must be activated using the "use_anon_host" variable. Actually using "use_proxy"
3. In metronome_conf_host.master, "proxy65_interfaces" and "proxy65_ports" must be defined in metronome_conf_global.master https://prosody.im/doc/modules/mod_proxy65
3. In metronome_conf_global.master, "bosh_ports" have no effect because this option was removed in prosody v0.9 https://prosody.im/doc/modules/mod_bosh

I can assume that you make this implementation for an older version of metronome/prosody?
Are you planning to release/commit the 1.1 version? (I need the Account management method (Normal / By Mail Domain)) feature.

Regards,
Michael Epstein

 

Hi Michael,
I'm glad you like it.
You're right, my implementation is from march '15. It seemed to be a high chance for a new release in the first month of the last year so I finished my commits by then. For 1.1 I wanted to wait until 1.0 is out with the new release to fix bux first before I start 1.1 development.
There is still another bug I'm working on at the moment:
Metronome should spawn a process for the auth script to work with, but on systems installed with perfect server setup tutorial it doesn't.
So at the moment I try to fix the bugs first.
The timing of this major release is really bad for me, because I start my bachelor theses this week. So it unfortunatly will take some time to fix the bugs. Furthermore I don't think that I will be able to place a merge request for 1.1 this year.
But I definitely will continue my work as soon as the known bugs and problems are removed.

Regards,
Michael

 

Hi guys,
I installed several weeks back "The Perfect Server - Ubuntu 16.04" with Metronome, and I spent a lots of time trying to connect to it from XMPP client without any luck, I checked for info everywhere even the xmpp core RFCs no luck, I would really appreciate any help I can get, here's my configuration:
I setup the DNS for 2 srv records and 1 txt record, also I have setup an XMPP domain & account on my ISPconfig, when I connect from an Xmpp client (Pidgin) to my server I get the following error:
====================================
(12:20:04) jabber: Recv (ssl)(137): <iq id='purple926531ff' type='error'><error type='cancel'><service-unavailable xmlns='urn:ietfarams:xml:ns:xmpp-stanzas'/></error></iq>
(12:20:04) connection: Connection error on 04675888 (reason: 0 description: Service Unavailable)
========================================

Best regards,
Charles

 

Hi ccoudsi,
As Michael say, metronome should spawn a process for the auth but it doesn't.
I spend 5 days reading, changing config..... and nothing. I think the problem is the compatibility for mod_auth_external and the actual metronome version. I ended uninstalling metronome and installing Prosody 0.10. Now everything is working fine, but I have to configure everything by hand and use another auth method, mail account.

I really appreciate Michael and ISPConfig team work, but this metronome implementation is unusable.

 

Hi Michael,
Thaks for your reply, I thought I was doing something wrong !!
Can you please share how did you "use another auth method, mail account." ??
Thanks

 

Hi ccoudsi,
With Prosody, you can use other plugins for authentication. Please check https://hg.prosody.im/prosody-modules/file (all the mod_auth_*)
Right now Im using mod_auth_imap:

Code:

authentication = "imap"
auth_append_host = true;
imap_auth_host = "localhost";
auth_imap_verify_certificate = false;
imap_auth_port = 993;

 

I am still hunting for the bug with the developer of metronome. Hope, we will have a solution soon.
If you still have metronome installed, please try the following command as root:
It should spawn the auth script that reads single lines in the form

Code:

isuser:username:domain
auth:username:domain:password

(replace username, domain and password with real xmpp user credentials)
It should answer to each command with 0 or 1 and also write to /var/log/metronome/auth.log (without password). If it does, the auth script is working as expected and your bug is the same I am working on at the moment.

And by the way, mod_auth_imap should also work with metronome as metronome is a prosody fork. Download it to /usr/lib/metronome/isp-modules and edit the generated host configs in /etc/metronome/hosts/ to use it. Afterwards run 'metronomectl restart' to restart with imap authentication while we fix the bug with the isp authentication.

 

Hi Michael,
I made this test some time ago and when I manually execute the .sh script, it works, and I get the 0/1 response, but with metronome I never pass the auth process.

Now that I'm running Prosody 0.10, I test the ispconfig script (only for testing purpose) with the auth external module and the script is executed by prosody, but prosody don't like the script response and I can't restart prosody anymore with command "service prosody restart" or "prosodyctl restart", I have to KILL the .sh process first.

Other thing to note (don't know if usefull for you):
With auth imap, when I execute "netstat -tupln | grep 52" I see for example:

Code:

tcp        0      0 0.0.0.0:5280            0.0.0.0:*               LISTEN      32945/lua5.1
tcp        0      0 0.0.0.0:5281            0.0.0.0:*               LISTEN      32945/lua5.1
tcp        0      0 0.0.0.0:5222            0.0.0.0:*               LISTEN      32945/lua5.1
tcp        0      0 0.0.0.0:5269            0.0.0.0:*               LISTEN      32945/lua5.1

but when I use the auth external and after the first login attemp, I see something like this:

Code:

tcp        0      0 0.0.0.0:5280            0.0.0.0:*               LISTEN      32945/bash
tcp        0      0 0.0.0.0:5281            0.0.0.0:*               LISTEN      32945/bash
tcp        0      0 0.0.0.0:5222            0.0.0.0:*               LISTEN      32945/bash
tcp        0      0 0.0.0.0:5269            0.0.0.0:*               LISTEN      32945/bash

 

Hi Guys,
I still have Metronome installed and I'm using it with "internal_hashed", I created user accounts and folders manually and it seems working fine, now I'm trying to get "stun service" working for audio & video, no luck.

Last night I enabled "external" authentication and added 1 xmpp user account (user1: mydomain.com) to ISP config, I tried to run the commands above as root I got (Command not found), from which folder to run the commands??
/opt/metronome # isuser:username:domain
/opt/metronome # auth:user1:mydomain.com:Mypassword

Cheers,

 

1- STUN/TURN:

Code:

apt-get install rfc5766-turn-server

Edit /etc/default/rfc5766-turn-server and uncomment:

Code:

TURNSERVER_ENABLED=1

Edit /etc/turnserver.cfg and uncomment/set:

Code:

lt-cred-mech
use-auth-secret
static-auth-secret     YoUrSecReTAuthStrinG
realm=turn.your.domain
cert=/path/to/cert.crt
pkey=/path/to/key.key

Restart:

Code:

service rfc5766-turn-server restart

Create DNS A record for turn host and open port 3478 and 5349 TCP/UDP

2- You must execute the authenticate_isp.sh script and then send the auth params

 

CubAfull,
Thank you, I will give it a try.
I was looking at the sample configuration from metronome, I thought I can just do the following:
Enable --> "extdisco"; -- External Service Discovery
Add -->

Code:

-- External Service Discovery (mod_extdisco)
external_services = {
    ["stun.mydomain.com"] = {
        [1] = {
            port = "3478",
            transport = "udp",
            type = "stun"
        },


        [2] = {
            port = "3478",
            transport = "tcp",
            type = "stun"
        }
    }
};

Then open port 3478, and create "A" record for stun.Mydomain.com

Cheers,

 

Oh sorry... I forgot to say that to make it work with Prosody I have to use the "turncredentials" module and set:

Code:

turncredentials_secret = "YoUrSecReTAuthStrinG";
turncredentials_host = "turn.your.domain";
turncredentials_port = "3478";
turncredentials_ttl = "3600";

This module set the "External Service Discovery" and allow only registered users to use the STUN/TURN server.

 

CubAfull,
Thanks for the info I'll give it a try.

Michael,
Here's my test results, I renamed the user & domain:

Code:

==================
root@server1:/usr/lib/metronome/isp-modules/mod_auth_external# ./authenticate_isp.sh
isuser:username:domain
0
auth:user1:mydomain.com:mypassword
1
^C
root@server1:/usr/lib/metronome/isp-modules/mod_auth_external#
========================
Auth LOG file :
Date: Tue Nov 15 09:48:29 PST 2016 Action: isuser User: username Host: domain
ISUSER FAILED
Date: Tue Nov 15 09:48:58 PST 2016 Action: auth User: user1 Host: mydomain.com
AUTH OK
=========================

 

Hi CubAfull,
I tried to install TURN server I got the following error message:

Code:

root@server1:/home/ccoudsi# apt-get install rfc5766-turn-server
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package rfc5766-turn-server is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'rfc5766-turn-server' has no installation candidate
root@server1:/home/ccoudsi#

Cheers,