hi, I'm interested in push up the feature request here: https://git.ispconfig.org/ispconfig/ispconfig3/-/issues/3875 "Limit number of mails sent by user per hour/day" posted by m4recek. How can I proceed? Stefano Sorry if there is a best place for this request, i serched the forum but I can't find a better place where to post it.
Hello Stefano, it is important to know if you are using rspamd or amavis as filter. In rspamd you can (relatively) easily configure that kind of "group" basis. Let's say "basic" 200/h, premium "5000/h", limited "10/h" etc. Of course without ISPConfig integration you'd have to edit the list files for each group by hand (simple text-file with user/domain in it)
Hello @stefano.b, I've been planning to implement this with postfwd eventually, so it probably will get done at some point. It certainly won't be in 3.2. If you want to sponsor getting to it sooner than later I'd be glad to discuss it, as well as your requirements (in particular the OS version).
Hi @Croydon, thank you for your reply and suggestion, the filter was Amavis, but now I set up Rspamd, I readed the documentation about the ratelimit of Rspamd. In ISPConfig words, it's possible to set up a server in which every Client and/or Web Domain has his limit? So if we have 2 clients with total of 3 web domains every web site can send its newsletter (monitored and controlled by Rspamd) without generate trouble for other domains? before i finish to read the documentation have you any hint about where I found this files? have a nice day S.
hi @Jesse Norell, yes I'd like to get it sooner than later, so let's me know. For my requirements I'm testing on a Test VPS, the OS version is: CentOS 7.4 x64 Linux web.xxx.com 3.10.0-1062.7.1.el7.x86_64 #1 SMP Mon Dec 2 17:33:29 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux ISPConfig Version: 3.1dev have a nice day S.
Sadly not. It is a custom setup that I use. I set it up like that: 1. Config in /etc/rspamd/local.d/ratelimit.conf Code: custom_keywords = "/etc/rspamd/my_ratelimit.lua"; 2. In that mentioned file: Code: function table_print (tt, indent, done) done = done or {} indent = indent or 0 if type(tt) == "table" then local sb = {} for key, value in pairs (tt) do table.insert(sb, string.rep (" ", indent)) -- indent it if type (value) == "table" and not done [value] then done [value] = true table.insert(sb, key .. " = {\n"); table.insert(sb, table_print (value, indent + 2, done)) table.insert(sb, string.rep (" ", indent)) -- indent it table.insert(sb, "}\n"); elseif "number" == type(key) then table.insert(sb, string.format("\"%s\"\n", tostring(value))) else table.insert(sb, string.format( "%s = \"%s\"\n", tostring (key), tostring(value))) end end return table.concat(sb) else return tt .. "\n" end end function to_string( tbl ) if "nil" == type( tbl ) then return tostring(nil) elseif "table" == type( tbl ) then return table_print(tbl) elseif "string" == type( tbl ) then return tbl else return tostring(tbl) end end local custom_keywords = {} local d = {} d['limit5k'] = rspamd_config:add_map({ ['url'] = '/etc/rspamd/ratelimit_5k.map', ['type'] = 'set', ['description'] = 'Incoming 5k limit' }) d['limit5k_out'] = rspamd_config:add_map({ ['url'] = '/etc/rspamd/ratelimit_5k_out.map', ['type'] = 'set', ['description'] = 'Outgoing 5k limit' }) d['limit1k'] = rspamd_config:add_map({ ['url'] = '/etc/rspamd/ratelimit_1k.map', ['type'] = 'set', ['description'] = 'Incoming 1k limit' }) d['limit1k_out'] = rspamd_config:add_map({ ['url'] = '/etc/rspamd/ratelimit_1k_out.map', ['type'] = 'set', ['description'] = 'Outgoing 1k limit' }) custom_keywords.customrl = function(task) local rspamd_logger = require "rspamd_logger" local mail_to = task:get_principal_recipient() local mail_from_tbl = task:get_from("mime") local mail_from = nil local from_domain = nil if mail_from_tbl and mail_from_tbl[1] and mail_from_tbl[1].addr then mail_from = mail_from_tbl[1].addr if mail_from_tbl[1].domain then from_domain = "@" .. mail_from_tbl[1].domain end else rspamd_logger.infox(rspamd_config, "Mail from is %s", to_string(mail_from_tbl)) end local mail_auth = task:get_user() local to_domain = string.match(mail_to, ".*(@.*)") local crl = "200 / 1h" rspamd_logger.infox(rspamd_config, "Mail (auth %s) from %s (%s) to %s (%s) limit check coming.", mail_auth, mail_from, from_domain, mail_to, to_domain) if mail_auth then if d['limit5k_out']:get_key(mail_auth) then crl = "5000 / 1h" rspamd_logger.warnx(rspamd_config, "User %s is in limit5k_out map. Limit is %s", mail_auth, crl) return "my_rlo_" .. mail_auth, crl elseif d['limit1k_out']:get_key(mail_auth) then crl = "1000 / 1h" rspamd_logger.warnx(rspamd_config, "User %s is in limit1k_out map. Limit is %s", mail_auth, crl) return "my_rlo_" .. mail_auth, crl end end if mail_from then if d['limit5k_out']:get_key(mail_from) then crl = "5000 / 1h" rspamd_logger.warnx(rspamd_config, "%s is in limit5k_out map. Limit is %s", mail_from, crl) return "my_rlo_" .. mail_from, crl elseif d['limit1k_out']:get_key(mail_from) then crl = "1000 / 1h" rspamd_logger.warnx(rspamd_config, "%s is in limit1k_out map. Limit is %s", mail_from, crl) return "my_rlo_" .. mail_from, crl end end if mail_to then if d['limit5k']:get_key(mail_to) then crl = "5000 / 1h" rspamd_logger.warnx(rspamd_config, "%s is in limit5k map. Limit is %s", mail_to, crl) return "my_rli_" .. mail_to, crl elseif d['limit1k']:get_key(mail_to) then crl = "1000 / 1h" rspamd_logger.warnx(rspamd_config, "%s is in limit1k map. Limit is %s", mail_to, crl) return "my_rli_" .. mail_to, crl end end if from_domain then if d['limit5k_out']:get_key(from_domain) then crl = "5000 / 1h" rspamd_logger.warnx(rspamd_config, "%s is in limit5k_out map. Limit is %s", from_domain, crl) return "my_rlo_" .. from_domain, crl elseif d['limit1k_out']:get_key(from_domain) then crl = "1000 / 1h" rspamd_logger.warnx(rspamd_config, "%s is in limit1k_out map. Limit is %s", from_domain, crl) return "my_rlo_" .. from_domain, crl end end if to_domain then if d['limit5k']:get_key(to_domain) then crl = "5000 / 1h" rspamd_logger.warnx(rspamd_config, "User %s is in limit5k map. Limit is %s", to_domain, crl) return "my_rli_" .. to_domain, crl elseif d['limit1k']:get_key(to_domain) then crl = "1000 / 1h" rspamd_logger.warnx(rspamd_config, "User %s is in limit1k map. Limit is %s", to_domain, crl) return "my_rli_" .. to_domain, crl end end rspamd_logger.warnx(rspamd_config, "Mail from %s to %s not found in maps. Limit is %s", mail_from, mail_to, crl) return crl end return custom_keywords I am no expert in LUA programming language, so there might be much better solutions. 3. As you can see in LUA file there are mentioned different "map" files, e. g. /etc/rspamd/ratelimit_5k_out.map Each of that files is a simple text file with content like: Code: @domain1.com @domain2.com [email protected] That's all. You could extend that to make it even more dynamic or "fine-grained".
A more dynamic approach would be to use key-value-maps instead of simple sets. So you would only need 2 maps (incoming/outgoing) and could insert domain or user to limit string in there. Have not tried that, yet.
For outbound (from your clients) mail I had in mind server-specific default limits for mail from a domain and from a mailbox, with ability to override that in domain/mailbox settings, does that meet your needs? Ie. I did not plan to implement per-client limits. Inbound would have some server-wide limits, not customizable (via gui) for individual sender ip addrs/domains. There would need to be some hook for custom postfwd config (either local file that is included, or maybe post raw postfwd conf into server config section; probably the former.
Hi @Jesse Norell, Yes, I think so. In any case this is the scenario: some of my clients use their CMS/software for sending their newsletters. I set up both the client software (for lower throughput) and the Control Panel Server for limit the number of mails they can send. I need a per domain (or mail) control/limit so that I can set up the ISPConfig for control (and maybe manage?) the sending process nomatter what the client software try to do. Please tell me if it Is this clear anought. Personally I'm not interessed in control the incoming mail, but it's ok this process too. have a nice day S. PS. I never asked for a customization to a GPL community software, so I don't know how procede, please tell me some extra data here o in private so I can understand if I can do it alone or I need help from some other users.
Hi @Croydon, If I understood the logic of this code, here there is not control, but "only" a writing log. Is it correct? have a nice day S.
ISPConfig is not GPL Software, it is BSD Licensed. The BSD license is a liberal OpenSource License, so you can extend it and either keep your changes for yourself or provide them to the community, that's up to you.
hi @till, thank you for your reply and the correction, what I meant was that "it is GPL compliant" as we can see here. In any case, we are talking about the possibility that the personalization is made by a ISConfig Develooper, and I (and some other users) can sponsor this features. For this procedure I asked some more info. have a nice day S.
The issue is that: BSD license is GPL compliant, but GPL is not BSD compliant, which means, if someone would choose a GPL compliant license that is not the BSD license or MIT license, then the code may not be combined with ISPConfigs existing code.
Hello @stefano.b, I've wrapped up my work for 3.2 now, if you want to discuss implementing postfwd I can send you a PM. I think making a contribution to the ISPConfig project is what you have in mind, not just local customization, but to be clear I would *much* prefer that route (it would be less work, and subsequently supported by the project, and I wouldn't have to re-implement it for the project later .
Hi @Jesse Norell, yes absolutely, I'm interested in implement this function for the ISPconfig's community Please send me a PM. have a nice day S.
I made some changes to the script from @Croydon. Code: mkdir -p /etc/rspamd/ratelimit touch /etc/rspamd/ratelimit/{limits_auth,limits_in,limits_out} And set the global values crl_out and crl_in (default is nil) Format of the auth-files: Code: SOURCE/DESTINATION:LIMIT_IN:LIMIT:OUT For a ratelimit different to the defaults: Limit outbound to 5 / minute to example.com: Code: @example.com::5/1m Limit inbound to 1 / minute from [email protected]: Code: [email protected]:1/1m: Limit in- and out for the domain example.com: Code: @example.com:1/1m:5/1m Add domain / addresses to the map-files limits_auth limit_auth_out.map and is used for authenticated senders only. Code: function table_print (tt, indent, done) done = done or {} indent = indent or 0 if type(tt) == "table" then local sb = {} for key, value in pairs (tt) do table.insert(sb, string.rep (" ", indent)) -- indent it if type (value) == "table" and not done [value] then done [value] = true table.insert(sb, key .. " = {\n"); table.insert(sb, table_print (value, indent + 2, done)) table.insert(sb, string.rep (" ", indent)) -- indent it table.insert(sb, "}\n"); elseif "number" == type(key) then table.insert(sb, string.format("\"%s\"\n", tostring(value))) else table.insert(sb, string.format("%s = \"%s\"\n", tostring (key), tostring(value))) end end return table.concat(sb) else return tt .. "\n" end end function to_string( tbl ) if "nil" == type( tbl ) then return tostring(nil) elseif "table" == type( tbl ) then return table_print(tbl) elseif "string" == type( tbl ) then return tbl else return tostring(tbl) end end function Split(s, delimiter) result = {}; for match in (s..delimiter):gmatch("(.-)"..delimiter) do table.insert(result, match); end return result; end local custom_keywords = {} local rl_auth = {} local rl_auth_limits = {} local rl_auth_out_limits = {} local rl_in = {} local rl_in_limits = {} -- auth user local cfg_file = io.open("/etc/rspamd/ratelimit/limits_auth", "r") for line in cfg_file:lines() do split_string = Split(line, ':') local limit_in = '' local limit_out = '' local temp_in = split_string[2]:gsub('%s+', '') -- trim string local temp_out = split_string[3]:gsub('%s+', '') -- trim string if temp_in ~= '' then limit_in = split_string[2] else limit_in = nil end if temp_out ~= '' then limit_out = split_string[3] else limit_out = nil end rl_auth_limits[split_string[1]] = { ['limit_in'] = limit_in, ['limit_out'] = limit_out } -- maps rl_auth[split_string[1]] = rspamd_config:add_map({ ['url'] = '/etc/rspamd/ratelimit/maps/limit_auth.map'; ['type'] = 'set', ['description'] = 'rl_auth for ' .. split_string[1] }) end io.close(cfg_file) -- auth user outbound local cfg_file = io.open("/etc/rspamd/ratelimit/limits_out", "r") for line in cfg_file:lines() do split_string = Split(line, ':') local limit_in = '' local limit_out = '' local temp_in = split_string[2]:gsub('%s+', '') -- trim string local temp_out = split_string[3]:gsub('%s+', '') -- trim string if temp_in ~= '' then limit_in = split_string[2] else limit_in = nil end if temp_out ~= '' then limit_out = split_string[3] else limit_out = nil end rl_auth_out_limits[split_string[1]] = { ['limit_in'] = limit_in, ['limit_out'] = limit_out } -- maps rl_auth[split_string[1]] = rspamd_config:add_map({ ['url'] = '/etc/rspamd/ratelimit/maps/limit_auth_out.map'; ['type'] = 'set', ['description'] = 'rl_auth for ' .. split_string[1] }) end io.close(cfg_file) -- inbound local cfg_file = io.open("/etc/rspamd/ratelimit/limits_in", "r") for line in cfg_file:lines() do split_string = Split(line, ':') local limit_in = '' local limit_out = '' local temp_in = split_string[2]:gsub('%s+', '') -- trim string local temp_out = split_string[3]:gsub('%s+', '') -- trim string if temp_in ~= '' then limit_in = split_string[2] else limit_in = nil end if temp_out ~= '' then limit_out = split_string[3] else limit_out = nil end rl_in_limits[split_string[1]] = { ['limit_in'] = limit_in, ['limit_out'] = limit_out } -- maps rl_in[split_string[1]] = rspamd_config:add_map({ ['url'] = '/etc/rspamd/ratelimit/maps/limit_in.map'; ['type'] = 'set', ['description'] = 'rl_in for ' .. split_string[1] }) end io.close(cfg_file) custom_keywords.customrl = function(task) local rspamd_logger = require "rspamd_logger" local mail_to = task:get_principal_recipient() local mail_from_tbl = task:get_from("mime") local mail_from = nil local from_domain = nil if mail_from_tbl and mail_from_tbl[1] and mail_from_tbl[1].addr then mail_from = mail_from_tbl[1].addr if mail_from_tbl[1].domain then from_domain = "@" .. mail_from_tbl[1].domain end else rspamd_logger.infox(rspamd_config, "Mail from is %s", to_string(mail_from_tbl)) end local mail_auth = task:get_user() local to_domain = string.match(mail_to, ".*(@.*)") -- default values crl_out = nil crl_in = nil if mail_auth then -- limit auth user local check = nil rspamd_logger.debugx(rspamd_config, "Mail (auth %s) from %s (%s) to %s (%s) limit check for out coming.", mail_auth, mail_from, from_domain, mail_to, to_domain) if rl_auth[mail_auth] then if rl_auth[mail_auth]:get_key(mail_auth) then check = rl_auth_limits[mail_auth].limit_out rspamd_logger.warnx(rspamd_config, "User %s is limited to %s", mail_auth, check) if check ~= nil then return "my_rlo_" .. mail_auth, check end end if mail_to then -- limit destination email if rl_auth[mail_to] then if rl_auth[mail_to]:get_key(mail_to) then check = rl_auth_out_limits[mail_to].limit_out rspamd_logger.warnx(rspamd_config, "Mail to (mail_to %s) is limited to %s", mail_to, check) if check ~= nil then return "my_rlo_" .. mail_to, check end end end end if to_domain then -- limit destination domain if rl_auth[to_domain] then if rl_auth[to_domain]:get_key(to_domain) then check = rl_auth_out_limits[to_domain].limit_out rspamd_logger.warnx(rspamd_config, "Mail to (to_domain %s) is limited to %s", to_domain, check) if check ~= nil then return "my_rlo_" .. to_domain, check end end end end rspamd_logger.debugx(rspamd_config, "User %s has no custom limit. Limited to default %s", mail_auth, crl_out) return crl_out end else -- limit incoming mails local check = nil rspamd_logger.debugx(rspamd_config, "Mail from %s (%s) to %s (%s) limit check for in coming.", mail_from, from_domain, mail_to, to_domain) if mail_from then if rl_in[mail_from] then if rl_in[mail_from]:get_key(mail_from) then check = rl_in_limits[mail_from].limit_in rspamd_logger.warnx(rspamd_config, "Mail from (mail_from %s) is limited to %s", mail_from, check) if check ~= nil then return "my_rlo_" .. to_domain, check end end end end if from_domain then if rl_in[from_domain] then if rl_in[from_domain]:get_key(from_domain) then check = rl_in_limits[from_domain].limit_in rspamd_logger.warnx(rspamd_config, "Mail from (from_domain %s) is limited to %s", from_domain, check) if check ~= nil then return "my_rlo_" .. to_domain, check end end end end rspamd_logger.debugx(rspamd_config, "Mail from %s (%s) to %s (%s) has no custom limit. Limited to default %s", mail_from, from_domain, mail_to, to_domain, crl_in) return crl_in end end return custom_keywords
