Abusive blocking from SpamRats

Discussion in 'Technical' started by JohnEdward, Jan 26, 2023.

  1. JohnEdward

    JohnEdward New Member

    Hi.(My apologies for my bad English. It is not my native language)
    I need your advice and your guidance, how to approach the situation and how to proceed further.
    I have two separate servers, with Debian 11 and ISPConfig 3.1
    Recently, a few days ago, a user sent an email to a new customer, and I instantly received a rejection email.
    That email was a reply, after the customer placed a commercial order on our website.

    "office@customerdomain. dot ro>: host mail.customer dot ro[77.81.xx.xx] said: 554 5.7.1 Service
    unavailable; Client host [11.22.33.44] blocked using spam.spamrats dot com;
    SPAMRATS IP Addresses See: www.spamrats dot com/bl?11.22.33.44 (in
    reply to RCPT TO command)"

    I checked the logs, checked the blacklists and found that everything is fine except for spamrats, which did not necessarily block our e-mail servers, but the entire class of IPs of our ISP.

    I had a discussion with our ISP and he told me, after the checks, that everything is clean. No attack, no abuse, no infection on that class of ips.

    I have checked my IP in SpamRATS and the result is the following:

    Standards Compliance:

    Does IP Address resolve to a reverse hostname... Passed!

    Does IP Address comply with reverse hostname naming convention...Passed!

    List Status:

    RATS-Dyna - On the list. Worst Offender Alert

    RATS-NoPtr - Not on the list

    RATS-Spam - Not on the list


    No delisting method works on the SpamRats website.I wrote to them on the contact page and they did not reply. I sent them an email from gmail to abuse@spamrats dot com and they didn't reply.

    Some reviews on the Internet about SpamRats look like this:
    "SpamRats is a very unreliable blacklist. They basically ignore everyone including ISPs, so there are always a good number of innocent sites listed by their system.

    Ignore or remove Spamrats and use the more reliable lists like Spamhaus, SpamCop, Truncate, UCEprotect."

    "SpamRats is a cheating "company" that is adding whole IP ranges and after than they try to sell you commercial services provided by their official companies ... if you write them an email with demand to stop illegal discreditation of the good name of your company or you will sue them they will remove it within an hour ... personal experience"

    OK until this step. I don't use SpamRats in SpamAssassin.

    In PostFix main.cf i have this line:
    smtpd_client_restrictions = check_client_access proxy:mysql:/etc/postfix/mysql-virtual_client.cf, permit_inet_interfaces, permit_mynetworks, permit_sasl_authenticated, check_client_access regexp:/etc/postfix/rbl_override, reject_rbl_client sbl.spamhaus org, reject_rbl_client xbl.spamhaus org, reject_rbl_client rbl.abuse ro, reject_rbl_client zen.spamhaus org, reject_rbl_client bl.spamcop net, reject_rbl_client psbl.surriel com, reject_rbl_client dnsbl-1.uceprotect net, reject_rbl_client cbl.abuseat org, reject_rbl_client dnsbl.justspam org, reject_rbl_client bl.mailspike net, reject_rbl_client spam.dnsbl.anonmails de, reject_rbl_client ix.dnsbl.manitu net, reject_rbl_client truncate.gbudb net, reject_rbl_client bl.blocklist. de, reject_rhsbl_client hostkarma.junkemailfilter com=127.0.0.2, reject_rhsbl_sender hostkarma.junkemailfilter com=127.0.0.2, reject_unauth_pipelining, permit

    Our mail servers meets all the technical conditions of operation, in terms of national and international regulations.
    Our email server, have DNS PTR registration, SpamAssassin filters, Mail Antivirus, SPF valid DNS registration, DKIM and dmarc record.
    We use our mail servers only for communication inside the organization and for communication with our clients on a 1:1 basis.
    We do not have mass-email campaigns, and we don't do either advertise by email.

    Even if I have the e-mail server and the dns server configured correctly, I can expect that the first mail sent to a new customer will end up in spam.
    Possibly..., even if it hasn't been the case so far.
    But in this situation it was like a trap. First attempt - first shot and that's it. We are in the blacklist.
    Why? What is the clear reason? Evidence of spam?
    We can ignore SpamRats, but what do we do about communicating with our future clients who use SpamTrap on their email servers?
    We believe that this will affect the smooth running of the business we own.
    We had e-mail services at Google for domain, Other domains, we had them at other hosting providers with Cpanel, but we did not have clear control over e-mail whitelist customers, and we were not receiving some e-mails from customers. That's why we chose to host our server ourselves.

    Since SpamRats services are hosted in Canada, we would like to file a complaint with the appropriate authorities in Canada:
    ised-isde.canada dot ca/site/competition-bureau-canada/en/restrictive-trade-practices

    I would still like to hear your opinion. There may be a simpler solution.
     
  2. pyte

    pyte Well-Known Member HowtoForge Supporter

    There most likly is a good reason why the subnet your servers IP belongs to got blacklisted. They don't do this just cause the feel like it.

    As your IP got listed on the RATS-dyna list you may want to read what they say about the removal from that list:
    Where are your servers located? Are these hosted servers in a datacenter or locally? And if they are locally, is this a normal ISP Internet Connection that connects them to the internet?
     
  3. JohnEdward

    JohnEdward New Member

    Do you know them well? Are you ready to give their actions a boost of confidence?
    I think they are abusing. They are not a regulatory or certification authority, but they pretend to be and want to cheating users.
    They have not presented a clear reason for this blocking and refuse to communicate.

    I already wrote when opening this thread:
    "..
    Does IP Address resolve to a reverse hostname... Passed!
    Does IP Address comply with reverse hostname naming convention...Passed!
    Our mail servers meets all the technical conditions of operation, in terms of national and international regulations.
    Our email server, have DNS PTR registration, SpamAssassin filters, Mail Antivirus, SPF valid DNS registration, DKIM and dmarc record."

    Our mail servers are hosted inside our company.
    I am responsible for two IP classes: one /29 and one /28, which the ISP provided us based on a business subscription.
    Is this a normal ISP Internet Connection?
    I am editing/correcting a question I wrote above:
    We can ignore SpamRats, but what do we do about communicating with our future clients who use SpamRats on their email servers?
     
  4. pyte

    pyte Well-Known Member HowtoForge Supporter

    They are one the scene for over 15 years and sponsored by some reputable organisations. I don't think they just "abuse" anyone just because they feel like, that's absurd.

    So checking the reverse DNS entries for the IPs results in the hostnames of your mailserver?

    Who owns the /29 and /28? Who is lsited in the Ripe DB your company or your ISP?
    Did you aquire these subnets just recently?
     
  5. JohnEdward

    JohnEdward New Member

    "Reputable" organizations are reputable, until proven otherwise.
    And anyone can apply a fake portfolio on the website page.
    They are not a regulatory or certification authority.
    Their actions, this labeling, cause us damages and losses in our businesses.

    So checking the reverse DNS entries for the our IPs, results in the hostnames of our mailserversȘ mail.ourdomain1.tld1 and mail.ourdomain2.tld2
    I don't own the those /29 and /28 ip classes. These are allocated by the ISP to us, to use them as we wish. So I am responsible for their use.
    Could you please stop trolling this thread? I would like to know everyone's opinion, especially the experts here, not just yours. You only have an opinion, I understand it, but I have my opinion.
    If we assume that spam is sent from our servers, why are we green/clean at all providers but not at SpamRats.
    It is SpamRats, more valid and famous than, Barracuda Reputation Block List or Blocklist.de or Spamcop or Spamhaus ?? I do not believe that!
    I have no evidence to prove that you are guilty, but I declare you guilty, and you have to provide evidence that you are not. But I don't allow you to contact me and defend yourself, unless you activate a paid subscription from our website.
    What is this called? Scam? I see it as a scam
     
  6. JohnEdward

    JohnEdward New Member

    Now it appears differently when checking on the SpamRats site:
    Standards Compliance
    Does IP Address resolve to a reverse hostname... Passed!

    Does IP Address comply with reverse hostname naming convention... Passed!

    List Status
    RATS-Dyna - Not on the list.

    RATS-NoPtr - Not on the list.

    RATS-Spam - On the list. Worst Offender Alert.

    RATS-Auth - Not on the list.
     
  7. pyte

    pyte Well-Known Member HowtoForge Supporter

    I was trying to help and understand your envoirenment better to may find the reason/a possability why spamrats marked this subnet. All BL services work like this and just because you are in 1 list doesn't mean you have to be on another... In most cases my servers are reported by ucprotect from time to time.
    And just because you are listed on one of these lists doesn't mean that all your valid mails get rejected instantly. No one in there sane mind would configured their spamfilter to reject mails because the sender is listed on one list.

    But yea good luck, i'm out. Your attitude ist bothering me.
     
    Last edited: Jan 26, 2023
    30uke likes this.
  8. JohnEdward

    JohnEdward New Member

    My email servers are not reported by uceprotect. They were never reported.
    You can check above which providers I use in SpamAssassin.
    But I had an email blocked in 18 August 2022 by mail.de, which asked for a paid subscription
    to include me in a "Tested Data Protection" whitelist, which included the famous Adobe company. blah blah..
    I refused to pay something like that. Somehow this association of SpamRats and Mail.de seems dubious.
    (It is on the official website of SpamRats in the portfolio)
     
  9. pyte

    pyte Well-Known Member HowtoForge Supporter

    spamrats blocks whole /24 subnets. They check the whole subnet for valid rdns records. Make sure your subnets have rDNS for all IPs and request a delist.

    May not a good practice but it is what it is. Like i said, i can't imagine anyone rejecting a mail based only on the fact that the ip is listed on 1 blacklist.
    You can't do anything about this than comply with their requirements and request a delist or change your IP.
     
  10. JohnEdward

    JohnEdward New Member

    I would like to know how many of you can access this registration link at mail.de (I tried from Vodafone, Orange and MyISP):
    https://registrierung.mail.de/de/

    Every time, from any network, I get the same message:
    "Netzwerk gesperrt!
    Zugriffe aus dem von Ihnen genutzten Netzwerk sind zur Zeit gesperrt.
    Da die Spamrate aus dem von Ihnen genutzten Netzwerk überdurchschnittlich hoch ist, wurde Ihr Zugriff auf unsere Seiten gesperrt. Wir bitten um Ihr Verständnis."

    "Network blocked!
    Access from the network you are using is currently blocked.

    Since the spam rate from the network you use is above average,
    your access to our site has been blocked. We ask for your understanding."

    It seems that something is wrong with the SpamRats they are affiliated with. This is how this blacklist is explained.
     
  11. till

    till Super Moderator Staff Member ISPConfig Developer

    Just tested it, works fine from here. But I connected from a german network provider, so maybe this makes the difference.
     
  12. JohnEdward

    JohnEdward New Member

    As a German mail provider, it is your right to offer your services only to German citizens.
    But on their website, they do not specify this, but more than that, they formulate an abusive and baseless accusation:
    "Since the spam rate from the network you use is above average,
    your access to our site has been blocked. We ask for your understanding."

    What spam rate? When? How? Can you prove? Where can it be checked?
    And again: Are you an authority for certification and regulation of internet services? In Europe, IP class abuses are reported to RIPE.
    How dare you think you are RIPE?

    I personally use fail2ban, and I report attacks from various IPs to blocklist.de and AbuseIPDB, with clear logs: date, time and reason for blocking.
    The user of the accused IP can check the reason for the act.

    This is not the way to go. It is a clear abuse when you block classes of ips or countries, presumptively,
    assuming the reason that you will probably do this or that, but you haven't done it yet and even more so that you probably won't.

    By the way, I am ethnically German, although my grandparents and parents were not part of a larger community, that's why I didn't study/learn German.
    I specify this so that it is not understood that I have something against the Germans.
     
  13. MaxT

    MaxT Member HowtoForge Supporter

    time ago I was forced to delete the SpamRats RBL although not because their fault. Some legitimate senders in my country are using a very big host company who allow websites with marketing commercial activity to be mixed with the common customers into the same network segments. I wonder if this could be your case.

    your problem is a bad thing. Although the critique is difficult, because in example I do the same by my own with many marketing companies like mailchimp which frequently are not covered by the RBL lists (strange? o_O ). Periodically I run a routine to detect IPs from marketing messages and I block the whole network segment belonging to one ip. I have an accumulated list with near 1400 cdir segments blocked in a preventive way, and no problem to receive legitimate messages.

    Maybe you can consider searching for another host company. Many offer some days to return the money. In that time you can build an easy RBL routine to know if your new IP can have blacklisted neighbors in the same segment.
     
    Gwyneth Llewelyn and JohnEdward like this.
  14. JohnEdward

    JohnEdward New Member

    I also use lists with malicious/attackers IPs, which I import and insert into the firewall. Nothing wrong with this.
    The problem, in essence, is with the rbl provider, which does not allow communication with it,
    which does not allow the right to add small exceptions.
    Do the innocent customers who want to use their blacklists know that the lists are abusively exaggerated?

    We host our own servers and services under our own banner, inside our company.
    We do not use third party hosting companies.
    We had, we tried and they did not satisfy our requirements. More precisely at the email services/server.
    We chose to build/host/manage our own servers with Debian 11 and ISPConfig. Everything has been almost perfect so far.

    The small pool of /28 IPs that was allocated to us for use by the ISP is part of a pool of /23.
    If we divide that range of IPs /23 into two /24,
    The first pool /24 contains IPs with problems at rbl providers.
    The second pool /24, which also includes our subnet, is clean. I did not find any ip with problems in a BulkBlackList check.
    So there is a pool/ip range eligible for a whitelist.
    When we started using the server with ISPConfig, in July 2022, we had a presumptive blacklist at Microsoft Office 365 AntiSpam.
    We opened a dispute, we communicated with them, and they inserted our IPs into the whitelist.

    I think that, as an RBL provider, in this case SpamRats, if you sell services for money, you ask for that money for the RBL filters, for their use.
    I don't think it's ethical to ask for money as a ransom/redemption.
    Especially if you are not guilty of anything. I see this as a method of being allowed to spam for a fee.

    There is a clean network segment that we belong to, a network segment that can be entered as an exception in the white list(That pool /24).
    Only Goodwill and communication with the RBL providers should be. But that is missing in this case.
     
  15. I know I'm piping in late, but I just wanted to give @JohnEdward a hug and tell him that I know perfectly well how he feels!

    SpamRATS are not the only annoying RBL provider out there. In my continuous task of getting mail to flow to and from my single-server-only-for-non-profits-and-friends-micro-hosting-provider, I've been on a long journey, going from cheap VPS providers to cheap bare metal providers, first in the US, later in Europe. Why cheap? Well, I don't get paid, so I have to get the best machine I can afford for the lowest possible price. It's not easy — but doable — and I love challenges!

    The problem is just that cheap providers are, well... cheap for everybody, most notoriously for all sorts of malicious agents, spammers, scammers, script kiddies, wannabe hackers, botnet operators, Tor node operators, BitTorrent operators, IPFS node operators, and cryptocurrency miners of all sizes and descriptions... not to mention ****, "dating" services, Ponzi schemes, and everything out there which is not quite legal but also not blatantly illegal.

    Well, and the odd legitimate service here and there.

    As such, getting banned or listed on a RBL is very easy.

    I used to host with the giant French operator OVH (they claim to be the largest in Europe). Because they're so big, they have tons of surplus hardware, which they rent for cheaper and cheaper prices, until eventually the machine finally dies. But before that happens, you'll get a rather good price for what used to be a high-performance server half a decade ago — more than enough for most purposes!

    Because everybody knows that, OVH is crammed full with the semi-legal, quasi-legal, and probably-not-legal hosting providers out there. And, as a consequence, most get banned on all sorts of lists — RBL, AbuseIPDB, you name it, they'll be placed there in no time.

    When SpamRATS got an email originating from one of my users who was technically "spamming" (they were unaware of it; their own computer had been infected with viruses before their anti-virus scanner figured it out), they saw that my reverse IP address was actually part of the OVH "scamnet" and just blocked it. One offense was enough.

    But they were not the only ones. A certain popular Swiss-based RBL went even further: they pre-blocked entire networks, including OVH among many others. Their reasoning was the following: if they got a series of spam messages from a single IP address, that address would be placed on a list. If the spam messages come from several systems in a /24 block, well, then the whole /24 block gets in the list. And so forth. When an operator has several "independent" networks, they block them all. If the operator has a whole autonomous system (ASN), well, then it's the whole ASN (!) that gets blocked. And giant operators such as OVH, who have several autonomous systems, get everything blocked — each and every one of their servers.

    Now, OVH — as an example! — as said, operates servers at different tiers. At the top tier their prices are exactly the same as other regular co-location providers — i.e. expensive :) Nevertheless, these also get blocked — it matters little if some of those servers are hosting, say, national French newspapers or even websites for a local municipality. All get blocked.

    They do have a form to unblock your IP address. But that works just up to a point. If you are part of a huge section of the Internet that they deem to be "too full of spammers", then they require you to pay for the privilege of them opening up a "hole" in the long list of IP addresses — an exception, that is. That still doesn't "guarantee" delivery — you might get black-listed elsewhere, of course — and they allow your IP address to be removed exactly once. On the second email spam received from your IP address, even if you're paying for the privilege of being on their "special whitelist", your account will be canceled (NO refunds!), and your address will go into the special blacklist reserved for the Worst Possible Offenders (like SpamRATS). And they will also make sure that they share that information with all other RBL operators (it will be up for those to do the same as well, of cours).

    Now, suppose that, like myself, you are a legitimate user of OVH. You don't really control the IP address you've got, but since you're on OVH's fully-blacklisted-by-default list, you have no other choice but to pay to get an exception for your address. But they might refuse it — on the basis that this IP address had already been flagged as a source of spam in the past, and obviously they have no way to know if you're the same person who managed the server which used to have that IP address or not. They err on the side of caution — since you're on OVH anyway, it's likely that if you're not the same spammer, you'll just be a new spammer, so you'll get pre-blocked by default — "Guilty by Association", so to speak.

    They actually have the nerve to have a notice saying (I'm paraphrasing, I haven't visited their site recently) that if you're really keen to remain on a "cheap hosting provider with a bad reputation" it's all your fault. Just move to a more "decent" provider. If you're serious about your business, you should better not be on OVH anyway. But if you do not move away from OVH, then you have your own reasons for that choice, and it's up to them to pre-label you as a "potential spammer" and simply refuse to accept your money to open an exception in their list.

    Neat, right? The worse is that they're not the only ones with that policy. Many do their actual business by charging fees to legitimate operators to get removed from their blacklists! The problem is that there are so many RBLs out there that it starts to become very expensive to pay everybody to get out of those lists... and neither guarantees anything.

    (Note: I moved since then to Hetzner in Germany. To no avail: few weeks later, they blocked my new IP address at Hetzner as well. I guess that Hetzner's reputation is as bad as OVH's. No wonder — their prices are roughly the same, but Hetzner has far better machines for the same monthly fee. Too bad!)

    Case #2. You are connected via fibre to a reputable ISP, never having any issue with them whatsoever. By sheer coincidence, you also have a server running on their data centres, where they have special packages for existing customers. And one day, out of nothing, your address gets blocked on a blacklist as well.

    Your server is "clean" — you check and double-check everything, look up the logs, and nothing — no spam entered, much less left your system. So, you get in touch with the RBL who blacklisted your IP address. And they will just shrug it away, saying that all addresses from your ISP were blocked. You ask them why, and even bother to explain that this is not really a low-end, cheap service provider. They couldn't care less. They just noticed that a huge number of Windows computers running at people's homes and connecting to fibre were infected by a new variant of a botnet, and had been spamming half the world for a while (until the ISP put a stop to it — but then it was already too late, the damage had been done). Therefore, they decided to block all addresses that are even remotely connected through that ISP: home users, business networks, commercial hosting providers who co-locate at their data centres... everybody. No exceptions.

    And, indeed, not shortly afterwards, you start getting the weirdest error messages on your personal computer. Cloudflare starts verifying your connections to half the websites you visit. Netcraft, and then the viewer itself, start giving you all sorts of error messages. Popular websites where you had been registered now refuse to log you in because your connection comes from a "unreputable source".

    Granted... at some point, it will be the ISP itself making their voice heard and demanding that these guys stop fooling around with their customers, or face them in court. They assure them that whoever had been infected with the botnet virus is now "clean" again, because they made sure this happened. Etc. etc. blah blah — in short, they pulled their weight, and eventually (reluctantly!) the RBL maintainers will release the whole block from the blacklist (at least... for a while).

    In late 2021/2022, during the pandemic, this went so far that Google themselves were being blacklisted by many RBLs. I just noticed if because my users were saying that "our mail server is refusing connections from Google, saying they're spam!". And they were right! Postfix & Rspamd were doing their job exactly as configured, checking Google's outgoing email servers for their IP addresses, and finding them in some blacklists. Oh, most were just temporarily listed; but that was enough to bounce a lot of messages.

    The result? Why, naturally, I was blamed for "having some kind of misconfiguration" because only the mailboxes in my server were affected. Those who used Gmail accounts between themselves never saw any messages flagged as "spam"; so, obviously, it must have been my server which was "wrong". Some even abandoned my (free) services and decided to pay for a Google Workspaces account — because they (correctly) concluded that "Gmail mailboxes are never affected by messages flagged as spam sent between each other, so, since all my business contacts use Gmail, I have no other choice to do the same".

    Well, that explains why Google keeps adding more and more Gmail mailboxes :)

    To be fair to Google (as far as they deserve it), they didn't get blacklisted because of Google's lack of oversight, or, worse, as a deliberate evil plot to make email communications with independent mailbox hosting providers next-to-impossible in order to get more accounts. In fact, it was rare than more than one or two of their outgoing mail servers were temporarily blocked, for a day or two. Although obviously Google will not say so, it's likely that some spammer had figured out a new way of using Gmail to send out much more spam than usually, and, as a consequence, until Google's automatic spam-blocking tools kicked in, enough spams were delivered from two or three of their outgoing servers... enough to trigger the blacklisting by some trigger-happy RBL providers. Google fixes those cases very quickly in most cases, but not quickly enough to get unlisted from a RBL (or several ones, who shared blacklisting data among themselves) — therefore, bouncing back all emails having a Gmail box as a recipient, before they even left my mail server.

    This has consequences. In my case, for instance, during these outbursts, I got some emails from the WordPress teams — which bounced back. I never received them, so I didn't acknowledge whatever was being requested in those messages. And, because of that, my WordPress plugins got removed from their public library, because one of the conditions of having the plugins hosted there is "to be available via email 24h/7" — no bouncing allowed!

    To try to avoid that, I had to stop relying on so many RBLs, just a selected few, and explicitly added Google, Facebook, WordPress and a few others on my server's own whitelists. Thus, even if one of the RBLs I used did flag one of them as "source of spam", I would silently ignore it and deliver locally anyway.

    And, finally, there was the last straw which made me dump all RBLs except three.

    As you're probably aware, almost all these RBL providers have a "free tier" (for small-scale mail providers) and a "paid tier". The difference is basically the limit on the number of DNS requests you can make per unit of time (some do the metering per hour, some per 24h-day, some per month...). This makes sense — the whole purpose of having RBLs is to cover as many mail providers as possible (big and small!), and that means that the "small fry", who cannot afford to pay hefty fees, should get at least minimum, basic protection, both from getting spammed as well as being (involuntarily) the source of spam.

    Now, a recent trend worldwide has been a rethinking of how the whole of DNS works. In the earliest days, only "wealthy" providers would run their own DNS servers; smaller ones would just use a caching nameserver, and forward all requests to the "big" domain name servers out there. Then the trend started to change, and even the small providers started to run their own authoritative DNS servers (thus the reason why ISPConfig comes with so many DNS tools!), just because they could. It was also a simple way for smaller providers to avoid using the often very slow and overburdened upstream DNS servers from their co-location facilities.

    But DNS grew — in all possible ways — and getting access to superfast DNS became imperative for everybody, consumers and small providers alike. As the large providers' DNS servers became worse and worse — or unreliable — and one's own DNS server was starting to crumble under the demands (if not the bandwidth!), people started replacing their DNS servers with a new generation of cloud-based services, such as 8.8.8.8 from Google (or, if you're paranoid about what Google does with the requests they receive from your system, you can always use 1.1.1.1 from Cloudflare or 9.9.9.9 from Quad-9). This was a blessing for many — well, it still is! — but created a problem for RBL providers: how could they limit the "free tier" that comes from the small providers, since now everybody apparently seems to make requests from Google, Cloudflare or Quad-9?

    There are a lot of different ways to deal with this issue, but most RBL maintainers just implemented the easiest method: just forbid requests coming in from these public DNS servers. Simple.

    In fact, while requests coming from the major public DNS services might be pre-excluded, they're not the only ones. Big providers — such as OVH or Hetzner, but any sufficiently large organisation will be in the same category — will also aggregate all requests from all their customers, funneling them to the RBL servers as if they come from a single DNS server. One that clearly makes far too many requests to be put into the "free tier". In some very rare cases, the ISP itself might be willing to pay to the RBL operators to be able to handle DNS requests on behalf of their customers; but that will be rare, since they already give them at least three options — use their cusomers-only DNS servers (which will be rate-limited or blocked very quickly), use one of the popular Big Public DNS providers (Google, Cloudflare, Quad-9... and several others), or, well, run your own DNS server instead.

    DNS is a critical element in one's Internet connectivity. You can afford to have mail down for a short period — it will be queued and delivered with a delay, but delivered nonetheless. You might not be able to keep the Web down for long — unless you have a CDN or some sort of front-end server to cache your users' content on their behalf; then, you can also rely on those to provide at least some measure of service connectivity even during a more extended period of time.

    But you cannot connect to the wide Internet without a fully operational DNS. And the faster you can guarantee that your DNS replies to requests, the higher the quality of your overall connection. This is unavoidable, due to the way the Internet works. Even email — aye, it's true that you can merely enqueue email for later delivery, but if you wish to scan those emails for legitimacy, you will have to have a fully-working DNS. And that effectively means running your own DNS server.

    Or, well, dropping out of those nifty free RBL providers...

    Now, I have personally no issue in using ISPConfig3's own DNS management tools in order to run everything from the ISPConfig3 admin panel; but I'm also quite aware that I simply cannot "compete" in providing quality DNS service if I run my own, publicly-exposed DNS server. I'm also no stranger to debugging complex DNS misconfigurations; but why should I bother wasting my time doing that, if there are whole teams out there doing this full-time, and allowing you to connect through their DNS servers for free? Is it worth to drop the ease of configuration of Google's or Cloudflare's public DNS service — not to mention their blindingly fast DNS query resolution, which happens on the edge of their network, thus replying to local queries as fast as light permits? You cannot even come close to "compete" with that (even assuming you would want to do it!)

    When it came to choosing between continuing to have a super-powerful DNS server making all requests (and replies!) on behalf of my Internet-facing services, or dropping whole RBL lists altogether — because they won't provide any free service through the public DNS servers that I use — well, my choice was clear: it meant relying more on Rspamd's automatic abilities to figure out what is spam and what isn't, and rely less on external lists, compiled by humans, to decide which emails to keep, and which to drop...
     
    Taleman likes this.
  16. What we need is to establish a network of independent trusted providers to exchange such Realtime Blackhole Lists among ourselves. A bit like what AbuseIDPB does.

    I also like the reverse concept: a repository of whitelisted information, provided by https://www.dnswl.org/, entirely run by volunteers. Most entries there were obviously created using their self-service system (i.e. you basically whitelist yourself and commit to abide by their standards as well). In order to discourage spammers to whitelist themselves, they take a long time in approving new entries. How long? Well, that depends: the less email traffic you have, the longer it takes, because what they will do first is to generate some statistics about how much spam is sent (or seen as coming from) your email servers, compared to legitimate "ham" (i.e. not spam). I suppose such statistics also include things like monitoring your servers to see how often they are caught by any blacklist, and how long they spend being in such blacklists. Then the gathered information is reviewed by a human, which may or may not approve. They claim that it might take "several weeks" (thus making this service not interesting to those spammers who register a domain, do highly intense spamming for a few hours, drop the domain, and start again later with a fresh domain, from a different location). In my case it took "several months" and I'm still not quite sure if I'm really being considered to be whitelisted, or if I'm just on the "internal list of those who await approval from a human" :)

    All such systems, of course, are prone to gaming, abuse, and corruption/bribery. But you can narrow that down a bit, using a free-to-use API which requires previous registration to use (as opposed to free-for-all DNS queries... faster, but way too public). Perhaps with some web-of-trust thing thrown in, such as using OpenPGP to exchange keys, trusting only the reports from some but not all participants. Or even going as far as using Keybase to show proof of existence across several different social networks, websites, and services. You can make things slightly less prone to abuse and not overly cumbersome; the key here is to use the information from others merely as advisory, with a "credibility rating" that depends on how much you trust the other parties to send you correct information.
     
    MaxT likes this.

Share This Page