GEEZ Yet ANOTHER strange problem!!!

this morning my server seemed working fine.
then at about 830am I noticed - one of my domains was missing!
I could ping everyone else EXCEPT cdbsystems.com
now - cdbsystems.com is important - ns10.cdbsystems.com is my main name server!
so... putting on my detective hat (and worried Till might be busy these holidays!)
I systemctl restart named
no help. same problem. I noticed in the systemctl status named:

Code:

 named.service - Berkeley Internet Name Domain (DNS)
   Loaded: loaded (/usr/lib/systemd/system/named.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2020-12-28 08:40:05 EST; 2h 11min ago
  Process: 1684058 ExecStop=/bin/sh -c /usr/sbin/rndc stop > /dev/null 2>&1 || /bin/kill -TERM $MAINPID (code=exited, status=0/SUCCESS)
  Process: 1587533 ExecReload=/bin/sh -c if /usr/sbin/rndc null > /dev/null 2>&1; then /usr/sbin/rndc reload; else /bin/kill -HUP $MAINPID; fi (code=exited, status=0/SUCCESS)
  Process: 1684076 ExecStart=/usr/sbin/named -u named -c ${NAMEDCONF} $OPTIONS (code=exited, status=0/SUCCESS)
  Process: 1684073 ExecStartPre=/bin/bash -c if [ ! "$DISABLE_ZONE_CHECKING" == "yes" ]; then /usr/sbin/named-checkconf -z "$NAMEDCONF"; else echo "Checking of zone files is disabled"; fi (code=exited, s> Main PID: 1684077 (named)
    Tasks: 67 (limit: 1648793)
   Memory: 162.3M
   CGroup: /system.slice/named.service
           └─1684077 /usr/sbin/named -u named -c /etc/named.conf

Dec 28 10:51:10 ns10.cdbsystems.com named[1684077]: client @0x7f0f5800c140 172.68.53.174#29454 (ns1.cdbsystems.com): query (cache) 'ns1.cdbsystems.com/AAAA/IN' denied
Dec 28 10:51:22 ns10.cdbsystems.com named[1684077]: client @0x7f102c8a21e0 107.87.146.38#36388 (ns10.cdbsystems.com): query (cache) 'ns10.cdbsystems.com/A/IN' denied
Dec 28 10:51:22 ns10.cdbsystems.com named[1684077]: client @0x7f102ca887b0 107.87.146.38#36388 (ns10.cdbsystems.com): query (cache) 'ns10.cdbsystems.com/AAAA/IN' denied
Dec 28 10:51:22 ns10.cdbsystems.com named[1684077]: client @0x7f102cafe630 107.87.146.38#36388 (ns10.cdbsystems.com): query (cache) 'ns10.cdbsystems.com/AAAA/IN' denied
Dec 28 10:51:22 ns10.cdbsystems.com named[1684077]: client @0x7f102c81d790 107.87.146.38#36388 (ns4.cdbsystems.com): query (cache) 'ns4.cdbsystems.com/AAAA/IN' denied
Dec 28 10:51:22 ns10.cdbsystems.com named[1684077]: client @0x7f102cb91c50 107.87.146.40#27279 (ns10.cdbsystems.com): query (cache) 'ns10.cdbsystems.com/A/IN' denied
Dec 28 10:51:22 ns10.cdbsystems.com named[1684077]: client @0x7f102ca887b0 107.87.146.40#27279 (ns10.cdbsystems.com): query (cache) 'ns10.cdbsystems.com/AAAA/IN' denied
Dec 28 10:51:22 ns10.cdbsystems.com named[1684077]: client @0x7f102c9f5190 107.87.146.40#27279 (ns4.cdbsystems.com): query (cache) 'ns4.cdbsystems.com/AAAA/IN' denied
Dec 28 10:51:22 ns10.cdbsystems.com named[1684077]: client @0x7f102c81d790 107.87.146.40#27279 (ns10.cdbsystems.com): query (cache) 'ns10.cdbsystems.com/AAAA/IN' denied
Dec 28 10:51:33 ns10.cdbsystems.com named[1684077]: client @0x7f102cae0e90 184.188.68.11#58736 (ns9.cdbsystems.com): query (cache) 'ns9.cdbsystems.com/AAAA/IN' denied

so - looked up and found maybe it was due to recursion being denied.
I edited named.conf and changed allow-recursion {any;} (or whatever typing!)>
still could not access domain but now systemctl status named returned:

ouch! now I was poking around in etc/named and I pulled up named.conf.local and I was blown away!
there was NO entry for cdbsystems.com!!!!!
I added manually:
zone "cdbsystems.com" {
type master;
allow-transfer {35.169.39.140;216.117.186.93;};
also-notify {35.169.39.140;};
file "/var/named/pri.cdbsystems.com";
};

systemctl restart named
and now cdbsystems was operational.
now this merits a big WTF?????
how could the zone entry have been deleted between 6am and 830 or so? what could have done it??
and of course I'm VERY concerned - will it happen again??
sounds like an ispconfig problem - somewhere.
I checked last and no extra root accesses to server, and what hacker would delete ONE zone and leave the rest???
what can I do to troubleshoot? I'm obviously concerned this might happen -- oh again as I'm typing???
o Till, I truly need you now!
cdb.

 

i look at the datalog records and nothing says 'lets delete the cdbsystems.com entry from named.conf.local'
what am I missing?
and Th0m your help most welcome.

also do I need a reverse zone somewhere? used to have that back in the old days.

and not sure I understand Th0m. I never DELETED the cdbsystems.com zone. it was in the named.conf.local file at 630am this morning and gone at 8am. cdbsystems.com has ALWAYS been in my DNS lists under Ispconfig. I didnt add it - its been there since day 1! its only the entry in named.conf.local that got deleted.
by whom? and why? inquiring minds and all that.
happy german holidays!

 

the only data log entries are:
2020-12-29 03:30 ns10.cdbsystems.com Update dns_soa
2020-12-28 11:14 ns10.cdbsystems.com Update dns_soa
2020-12-28 03:30 ns10.cdbsystems.com Update dns_soa
2020-12-28 03:30 ns10.cdbsystems.com Update dns_soa
2020-12-28 03:30 ns10.cdbsystems.com Update dns_soa
2020-12-28 03:30 ns10.cdbsystems.com Update dns_soa
2020-12-28 03:30 ns10.cdbsystems.com Update dns_soa

now I've been updating various of my dns entries because the zone settings say ns9 is the nameserver (which no longer exists).
I've been slowly changing ns9 to ns10 and saving the entry.
doing anything wrong? and surely nothing to cause cdbsystems.com (but nothing else) to be removed from named.conf.local!

also doing digs I'm getting ns10.cdbsystems.com as a non-authoritative server. surely it should be authoritative?