switch dns replication method

I already have two dedicated dns servers running, ns1 and ns2 where ns2 is configured in ispconfig as a mirror of ns1.

i would like to change this to remove the mirroring and use the existing ns2 server as a secondary dns (slave) server so that I can start using DNSSEC.

i would like to do this without interrupting the dns service on ns2 or adding (even temporarily) a 3rd dns server. is this possible?


my thoughts on this are currently:
update xfer and also_notify in the templates.
edit dbispconfig master database, add the ip of ns2 to xfer and also_notify fields of the dns_soa table.
resync dns service on ns1
remove 'is mirror of server ns1' from system -> server services -> ns2
(hope this doesn't remove pri.<domain> zones from ns2, or stop bind from answering requests to this nameserver. )
create secondary dns zones in ispconfig (probably gui. can i add it directly in mysql copying relevant fields from dns_soa? or use api? i can see adding a zone using dns_zone_add, but that appears to be just for the primary zone, i don't see any api stuff for secondary zones)
resync dns service on ns2.


am i missing anything?

 

You can't do this without disrupting ns2.
Mainly because you can't have a master (through the mirror) and a slave zone for the same domain together.

I hope you haven't got a lot of zones.
Disruption for just a short amount of time shouldn't be that big of an issue.
Stop the dns service on ns2 and everyone will use only ns1 to resolve.
Remove the mirror in ISPC, delete the master zones on ns2 and add slave zones as quickly as posible.

 

damn..
was hoping that breaking the mirror without resyncing dns on ns2 would still leave ns2 fully working..
which should be able to host both primary and secondary zone files, just not both zones of the same domain..
and that then creating a new secondary zone for a domain in ispconfig would then both remove the primary zone file on ns2 and create the slave zone file on ns2.. thus avoiding having both primary and slave zones of a domain on the same bind server simultaneously and thereby work without interruption.

there's not too many domains, but still enough that it'll be tedious and time consuming to do manually.

i guess the other option then is can i just:
add the ns2 ip's to the xfer and also_notify fields in dns_soa on the master
break the mirroring
resync the dns on both ns1 and ns2 (i'm assuming this will automatically remove all the pri zones on ns2)
create all the secondary zones in dns_slave directly in mysql on the master
resync the dns on ns2

as far as i can see, creating the secondary zones directly in mysql should be ok, dns_slave doesn't appear to be reliant on other tables, so as long as i match up sys_userid, sys_groupid, and origin with the entries in dns_soa it shouldn't create any problems
.@till can you confirm it's safe to create secondary zones this way?
if so, that should greatly minimize any interruption on ns2.

 

You should be able to break the mirror and change master zones to slave zones one by one.
There will however be outage on ns2 for a zone when you switch from master to slave on that server.

And there is a big 'but'!
Master zones are updated every once in a while automatically.
When you have broken the mirror the master zones on ns 1 and ns2 will become out of sync after a while.
As long as records aren't updated/changed within ISPC this should not be a problem, but (online) dns checks will almost certainly result in errors. At least out of sync serials but probably more.

 

i've got a mysql query i can run..

worked it out on a standalone db dump to be safe.
* assuming all dns zones in ispconfig are to be switched from ispconfig mirrored to bind master/slave

Code:

insert into dns_slave (sys_userid, sys_groupid, sys_perm_user, sys_perm_group, origin, server_id, active, xfer) select A.sys_userid, A.sys_groupid, A.sys_perm_user, A.sys_perm_group, A.origin, B.server_id, A.active, A.xfer from dns_soa A join server B where B.server_name = '<fqdn of secondary dns server>';

then

Code:

update dns_slave set ns = '<comma separated list of ip's of primary dns server>';

might be able to get it all into one query.. but can't be bothered, will take longer to work out than it would to just run these.

break mirroring, run mysql queries, resync dns on secondary..
should be quick....

 

since i have to do this sometime this year as well i am wondering: Are there any additional steps beyond the ones you have shown? Does this work in a timely manner with 1000+ zones?

 

i don't know yet.. not actually done it yet..

i think it'll work fine, but still worried about it going wrong.. so i'm going to try it at a quiet, out-of-hours time.. probably early sunday morning.... (and backup dbispconfig on the master and both ns servers first.. and probably snapshot all 3 vm's first as well.. not that i'm paranoid or anything... )

will update here when it's done...

 

Don't worry i would do the same thing

Thank you!

 

ok.. that doesn't work..
will explain everything done and what happened a little later.. just gonna go have a big unhealthy fried breakfast to restore my mood.

 

yep.. did that first, put all 3 ip's of the secondary dns server there. internal ip4, public ip4, and public ip6.
also put all 3 there for also_notify.

*related point. in the sql query above, leave out the xfer bit otherwise it also copies all those to the dns_slave table and then the xfer field needs to be cleared again in there.

to be fair.. dns resolution continued working properly on both dns servers throughout everything. so at least it seems very difficult to break the dns service.

i set xfer and also_notify on all dns_soa records, broke mirroring. run mysql queries to create dns_slave entries for all zones. cleared xfer fields on dns_slave records, and then tried resyncing dns on ns1 first.
resync completed, everything seemed fine.. so i tried going to resync dns records on secondary, it wasn't listed as a server option.
tried resync 'all services' on ns2 instead.. that completed, seemed to only do client/seller records..
tried ispconfig_update.sh --force on ns2, completed fine.
still no option to sync 'dns records' on ns2, i'm assuming this is normal for secondary (non-mirrored) dns.
resynced all services on ns2 again instead..
still only done client/reseller records.
/etc/bind/slave directory is empty.

deleted one of the secondary zones in gui, and recreated it through the gui.
it created the slave entry in /etc/bind/named.conf.local, but it still also had the primary zone for the domain configured in there as well.
/etc/bind/slave was still empty.
i tried created a new TXT record in the dns zone, see if an update to the zone on the primary would force it's sync.
record was added to primary zone on ns1 fine, /etc/bind/slave on secondary was still empty.

at this point, i decided to go back to how everything was before and give myself more time to think this through, so i restored the dbispconfig database on the master, ns1, and ns2 servers and resynced records on ns1 again, checking all primary zones on both ns1 and ns2 were fully recreated from the db data.

 

yes, i did disable mirroring under server services for ns2.
dns was definitely active as a service on that server before i disabled mirroring.
i believe it was still active as a service afterwards but can't remember definitively now, i know i didn't manually uncheck the service setting.

can't do anything on the servers at the moment, will try again later today.

 

well, i've tried breaking the mirroring again.
ns2 is still has dns enabled in system - server services
it definitely doesn't show up as an option under tools - resync - dns records.

also, not sure if they were out-of-line before, but after restoring dbispconfig on master, ns1 and ns2, the update numbers in the server tables are now out of whack.


updated on master dbispconfig:

Code:

+-----------+---------------------------------+---------+
| server_id | server_name                     | updated |
+-----------+---------------------------------+---------+
|         1 | master.simplified.host          |   26760 |
|         2 | ns1.simplified.host             |   29012 |
|         3 | mail-01.simplified.host         |   26748 |
|         4 | web101.simplified.host          |   26745 |
|         5 | ns2.simplified.host             |   29012 |
|         6 | nbg-vps1.simplified.host        |   26745 |
|         7 | cfishost.theaccessibleweb.co.uk |   26767 |
|         8 | nbg-vps2.simplified.host        |   26745 |
|        11 | nbg-vps3.simplified.host        |   26745 |
|        12 | nbg-web101.simplified.host      |   29016 |
+-----------+---------------------------------+---------+

on ns1 and ns2 databases the updated number is 29012.

and i guess the other servers picked up all the dns resyncs from before i restored dbispconfig.
on eg mail-01 database the updated number is 29234.

i originally just ran 'select * from server' on the master and used the number on the last entry as the number to update the field in dbispconfig on nbg-web101.simplified.host to 26760, not realising in my haste that each server had it's own 'updated' number, it now says updated on that host is 29016

i hadn't taken backups of dbispconfig on anything other than master, ns1 and ns2 before trying to break mirroring the 1st time.. didn't thiink it would be necessary.

advice?

 

ok, tried some more stuff.. i set the updated field on each server to match it's corresponding updated number in the master db.

again i tried breaking mirroring, still no option to resync dns records on just ns2, so i enabled mirroring again, without restoring any db's. and resynced all services on ns1.

this changed the updated number on every server, and the master shows them all on the same number:

Code:

+-----------+---------------------------------+---------+
| server_id | server_name                     | updated |
+-----------+---------------------------------+---------+
|         1 | master.simplified.host          |   31323 |
|         2 | ns1.simplified.host             |   31323 |
|         3 | mail-01.simplified.host         |   31323 |
|         4 | web101.simplified.host          |   31323 |
|         5 | ns2.simplified.host             |   31323 |
|         6 | nbg-vps1.simplified.host        |   31323 |
|         7 | cfishost.theaccessibleweb.co.uk |   31323 |
|         8 | nbg-vps2.simplified.host        |   31323 |
|        11 | nbg-vps3.simplified.host        |   31323 |
|        12 | nbg-web101.simplified.host      |   31323 |
+-----------+---------------------------------+---------+

i've gone to each servers dbispconfig db and check the updated number on each server.. and they all match up, 31323.

except on ns2. that has two entries in the server table:

Code:

+-----------+---------------------+---------+
| server_id | server_name         | updated |
+-----------+---------------------+---------+
|         2 | ns1.simplified.host |   28994 |
|         5 | ns2.simplified.host |   31323 |
+-----------+---------------------+---------+

no idea how or why ns1 on there is different, i'm minded to just set it to 31323, but not sure if that'll cause more problems.


also, not sure if it's a problem, or indicative of other/bigger issues on ns2, but the in the gui, under system -> server ip addresses.
it lists internal, public ip4, public ip6 addresses and names of servers, but for ns2 it only lists the ip's, the server field is blank and no matter what i do, i can't get it to put the server fqdn there.

 

ok.. i've had some time to play around with this a bit more, and i think i know why i couldn't run a resync on ns2..

i've installed a 3rd ns server, i thought i'd try another one, see if the issue was a problem on ns2. but i also thought i'd try out a few other things before configuring it as a mirror, letting it get all the zones, and then breaking mirroring..

firstly. i just went to tools - resync to see if it shows up there as an option to resync under 'dns records', but nothing new only the ns1 server.
i added the ip's of the new (3rd) nameserver to the 'allow zone tranfers to' and 'also_notify' fields of a zone.
ip's added are internal, public ip4, and public ip6.. ( 10.0.0.13,157.180.20.241,2a01:4f9:c012:712c::1 )
and created a secondary zone for a domain on the new (3rd) nameserver..
the slave zone config in named.conf.local was created properly, and the secondary zone in /etc/bind/slave was created properly.
still nothing for this server in the gui under tools - resync - dns records, still only showing ns1
new (3rd) nameserver, and ns2 do both show up as syncable servers under tools - resync - 'all services'

i then deleted the secondary dns zone in the gui, the config was removed from the named.conf.local file and the slave zone file was deleted properly.

i then created a completely new primary dns zone, on the new (3rd) nameserver (not on ns1) and it created the correct config in named.conf.local, and created the primary zone file in /etc/bind.
with this in place, the new (3rd) nameserver shows up, along with ns1, under tools - resync - dns records.

so tools - resync - dns records will only be an option when a primary zone exists on a dns server.


i also tried resyncing all services on the new 3rd nameserver with only a secondary zone configured on there, which apparently resynced only the client and reseller details, even though the 'client and reseller' checkbox was unticked..
the dns part of all the service resyncs was:
Resynced DNS zone
Nothing found

so if a dns server only has secondary zones active on it, ispconfig doesn't find anything at all dns related to sync, presumably because this is supposed to be handled by bind, even the timestamps on named.conf.local and /etc/bind/slave/sec.domain.tld were unchanged, so it's not even rebuilding the named.conf.local slave configuration data on the secondary server from any db data.

so ispconfig resync tools is currently only usable for primary dns zones.

i'll try more about creating secondary zones via mysql and seeing if bind will pick up and sync those..
and if it does.. i'll look more into the easiest way to change dns from ispconfig mirroring to bind master/slave.
but that will probably all be tomorrow now.. going to be leaving the office in a few minutes...

 

been playing around with this more today... and it seems that populating the dns_slave table on the master using mysql is a waste of time.
nothing can be done with resyncing, updating a record in a primary zone etc to get any secondary zone data onto the actual nameserver instance.
looks like the only way to configure secondary zones is to manually create them via the gui.
possibly can be done via the api, but a don't see any docs/examples for it in the install tarball, i can see docs for dns_slave_delete and dns_slave_get, but no dns_slave_add.


now got to the point where i've played with renaming the newly installed nameserver, which seems to be a nice, fairly simple and trouble free process.
so now it seems options are
method 1.
break mirroring of ns1 by ns2.
clean dns_soa, dns_rr tables of ns2.
possibly resync all services on ns2.. not sure this is necessary - would remove all zone files and config in named.conf.local. definitely breaking dns responses from this server.
manually create all secondary dns zones.

method 2.
manually create all secondary dns zones on new ns server.
break mirroring of ns1 by ns2. (not sure this is necessary, just deleting the ns2 server from ispconfig may be enough)
delete ns2 server in ispconfig ( this should delete the iscpcrv## users of ns2 on master mysql)
remove root mysql access from ns2 on master
rename new ns server to ns2
update A/AAAA records for ns2 in dns. update glue records for ns2 with domain registrar


i'm more inclined to going with method 2.. i know it's a few more steps.. but each step is a known quantity, plus it allows me to keep the old ns2 in place and answering dns requests whilst i rename the new nameserver to ns2 and update ip's, and also leave the old ns2 running whilst the A, AAAA and glue record changes propagate.
i can take my time creating all the secondary zones on the new nameserver before doing anything affecting that affects ns2, i don't have to rush through creating all the secondary zones, knowing that ns2 is not responding to dns requests like i would using method 1


all that said, it definitely seems like there should be a simpler, quicker, more automated way to switch a nameserver from ispconfig server mirroring to being a bind slave.

 

ok.. just noticed one possible future problem on the live system..
in the zone template settings, in the fields options, the DNSSEC checkbox option doesn't exist, it's also not available when actually adding a zone using the wizard, or manually, or in the zone settings tab of an existing domain.
i'm assuming (hoping) that this is just because server mirroring of the primary nameserver is already in use..

am i right and the dnssec option will become available once server mirroring is stopped? would rather know before i actually break any mirroring again..