Bind DNS rate-limit and values for responses-per-second and window

I have a very close configuration (No Window 5 value) on an operational bind 9 server which is suffering an Amplification Attack. I have done DNS Packet Capture and the Server was correctly answering the legitimate queries and not responding to the Attack queries. This was a 5000 packet capture over a period of about 30 minutes. These values have completely thwarted the Attack from my point of view.

You should read the administrator reference manual for BIND 9.9.

Basically, responses-per-second is the number of identical replies that can be sent to one single destination, per second. The definitions are tricky.

A single destination is a block of network addresses, of the size configured in ipv4-prefix-length or ipv6-prefix-length as applicable. So, if the ipv4-prefix-length is 24, and both 192.0.2.1 and 192.0.2.2 are querying the DNS server, they will share this quota and can only send so many queries between the two of them.

Identical replies are replies to queries for a particular RRtype for a particular existent name, or for a nonexistent name. The following queries are all distinct:

IN A example.net.
IN A www.example.net.
IN AAAA example.net.
IN A nonexistent.domain.example.net.

However, all of the following queries are identical (assuming nonexistent.domain.example.net. etc. live up to their names):

IN A nonexistent.domain.example.net.
IN A nonexistent.domain2.example.net.
IN SOA other.nonexistent.domain.example.net.

window complicates things a little more still. It is the number of seconds for which quota can be banked. Multiplying window and responses-per-second gives the maximum by which any quota can be positive, or in more basic terms, the burst capacity.

To give a catch-all example:

You are the nonrecursing, authoritative nameserver for example.net.. Imagine no DNS traffic was seen at all in the past 10 seconds, and the configuration in the question applies globally. The following events happen sequentially:

1. Host 198.51.100.1 sends 100 queries for IN NS example.net.. 25 will be allowed, and the remaining 75 will be ignored.
2. Host 198.51.100.1 sends 100 queries for IN A nonexistent.example.net.. 25 will be allowed, and the remaining 75 will be ignored.
3. Host 198.51.100.1 sends 1 query for IN MX nonexistent-domain.example.net. It will be ignored since the limit for nonexistent domains has been reached.
4. Host 198.51.100.1 sends 1 query for IN A example.net.. It is allowed.
5. Hosts 192.0.2.1 through 192.0.2.50 each send a single query for IN NS example.net.. 25 of them get replies and the remaining 25 are ignored; the quota for 198.51.100.0/24 does not apply to these hosts, but they share the quota for 192.0.2.0/24.
6. One second passes
7. Hosts 192.0.2.26 through 192.0.2.50 repeat their query IN NS example.net.. 5 of them get replies and the remaining 20 are ignored, since the quota is only replenished by 5 queries per second.

I don't think its a good idea to rate limit, ask yourself : do you rate limit the webserver responses too? Why do you think DNS responses are less important than webserver responses?
And even if you rate limit, that 5 req/sec sounds very low.

iptables -A INPUT -p udp --dport 53 -m recent --set --name dnslimit
iptables -A INPUT -p udp --dport 53 -m recent --update --seconds 60 --hitcount 11 --name dnslimit -j DROP 

IPtables can work just as well. Keeps the traffic out of the service completely if an attack is found.