ModSecurity Anti-Automation Rule Set for Detecting Denial of Service (DOS) Attacks, Part 1

In a previous article, I introduced ModSecurity as an effective web application firewall.  One of the things I like about ModSecurity is that it comes with a set of core rules (CRS) for detecting and handling various nefarious web activities like XSS and SQL injection.  I was thrilled to learn that CRS also comes with a set of rules for detecting DOS attacks.  I was curious to learn how the anti-DOS rules work.  I have stared at them long enough now that I'm beginning to see the light at the end of the tunnel.  In this article, I'll give an overview of how the DOS rules work.  In the next article, I'll dive a bit deeper by patching a race condition bug in the rules.

For reference, here are the rules from modsecurity_crs_11_dos_protection.conf:

#
# Anti-Automation rule set for detecting Denial of Service Attacks. 
#
#
# Enforce an existing IP address block and log only 1-time/minute
# We don't want to get flooded by alerts during an attack or scan so
# we are only triggering an alert once/minute.  You can adjust how often
# you want to receive status alerts by changing the expirevar setting below.
#
SecRule IP:DOS_BLOCK "@eq 1" "chain,phase:1,id:'981044',drop,msg:'Denial of Service (DoS) Attack Identified from %{tx.real_ip} (%{tx.dos_block_counter} hits since last alert)',setvar:ip.dos_block_counter=+1"
SecRule &IP:DOS_BLOCK_FLAG "@eq 0" "setvar:ip.dos_block_flag=1,expirevar:ip.dos_block_flag=60,setvar:tx.dos_block_counter=%{ip.dos_block_counter},setvar:ip.dos_block_counter=0"

#
# Block and track # of requests but don't log
SecRule IP:DOS_BLOCK "@eq 1" "phase:1,id:'981045',t:none,drop,nolog,setvar:ip.dos_block_counter=+1"

#
# skipAfter Check
# There are different scenarios where we don't want to do checks -
# 1. If the current IP address has already been blocked due to high requests
# In this case, we skip doing the request counts.
#
SecRule IP:DOS_BLOCK "@eq 1" "phase:5,id:'981046',t:none,nolog,pass,skipAfter:END_DOS_PROTECTION_CHECKS"

#
# DOS Counter
# Count the number of requests to non-static resoures
# 
SecRule REQUEST_BASENAME "!\.(jpe?g|png|gif|js|css|ico)$" "phase:5,id:'981047',t:none,nolog,pass,setvar:ip.dos_counter=+1"

#
# Check DOS Counter
# If the request count is greater than or equal to user settings,
# we then set the burst counter
# 
SecRule IP:DOS_COUNTER "@gt %{tx.dos_counter_threshold}" "phase:5,id:'981048',t:none,nolog,pass,t:none,setvar:ip.dos_burst_counter=+1,expirevar:ip.dos_burst_counter=%{tx.dos_burst_time_slice},setvar:!ip.dos_counter"

#
# Check DOS Burst Counter and set Block
# Check the burst counter - if greater than or equal to 2, then we set the IP
# block variable for 5 mins and issue an alert.
#
SecRule IP:DOS_BURST_COUNTER "@ge 2" "phase:5,id:'981049',t:none,log,pass,msg:'Potential Denial of Service (DoS) Attack from %{tx.real_ip} - # of Request Bursts: %{ip.dos_burst_counter}',setvar:ip.dos_block=1,expirevar:ip.dos_block=%{tx.dos_block_timeout}"

SecMarker END_DOS_PROTECTION_CHECKS

You can tailor the burst detection patterns by editing the file at /etc/httpd/modsecurity.d/modsecurity_localrules.conf:

#
# -- [[ DoS Protection ]] ----------------------------------------------------------------
#
# If you are using the DoS Protection rule set, then uncomment the following
# lines and set the following variables:
# - Burst Time Slice Interval: time interval window to monitor for bursts
# - Request Threshold: request # threshold to trigger a burst
# - Block Period: temporary block timeout
#
SecAction \
  "id:'900015', \
  phase:1, \
  t:none, \
  setvar:'tx.dos_burst_time_slice=60', \
  setvar:'tx.dos_counter_threshold=100', \
  setvar:'tx.dos_block_timeout=600', \
  nolog, \
  pass"

The anti-DOS rules count the number of requests for non-static resources made by an IP address.  By default, if the count exceeds 100 requests / minute, then the IP address is blocked for 10 minutes.  During the block period, any requests made by the IP address are dropped.  The counter starts afresh when the block period expires.  For reporting, the IP address is logged when it has just been blocked.  During the block period, the number of new requests by the IP address is logged every minute.

To implement this logic, the rules use 5 internal variables.  The dos_counter variable counts the number of requests during the time when client is not blocked.  In pseudo-code, the variable with its default value is:

dos_counter = 0

The dos_block_counter variable counts the number of requests during the time when client is blocked:

dos_block_counter = 0

The dos_burst_counter variable keeps track if a burst had occurred (whenever value is 2).  The value resets to 0 upon expiration time.

struct dos_burst_counter
    value = 0
    expiration = +inf

The dos_block variable indicates if an address is currently blocked (whenever value is 1).  The value resets to 0 upon expiration time.

struct dos_block
    value = 0
    expiration = +inf

The dos_block_flag variable controls the frequency of log updates during block period.  The value resets to 0 upon expiration time.

struct dos_block_flag
    value = 0
    expiration = +inf

The burst detection pattern is set with the 3 parameters:

DOS_COUNTER_THRESHOLD = 100
DOS_BURST_TIME_SLICE = 60
DOS_BLOCK_TIMEOUT = 600

In pseudo-code, the 6 anti-DOS rules are encapsulated in a function:

function process(request, response)
    //
    // Rule 981044 - log new requests by the blocked IP address every minute
    //
    if dos_block.value == 1 && dos_block_flag.value == 0
        dos_block_flag.value = 1
        dos_block_flag.expiration = unix_time() + 60
        print("DOS attack from %s %d hits since last alert", request.ip, dos_block_counter)
        dos_block_counter = 0

    //
    // Rule 982045 - drop request if IP address is blocked
    //
    if dos_block.value == 1
        dos_block_counter++
        response.drop()

    //
    // Rule 981046 - skip the following rules
    //
    if dos_block.value == 1
        return

    //
    // Rule 981047 - increment request counter
    //
    if request.non_static_resource
        dos_counter++

    //
    // Rule 981048 - check request counter against threshold
    //
    if dos_counter > DOS_COUNTER_THRESHOLD
        dos_burst_counter.value++
        dos_burst_counter.expiration = unix_time() + DOS_BURST_TIME_SLICE
        dos_counter = 0

    //
    // Rule 981049 - check if a burst had occurred
    //
    if dos_burst_counter.value >= 2
        dos_block.value = 1
        dos_block.expiration = unix_time() + DOS_BLOCK_TIMEOUT
        print("Potential DOS attack from %s - %d bursts, request.ip, dos_burst_counter.value)

Every time a web request comes in, the process() function is invoked.  In addition, every second or less, a background process calls the following function to reset any variables with pending expiration.

function expire()
    int time_now = unix_time()

    if time_now >= dos_block.expiration
        dos_block.value = 0
        dos_block.expiration = +inf

    if time_now >= dos_burst_counter.expiration
        dos_burst_counter.value = 0
        dos_burst_counter.expiration = +inf

    if time_now >= dos_block_flag.expiration
        dos_block_flag.value = 0
        dos_block_flag.expiration = +inf

Barring any issues with concurrency, the above pseudo-code should mimic the behavior of the anti-DOS rules but in a more readable and comprehensible manner.  In the next article, I'll dive a bit deeper by patching a race condition bug in the rules.