Failed deleting collection (name "ip", key “X.X.X.X_"): Internal error
I was concerned that if the collection file was not being cleaned up automatically by Apache or ModSecurity, the file may grow to an astronomical size over time causing other problems. While I wasn't able to find a definitive cause for the error by searching online, I stumbled upon 3 proposed workaround solutions:
- Use memcache for collections
- Set SecCollectionTimeout to a small value such as 600 (default is 3600 seconds)
- Install and run modsec-sdbm-util in a separate process to clean up the collection file regularly
For the record, I took the steps below to install modsec-sdbm-util on Amazon Linux AMI running Apache 2.4 with ModSecurity 2.8.
First, install all pre-requisite libraries and tools:
sudo su yum install libtool autoreconf autoheader automake autoconf apr-util-devel
Then, download and install modsec-sdbm-util to a directory.
git clone https://github.com/SpiderLabs/modsec-sdbm-util.git cd modsec-sdbm-util ./autogen.sh ./configure make install
Check that it’s installed successfully by running (assuming ip.pag file is in /tmp):
/usr/local/modsecurity/bin/modsec-sdbm-util -s /tmp/ip.pag
If everything is okay, it should output a status report similar to:
Opening file: /tmp/ip.pag Database ready to be used. [|] 10 records so far. Total of 17 elements processed. 0 elements removed. Expired elements: 7, inconsistent items: 0 Fragmentation rate: 41.18% of the database is/was dirty data.
Set up a cron job to run modsec-sdbm-util every half hour or so to remove expired elements from the collection file.
*/30 * * * * /usr/local/modsecurity/bin/modsec-sdbm-util -k /tmp/ip.pag &> /dev/null
This should do it! (Cross my fingers.)