Splunk only lets you mark data as deleted (using the
| delete command in a search query and the
can_delete role) but this does not free up disk space. You can delete all of the data in an index using
splunk clean , but that's no good if you only want to get rid of a subset of the data.
The solution is to export the data in each of your index's database buckets to CSV, which excludes deleted items, and re-import them. As database buckets are split into manageable chunks, this will even work if you're perilously low on disk space! :)
#!/bin/bash exporttool="splunk cmd exporttool" importtool="splunk cmd importtool" # Set this to the correct directory for the index you're cleaning up for bucket in /opt/splunk/var/lib/splunk/defaultdb/db/*; do echo $bucket $exporttool $bucket - -csv meta::all | $importtool $bucket.new - splunk-optimize -d $bucket.new splunk-optimize-lex -d $bucket.new find $bucket.new/rawdata -name '[0-9]*[0-9]' -size +1k -print0 | xargs -0 -r gzip -v9 rm -r $bucket && mv $bucket.new $bucket done
I've written an article on using Apache 2.4's forms-based authentication for Splunk Free. This is much fancier and nicer than HTTP BASIC authentication!