As I mentioned before once you export firewall logs into human-readable format you can do lots of interesting things – for example script that gives statistics of how many times each Security rule was hit .
Be aware that this counts explicit Security rules only – i.e. the ones you see in Security tab of the Smartdashboard. No other rules you usually see in Smartview Tracker are counted – e.g. SmartDefense,Web Filtering etc. Also afterwards I sort it by number of hits to see what rules are used most:
awk -F\; ' {match($0,/rule: +([0-9]+)/,rules);rule_count[rules[1]]++} END {for (rule_number in rule_count) print " Rule number: " rule_number " Hits: " rule_count[rule_number]}' ./fw.log.txt | sort -n -k5
Rule number: Hits: 1197330 Ignore this line as it counts non-matched lines I dont want to filter with additional conditions and added time processing Rule number: 2 Hits: 9 Rule number: 5 Hits: 366 Rule number: 11 Hits: 12296 Rule number: 9 Hits: 14457 Rule number: 0 Hits: 17094 Rule number: 1 Hits: 44066 Rule number: 7 Hits: 233643 Rule number: 10 Hits: 366275 Rule number: 6 Hits: 424639
Update 2012 Below is the script to use Rule ID instead of Rule sequential numbers – this way changing rules order will not affect statistics. The script matches also non-security rules – e.g. email session id, that are a bit shorter then Rule ID, but I didn’t want to slow down the processing with additional formatting .
awk -F\; ' {match($0,/{([[:print:]]+)}/,rules);rule_count[rules[1]]++} END {for (rule_number in rule_count) print " Rule number: " rule_number " Hits: " rule_count[rule_number]}' ./fw.log.txt | sort -n -k5
Rule number: D199972C-ED3E-4EB4-8B83-813333156D18 Hits: 175 Rule number: 85A905A7-951E-4100-A4BA-E13333151D29 Hits: 219 Rule number: 81333316-E942-4313-BB7D-E1333315802F Hits: 1519 Rule number: 71333215-2DB5-4A3A-95BC-5080AD0F5564 Hits: 2298 Rule number: 11331315-AE52-44E0-A42A-711029B5768E Hits: 3755 Rule number: 01333315-D290-4B05-AFE7-23BF24D889FF Hits: 4116 Rule number: 121FA62F-3885-4328-8090-BF1333315eB1 Hits: 399793 Rule number: FE40E076-BAEB-4979-8E41-5EF1333315e6 Hits: 440101 Rule number: BB3F6772-4D38-4D5A-952A-301333315de8 Hits: 1354341 Running time for a file of 900 Mb with 4.7 million records real 5m50.287s user 4m22.890s sys 0m3.190s
Hi,
I’m running this command, but do not get the result of all rules only the total hits.
I need to change some syntax for that to happen?
I’m just getting this result:
Rule number: Hits: 1565351
Thanks, Junior Toledo
Well , strange – I run today this script against NGX R65 and R70.10 and had expected results. Hard to say why it doesnt work for you w/o looking at the log file format you use, so …
BTW I think of rewriting this script to calculate hit counts based on rules ID and not numbers that change after you add/remove rules, so watch for update
Yuri- Did you ever get around to rewriting for rules ID?
It is still on my to-do list , but thanks for reminding , I’ll try to find some time , may be next week
Yuri
Hi
Any luck on rewriting it?
Cheers
Balaji
Hi,
you might want to take a look at: http://code.google.com/p/frala/
best regards
rafal
Thanks , i will have a look at this
Hi Yuri,
Very useful. I would like to know what I need to change/adjust if I want to have rule names displayed instaead of UIDs?
I tried things but without success.
Many thanks.
Thx ,
what do you mean by ‘rule name ‘ ?
Hi Yuri,
Thank you for providing a feedback. Much appreciated.
Checkpoint R70.40 and R71.40 versions are implicated here.
You can give a name to each security rule in your policy.
When “fwm logexport” a fw1 log file and opens it with a text editor the first line shows all fields that can be display. see below.
num;date;time;orig;type;action;alert;i/f_name;i/f_dir;product;log_sys_message;inzone;outzone;rule;rule_uid;rule_name;service_id;src;dst;proto;etc …
I would like the “rule_name” field to be displayed instead of the “rule_uid” according to the hits count.
I hope to have provided a proper answer?
Best regards.