yurisk.info

Yuri Slobodyanyuk's blog on IT Security and Networking sharing experience and expertise

awk weekly – Security rule hits statistics . Checkpoint

As I mentioned before once you export firewall logs into human-readable format you can do lots of interesting things – for example script that gives statistics of how many times each Security rule was hit .
Be aware that this counts explicit Security rules only – i.e. the ones you see in Security tab of the Smartdashboard. No other rules you usually see in Smartview Tracker are counted – e.g. SmartDefense,Web Filtering etc. Also afterwards I sort it by number of hits to see what rules are used most:

awk -F\; ' {match($0,/rule: +([0-9]+)/,rules);rule_count[rules[1]]++} END {for (rule_number in rule_count) print " Rule number: " rule_number " Hits: " rule_count[rule_number]}' ./fw.log.txt | sort -n -k5
Rule number:  Hits: 1197330  Ignore this line as it counts non-matched lines I dont want to filter with additional conditions and added time processing
 Rule number: 2 Hits: 9
 Rule number: 5 Hits: 366
 Rule number: 11 Hits: 12296
 Rule number: 9 Hits: 14457
 Rule number: 0 Hits: 17094
 Rule number: 1 Hits: 44066
 Rule number: 7 Hits: 233643
 Rule number: 10 Hits: 366275
 Rule number: 6 Hits: 424639 
Update 2012 Below is the script to use Rule ID instead of Rule sequential numbers – this way changing rules order will not affect statistics. The script matches also non-security rules – e.g. email session id, that are a bit shorter then Rule ID, but I didn’t want to slow down the processing with additional formatting .
awk -F\; ' {match($0,/{([[:print:]]+)}/,rules);rule_count[rules[1]]++} END {for (rule_number in rule_count) print " Rule number: " rule_number " Hits: " rule_count[rule_number]}' ./fw.log.txt | sort -n -k5
Rule number: D199972C-ED3E-4EB4-8B83-813333156D18 Hits: 175
 Rule number: 85A905A7-951E-4100-A4BA-E13333151D29 Hits: 219
 Rule number: 81333316-E942-4313-BB7D-E1333315802F Hits: 1519
 Rule number: 71333215-2DB5-4A3A-95BC-5080AD0F5564 Hits: 2298
 Rule number: 11331315-AE52-44E0-A42A-711029B5768E Hits: 3755
 Rule number: 01333315-D290-4B05-AFE7-23BF24D889FF Hits: 4116
 Rule number: 121FA62F-3885-4328-8090-BF1333315eB1 Hits: 399793
 Rule number: FE40E076-BAEB-4979-8E41-5EF1333315e6 Hits: 440101
 Rule number: BB3F6772-4D38-4D5A-952A-301333315de8 Hits: 1354341
Running time for a file of 900 Mb with 4.7 million records
real    5m50.287s
user    4m22.890s
sys     0m3.190s


12 Comments

  1. Hi,

    I’m running this command, but do not get the result of all rules only the total hits.
    I need to change some syntax for that to happen?

    I’m just getting this result:
    Rule number: Hits: 1565351

    Thanks, Junior Toledo

  2. Yuri

    September 7, 2010 at 7:08 pm

    Well , strange – I run today this script against NGX R65 and R70.10 and had expected results. Hard to say why it doesnt work for you w/o looking at the log file format you use, so …
    BTW I think of rewriting this script to calculate hit counts based on rules ID and not numbers that change after you add/remove rules, so watch for update

  3. Yuri- Did you ever get around to rewriting for rules ID?

  4. Yuri

    March 4, 2011 at 2:18 pm

    It is still on my to-do list , but thanks for reminding , I’ll try to find some time , may be next week
    Yuri

  5. Hi

    Any luck on rewriting it?

    Cheers
    Balaji

  6. Hi,
    you might want to take a look at: http://code.google.com/p/frala/

    best regards
    rafal

  7. Yuri

    February 20, 2012 at 6:23 pm

    Thanks , i will have a look at this

  8. Hi Yuri,

    Very useful. I would like to know what I need to change/adjust if I want to have rule names displayed instaead of UIDs?
    I tried things but without success.
    Many thanks.

  9. Yuri

    February 24, 2012 at 10:53 pm

    Thx ,
    what do you mean by ‘rule name ‘ ?

  10. Hi Yuri,

    Thank you for providing a feedback. Much appreciated.
    Checkpoint R70.40 and R71.40 versions are implicated here.
    You can give a name to each security rule in your policy.
    When “fwm logexport” a fw1 log file and opens it with a text editor the first line shows all fields that can be display. see below.

    num;date;time;orig;type;action;alert;i/f_name;i/f_dir;product;log_sys_message;inzone;outzone;rule;rule_uid;rule_name;service_id;src;dst;proto;etc …

    I would like the “rule_name” field to be displayed instead of the “rule_uid” according to the hits count.
    I hope to have provided a proper answer?
    Best regards.

  11. Is there any command which can be used in Cisco 9K router to print particular column?

  12. Hi, is there way that we can find the rules which is not being used. like in cisco we can find the hit counts of each rules. let’s in cisco ACL count is 0 which is not being used anymore. in that similar way, is it possible to list out the rules which is not being used.

Comments are closed.

© 2016 yurisk.info

Theme by Anders NorenUp ↑