yurisk.info

Yuri Slobodyanyuk's blog on IT Security and Networking sharing experience and expertise

See what your users are doing – awk one-line scripts to parse eSafe logs

As most of the posts here this one is also inspired by a client. There was an unassuming shy and not making any troubles eSafe 8.5 appliance XG-200. Then one day Security Admin of the company complained to me about ‘high CPU utilization’ – getting somewhere up to 60% . eSafe looked absolutely fine and doing its work.
Also I noticed it was kinda working hard , nothing special but just general feeling that it handles a lot of load. And boy was I right – it was doing 200 Mb of logs per day . Given the number of stations in LAN and working hours that was huge. I looked with awk at the logs and happily updated the SecAdmin that eSafe is doing its work and blocks all the users trying frantically to visit various porno sites ignoring the “Site is blocked ..”message by the eSafe. “What? my users to pron sites, can’t be, can you show me the logs, who does it ?” . No problem, and so the awk one-liners you see below were written to parse esafe Aladdin logs to get some insight. Also at the end of the post see eSafe log format in case you want to develop your own scripts.
All logs are located at /opt/eSafe/eSafeCR/SessionLog/
One-liner number one – Gather IPs that sent spam , count number of spam messages per IP, sort the list in ascending order .

# awk -F"|" '$4~SMTP && ( $6~/Spam blocked/ || $6~/Mail rejected/ ) { print $11} ' *.log | sort -n | uniq -c | sort -n | tail -10
29 80.179.60.37
32 41.59.0.205
41 74.63.80.6
41 83.16.167.14
48 125.163.188.31
54 125.163.192.184
57 113.168.9.224
78 89.223.56.16
80 212.143.70.26
104 212.143.70.27

One-liner number two – Mail sender fileds of spam messages , just for fun , no real value for security purpose:

# awk -F"|" ' $4~SMTP && ( $6~/Spam blocked/ || $6~/Mail rejected/ ) { print $15} ' *.log | sort | uniq -c | sort -n
8 Stephan@117.40.136.73
10 bsb@bsbinfo.in
13 info@all-free.co.il
13 Janette@2.90.58.204
15 Ronnie@178.34.19.174
17 KellieClements@cramerspointmotel.com
22 notifs@m.snapinteractiveapps.com
60 ezrachmudag26@gmail.com
25 Simone@187.63.223.21
102 Angelo@31.subnet125-163-188.speedy.telkom.net.id

One-liner to see all the blocks/rejects reasons and respective statistics.

# awk -F"|"' {print $6}' *.log | sort -k1,1 | uniq -c
8 Application blocked
21967 File allowed
360 File blocked
114891 File clean
1731 File modified to remove malicious content
3650 Mail clean
111 Mail modified to remove malicious content
13 Mail rejected #912 – Anti-spoofing – Mail rejected. Attempt to impersonate a local user
164 SMTP error
803 Spam blocked

Now let’s move to HTTP browsing.
One-liner number four – blocked access to websites : number of blocked attempts per website, hostname of the website, internal LAN IP of PC that tried to access the resource.
I do not bring examples here as they are quite embarrassing, even to be brought anonymously, so just trust me – run it on your esafe and you will blush.

# awk -F"|" '$4~HTTP && /File blocked/ { print $7,$17} ' *.log | sort -k1,1 | uniq -c | sort -n -k1,1

Same as above but with full path to the prohibited file.

# awk -F"|" '$4~HTTP && /File blocked/ { print $8,$17,$11} ' *.log | sort -k1,1 | uniq -c | sort -n -k1,1

And finally as promised the format of eSafe logs. All the fields in logs are separated by vertical bar (as you probably guessed awk –F”|” accounts for that). All the fields are present, while irrelevant fields are empty. So it is really scripting-friendly. I broke down the fields into separate lines with field number of each field. Enjoy.

# awk -F"|" ' { for (i=1;i<=NF;i++) print i,$i}' header.txt
1 Date (yyyy-mm-dd HH:mm:ss)
2 eSafe name
3 Record ID
4 ProtocolType
5 Method
6 Event
7 URL host
8 File Name\Mail Subject
9 File Type
10 #File Size
11 Source IP
12 Destination IP
13 #VLAN
14 #Port
15 Mail Sender
16 Mail Recipients
17 URL category
18 User
19 LDAP domain
20 Host
21 Decision By
22 Profile
23 Policy
24 #Policy ID
25 Details
26 Extended result
27 SessionID
28 MessageID
29 #Rule
30 #File Binary Family
31 File container path
32 File name
33 #File parameter
34 #Engine code
35 #Activity code
36 Blocked URL category code
37 URL category mask
38 Result name
39 #Result code
40 #Server type ID
41 #Application code
42 #Action
43 #Risky
44 #Source IP
45 #Destination IP
46 #MachineIP
47 #Duration
48 #AID
49 Referrer
50 UUID
51 #Has CMF
52 Date
53 Time
54 #Mail status
55 DLP profile


1 Comment

  1. Hi, great job concerning “field number”

    Thanks

Comments are closed.

© 2016 yurisk.info

Theme by Anders NorenUp ↑