yurisk.info

Yuri Slobodyanyuk's blog on IT Security and Networking sharing experience and expertise

Category: Uncategorized (page 2 of 3)

See what your users are doing – awk one-line scripts to parse eSafe logs

As most of the posts here this one is also inspired by a client. There was an unassuming shy and not making any troubles eSafe 8.5 appliance XG-200. Then one day Security Admin of the company complained to me about ‘high CPU utilization’ – getting somewhere up to 60% . eSafe looked absolutely fine and doing its work.
Also I noticed it was kinda working hard , nothing special but just general feeling that it handles a lot of load. And boy was I right – it was doing 200 Mb of logs per day . Given the number of stations in LAN and working hours that was huge. I looked with awk at the logs and happily updated the SecAdmin that eSafe is doing its work and blocks all the users trying frantically to visit various porno sites ignoring the “Site is blocked ..”message by the eSafe. “What? my users to pron sites, can’t be, can you show me the logs, who does it ?” . No problem, and so the awk one-liners you see below were written to parse esafe Aladdin logs to get some insight. Also at the end of the post see eSafe log format in case you want to develop your own scripts.
All logs are located at /opt/eSafe/eSafeCR/SessionLog/
One-liner number one – Gather IPs that sent spam , count number of spam messages per IP, sort the list in ascending order .

# awk -F"|" '$4~SMTP && ( $6~/Spam blocked/ || $6~/Mail rejected/ ) { print $11} ' *.log | sort -n | uniq -c | sort -n | tail -10
29 80.179.60.37
32 41.59.0.205
41 74.63.80.6
41 83.16.167.14
48 125.163.188.31
54 125.163.192.184
57 113.168.9.224
78 89.223.56.16
80 212.143.70.26
104 212.143.70.27

One-liner number two – Mail sender fileds of spam messages , just for fun , no real value for security purpose:

# awk -F"|" ' $4~SMTP && ( $6~/Spam blocked/ || $6~/Mail rejected/ ) { print $15} ' *.log | sort | uniq -c | sort -n
8 Stephan@117.40.136.73
10 bsb@bsbinfo.in
13 info@all-free.co.il
13 Janette@2.90.58.204
15 Ronnie@178.34.19.174
17 KellieClements@cramerspointmotel.com
22 notifs@m.snapinteractiveapps.com
60 ezrachmudag26@gmail.com
25 Simone@187.63.223.21
102 Angelo@31.subnet125-163-188.speedy.telkom.net.id

One-liner to see all the blocks/rejects reasons and respective statistics.

# awk -F"|"' {print $6}' *.log | sort -k1,1 | uniq -c
8 Application blocked
21967 File allowed
360 File blocked
114891 File clean
1731 File modified to remove malicious content
3650 Mail clean
111 Mail modified to remove malicious content
13 Mail rejected #912 – Anti-spoofing – Mail rejected. Attempt to impersonate a local user
164 SMTP error
803 Spam blocked

Now let’s move to HTTP browsing.
One-liner number four – blocked access to websites : number of blocked attempts per website, hostname of the website, internal LAN IP of PC that tried to access the resource.
I do not bring examples here as they are quite embarrassing, even to be brought anonymously, so just trust me – run it on your esafe and you will blush.

# awk -F"|" '$4~HTTP && /File blocked/ { print $7,$17} ' *.log | sort -k1,1 | uniq -c | sort -n -k1,1

Same as above but with full path to the prohibited file.

# awk -F"|" '$4~HTTP && /File blocked/ { print $8,$17,$11} ' *.log | sort -k1,1 | uniq -c | sort -n -k1,1

And finally as promised the format of eSafe logs. All the fields in logs are separated by vertical bar (as you probably guessed awk –F”|” accounts for that). All the fields are present, while irrelevant fields are empty. So it is really scripting-friendly. I broke down the fields into separate lines with field number of each field. Enjoy.

# awk -F"|" ' { for (i=1;i<=NF;i++) print i,$i}' header.txt
1 Date (yyyy-mm-dd HH:mm:ss)
2 eSafe name
3 Record ID
4 ProtocolType
5 Method
6 Event
7 URL host
8 File Name\Mail Subject
9 File Type
10 #File Size
11 Source IP
12 Destination IP Continue reading

RBLs – more hassle than benefit

as I wrote earlier Google mail servers got blacklisted , but it was only the beginning. Now I got complains from the client that hotmail.com users also get mail bounced because of this list . So, as running after each and every IP blocked by this list endlessly wasn’t the best use of my time, I disabled this specific RBL in eSafe. So far no complains neither about bounced mail nor increased spam. The screenshot below shows where to find this RBL in the eSafe.
How to disable specific RBL server

Alert on change of SOA in domain

This comes from unpleasant experience of mine. One of my clients’ domain records (MX for the case involved) was mistakenly changed. While it was a human error and trying to fix humans is rolling the rock of Sisyphus,damage would be much lesser had I known about the change immediately . To take care of this side of the story I wrote awk one-liner that when invoked by cron compares SOA of the domain with the one saved locally in the file. If there is a discrepancy the mail is sent.
I tried to find more elegant solution on Google but found zillions of tools too complex for such a simple task.

#!/bin/bash
awk ' BEGIN {"dig +short soa yurisk.info"| getline
SOA_NOW=$3
getline SOA < "serial-yurisk.info"
if (SOA_NOW != SOA) { print (" mail -c yurisk@yurisk.info -s \"SOA of domain yurisk.info has changed\" president@whitehouse.gov") | "/bin/bash" }}'
serial-yurisk.info:
2010012000

Quick and dirty way to bypass eSafe inspection

There are times when you need to make some website work immediately while it is being blocked by eSafe for some (many) reasons. And you just don’t get it working the educated way – adding to white/exclude lists, changing script/category block options etc.
For the cases just like that Aladdin have provided us with Exclusion List in NitroInspection Configuration . It basically means traffic to/from the IP addresses you put into this list will be COMPLETELY ignored by eSafe scanning engine, and will be moved from interface to interface at the NIC driver speed.
To get there you go to Options->-NitroInspection Configuration->-Exclusion list->Add
In example below I add facebook.com IP range to such exclusion list.
NitroInspection Exclusion list screenshot

Increase log size in eSafe

Session logs in eSafe are  essential for debugging  . By default ,nevertheless each Session log file is limited to 100 megabytes in size , after reaching this limit eSafe stops writing the Session logs until the next log rotation – that is midnight.

To fix this , edit  the file /opt/eSafe/eSafeCR/esafecfg.ini:

[ALERT GENERAL]  
Size limit=2
Last overflow=0
Minimum free disk space=2000
Block if internal error=1
File name=^M
Report days=10
Session log days=7    = >  Session log days= 365
Report max length=100
Session log max length=100  =>  Session log max length=500
Log sessions=1
Detailed log sessions=0
Log System Info Interval=10
MMS block if internal error=1
SessionLog To EventLog=0

eSafe has iptables too ….

Did you know that eSafe  software is based on RedHat Enterprise  Linux (RHEL) ? Of course you did.
But what does it mean? It means that all (or almost all) tools/utilities/programming logic of the Linux is at your fingertips.
Let’s take for example software eSafe runs on the boot

 [root@esafe root]# chkconfig –list
rdisc                          0:off   1:off   2:off   3:off   4:off   5:off   6:off
anacron                   0:off   1:off   2:off   3:off   4:off   5:off   6:off
kudzu                        0:off   1:off   2:off   3:on    4:on    5:on    6:off
syslog                       0:off   1:off   2:on    3:on    4:on    5:on    6:off
network                   0:off   1:off   2:on    3:on    4:on    5:on    6:off
random                    0:off   1:off   2:on    3:on    4:on    5:on    6:off
saslauthd                 0:off   1:off   2:off   3:off   4:off   5:off   6:off
microcode_ctl      0:off   1:off   2:on    3:on    4:on    5:on    6:off
irqbalance              0:off   1:off   2:off   3:on    4:on    5:on    6:off
smartd                     0:off   1:off   2:off   3:off   4:off   5:off   6:off
atd                             0:off   1:off   2:off   3:on    4:on    5:on    6:off
log2trap                   0:off   1:off   2:off   3:off   4:off   5:off   6:off
sshd                          0:off   1:off   2:on    3:on    4:on    5:on    6:off
crond                       0:off   1:off   2:on    3:on    4:on    5:on    6:off
ntpd                         0:off   1:off   2:off   3:off   4:off   5:off   6:off
iptables                   0:off   1:off   2:on    3:on    4:on    5:on    6:off
webmin                   0:off   1:off   2:on    3:on    4:off   5:on    6:off
esafe                        0:off   1:off   2:off   3:on    4:on    5:on    6:off

Today I played with just one of them – iptables. First things first – eSafe is not a firewall . You don’t usually give
bunch of papers (called money) to use eSafe for the function that any Pentium 4 PC can do for the fraction of the price. So, it
is not supported and not to be used as the mainstream feature. Nevertheless it is here and may come handy in some situations.

In the output of chkconfig –list  chckconfig showed that whenever eSafe is on, iptables is on as well. The default iptables policy is ALLOW ANY ANY :
[root@esafe root]# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Few observations:
– To block incoming connections destined for the eSafe itself you use INPUT chain, FORWARD chain has no meaning in bridged eSafe Gateway (in routing Gateway mode it  would be applicable I guess)
– I tried few basic rules on the eSafe Hellgate 200 while turning on High-Debug mode , and while machine was loaded by debug
mode , using iptables filtering did not add visible overhead to it.
In most of the deployment scenarios eSafe is installed between internal interface of the firewall and LAN. It means from outside it is protected quite well whereas from LAN it is wide open to any trouble coming in. eSafe has its own means of  limiting access to it from certain IPs.
To limit access to eConsole – you go in eConsole to Options -> Access and Permissions -> configure user (by default admin will be there)  and IP from which this user will be granted access.
To limit access to WebGUI (HTTPS) : in the WebGUI go to Settings -> Access Control ->  put IP to allow it access in “eSafe Appliance accepts connections from:”
To limit access by SSH use Linux means (there are few ways but I show just one of them) :
 vi /etc/ssh/sshd_config
# add at the end the following line to limit ssh access to 10.99.99.150 only:
AllowUsers   *@10.99.99.150
Or, using wildcards to the whole network only:
AllowUsers   *@10.*.*.*

All the above is correct and fine BUT – what if you need to temporarily limit access and not permanently ? All the above are permanent changes that will survive  reboot and  if you make a mistake you grant someone a visit to the appliance for  console connection. On the other hand you may use iptables to achieve the same access control and should something go wrong reboot will return all back to normal.

So, let’s go:

– I don’t like when debugging some complicated issue after logging off from eConsole I can’t access it again as someone from LAN already logged in :
1) Grant your IP the access to econsole:
[root@esafe root]# iptables -I INPUT    -p tcp -s 10.99.99.150 --dport 43969:43982 -j ACCEPT
2) Deny anyone else
[root@esafe root]# iptables -I INPUT 2 -p tcp --dport 43969:43982 -j DROP
– SSH in general isn’t something  a client should have access to :
1) First allow youself access:
 [root@esafe root]# iptables -I INPUT -p tcp --dport 22 -s 10.99.99.150  -j ACCEPT
2) Then deny anyone else:
[root@esafe root]# iptables -I  INPUT 2 -p tcp --dport 22   -j DROP

3) Restart ssh daemon (it won’t disconnect your current session):

[root@esafe root]# service sshd restart

[root@esafe root]# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     tcp  —  10.99.99.150         anywhere           tcp dpt:ssh
DROP       tcp  —  anywhere             anywhere           tcp dpt:ssh

 
Those  addicted to iptables know that some spicy features come with modules and kernel options set at compile time.
To give you the taste of  what is included in eSafe iptables, the listing follows:

/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/arp_tables.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/arpt_mangle.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/arptable_filter.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ip_conntrack.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ip_conntrack_amanda.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ip_conntrack_ftp.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ip_conntrack_irc.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ip_conntrack_tftp.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ip_nat_amanda.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ip_nat_ftp.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ip_nat_irc.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ip_nat_snmp_basic.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ip_nat_tftp.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ip_queue.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ip_tables.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipchains.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipfwadm.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_DSCP.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_ECN.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_LOG.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_MARK.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_MASQUERADE.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_MIRROR.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_REDIRECT.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_REJECT.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_TCPMSS.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_TOS.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_ULOG.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_ah.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_conntrack.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_dscp.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_ecn.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_esp.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_helper.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_length.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_limit.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_mac.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_mark.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_multiport.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_owner.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_pkttype.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_recent.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_state.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_tcpmss.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_tos.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_ttl.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/ipt_unclean.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/iptable_filter.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/iptable_mangle.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv4/netfilter/iptable_nat.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/ah6.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/esp6.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/ip6_tunnel.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/ipcomp6.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/ipv6.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/netfilter/ip6_tables.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/netfilter/ip6t_LOG.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/netfilter/ip6t_MARK.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/netfilter/ip6t_ah.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/netfilter/ip6t_dst.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/netfilter/ip6t_esp.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/netfilter/ip6t_eui64.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/netfilter/ip6t_frag.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/netfilter/ip6t_hbh.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/netfilter/ip6t_hl.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/netfilter/ip6t_ipv6header.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/netfilter/ip6t_length.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/netfilter/ip6t_limit.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/netfilter/ip6t_mac.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/netfilter/ip6t_mark.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/netfilter/ip6t_multiport.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/netfilter/ip6t_owner.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/netfilter/ip6t_rt.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/netfilter/ip6table_filter.o
/lib/modules/2.4.21-47.ELsmp/kernel/net/ipv6/netfilter/ip6table_mangle.o

Website/malware categorization in eSafe

If some website gets blocked by eSafe for being categorized wrongly you
may fix it actually very simple. You enter the link below and change the website category; this takes some time , usually from few hours up to a day,for the change to take effect. If website has no category already then update takes effect fast.

filterdb.iss.net/urlcheck/

To see what each category includes:
www-935.ibm.com/services/us/index.wss/detail/iss/a1029077?cntxt=a1027244

When you want to report an item that was falsely detected as virus/malware by
eSafe you should send your request to :Nowhere, eSafe ceased to exist as a product

Older posts Newer posts

© 2016 yurisk.info

Theme by Anders NorenUp ↑