yurisk.info

Yuri Slobodyanyuk's blog on IT Security and Networking sharing experience and expertise

Page 20 of 24

ARP table overflow in Checkpoint and Linux in general

Not specific to the Checkpoint but rather any Linux-based system issue, still people often
forget  about that and look for the Checkpoint-specific solutions to that , so to help with  this search I wrote the note 
how  to fix it  below:
Problem  usually shows itself in randomly distributed inability of stations to pass the firewall, slowness and other network problems follow.
In /var/log/message you see the following record:

kernel: Neighbour table overflow.
That means ARP table has reached its maximum allowed limit and no new ARP entries are being learnt.

You can either find reason for sudden ARP requests influx or adjust ARP table limits accordingly.
You adjust ARP table limits either editing  this file (then change survives reboot):

/etc/sysctl.conf
If not present add these lines at the end, and try not to delete by mistake anything:
net.ipv4.neigh.default.gc_thresh1 = 1024
net.ipv4.neigh.default.gc_thresh2 = 4096
net.ipv4.neigh.default.gc_thresh3 = 16384

 – Then issue command:
  # sysctl -p
– Or if you want to increase it temporarily until reboot:
#echo 1024 > /proc/sys/net/ipv4/neigh/default/gc_thresh1
#echo 4096 > /proc/sys/net/ipv4/neigh/default/gc_thresh2
#echo 16384 > /proc/sys/net/ipv4/neigh/default/gc_thresh3

And the short explanation follows.
gc in the above means Garbage Collector (GC).
net.ipv4.neigh.default.gc_thresh1  – sets minimum number of ARP entries in the cache.
Until this value is reached GC doesnt run at all.
net.ipv4.neigh.default.gc_thresh2  – sets soft maximum number of ARP entries in the cache.
GC allows ARP cache to pass this limit for 5 seconds and then starts cleaning.
net.ipv4.neigh.default.gc_thresh3  –  sets hard limit of ARP entries in the cache.
After it is reached no more ARP entries are being added.

Increase log size in eSafe

Session logs in eSafe are  essential for debugging  . By default ,nevertheless each Session log file is limited to 100 megabytes in size , after reaching this limit eSafe stops writing the Session logs until the next log rotation – that is midnight.

To fix this , edit  the file /opt/eSafe/eSafeCR/esafecfg.ini:

[ALERT GENERAL]  
Size limit=2
Last overflow=0
Minimum free disk space=2000
Block if internal error=1
File name=^M
Report days=10
Session log days=7    = >  Session log days= 365
Report max length=100
Session log max length=100  =>  Session log max length=500
Log sessions=1
Detailed log sessions=0
Log System Info Interval=10
MMS block if internal error=1
SessionLog To EventLog=0

Increase and rotate SSH log files in Checkpoint

Log is knowledge (of who did what) and knowledge is power. All modern Operating Sytems today provide extensive logging facilities, and Linux on which Checkpoint products are based is no exception.
Checkpoint have own logging capabilities where logs are enabled in Security rulebase. And that’s fine, but SSHD daemon also has  logging and rotating feature which by default is limited to retain only last 4 logfiles.
I found it very useful to have ssh access logs for a longer period, especially when client also has access to the firewall and does changes on his/her own.
To tune parameters of the SSH logging edit /etc/cpshell/log_rotation.conf (no need to restart anything) :

# cat /etc/cpshell/log_rotation.conf
# File    max  size    backlog
# By default max size is 65536 bytes and backlog (how many files to rtain) is 4, I usually change it to the values before:
/var/log/messages 65536 256
/var/log/routing_messages 64536 256
/var/log/wtmp 65536 256
/var/log/lastlog 262400 256
/var/log/secure 64536 256
$CPDIR/log/cpstart.log 1048576 4
# Files after this line will not be shown by log command
/var/log/CPbackup.elg 64536 4
/var/CPbackup/log/backup_logs.elg 64536 4
$FWDIR/log/fwd.elg 1048576 4
$FWDIR/log/dtlsd.elg 1048576 4

< CUT here —————————– >

fw monitor command reference

This is a quick reference sheet of all usable options for the fw monitor tool .At the end I put a list of fw monitor examples. The previous experience with the tool is assumed, i’ll just say that if you are serious about debugging Checkpoint products learn it and learn it well.
By default the fw monitor sniffing driver is inserted into the 4 locations on
the Firewall kernel chain .
Here they are:

 i (PREIN) – inbound direction before firewall Virtual
Machine (VM, and it is CP terminology) . Most important fact to know about that
is that this packet capturing location shows packets BEFORE any security rule
in the policy is applied. That is, no matter what rules say a packet should at
least be seen here, this would prove that packets actually reach the firewall
at all.
 I (POSTIN) – inbound direction after firewall VM.
 o (PREOUT) – outbound direction before firewall VM,
 O (POSTOUT) – outbound direction after firewall VM.

You can change point of insertion within the fw chain with :

# fw monitor –p<i|I|O|o> <where to
insert>

easiest way to specify where to insert is to first see the chain:
# fw ctl chain
then give relative to any module you see there <+|->module_name

Now the usage itself:

# fw monitor
Usage: fw monitor [- u|s] [-i] [-d] [-T] <{-e
expression}+|-f <filter-file|->> [-l len] [-m mask] [-x offset[,len]]
[-o <file>] <[-pi pos] [-pI pos] [-po pos] [-pO pos] | -p all [-a
]> [-ci count] [-co count]

Round up of options:

-m mask , which point of capture is to be displayed, possible: i,I,o,O
-d/-D debug output from fw monitor itself, not very useful IMO.
-u|s print also connection/session Universal ID
– i after writing each packet flush stdout
-T add timestamp, not interesting
-e expr expression to filter the packets (in detail later)
-f filter_file the same as above but read expression from file
-l <len> packet length to capture

Expressions
On the very low level fw monitor understands byte offsets from the header
start. So to specify for example 20th byte of the IP packet (that is source IP)
you can just use:

# fw monitor -e 'accept [12,b]=8.8.8.8;'

Where:
12 – offset in bytes from the beginning of the packet
b – mandatory, means big endian order.
4 – not seen here but size (in bytes) of how many bytes to look for from the
starting offset (default is 4 )

To look for source port 53 (UDP/TCP) in raw packet:

# fw monitor -m i -e 'accept [20:2,b]=53;'
Here I say to fw monitor to look at 2 bytes at offset 20.

While this way of looking at packets is the most general and therefore includes
all cases, you rarely have the need for such a granular looking glass. In 99%
of the cases you will be doing alright with a limited known set of expressions.
Just for that Checkpoint defined and kindly provided us in every Splat
installation with definition files that give meaningful synonyms to the most
used patterns. There are few definition files but they circularly reference
each other providing multiple synonyms for the same pattern.
I put all those predefined patterns in the list below for the easy to use
reference.

Summary table of possible expressions to be fed to the fw
monitor
Specifying
Hosts
 host(IP_address)  to or from this host
 src=IP_address  where source ip = IP_address
 dst=IP_address  where destination ip = IP_address
 net(network_address,netmask)  to or from this network
 to_net(network_address,netmask)  to this network
 from_net(network_address,netmask)  from this network
 
 Specifying ports
 port(port_number)  having this source or destination port
 sport=port_number  having this source port
 dport=port_number  having this destination port
 tcpport(port_number)  having this source or destination port that is also TCP
 udpport(port_number)  having this source or destination port that is also UDP
 
 Specifying protocols  
 ip_p=<protocol_number_as_per_IANA>  this way you can specifiy any known protocol by its registered
number in IANAFor detailed list of protocol numbers see www.iana.org/assignments/protocol-numbers
 icmp  what it says , icmp protocol
 tcp  TCP
 udp  UDP
 
 Protocol specific oprions  
 IP
 ip_tos = <value>  TOS field of the IP packet
 ip_len = <length_in_bytes>  Length of the IP packet in bytes
 ip_src/ ip_dst = <IP_address>  Source or destination IP address of the packet
 ip_p =<protocol_number_as_per_IANA>  See above
 ICMP
  echo_reply  ICMP reply packets
  echo_req  Echo requests
  ping  Echo requests and echo replies
  icmp_error  ICMP error messages (Redirect,Unreachables,Time exceeded,Source
quench,Parameter problem)
  traceroute  Traceroute as implemented in Unix (UDP packets to high ports)
  tracert  Traceroute as implemented in Windows (ICMP packets , TTL
<30)
  icmp_type = <ICMP types as per RFC>  catch packets of certain type
  icmp_code = <ICMP type as per RFC>  catch packets of certain code
  ICMP types and where applicable respective codes:ICMP_ECHOREPLY
ICMP_UNREACH
      ICMP_UNREACH_NET
      ICMP_UNREACH_HOST
      ICMP_UNREACH_PROTOCOL
      ICMP_UNREACH_PORT
      ICMP_UNREACH_NEEDFRAG
      ICMP_UNREACH_SRCFAIL
ICMP_SOURCEQUENCH
ICMP_REDIRECT
      ICMP_REDIRECT_NET
      ICMP_REDIRECT_HOST
      ICMP_REDIRECT_TOSNET
      ICMP_REDIRECT_TOSHOST
ICMP_ECHO
ICMP_ROUTERADVERT
ICMP_ROUTERSOLICIT
ICMP_TIMXCEED
      ICMP_TIMXCEED_INTRANS
      ICMP_TIMXCEED_REASS
ICMP_PARAMPROB
ICMP_TSTAMP
ICMP_TSTAMPREPLY
ICMP_IREQ
ICMP_IREQREPLY
ICMP_MASKREQ
ICMP_MASKREPLY
  icmp_ip_len = <length>  Length of ICMP packet
 icmp_ip_ttl = <TTL>  TTL of ICMP packet, use with icmp protocol otherwise will catch ANY
packet with TTL given
 < cut here—-bunch of other icmp-related fields
like ID ,sequence I don’t see any value in bringing here–>
 
 TCP
 syn  SYN flag set
 fin  FIN flag set
 rst  RST flag set
 ack  ACK flag set
 first  first packet (means SYN is set but ACK is not)
 not_first  not first packet (SYN is not set)
 established  established connection (means ACK is set but SYN is not)
 last  last packet in stream (ACK and FIN are set)
 tcpdone  RST or FIN are set
 th_flags – more general way to match the flags inside
TCP packets
 th_flags = TH_PUSH  Push flag set
 th_flags = TH_URG  Urgent flag set
 UDP
 uh_ulen = <length_in_bytes>  Length of the UDP header (doesnt include IP header)

 

And the last thing to remember before we move to examples – expressions support logical operators and numerical values support relative operators:

and – logical AND
or – logical OR
not – logical NOT
> MORE than
< LESS than
>= MORE than or EQUAL to
<= LESS than or EQUAL to
You can combine logical expressions and influence order by using ()

Below is laundry list of examples to showcase the reference table above.

# fw monitor -m i -e 'accept host(208.44.108.136) ;'
# fw monitor -e 'accept src=216.12.145.20 ;'  packets where source ip = 216.12.145.20
# fw monitor -e 'accept src=216.12.145.20 or dst= 216.12.145.20;'  packets where source or destination ip = 216.12.145.20
# fw monitor -e 'accept port(25) ;'  packets where destination or source port = 25
# fw monitor -e 'accept dport=80 ;'  packets where destination port = 80
#fw monitor -e 'accept sport>22 and dport>22 ; '  packets with source and destination ports greater than 22
# fw monitor -e 'accept ip_len = 1477;'  packets where their length equals exactly 1477 bytes
# fw monitor -e 'accept icmp_type=ICMP_UNREACH;'  ICMP packets of Unreachable type
# fw monitor -e 'accept from_net(216.163.137.68,24);'  packets having source IP in the network 216.163.137.0/24
# fw monitor -e 'accept from_net(216.163.137.68,24) and port(25) and dst=8.8.8.8 ;'  packets coming from network 216.163.137.0/24 that are destined to the host 8.8.8.8 and hving source or destination port = 25
# fw monitor -m i -x 40,450 -e 'accept port(80);'  incoming packets before any rules are applied also
display contents of the packet starting at 40th byte of 450 bytes length

# fw monitor -m i -pi -ipopt_strip -e 'accept host(66.240.206.90);'  incoming packets from/to host 66.240.206.90 , insert sniffer before module named ipopt_strip
# fw monitor -D -m i -pi -ipopt_strip -e 'accept host(66.240.206.90);'  same as above but add debug info

Resources:
PDF version of fw monitor command reference

Checkpoint Connectra and Internet Explorer 7 and 8

With the arrival of Internet Explorer 7 and 8 remote users connecting to anything SSL-related
have to explicitly click on the warning message link to continue browsing . It also includes
Remote users connecting by SSL to Checkpoint Connectra that works with self-signed certificate .
Sounds like a minor nuisance but … certificate error HTML page that Connectra presents to the
connecting user is too small to click there on anything and to make things worse is NOT resizable.
There are probably more sofisticated ways to solve it , but I solved it simply –
edited HTML template Connectra uses to create this page.

Below I changed resizable from “0” to “1”
[Expert@fw]# pwd
/var/opt/CPsuite-R65/fw1/conf/extender

[Expert@fw]# grep ‘resizable’ index.html
                        extender = window.open(‘pre_login.html’,’pre_login’,’width=155,height=133,menubar=0,resizable=1,scrollbars=0,status=0,titlebar=no’);
                        extender = window.open(‘extender.html’,’pre_login’,’width=438,height=324,menubar=0,resizable=1,scrollbars=0,status=0,titlebar=no’);
        help_win = window.open( “help.html”, “Help”, “toolbar=0,status=0,width=800,height=600,scrollbars=0,resizable=0” );

PS. Of course you always have the option of buying a certificate for the Connectra from recognized authority , say Thawte, then IE will not pop up the warning window.

copy http flash – download from HTTP server to the Cisco router

 The feature to download anything (mostly used to download IOS images) from remote HTTP server to the cisco router has
been with us for years, yet there are few caveats to be aware of before using it.
The command itself is pretty simple:
Router# copy http[:full URI specification]  flash[: local path to save the file]

The facts you should know:

– router is first doing resolving of the domain name to the IP, then uses this IP as Host header in the  communication with
the remote HTTP server. This is important when you try to download something from the webserver already configured
for the Virtual hosts. Because then webserver looks at this header and searches for the matching local file according to
its internal logic.
For example if using Apache configured for named Virtual hosting you should put the file to be downloaded in
the default Virtual host, i.e. first virtual host in the Apache configuration file. Let’s look at the example.
Here we have  the partial Apache config file :
#The file we want to download is in /usr/local/apache2/htdocs/mrtg/test.bin

#Here comes the 1st VirtualHost entry
<VirtualHost *:80>
   ServerAdmin  admin@yurisk.net
   DocumentRoot “/usr/local/apache2/htdocs/mrtg”
# as this this the 1st Virtual Host entry server names below are irrelevant for our case
   ServerName mrtg.yurisk.info
   ServerAlias mrtg. yurisk.net
   ErrorLog “logs/mrtg.yurisk.info-error_log”
   CustomLog “logs/mrtg.yurisk.info-custom_log” common
<Directory />
   Options FollowSymLinks
   AllowOverride None
#Here I set up a basic authentication with local user/pass file, you may omit this
       AuthType  Basic
       AuthName  “By My Invitation only :)”
       AuthUserFile /usr/local/apache2/passwords
       Require valid-user
       Options None
#Uncomment below if not using the authentication
#    Order allow,deny
#    Allow from any

</Directory>

<VirtualHost *:80>
——-Cut here – many more virtual hosts ——

– while using TCP with built-in packet verification generally prevents damaged downloads , it is always a good idea to verify with md5
sum the downloaded file. The command:

#verify /md5 flash:<downloaded file name>
– This command also supports copying from HTTPs, but it would add unwanted SSL encrypt/decrypt overload
so I haven’t tested it , yet.

Now the real life example:

  Tair#copy http://qwerty:12345@ 214.90.51.41/test.bin flash
Destination filename [test.bin]?
Loading http:// qwerty:12345@ 214.90.51.41/test.bin
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
5120000 bytes copied in 17.924 secs (285651 bytes/sec)
Tair # verify /md5 flash:test.bin
…………………………………………………………………………………………………………………………………………………………………….

……………………………………………………………………………………………………………………………………………………………………..

……………………………………………………………………………………………………………………………………………………………………..

……………………………………………………………………………………………………………………………………………………………………..

……………………………………………………………………………………………………………………………………………………………………..

………………………………………………………………………………………………………………………………………………………………………

…………………………………………………………………………………………………………..Done!
verify /md5 (flash:test.bin) = e8c39d44aafc82b035dfc7ad16fc2183

SSH login alert by mail Linux or Unix based systems

As you may have noticed many security-related software/appliances are based on Linux or Unix operating
systems in their variety. And as the logical consequence of that remote managing of such devices is done with OpenSSH
package  .  What is lacking in these applications built on Linux/Unix platforms is alerting in real–time on successful
SSH login to the system . e.g eSafe can alert only on login to the software itself (i.e. econsole), the same goes for the Checkpoint firewall
. On the other hand SSH login to the system ultimately means superuser/root access that gives control over the whole
system. To fix it I wrote the following script. This script sends mail to predefined email address each time someone
successfully logs in by SSH to the machine.
I take advantage here of the built-in feature of the  OpenSSH daemon – if you create text file containing commands (as if you typed
them on the command line), and name it either /etc./ssh/sshrc or <user home dir>/.ssh/rc , these commands in file will be run each time user logs in through SSH daemon to the system.
The file has to be readable by the user logging in through SSH.
Note 1:
file /etc/ssh/sshrc is applied globally to any user logging in, unless:
Note 2:
file <user home dir>/.ssh/rc   overrides action of /etc/ssh/sshrc . Caveat here – it is enough for a user to put in his home .ssh directory
empty file named rc and it will disable /etc/ssh/sshrc including mail alerts sent from it. Actually it is not that big of an issue as you may
create rc file in the home directory of the user yourself, give it 644 permissions and while user will know what is going on when doing ssh login he/she won’t be able to do anything about that.

So to script itself.
Here:
yurisk@yurisk.info –   mail to which I get mail alert
mail.yurisk.info   –   mail server that accepts mails destined for yurisk.info domain (its MX record)
SENDING_HOST   –  hostname of sending host, will be included in the subject so later I can create mail inbox rule to pay appropriate attention   to such mails
USER_ID     – output of the #id command so I will also be able to filter incoming messages on the user logged in

freeBSD# cat /etc/ssh/mail_alert.awk
BEGIN {
# Set up some info to be included in the mail
# As you see I prefer to use absolute pathnames , but you don’t have to
# Find the hostname to which SSH login happened , to be included in the Subject
"/bin/hostname" | getline SENDING_HOST
# FInd ID of logged
"/usr/bin/id" | getline
USER_ID = $1
SMTP = "/inet/tcp/0/mail.yurisk.info/25"
RS = ORS = "\r\n"
print "helo yurisk.info"     |& SMTP
SMTP                       |& getline
print "mail from: <yurisk@yurisk.info>" |& SMTP
SMTP                       |& getline
print  "rcpt to: <yurisk@yurisk.info>"  |& SMTP
SMTP                       |& getline
print   "data"             |& SMTP
SMTP                       |& getline
print  "Subject:SSH login alert – user " USER_ID “logged in " SENDING_HOST  |& SMTP
print                       |&  SMTP
"/usr/bin/w" | getline
print  $0                  |& SMTP
print   " He is most free from danger, who, even when safe, is on his guard  "               |& SMTP

print   "  "               |& SMTP
print  "."                 |& SMTP
print                      |& SMTP

print  "quit"              |& SMTP

}
– Now the file that is checked on each login for commands ( I put both files  in /etc/ssh/) :
freeBSD# cat /etc/ssh/sshrc
awk -f /etc/ssh/mail_alert.awk > /dev/null
Note for FreeBSD (I guess any *BSD) users:  in *rc file above you will have to replace awk with gawk, as in *BSD systems awk behaves as the old-style Unix awk that has no bidirectional pipe to connect to mail server.

PS. You might be asking why awk here ? True, Linux/Unix have perfect tool for sending mails called #mail, but I did it with awk
for  a reason – not on every (especially if hardened) system you will find mail/telnet/etc utilities with which sending mails is more simple and more reliable. The biggest one is Checkpoint firewall – it has NO mail or telnet clients, neither scripting language beyond AWK and Bash.

The downside of awk is that it is not perfect for more or less complex protocols. So script may stuck / send commands too fast/ etc and therefore be disconnected by the server.

Also if mail server uses greylisting – this script won’t understand it. So check it in interactive session before using. If time permits later I will polish it a bit to count for such cases.

BTW If you haven’t noticed eSafe has full-blown scripting languages installed – Perl and Python . With these you are limited by your imagination only.

« Older posts Newer posts »

© 2016 yurisk.info

Theme by Anders NorenUp ↑