yurisk.info » IOS Cisco

Difference between ebgp-multihop and ttl-security.

Yuri — Fri, 26 Feb 2010 18:39:12 +0000

Once upon a time reading some CCIE paper at work I asked myself a question : “Why would someone bother to invent ttl-security and even write RFC http://tools.ietf.org/html/rfc5082 on it when multi-hop EBGP feature provides the same end result ?” .
The results of my busy/doing-nothing activity I present here.
First some background. For some (unknown to me) reasons BGP peering was envisioned as TCP connection between directly connected routers, by default. To proceed with this design (worth checking BGP RFCs if it was actually an obligation) vendors (Cisco,Juniper and even Fortinet) implemented all BGP protocol communication using TTL=1 in TCP packets being exchanged. As the logical consequence of this if a router was placed more than 1 hop away from its peer BGP session could not be established. To provide for such set ups when peers are many hops away the ebgp-multihop term was coined – on configuration level you can specify that BGP peer is that hops far away .
What happens in fact is that when you specify such multi-hop BGP peer the router starts sending BGP packets with TTL being equal to the number of hops you set . That means if I set peer to be 3 hops away and some attacker tries to spoof legit peer’s IP but is 4 hops away – such attack won’t succeed cause my router will receive spoofed BGP packets ok but will send replies with TTL of 3 which will expire just 1 hop away from the attacker.
Questionable , but security . So why ttl security?
This feature indeed enforces that BGP peer is no more than given hops away . And here comes the difference – it enforces it inbound . It works this way – after you enable ttl security on the BGP peer session and specify how many hops away this peer is allowed to be, your router
checks incoming TCP packets from this peer and does this simple calculation ; configured value <= 255 – hops-away-to-peer , if it holds true your router goes on with establishing BGP session , if not – session is shut down. Regarding outgoing TTL values – may be it is Cisco-only thing, may be not , but the moment you enable ttl security for some BGP peer on Cisco the router itself starts sending BGP-related packets to this peer with initial ttl being equal to 255. I guess it is logical that if you enforce on your side ttl security the peering side will want to do the same.

When ttl rule is broken we see in the debug session:
Dec 27 19:08:04.103: %BGP-4-INCORRECT_TTL: Discarded message with TTL 1 from 124.2.11.15
And neighbor status is:
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
124.2.11.15 4 13462 33 63 0 0 0 00:04:31 Idle

#sh ip bgp neighbors 124.2.11.15
BGP neighbor is 124.2.11.15, remote AS 13462, external link
BGP version 4, remote router ID 0.0.0.0
BGP state = Closing

Capture packets at IOS Cisco router or finally we have a sniffer

Yuri — Mon, 01 Feb 2010 07:33:06 +0000

Finally it is here – built-in sniffer on the Cisco IOS platform ! Starting IOS 12.4(20) release Cisco introduces brand new feature
called Embedded Packet Capture (EPC) that allows us to capture raw packets on the Cisco router and then later analyze it offline.
It can capture any traffic passing through the router, destined to it, or originated from it . The captured packets are stored in DRAM
of the router from where you can upload the capture file using HTTP/SCP/HTTPS/TFTP/FTP anywhere and then dissect it. The capture
is stored using PCAP format , so any protocol dissector will understand this file, including the favorite one WIreshark/Ethereal.
Now some limitations:
– CEF has to be enabled on the router;
– The capture is stored in the DRAM , so you’d better have enough of it;
– While no maximum capture buffer or packet size is stated I guess it depends on the platform (see tests below);
– IOS has to be 12.4(20) or higher.
Let’s now look at steps to configure the capture on the router and then look at the results.
Configuration involves 5 steps:

Create named capture buffer in router memory (including filters what to capture and what not to). Multiple buffers simultaneously are supported;
Create named capture point , again, multiple capture points active at the same time are possible. Using multiple capture buffers and capture points gives us full flexibility in the process – I can say capture packets at the same time inbound on incoming interface and
store it to the memory buffer A while the same traffic going outbound on outgoing interface capture to another buffer B and have this
way capture of the same traffic at 2 distinct points on the router. Your imagination is the limit here.
Associate capture buffers with capture points;
Start/stop capture;
Export captured packets as PCAP file elsewhere or see it in raw format on the router itself (in case binary is your first language).

Now I will walk through configuring,all this is being done on Cisco 2821 (250 Mb of DRAM). IOS is being Cisco IOS Software, 2800 Software (C2800NM-IPBASEK9-M), Version 12.4(24)T1, RELEASE SOFTWARE (fc3)
1) Create named capture buffer in memory. Packets are stored there, as this is DRAM storage if router does restart all capture data will be lost. You also specify filter for which packets are to be captured , if none given it will capture ALL packets at the
capture point. Not surprisingly for filtering you use access-lists, standard or extended, named or numbered.

In my testing I am trying to capture all SMTP traffic passing through the interface Giga0/1. Accordingly the ACL for it will be:

Eldorado(config)#ip access-list extended MAIL_TEST
Eldorado (config-ext-nacl)# permit tcp any any eq smtp

Now I create capture buffer in the memory:

Eldorado #monitor capture buffer MAIL filter access-list MAIL_TEST

NOTE: Fo the particular platform (cisco 2821) the limits and defaults for the buffer are these:

Eldorado#monitor capture buffer MAIL size ?
<1-5127gt; Buffer size in Kbytes : 512K or less (default is 256K)

NOTE 2: In Cisco.com documentation this and other commands related to capture have options that trying to use them gave error.
For example Command reference gives option to configure length of the packet to be captured (instead of default 68 bytes) :

Eldorado #monitor capture buffer MAIL length
^
% Invalid input detected at ‘^’ marker.

2) Creating capture point (i.e. where to capture packets on the router):

Eldorado# monitor capture point ip cef GIGA GigabitEthernet0/1 both

Here I specify interface GigabitEthernet0/1 as point of capture and also set that traffic is to be captured in both directions (or you can use in/out instead)

3) Associate capture buffer with capture point (it does not start capture yet):

Eldorado#monitor capture point associate GIGA MAIL

4) Start capturing packets:

Eldorado#monitor capture point start GIGA

4.1)Stop capture (optional) , you can export capture in the next step without stopping it:

Eldorado# monitor capture point stop GIGA

5) Export captured packets as file to external server , here I use SCP as protocol:

Eldorado#monitor capture buffer MAIL export scp://rumba@216.163.142.1:/capture.cap
Writing capture.cap
Password:
Sink: C0644 309346 capture.cap
!!
Eldorado#

- Now you can see the capture file with Wireshark .
There is it to it.

Verifying.
– To see parameters of the capture:

Eldorado#show monitor capture buffer all parameters

 Capture buffer size (linear buffer)
Buffer Size : 262144 bytes, Max Element Size : 68 bytes, Packets : 0
Allow-nth-pak : 0, Duration : 0 (seconds), Max packets : 0, pps : 0
Associated Capture Points:
Configuration:
monitor capture buffer siz
Capture buffer MAIL (circular buffer)
Buffer Size : 512000 bytes, Max Element Size : 1024 bytes, Packets : 363
Allow-nth-pak : 0, Duration : 0 (seconds), Max packets : 0, pps : 0
Associated Capture Points:
Name : GIGA, Status : Inactive
Configuration:
monitor capture buffer MAIL size 500 max-size 1024 circular
monitor capture point associate GIGA MAIL
monitor capture buffer MAIL filter access-list MAIL_TEST
Eldorado#

- Seeing contents of the captured packets on the router :

# show monitor capture buffer MAIL dump

08:18:59.995 UTC Jan 25 2010 : IPv4 LES CEF    : Gi0/1 None

45514C50:                            002414F7              .$.w
45514C60: 2723001F 9E4cd37F 03e4cda dd379aaa  'A...F&...E..0W.
45514C70: dd379aaa dd379aaa  dd379aaa dd379aaa @.-.Eב#X.3,,.M%
45514C80: 03e4cda 03e4cda 03e4cda  03e4cda  ../..&....s@yh
45514C90: 00000204 23ee3444 000000             .....d.....

08:19:00.699 UTC Jan 25 2010 : IPv4 LES CEF    : Gi0/1 None

45514C50:                            002414F7              .$.w
45514C60: 03e4cda 03e4cda 03e4cda 03e4cda 'A...F&...E..KYj
45514C70: 03e4cda 03e4cda 03e4cda 03e4cda @.-.#4$f.%%
45514C80: 03e4cda 03e4cda 03e4cda 03e4cda../..'|S^^^0])
45514C90: 03e4cda 03e4cda 03e4cda 03e4cda..EHLO smtp02.bi
45514CA0: 03e4cda 03e4cda 03e4cda 03e4cda s.eu.blackberry.
45514CB0: 636F6D0D 0A00                        com...

IP Options are evil – drop them , drop them on Cisco Asa/IOS Microsoft ISA Juniper or Checkpoint

Yuri — Sat, 23 Jan 2010 19:51:22 +0000

As you probably noticed IP header has variable length placeholder for the IP Options field. It has been there since the beginning , once a good idea for debug now turned into trouble. RFC 791 states that hosts/routers supporting IP protocol must implement Ip Options filed . It is up to the vendor to decide what to do with this optional field, but it must understand it. Still, wouldn’t be a problem if not modern architecture of the routing equipment that was designed to do most efficiently Routing , i.e. pass from interface to interface gigabytes of traffic. Therefore routing functions are highly optimized and most of the time are implemented in hardware . All other types of traffic unfortunately are not, and in most of the cases processing , lets call it Control traffic, is being left to poor router CPU and done in software. That brought the troubles into the IP world – relatively small amounts of control traffic (including Ip Options packets) may bring down otherwise
powerful router in just minutes.
To prevent this attack vendors implemented protection measures to drop entirely or selectively IP packets that has Ip Options filed set. Below is quick cheat sheet how to do it in some gear :

Checkpoint firewall NG/NGX – packets with Ip Options are dropped by default except for the “Router Alert” option (0×94) for the IGMPv2 and PIM protocols [or so CP claim, will have to verify later] and not even logged. To start logging dropped packets go to Policy -> Global Properties -> Log and Alerts -> check Ip dropped packets : Log

There is a value related to it that is on by default : Global Properties -> SmartDashboard customization -> Advanced Configuration -> Configure -> Firewall 1 -> Stateful inspection -> enable_ip_options (check/uncheck) but unchecking it removes from firewall VM chain module that inspects these Options at all and all Ip Options packets are dropped . So all packets bearing Ip Options are happily dropped even before security rules , here:

[Expert@splat60]# fw ctl chain
in chain (9):
0: -7f800000 (9095dd60) (ffffffff) IP Options Strip (ipopt_strip)
1: – 1fffff6 (9095ee80) (00000001) Stateless verifications (asm)

Also Checkpoint say you can decide which Ip Options will be allowed later BUT only when installing the firewall: “The set of permitted options must be configured during installation … the enable_ip_options setting in SmartDashboard is then used to enable or disable this functionality. Contact Check Point support for instructions on configuring the set of allowed IP options.”

Microsoft ISA 2000 server:
– If Enable Packet Filtering is not checked then do it in IP Packet Filters -> Properties – > General tab. On the Packet Filters tab check Enable Filtering IP Options .
Microsoft ISA 2004 Server:
- IP options filtering is enabled by default
- Go to Configuration node of the server in question in Management console -> General -> Additional Security Policy
Define IP Preferences . Here you will have 3 options to deal with Ip Options packets:
a) Deny packets with any IP options;
b) Deny packets with selected IP options;
c) Deny packets with all except selected IP options
The same options are available in ISA 2006 , click on Configure IP Protection link – > IP Preference settings

IOS Cisco router :
see my other blog – to be filled later
Cisco ASA :
see my other blog – to be filled later

Juniper router:
You just add ip-options term to the filter and apply it to the interface of interest. In the example below I block only Route Record type of Ip Options, if you use any then it will block any type:

[edit firewall family inet filter NOICMP term 3]

firewall {
    family inet {
        filter NOICMP {
            term 1 {
                from {
                    address {
                        192.168.2.100/32;
                    }
                }
                then {
                    reject;
                }
            }
            term 2 {
                from {
                    ip-options route-record;
                }
                then {
                    reject;
                }
            }
            term 3 {
                from {
                    address {
                        192.168.2.0/24;
                    }
                }
                then accept;
            }
        }
    }
}

Apply to the interface:

interfaces {
    em0 {
        unit 0 {
            enable;
            family inet {
                filter {
                    input NOICMP;
                }
                address 192.168.2.133/24;
            }
        }
    }

Other possible arguments to ip-options clause:

set term 3 from ip-options ?

Possible completions:

              Range of values
  [                    Open a set of values
  any                  Any IP option
  loose-source-route   Loose source route
  route-record         Route record
  router-alert         Router alert
  security             Security
  stream-id            Stream ID
  strict-source-route  Strict source route
  timestamp            Timestamp

Windows 2008.
By default it doesnt allow/forward packets with Source Routing set, and that's good. For completeness
here is how to enable (or check whether it is enabled) source-routed forwarding:
BillG> netsh interface ipv4 set global sourceroutingbehavior=drop| forward| dontforward
- or-
Registry:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameter
Key: DisableIPSourceRouting
DWORD value: 0

Verify:
In Security any measure/protection/method is as good as the proof you can present that it actually works.
Windows:
- Ping with Record Route field set:
BillG> ping –r 9 192.2.2.1
- Ping with Strict Routing field set:
BillG> ping –k <1st_hop_router_IP> <2nd_hop_router_IP…>
- Ping with Loose Routing field set:
BillG> ping -j <1st_hop_router_IP> <2nd_hop_router_IP…>
- Ping with Timestamp option set:
BillG> ping –s 3 8.8.8.8
Linux:
- Ping with Record Route field set:
root@darktstar:~/nmap# ping -R 8.8.8.8
- Ping with Timestamp option set:
root@darkstar:~/nmap# ping -T tsonly 8.8.8.8
Linux,BSD,Unix :
This handy utility sends bunch of packets to the target to test what Ip Options the target supports:
freebsd# fragtest ip-opt 192.168.2.133
ip-opt: sec lsrr ts esec cipso satid ssrr
I run fragroute above against Juniper (8.3) that was configured in the example earlier to block only Record Route option, as you can see it is indeed missing in the output list that enumerates what Ip Options the target supports [ see Reference for fragroute details]

References for further details:
Juniper: JUNOS Enterprise Routing, 1st Edition, By Doug Marschke; Harry Reynolds, 2008
Microsoft ISA : Microsoft® ISA Server 2006 Unleashed ,By Michael Noel, 2007
Fragroute http://monkey.org/~dugsong/fragroute/
Windows 2008: Windows® Server 2008 TCP/IP Protocols and Services,By Joseph Davies, 2008

Cisco log: Missing cef table for tableid 65535 during CEF samecable event

Chen — Thu, 21 Jan 2010 10:39:19 +0000

Today I’ve noticed some strange error on my Cisco 1841 router :

%FIB-4-FIBCBLK: Missing cef table for tableid 65535 during CEF samecable event

After searching the net, i’ve found some Cisco bug that describes this.
“FIB-4-FIBCBLK errors with dns view
Symptoms

Message “%FIB-4-FIBCBLK: Missing cef table for tableid 65535 during CEF samecable event” displayed on the console logs.

Conditions

The message seems to be generated anytime a dns request is made to the router where the router then has to use the dns forwarder

Workaround
No workaround”
(Source)

This happens when you have DNS server on the device and it needs to grab the answer from the device configured DNS server (A.K.A DNS Forwarder), and each request will cause this error log.

According to Cisco, the affected device list does not include my MD release – 12.4(25b), however i do see it . (List)

List of IOS with the Fix :
12.2(33)XNE
12.4(24.6)T1
12.4(15)T9
12.2(32.8.11)SR183
12.2(32.8.1)REC186
12.4(20)T3
12.2(33.1.3)MCP5
15.0(1)M
12.4(24)T1
12.2(32.8.1)REE186
12.4(22)T2
12.4(22)MDA1
12.4(24)YG
12.4(24)GC1
12.4(22)XR
12.4(24)MD
12.4(22)YE2

copy http flash – download from HTTP server to the Cisco router

Yuri — Tue, 20 Oct 2009 19:06:33 +0000

The feature to download anything (mostly used to download IOS images) from remote HTTP server to the cisco router has
been with us for years, yet there are few caveats to be aware of before using it.
The command itself is pretty simple:
Router# copy http[:full URI specification] flash[: local path to save the file]

The facts you should know:

- router is first doing resolving of the domain name to the IP, then uses this IP as Host header in the communication with
the remote HTTP server. This is important when you try to download something from the webserver already configured
for the Virtual hosts. Because then webserver looks at this header and searches for the matching local file according to
its internal logic.
For example if using Apache configured for named Virtual hosting you should put the file to be downloaded in
the default Virtual host, i.e. first virtual host in the Apache configuration file. Let’s look at the example.
Here we have the partial Apache config file :
#The file we want to download is in /usr/local/apache2/htdocs/mrtg/test.bin

#Here comes the 1st VirtualHost entry

   ServerAdmin admin@yurisk.net
   DocumentRoot “/usr/local/apache2/htdocs/mrtg”
# as this this the 1st Virtual Host entry server names below are irrelevant for our case
   ServerName mrtg.yurisk.info
   ServerAlias mrtg. yurisk.net
   ErrorLog “logs/mrtg.yurisk.info-error_log”
   CustomLog “logs/mrtg.yurisk.info-custom_log” common

   Options FollowSymLinks
   AllowOverride None
#Here I set up a basic authentication with local user/pass file, you may omit this
       AuthType Basic
       AuthName “By My Invitation only ”
       AuthUserFile /usr/local/apache2/passwords
       Require valid-user
       Options None
#Uncomment below if not using the authentication
#    Order allow,deny
#    Allow from any

——-Cut here – many more virtual hosts ——

- while using TCP with built-in packet verification generally prevents damaged downloads , it is always a good idea to verify with md5
sum the downloaded file. The command:

#verify /md5 flash:
- This command also supports copying from HTTPs, but it would add unwanted SSL encrypt/decrypt overload
so I haven’t tested it , yet.

Now the real life example:

Tair#copy http://qwerty:12345@ 214.90.51.41/test.bin flash
Destination filename [test.bin]?
Loading http:// qwerty:12345@ 214.90.51.41/test.bin
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
5120000 bytes copied in 17.924 secs (285651 bytes/sec)
Tair # verify /md5 flash:test.bin
…………………………………………………………………………………………………………………………………………………………………….

……………………………………………………………………………………………………………………………………………………………………..

………………………………………………………………………………………………………………………………………………………………………

…………………………………………………………………………………………………………..Done!
verify /md5 (flash:test.bin) = e8c39d44aafc82b035dfc7ad16fc2183

Tracking the source of DOS attack with Cisco IOS

Yuri — Thu, 28 May 2009 11:23:47 +0000

Problem: Enterprise is under Denial Of Service Attack that brings down key elements of the business or the whole network at all.
To track the attacker is the first step in handling the attack and unless the flood is coming from inside (most probably not in a well managed LAN) you will need help of your Service Provider to find out the origin. Unfortunately Service Provider’s (SP) backbone is not well suited for such forensics, as its business role is
to provide uninterrupted connectivity to ALL the clients , not only you, so SP will not enable ACLs/ip accounting/Netflow on their backbone to identify where the attack is coming from . And if source Ip of the attack is spoofed you can’t do much .

For such cases Cisco came with the nice feature called
ip source tracking that will gather flow statistics for specific destination
IPs (of victim) and periodically will export them for viewing, and will do all this without overloading the backbone router it is enabled on (Of course relevant if your SP is using Cisco gear) . Here are details:

– Enable it globally for the victim IP , here IP being attacked is 63.45.33.22

Edge(config)#ip source-track 63.45.33.22

- If you want (and if this is being done by SP they will not) you may create log entries:
Edge1(config)#ip source-track syslog-interval 2
Then you will see in logs (good for reminding to disable this afterwards) :
May 28 10:55:47.105: %DOS_TRACK-5-CFG: IP Source Tracker configured for 1 hosts

- Also you may define how often to export gathered info to be viewed (seems to depend on the platform ) :

Edge(config)#ip source-track export-interval 60

- And finally , you see the data accumulated so far :

Edge#sh ip source-track
Address SrcIF Bytes Pkts Bytes/s Pkts/s
63.45.33.22 Fa0/0 141G 485M 8244 141

Most important here will be the Source interface (in this router there is only 1 ingress interface , in real backbone you will have few feeds) where you see most of the incoming traffic for this destination IP. Then you (SP) would go to the upstream router connected to this local interface, enable the same source tracking and so on. Up to the last point in the backbone where the attacking traffic enters
the backbone of SP out of some upstream SP . Then SP would have option to contact the abuse of this upstream provider for them to investigate the issue further, or at least divert the attack to the black hole at the entry point, so end client would not be affected at all.

Guarding against brute force attack on VTY in Cisco IOS

Yuri — Fri, 10 Oct 2008 09:24:03 +0000

Cisco starting IOS 12.3 introduced a simple but powerful feature to guard against brute force password guessing attack on remote access. The usual template followed when configuring VTY access is:
1) Configure ACL containing management IPs to be allowed to access the router through VTY
2) (Optional) Restrict VTY access protocol to ssh only (transport input ssh)
3) Apply this ACl to VTY : (config-line)# access-class in
4) (Optional) SIngle out one VTY line for a special remote access IP to be used if all VTY lines
are currently in use: (config)# line vty 4
Now I enhanced this template with following features:
#Blocks login for 300 seconds after 5 failed logins within 50 seconds time interval

login block-for 300 attempts 5 within 50
#apply specified ACl to VTY line when above event occurs, it is meant to exempt
#your managemnt IP form being blocked. After timed block expires this ACL gets removed
#from VTY and previous ACL that was applied before the event is reapplied back

login quiet-mode access-class anti-DOS

#Logging rate-limitation to prevent cluttering logs with failed attempts
login on-failure log every 10

ip access-list standard anti-DOS
permit 193.193.193.33
remark Deny VTY access to anyone else if brute-force logins take up all VTY lines

Another nice feature is delay between login attempts:
Sacramento(config)#login delay 2
Delay login is in seconds

Then in logs you will see the following failed attempts:

*May 2 02:04:14.105: %SEC_LOGIN-4-LOGIN_FAILED: Login failed [user: ] [Source: 62.141.52.141] [localport: 22] [Reason: Login Authentication Failed] at 05:04:14 Sat May 2 2009 *May 2 02:04:22.112: %SEC_LOGIN-1-QUIET_MODE_ON: Still timeleft for watching failures is 22 secs, [user: ] [Source: 62.141.52.141] [localport: 22] [Reason: Login Authentication Failed] [ACL: anti-DOS] at 05:04:22 Sat May 2 2009 *May 2 02:09:22.091: %SEC_LOGIN-5-QUIET_MODE_OFF: Quiet Mode is OFF, because block period timed out at 05:09:22 Sat May 2 2009