Fortigate BGP cookbook of example configuration and debug commands


Last updated: May 2020

BGP with two ISPs for multi-homing, each advertising default gateway and full routing table. Uses route-map, prefix list, weight
Prevent our Fortigate from becoming a transit AS, do not advertise learned via eBGP routes. Uses route-map, aspath-list
Force FG1 to advertise default route without having one in RIB and without using blackhole routing. Uses default-originate
Limit announced connected routes to 3.3.3.3 only. Uses route-map with redistribution
Secure BGP session between ISP1 and FG3 with one way hash. Uses MD5 authentication
Make sure we can see received routing advertisements before and after any filtering is applied. Uses soft reconfiguration
Set up BGP peering between FG3 and FG1 using loopback in FG3
[Remotely Triggered Black Hole Routing configuration]

BGP with two ISPs for multi-homing, each advertising default gateway and full routing table

Task: Configure 2 BGP peerings with different providers, each ISP advertising to us (FG3, AS 1680) both, default and Internet routes. Limit the learned routes from each ISP to default route only. Advertise to both ISPs our internal network of 10.10.10.1, making sure clients on the Internet prefer ISP1 (AS 111) to reach this network. Also we want to use ISP1 to reach the Internet, and only if it fails to use ISP2.

Solution.

The BGP configuration flow in general is:

fortigate bgp cookbook configuration flow

The topology of this case:

Fortigate BGP case 1 network topology diagram

FG3, AS1680:

  • Create prefix list to allow ONLY default route (0.0.0.0/0) and deny everything else.
config router prefix-list
    edit "accept-dflt-only"
        config rule
            edit 1
                set prefix 0.0.0.0 0.0.0.0
                unset ge
                unset le
            next
        end
    next
end
  • Prefer ISP1 to reach the Internet, having ISP2 as backup in case of failure. The easiest way to do so is via weight setting, which can be used inside config neighbor to set the weight for ALL routes learned from this neighbor. Or it can be used by first config route prefix-list to match specific route(s), then setting the weight for these specific matched routes inside config router route-map, which in turn will be applied to the neighbor. The other way would be to increase Local Preference of the routes learned from ISP1, but this would require to configure route-map, an additional extra step.
    Here we are not trying to prefer specific routes via ISP1 but all routes learned from it, so I will set weight on the neighbor.

  • The next step is to make sure my advertised route 10.10.10.1 is reachable via both ISPs, but is preferred by Internet clients via ISP1. Usually you do it by prepending your own AS number to the advertised route(s). I create route-map to do so:

config router route-map
    edit "prepend-out"
        config rule
            edit 1
                set set-aspath "1680 1680"
            next
        end
    next
end
  • Now I can configure both BGP peers on FG3, including redistributing the connected networks (here it is 10.10.10.1/32 of the loopback interface) to BGP:
config router bgp
    set as 1680
    config neighbor
        edit "12.12.12.12"
            set prefix-list-in "accept-dflt-only"
            set remote-as 111
            set weight 10
        next
        edit "13.13.13.6"
            set prefix-list-in "accept-dflt-only"
            set remote-as 222
            set route-map-out "prepend-out"
        next
    end
    config redistribute "connected"
        set status enable
    end

Verification.

As remote peers are not configured yet, the status will be oscillating between Active and Connect:

get router info bgp summary

FG3-AS1680 # get router info bgp summary
BGP router identifier 10.10.10.1, local AS number 1680
BGP table version is 1
1 BGP AS-PATH entries
0 BGP community entries

Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
12.12.12.12     4        111       0       0        0    0    0    never Active     
13.13.13.6      4        222       0       0        0    0    0    never Active     

...

Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
12.12.12.12     4        111       0       0        0    0    0    never Active     
13.13.13.6      4        222       0       0        0    0    0    never Connect   

FG1, FG6.
BGP settings of both peers are almost identical (except local to each AS number and FG3 peering IP) so I will list just FG1. One note: unlike in FG3, which distributes into BGP the directly connected loopback 10.10.10.1, I need both Fortigates here to advertise default route 0.0.0.0/0 which they don't have. As I mentioned in the Configuration Flow graph - BGP will only advertise routes present in the active routing table (RIB) by default. The Fortigate has 2 ways to circumvent this BGP standard requirement: we can announce the default route with capability-default-originate, and for other routes we can use set network-import-check disable. But I am not using either of them here.
To satisfy this condition, I add blackhole route to the 0.0.0.0/0 route, in Cisco world it is called "route to Null0". This adds 0.0.0.0/0 as static route which I can redistribute into BGP.
Note 1: Additionally, to simulate "Internet" IPs, I added 8.8.8.8 as loopback in both FG1 and FG6 and redistribute them via redistribute connected.

Note 2: Important point I glossed over in FG3 is router-id. Fortigate (as well as Cisco and most others) will take the highest IP address on the loopback interface available unless explicitly set. In this specific setup I have 8.8.8.8 address on both FG1 and FG6 set on their loopbacks to advertise them as "Internet" addresses to FG3. And this may cause a problem - if any BGP peer detects its own router-id coming from the peer, the BGP session will be torn down with NOTIFICATION sent. So, here it is a must, but generally is a good idea to set router-id manually to unique IP address. I will add unique router-id to FG3 and FG6.

When such situation of duplicate router-id happens, Fortigate will show the error:

BGP: 12.12.12.12-Outgoing [DECODE] Open: Invalid Router ID 8.8.8.8

FG1:

config router static
    edit 1
        set dst 0.0.0.0/0
        set blackhole enable
    next
end

Verification

Note: to save me typing, I add this alias to show routing table:

config system alias
    edit "rt"
        set command "get router info routing all"
    next
end

So when you see it in the output instead of the full command get router info routing all know it is an alias, and not a secret hidden command in Fortigate :).

# alias rt

Routing table for VRF=0
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default

S*      0.0.0.0/0 [10/0] is a summary, Null    <--- This is the default route we want to be present in the RIB and now it is.
C       2.2.2.2/32 is directly connected, Loop1
C       8.8.8.8/32 is directly connected, Loop2
C       13.13.13.0/24 is directly connected, port1
  • Now let's configre the BGP on FG1:
config router bgp
    set as 111
    set router-id 1.1.1.1
    config neighbor
        edit "12.12.12.3"
            set remote-as 1680
        next
    end
    config redistribute "connected"
        set status enable
    config redistribute "static"
        set status enable
    end

Verification

First, let's see if the BGP peering with two ISPs has been established (yes, it has).
On FG3:

FG3-AS1680 #  get router info bgp summary

BGP router identifier 10.10.10.1, local AS number 1680
BGP table version is 7
3 BGP AS-PATH entries
0 BGP community entries

Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
12.12.12.12     4        111     126     297        5    0    0 00:02:35        1
13.13.13.6      4        222     121     288        6    0    0 00:02:12        1

Total number of neighbors 2

Let's see if we are getting default route from both peers:

get router info bgp network 0.0.0.0/0

FG3-AS1680 # get router info bgp network 0.0.0.0/0
BGP routing table entry for 0.0.0.0/0
Paths: (2 available, best #2, table Default-IP-Routing-Table)     <--- Yes, we do
  Advertised to non peer-group peers:
   13.13.13.6                                                     <--- This is not good, read further why
  222
    13.13.13.6 from 13.13.13.6 (6.6.6.6)
      Origin incomplete metric 0, localpref 100, valid, external
      Last update: Wed May 20 12:06:00 2020

  111
    12.12.12.12 from 12.12.12.12 (1.1.1.1)         <--- default route from ISP1
      Origin incomplete metric 0, localpref 100, weight 10, valid, external, best  <--- preferred because its weight is 10
      Last update: Wed May 20 12:05:58 2020                                        <--- the 2nd ISP peer has weight not set, think 0

Now we need to make sure we advertise our network 10.10.10.1 to both peers:

This is what we advertise to ISP1:

FG3-AS1680 #get router info bgp neighbors 12.12.12.12 advertised-routes

BGP table version is 3, local router ID is 10.10.10.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight RouteTag Path
*> 3.3.3.3/32       12.12.12.3                         32768        0 ?
*> 10.10.10.1/32    12.12.12.3                         32768        0 ?
*> 12.12.12.0/24    12.12.12.3                         32768        0 ?
*> 13.13.13.0/24    12.12.12.3                         32768        0 ?

Total number of prefixes 4

Looks good - we advertise 10.10.10.1, as well as other directly connected networks from port1, port2, and loopback.

And what do we advertise to the ISP2?

FG3-AS1680 # get router info bgp neighbors 13.13.13.6 advertised-routes

BGP table version is 3, local router ID is 10.10.10.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight RouteTag Path
*> 0.0.0.0/0        13.13.13.3                            10        0 1680 1680 111 ?
*> 3.3.3.3/32       13.13.13.3                         32768        0 1680 1680 ?   <--- route-map prepends twice AS 1680 
*> 10.10.10.1/32    13.13.13.3                         32768        0 1680 1680 ?
*> 12.12.12.0/24    13.13.13.3                         32768        0 1680 1680 ?
*> 13.13.13.0/24    13.13.13.3                         32768        0 1680 1680 ?

Total number of prefixes 5

As you probably noticed there are too many routes! Indeed we have a problem here - instead of advertising just our (AS 1680) routes as we do to the ISP1, we advertise also the routes we learned from ISP1 (0.0.0.0/0)! We have become transit AS - if ISP2 does not filter incoming from us routes, their clients may potentially reach networks behind ISP1 via us, and for free. Also ISP1 may see our re-advertisements of their routes as BGP hijacking, and be very unhappy about that. To fix this issue we need to implement route filtering, be it on ISP1 & ISP2 sides, or on our outgoing advertisements. I will look into it in the next scenario.

The good news is that the route-map prepending AS 1680 to make ISP2 less preferred for our network works.

Let's have a look at the work the prefix-list filtering is doing on FG3. The BGP debug should show it:

FG3-AS1680 # diagnose ip router bgp level info

Here I set BGP debug level to INFO, as the default level of ERROR will not show enough information. Next I can run the debug.

diagnose ip router bgp all enable

Unfortunately as the BGP session is already established nothing really happens, so I clear ALL BGP sessions (not something you want to do on production Fortigate lightly):

exec router clear bgp all

BGP: 13.13.13.6-Outgoing [FSM] State: OpenConfirm Event: 26
id=20300 logdesc="BGP neighbor status changed" msg="BGP: %BGP-5-ADJCHANGE: neighbor 13.13.13.6 Up "   <--- The BGP session with ISP2 is established
BGP: 13.13.13.6-Outgoing [DECODE] Update: NLRI Len(15)
BGP: 13.13.13.6-Outgoing [FSM] State: Established Event: 27
BGP: 13.13.13.6-Outgoing [RIB] Update: Received Prefix 0.0.0.0/0                  <--- And here we can see prefix-list filtering
BGP: 13.13.13.6-Outgoing [RIB] Update: Prefix 13.13.13.0/24 denied due to filter  <--- in action , 0.0.0.0/0 is accepted but
BGP: 13.13.13.6-Outgoing [RIB] Update: Prefix 8.8.8.8/32 denied due to filter     <--- the rest of received routes are discarded
BGP: 13.13.13.6-Outgoing [RIB] Update: Prefix 2.2.2.2/32 denied due to filter

Disable all debug:

diagnose debug reset

Prevent our Fortigate from becoming a transit AS, do not advertise learned via eBGP routes.

As seen in the previous case, without any filtering on FG3 everything it learns from its BGP peers and is being installed in its routing table will be advertised to all the BGP peers.

We can prevent it in few ways:
- Filter outgoing advertisements to include only our networks by IPs (not very scalable, but granular)
- Filter outgoing advertisements using AS number (much more scalable, but not granular)

First is to explicitly allow our own networks in outgoing advertisements and block everything else. About blocking everything else - both prefix lists and ACLs have implicit deny any any, so it is not necessary to explicitly deny everything else.

Matching networks using prefix lists

Prefix lists use Prefix (network) and the Prefix Length (subnet mask length in bits) to look at when comparing the routes.
Some examples of using prefix lists:

Prefix What matches
0.0.0.0/0 le 32 Matches ANY prefix of ANY length
0.0.0.0/0 ge 24 le 24 Matches ANY network/prefix with subnet 24 bits long
0.0.0.0/0 ge 24 Matches any network with subnet mask of 24 bits or longer. Usually used by uplink providers to block incoming routes which are too specific, for preserving manageable size of the routing table.
0.0.0.0/0 Matches default route only
10.0.0.0/8 Matches
13.13.0.0/16 ge 25 Match any network prefix in range of 13.13.0.0/16 i.e. from 13.13.0.0 to 13.13.255.255, provided it also has bit mask length of 25 bits or longer. E.g. 13.13.123.0/25, 13.13.123.128/25 but not 13.13.0.0/24
13.13.0.0/16 ge 25 le 30 Match networks in range of 13.13.0.0 - 13.13.255.255 with bit mask of 25 or longer up to and including 30 bits.

The following prefix-list will allow just networks 10.10.10.1/32 and 3.3.3.3/32:

config router prefix-list
    edit "own-nets-only-out"
        config rule
            edit 1
                set prefix 10.10.10.1 255.255.255.255
                unset ge
                unset le
            next
            edit 2
                set prefix 3.3.3.3 255.255.255.255
                unset ge
                unset le
            next
        end
    next

What is left is to apply the prefix list outbound to both peers on FG3:

config router bgp
    set as 1680
    config neighbor
        edit "12.12.12.12"
            set prefix-list-in "accept-dflt-only"
            set prefix-list-out "own-nets-only-out"
            set remote-as 111
            set weight 10
        next
        edit "13.13.13.6"
            set prefix-list-in "accept-dflt-only"
            set prefix-list-out "own-nets-only-out"
            set remote-as 222
            set route-map-out "prepend-out"
        next
    end

The advertised routes before:

FG3-AS1680 # get router info bgp neighbors 13.13.13.6 advertised-routes

   Network          Next Hop            Metric LocPrf Weight RouteTag Path
*> 0.0.0.0/0        13.13.13.3                            10        0 1680 1680 111 ?
*> 3.3.3.3/32       13.13.13.3                         32768        0 1680 1680 ?
*> 10.10.10.1/32    13.13.13.3                         32768        0 1680 1680 ?
*> 12.12.12.0/24    13.13.13.3                         32768        0 1680 1680 ?
*> 13.13.13.0/24    13.13.13.3                         32768        0 1680 1680 ?

Total number of prefixes 5

FG3-AS1680 #get router info bgp neighbors 12.12.12.12  advertised-routes          

BGP table version is 3, local router ID is 10.10.10.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight RouteTag Path
*> 3.3.3.3/32       12.12.12.3                         32768        0 ?
*> 10.10.10.1/32    12.12.12.3                         32768        0 ?
*> 12.12.12.0/24    12.12.12.3                         32768        0 ?
*> 13.13.13.0/24    12.12.12.3                         32768        0 ?

Total number of prefixes 4

And after applying the prefix list:

FG3-AS1680 # get router info bgp neighbors 13.13.13.6 advertised-routes 

   Network          Next Hop            Metric LocPrf Weight RouteTag Path
*> 3.3.3.3/32       13.13.13.3                         32768        0 1680 1680 ?
*> 10.10.10.1/32    13.13.13.3                         32768        0 1680 1680 ?

Total number of prefixes 2


FG3-AS1680 # get router info bgp neighbors 12.12.12.12  advertised-routes

   Network          Next Hop            Metric LocPrf Weight RouteTag Path
*> 3.3.3.3/32       12.12.12.3                         32768        0 ?
*> 10.10.10.1/32    12.12.12.3                         32768        0 ?

Total number of prefixes 2

Also note that to ISP2 peer both networks are advertised with AS 1680 prepended, this is because route-map is applied latest in the incoming/outgoing routes processing.

Matching networks using AS PATH list

BGP advertised routes bear with them quite a lot of information on which we can match/filter and do other manipulation to our liking. One of the Well-known mandatory (i.e. present in EVERY route advertisement/withdrawal in BGP, the others being ORIGIN, and NEXT_HOP) is the AS_PATH attribute. So we can use it to allow advertising only our own routes with AS PATH lists. AS path lists use regular expressions to match the AS numbers in the path. The regex differs slightly from the familiar PCRE/sed. Special symbols understood:

Symbol What matches
. Any single character, including space
* Zero or more instance of preceding pattern
+ One or more instance of preceding pattern
? Zero or one instance of preceding pattern
^ Beginning of the string. Also can be used to negate inside class [^ ]
$ End of the string
_ (Underscore, special for AS Path lists) Matches comma, left brace ({), right brace (}), left parenthesis, right parenthesis, beginning of the string, end of the string, and a space.
[ ] Range of characters, can use - to skip specifying all the range.

Let's look at some examples of matching AS numbers.

AS PATH regex What matches
^$ Local routes only. In other words - match routes with empty AS path.
.* All and any routes
^111$ Routes originating from a directly attached peer, i.e routes that have just one AS number in their path. Here it is routes originated by ISP1 (AS 111).
_111$ Routes originated by the specified AS, but not necessarily learned directly from the source AS. If the given AS 111 advertises its routes to say AS 333, and we have peering with this AS 333, then we could learn routes from AS 333 that were originated by AS 111 and their AS path would look 333 111, and they would be matched. Also, we don't impose length limit on AS path here, so the path 777 999 333 111 would match as well.
_111_ Routes that passed on their way the specified AS, without looking in which order. This will match routes with AS paths like: 333 111 777 999, 111 777, 111 (see table above as _ will match as $ and ^ as well)

Now back to our FG3, let's create and apply AS path list filtering to advertise only our own nets to the BGP peers.

Step 1. Create aspath-list matching local routes:

config router aspath-list

    edit "LocalRoutesOnly"
        config rule
            edit 1
                set action permit
                set regexp "^$"
            next
        end
    next
end

Step 2. Create if needed (for ISP1) and/or edit existing route-map (for ISP2 there is already prepend-out for prepending AS) that uses the aspath-list for matching.

config router route-map
    edit "prepend-out"
        config rule
            edit 1
                set match-as-path "LocalRoutesOnly"    <-- adding the match for local routes only
                set set-aspath "1680 1680"
            next
        end
    next

    edit "LocalRoutesOut"
        config rule
            edit 1
                set match-as-path "LocalRoutesOnly"
            next
        end
    next
end

Step 3. Finally, apply route-map LocalRoutesOut to ISP1 and refreshing BGP sesison with ISP2 to activate the changes.

config router bgp
    set as 1680
    config neighbor
        edit "12.12.12.12"
            set prefix-list-in "accept-dflt-only"
            set remote-as 111
            set route-map-out "LocalRoutesOut"  <-- I removed here prefix-list ""own-nets-only-out"
" as unnecessary anymore
            set weight 10
        next
        edit "13.13.13.6"
            set prefix-list-in "accept-dflt-only"
            set remote-as 222
            set route-map-out "prepend-out"
        next
    end
    config redistribute "connected"
        set status enable
    end

Verification

ISP1, rotues received from FG3:

FG1-AS111 # get router info bgp neighbor 12.12.12.3 routes

   Network          Next Hop            Metric LocPrf Weight RouteTag Path
*> 3.3.3.3/32       12.12.12.3               0             0        0 1680 ?
*> 10.10.10.1/32    12.12.12.3               0             0        0 1680 ?
*  12.12.12.0/24    12.12.12.3               0             0        0 1680 ?
*> 13.13.13.0/24    12.12.12.3               0             0        0 1680 ?

ISP2:

FG6-AS222 # get router info bgp neighbors 13.13.13.3 routes

   Network          Next Hop            Metric LocPrf Weight RouteTag Path
*> 3.3.3.3/32       13.13.13.3               0             0        0 1680 1680 1680 ?
*> 10.10.10.1/32    13.13.13.3               0             0        0 1680 1680 1680 ?
*> 12.12.12.0/24    13.13.13.3               0             0        0 1680 1680 1680 ?
*  13.13.13.0/24    13.13.13.3               0             0        0 1680 1680 1680 ?

Matching networks using ACLs

Lastly, for the completeness sake, let's do the filtering with ACLs. The major issue with ACLs is that the pain is not worth the gain - they are not intuitive, and you spend more time calculating needed ACL wildcards than actually configuring them. And all that for no advantage whatsoever over the prefix lists. ACLs in BGP context appeared like 20 years ago in Cisco, before the prefix lists were available, and ever since they are supported for no obvious reason (to me, except one - for CCIE R&S exam where Cisco folks love to use them to confuse/make suffer the candidates). Fortinet, probably not to feel outdone, implemented it as well. I, personally, have never seen them being used in real life.

So, matching our loopback networks 3.3.3.3/32, 10.10.10.1/32, and directly attached 12.12.12.0/24, and 13.13.13.0/24 with ACls will look like:

config router access-list
    edit "own-nets-only"
        config rule
            edit 1
                unset prefix   <-- Funny thing, even though inside ACL, still Fortigate allows us to use
 prefix with subnet mask as in prefix-lists, i.e. 10.10.10.1/32 
                set wildcard 10.10.10.1 0.0.0.0
            next
            edit 2
                unset prefix
                set wildcard 3.3.3.3 0.0.0.0
            next
            edit 3
                unset prefix
                set wildcard 12.12.12.0 0.0.0.255
            next
            edit 4
                unset prefix
                set wildcard 13.13.13.0 0.0.0.255
            next
        end
    next
end

Step 2: create route-map to use the ACL (only for ISP1).

config router route-map
    edit "LocalOutACL"
        config rule
            edit 1
                set match-ip-address "own-nets-only"
            next
        end
    next
end

For ISP1 I will use the existing route-map "prepend-out".

Step 3: apply the route-map in outbound direction.

FG3-AS1680 # show router bgp
config router bgp
    set as 1680
    config neighbor
        edit "12.12.12.12"
            set prefix-list-in "accept-dflt-only"
            set remote-as 111
            set route-map-out "LocalOutACL"
            set weight 10
        next
        edit "13.13.13.6"
            set prefix-list-in "accept-dflt-only"
            set remote-as 222
            set route-map-out "prepend-out"
        next
    end

Force FG1 to advertise default route without having one in RIB and without using blackhole routing. Uses default-originate

Fortigate can advertise the default route to its peers, even if there is no such route in by using capability-default-originate enable command under the neighbor configuration.

FG1-AS111 # show router bgp
config router bgp
    set as 111
    set router-id 1.1.1.1
    config neighbor
        edit "12.12.12.3"
            set capability-default-originate enable    <-- This will cause FG1 to advertise 0.0.0.0/0
            set remote-as 1680
        next
    config redistribute "static"
    end

Verify:

FG1-AS111 # get router info bgp neighb 12.12.12.3 advertised-routes
   Network          Next Hop            Metric LocPrf Weight RouteTag Path
*> 0.0.0.0/0        12.12.12.12                   100  32768        0 i
*> 1.1.1.1/32       12.12.12.12                        32768        0 ?
*> 8.8.8.8/32       12.12.12.12                        32768        0 ?
*> 12.12.12.0/24    12.12.12.12                        32768        0 ?

Limit announced connected routes to 3.3.3.3 only. Uses route-map with redistribution

Redistribute statements under router BGP configuration support using route-maps to limit what routes get distributed into BGP and which do not. Let's limit the routes FG1 announces to just 3.3.3.3/32 of its loopback.

Step 1 Create prefix list to match the route:

config router prefix-list

    edit "allow-3.3.3.3-only"
        config rule
            edit 1
                set prefix 3.3.3.3 255.255.255.255
                unset ge
                unset le
            next
        end

Step 2 Update/create route-map to use the prefix-list.

    edit "redist-3.3.3.3-only"
        config rule
            edit 1
                set match-ip-address "allow-3.3.3.3-only"
            next
        end
    next

Step 3 Use this route-map.

config router bgp
    config redistribute "connected"
        set status enable
        set route-map "redist-3.3.3.3-only"
    end

Secure BGP session between ISP1 and FG3 with one way hash. Uses MD5 authentication

BGP has MD5 hashing to prevent adversary changes to the advertisements and potential DDoS attack by sending TCP RST packets (to sabotage an existing and legal session).
Naturally, you have to configure the same password on both BGP peers. The configuration itself is one line under neighbor configuration - password. E.g. FG3:

    config neighbor
        edit "12.12.12.12"
            set remote-as 111
            set route-map-out "LocalOutACL"
            set weight 10
            set password secretsuperpassword

More interesting though is to see what happens when misconfiguration occurs.

Case 1 FG3 has the password set, FG1 has not. As BGP RFC requires, the peer with BGP authentication enabled should drop and NOT acknowledge or give any other information when it receives unauthenticated packet. So, basically BGP sessions times out:

FG3-AS1680 # diagnose ip router bgp level info
FG3-AS1680 # diagnose ip router bgp all enable
FG1 (on which I initiated BGP sesison reset):

BGP: 12.12.12.3-Outgoing [ENCODE] Msg-Hdr: Type 3 <-- Type 3 is NOTIFICATION
BGP: %BGP-3-NOTIFICATION: sending to 12.12.12.3 6/0 (CeaseUnspecified Error Subcode) 0 data-bytes []

FG1-AS111 # BGP: [GRST] Timer Announce Defer: Check
id=20300 logdesc="BGP neighbor status changed" msg="BGP: %BGP-5-ADJCHANGE: neighbor 12.12.12.3 Down BGP Notification CEASE"
id=20300 logdesc="BGP neighbor status changed" msg="BGP: %BGP-5-ADJCHANGE: neighbor 12.12.12.3 Down User reset"
BGP: 12.12.12.3-Outgoing [FSM] State: Idle Event: 3   <-- This Fortigate falls back to Idle state


FG3-AS1680: (pretty much the same)

BGP: 12.12.12.12-Outgoing [ENCODE] Msg-Hdr: Type 3
BGP: %BGP-3-NOTIFICATION: sending to 12.12.12.12 4/0 (Hold Timer Expired/Unspecified Error Subcode) 0 data-bytes []
BGP: [GRST] Timer Announce Defer: Check
id=20300 logdesc="BGP neighbor status changed" msg="BGP: %BGP-5-ADJCHANGE: neighbor 12.12.12.12 Down Hold Timer Expired"
id=20300 logdesc="BGP neighbor status changed" msg="BGP: %BGP-5-ADJCHANGE: neighbor 12.12.12.12 Down BGP Notification FSM-ERR"

After that, there is nothing to point on mismatch in authentication session, but just time out:

FG1-AS111 #
BGP: 12.12.12.3-Outgoing [NETWORK] FD=21, Sock Status: 110-Connection timed out
BGP: 12.12.12.3-Outgoing [FSM] State: Connect Event: 18

FG3-AS1680 #
BGP: 12.12.12.12-Outgoing [NETWORK] FD=23, Sock Status: 110-Connection timed out
BGP: 12.12.12.12-Outgoing [FSM] State: Connect Event: 18

Case 2: One of the peers has wrong password set.

Well, here we have no clue from the Fortigate as well, just the same connection time out:

BGP: %BGP-3-NOTIFICATION: sending to 12.12.12.3 4/0 (Hold Timer Expired/Unspecified Error Subcode) 0 data-bytes []
BGP: [GRST] Timer Announce Defer: Check
id=20300 logdesc="BGP neighbor status changed" msg="BGP: %BGP-5-ADJCHANGE: neighbor 12.12.12.3 Down Hold Timer Expired"
id=20300 logdesc="BGP neighbor status changed" msg="BGP: %BGP-5-ADJCHANGE: neighbor 12.12.12.3 Down BGP Notification FSM-ERR"

BGP: 12.12.12.3-Outgoing [NETWORK] FD=21, Sock Status: 110-Connection timed out
BGP: 12.12.12.3-Outgoing [FSM] State: Connect Event: 18

Make sure we can see received routing advertisements before and after any filtering is applied. Uses soft reconfiguration

The inconvenience of not seeing received from a peer routes before we apply local manipulation/filtering actions can be fixed with soft reconfiguration, which is disabled by default. This feature, once enabled, forces Fortigate to keep in memory all received routes from the neighbor BEFORE any local filtering is being applied. The downside is that memory consumption goes up. Today, this functionality is only good as visual aid in debugging the changes situations because route refresh capability (details here RFC 2918 and RFC 7313) is by default enabled in Fortigate, so any changes to the BGP policy we make on Fortigate are applied almost immediately (few seconds delay).

Still, the feature is there and we can can enable with soft-reconfiguration enable.

FG3-AS1680 (neighbor) # show
config neighbor
    edit "12.12.12.12"
        set soft-reconfiguration enable
        set remote-as 111
        set route-map-out "LocalOutACL"
        set prefix-list-in "accept-dflt-only"
        set weight 10
    next

Now we can query for routes received from 12.12.12.12 (ISP1) BEFORE the policy accept-dflt-only is applied allowing just default route :

BEFORE filtering is applied:

FG3-AS1680 # get router info bgp neighbors 12.12.12.12  received-routes

   Network          Next Hop            Metric LocPrf Weight RouteTag Path
*> 0.0.0.0/0        12.12.12.12                            0        0 111 i
*> 1.1.1.1/32       12.12.12.12                            0        0 111 ?
*> 8.8.8.8/32       12.12.12.12                            0        0 111 ?
*> 12.12.12.0/24    12.12.12.12                            0        0 111 ?

Total number of prefixes 4

And AFTER:

FG3-AS1680 # get router info bgp neighbors 12.12.12.12  routes

   Network          Next Hop            Metric LocPrf Weight RouteTag Path
*> 0.0.0.0/0        12.12.12.12              0            10        0 111 i

Total number of prefixes 1

Set up BGP peering between FG3 and FG1 using loopback in FG3

In production you use loopbacks as source interface for BGP sessions mostly to ensure continuous BGP peering in case the physical link to the BGP peer goes down. In this case, if you have redundant links/paths to the same BGP peer via other, still functional interfaces, the BGP session will work uninterrupted.

In loopback as source interface case you have to account for 2 things:

  • Loopback is an interface by all means, so you have to add security rules to allow traffic (TCP port 179 in BGP's case) to/from it for BGP session to be established. The rule from loopback outbound is enough for Fortigate to be BGP client, always establishing connection to the peer.
  • Loopback adds 1 routing hop so for eBGP sessions you have to enable eBGP multihop for session to come up. You do it on the remote peer at least.

Configure FG3.
Configure security rule to allow outgoing from Loop2 connections:

configure firewall policy
        set srcintf "Loop2"
        set dstintf "port1"
        set srcaddr "all"
        set dstaddr "all"
        set action accept
        set status enable
        set schedule "always"
        set service "ALL"
    next

The BGP neighbor configuration to use Loop2 as the source interface:

config router bgp
    set as 1680
    config neighbor
        edit "12.12.12.12"
            set soft-reconfiguration enable
            set interface "Loop2"
            set prefix-list-in "accept-dflt-only"
            set remote-as 111
            set route-map-out "LocalOutACL"
            set update-source "Loop2"   <-- This causes FG3 to source BGP packets from Loop2
            set weight 10
        next

Configuration of FG1.
Create new peer with ip of 3.3.3.3 and add the multi-hop capability:

config router bgp
    set as 111
    set router-id 1.1.1.1
    config neighbor
        next
        edit "3.3.3.3"
            set capability-default-originate enable
            set ebgp-enforce-multihop enable
            set remote-as 1680
        next
    end

Verification

To see that FG3 is indeed client and FG1 (12.12.12.12) is server for this peering:

diagnose sys tcpsock | grep 179

FG3-AS1680 # diag sys tcpsock | grep 179
0.0.0.0:179->0.0.0.0:0->state=listen err=0 sockflag=0x1 rma=0 wma=0 fma=0 tma=0
3.3.3.3:21989->12.12.12.12:179->state=estabilshed err=0 sockflag=0x1 rma=0 wma=0 fma=0 tma=0
13.13.13.3:3345->13.13.13.6:179->state=estabilshed err=0 sockflag=0x1 rma=0 wma=0 fma=0 tma=0

Multi-hop neighbor enabled:

FG1-AS111 #  get router info bgp neighbors 
BGP neighbor is 3.3.3.3, remote AS 1680, local AS 111, external link
  BGP version 4, remote router ID 10.10.10.1
  BGP state = Established, up for 00:13:42
   ....
  External BGP neighbor may be up to 255 hops away. <-- Multi hop setting is active
Local host: 12.12.12.12, Local port: 179
Foreign host: 3.3.3.3, Foreign port: 21989

References

  • Config router bgp commands: https://docs.fortinet.com/document/fortigate/6.0.0/cli-reference/722994/router-bgp
  • Get commands for BGP https://docs.fortinet.com/document/fortigate/6.0.0/cli-reference/67947/router-info-bgp
  • Execute router set of commands: (https://docs.fortinet.com/document/fortigate/6.0.0/cli-reference/381055/router-clear-bgp)