NSX-T Active-Active Multi-site with Secondary Region Failover

NSX-T Active-Active Multisite in a Single Region and Failover to a Secondary Region Part 2.  

A guide to walkthrough the configuration of Multisite in Active-Active with failover.

In this article I will focus on the configuration, demonstration and testing of what was discussed in part 1 for NSX-T Multisite with a secondary availability zone which can be seen here; Part 1.

This article will not cover the base installation and configuration of NSX-T, the ToR's or the Core, it will focus on configuring and testing NSX-T Multisite as was discussed in part 1.

1. To begin with I will display the current interfaces and BGP neighbor configuration.

The current subnets being propagated from NSX-T can be seen below.

2. The image below shows all the routes the core switch has learnt.

Notice how the routes are learned from 4 different peers, which is in line with the 4 ToR's that are peering with the NSX-T T0 as well as the core switch.  Performing trace routes into the environment now will be balanced across all peers.  Keep in mind that I have configured the maximum-paths and multipath-relax on the core switch.  

Without these two commands, the path that the core deemed best was injected into the routing table and the same subnet would not presented from two different ASN's, ie 195.0.0.254/32 was only presented from 65200.

Now that we have routes appearing and all peering working as expected, it is time to configure BGP for Site-A to be preferred.  

3. T0 Gateways > Edit the T0 > Routing > IP Prefix Lists 

4. Below we create the route maps using the above prefix list and prepend the AS Path in SiteB-out.  To create a route map go into edit mode on your T0 and expand the routing section and click the hyperlink next to route maps.  The second route map is the inbound filter to set the local preference to 90, the default is 100 and the path with the higher preference wins.  In this case we are setting inbound routes from Site B to 90, and in doing so Site A has it's default of 100 still set.

5.  Apply the Route Map to the Site B peers, this can be done by going into the BGP neighbours, editing one of the peers, clicking on Out Filter, applying the list that was just created and then completing the same for the in filter.  Repeat this step for all peers that need to be prepended.  In this case it was all Site B peers.  Set the route maps according to the name, the SiteB-in is set as the in-filter and the SiteB-out is set as the out-filter.

6.  Now that is all done, let's have a look at the routing table on the core again.

 Here we can see that the routes being injected into the routing table are from the AS in Site A, as I prepended Site B's path to make it seem longer to the core.  The below image shows the routes still being advertised from the ToR's in Site B, however pay close attention to the path. 

The image below is the routes BGP has learned, notice here you see the prefixes listed multiple times learned from each peer.

 

This is an image of the routes on one of the edge nodes. 

8.  Next we will perform a quick failover test to show this working, I begin by pinging 195.0.0.254 from my desktop.  This will now route through the Edge's uplink interfaces in Site A.

9.  Shutdown the ToR's in Site A and below is the route table in the core after doing so.

10.  Below is the ping output, I dropped more packets than I should have as I hadn't configured BFD between the peers (you really should do this for faster failover).

11.  Finally, below is the trace route from the same desktop.

This concludes the second part of this article and hopefully has shed some light on how this all works.  If you would like some questions answered or need anything clarified, leave a comment!