Wednesday, August 27, 2014

Configuring Arista MLAG - Basic setup

Layer 2 All links forwarding none blocking ... Take that Spanning Tree!! (-- no offense Radia Perlman :) --) This is what you get with Arista's proprietary MLAG -- short for Multi-Chassis Link Aggregation. Although Spanning-Tree is extremely efficient at preventing loops and keeping your network healthy, it does come with a hefty price tag -- essentially blocking half of your uplinks. In this blogtorial, we will go through brief overview of spanning-tree and then deep dive into MLAG concepts, caveats, and configurations. As of this writing, MLAG is currently supported on Arista's 75xx, 7500E, 7048, 7150, 7050, 7050X, 7250X, and 7300X. If you are familiar with Cisco's proprietary vPC (Virtual Port-Channel) then most of this should be fairly straight forward.

Consider this traditional Layer 2 design where half of your links are blocked to prevent loops in the network.


Traditional Layer 2 STP network blocks half of the uplinks to prevent packets from looping the network. There are some ways to load balance and utilize all of your uplinks such as VLAN load-balancing, however these workarounds can get extremely hard to manage. This is where MLAG comes in and essentially lets us use 100% of our uplinks.

Consider the same topology, however this time with MLAG deployed. From the Access layer perspective the switch/hosts think they are connected to the same switch so all links (port-channeled) are forwarding without the fear of packets looping the network.



Now that we understand the basics of MLAG, let's dive into the configuration of MLAG.

First, let's verify and make sure the default control plane policy allows MLAG traffic to be handled by the CPU. Line 70 and 80 confirms that we have the correct ACE in place.

 mlag-1#show ip access-lists  
 IP Access List default-control-plane-acl [readonly]  
     statistics per-entry  
     10 permit icmp any any [match 38, 3 days, 8:36:07 ago]  
     20 permit ip any any tracked [match 450111, 0:00:00 ago]  
     30 permit udp any any eq bfd ttl eq 255  
     40 permit ospf any any  
     50 permit tcp any any eq ssh telnet www snmp bgp https msdp [match 9, 0:19:12 ago]  
     60 permit udp any any eq bootps bootpc snmp rip ntp [match 4, 3 days, 12:10:07 ago]  
     70 permit tcp any any eq mlag ttl eq 255 [match 75, 3 days, 9:42:37 ago]  
     80 permit udp any any eq mlag ttl eq 255  
     90 permit vrrp any any  
     100 permit ahp any any  
     110 permit pim any any  
     120 permit igmp any any  
     130 permit tcp any any range 5900 5910  
     140 permit tcp any any range 50000 50100  
     150 permit udp any any range 51000 51100  

Configure Port-Channel between the switches. Usually you would want to configure at least 2 links in the Port-Channels but 4 or 8 might be required if you have considerable amounts of data between the switches or for redundancy.

Port-channel between the switches

On both MLAG-1 and MLAG-2 switches
interface Ethernet 1 - 4 description port-channel channel-group 10 mode active interface port-channel 10 description port-channel switchport mode trunk

Configure VLAN used by MLAG and disable spanning-tree on this VLAN.

On both MLAG-1 and MLAG-2 switches
vlan 4094 trunk group mlagpeer interface port-channel 10 switchport trunk group mlagpeer no spanning-tree vlan 4094

We can safely disable spanning-tree because the trunk group ensures that VLAN 4094 will only traverse port-channel 10. Trunk group names must be identical on both switches for it to work properly.

Let's get the SVI configured on both switches so they can communicate.

 
 MLAG-1#  
  interface vlan 4094  
  description MLAG_VLAN  
  ip address 172.20.1.1/30  
  no shut  

 MLAG-2#  
  interface vlan 4094  
  description MLAG_VLAN  
  ip address 172.20.1.2/30  
  no shut  
 

Now the easy part on bringing up MLAG between the switches.


 MLAG-1#  
  mlag  
  local-interface vlan 4094  
  peer-address 172.20.1.2  
  peer-link port-channel 10  
  primary-priority 1  
  domain-id mlagcore  

 MLAG-2#   
  mlag  
  local-interface vlan 4094  
  peer-address 172.20.1.1  
  peer-link port-channel 10  
  domain-id mlagcore


Note: 
Primary-priority is a hidden command and it is needed for the deterministic election of MLAG Primary switch and lower priority wins. Commands related to spanning-tree comes from the MLAG primary switch. For example, when you configure spanning-tree vlan root priority on the MLAG secondary switch, the setting will not take effect.

Once the communication establishes MLAG will be active.

Verification

Notice 'mlag-1' switch shows that it is MLAG primary and that the status of the MLAG is active on both switches.


It is also recommended that both MLAG-1 and MLAG-2 have identical LAYER 2 commands such as STP root priority etc. Because if the primary switch were to reboot, the secondary switch will takeover and STP commands would be different resulting in a sub-optimal configuration.

Now let's configure MLAG port-channel on MLAG-1 and MLAG-2 switch and also configure regular port-channel on the access layer switch AXS1. Highly recommended to use LACP ACTIVE and not just channel-group mode on.

 
 mlag-1#  
  interface Ethernet11  
   description MLAG:PO_ASX1  
   no shutdown  
   channel-group 11 mode active  
  interface Port-Channel11  
   description MLAG:PO_ASX1  
   switchport mode trunk  
   mlag 11  



 mlag-2#  
  interface Ethernet11  
   description MLAG:PO_ASX1  
   no shutdown  
   channel-group 11 mode active  
  interface Port-Channel11  
   description MLAG:PO_ASX1  
   switchport mode trunk  
   mlag 11
  


 ASX1#
  interface Ethernet1/9  
   description MLAG:PO_TO_MLAG-1  
   switchport mode trunk  
   spanning-tree port type edge  
   channel-group 11 mode active  
  interface Ethernet1/11  
   description MLAG:PO_TO_MLAG-2  
   switchport mode trunk  
   channel-group 11 mode active  


Let's verify that our port-channels are up on MLAG-1 and MLAG-2 to the ASX1 switch.

 
 mlag-1#sh etherchannel detailed  
  Active Ports:  
     Port         Time became active    Protocol  Mode  
   -------------------- ------------------------ ------  
     Ethernet11        4:32:34          LACP    Active  
     PeerEthernet11    4:34:24          LACP    Active  


 mlag-1#show mlag interfaces detail  
                     local/remote  
  mlag     state  local  remote  oper  config  last change  changes  
 ------ ------------- ------- -------- ------- ---------- --------------   
   1  active-full  Po11  Po11  up/up  ena/ena  0:08:23 ago    24  

Notice that 'mlag-1#' shows 'PeerEthernet11' which is a good sign because it recognizes the other Ethernet11 on mlag-2 switch as a part of the port-channel.

 ASX1# show port-channel summary  
 Flags: D - Down    P - Up in port-channel (members)  
     I - Individual H - Hot-standby (LACP only)  
     s - Suspended  r - Module-removed  
     S - Switched  R - Routed  
     U - Up (port-channel)  
     M - Not in use. Min-links not met  
 --------------------------------------------------------------------------------  
 Group Port-    Type   Protocol Member Ports  
    Channel  
 --------------------------------------------------------------------------------  
 1   Po1(SU)   Eth   LACP   Eth1/9(P)  Eth1/11(P)  

Notice that from ASX1 perspective this is just a normal port-channel with no special configurations. As far as ASX1 is concerned both Eth1/9 and Eth1/11 are connected into the same switch.

Spanning-tree show commands on both mlag-1 and mlag-2 switch.

 
 mlag-1#sh spanning-tree vlan 101  
  Spanning tree instance for vlan 101  
  VL101  
   Spanning tree enabled protocol rapid-pvst  
   Root ID  Priority  32869  
        Address   021c.7309.791b  
        This bridge is the root  
   Bridge ID Priority  32869 (priority 32768 sys-id-ext 101)  
        Address   021c.7309.791b  
        Hello Time 2.000 sec Max Age 20 sec Forward Delay 15 sec  
  Interface    Role    State   Cost   Prio.Nbr Type  
  ---------------- ---------- ---------- --------- -------- --------------------  
  Po12       designated forwarding 1999   128.100 P2p  

 mlag-2#sh spanning-tree vlan 101  
  Spanning tree instance for vlan 101  
  VL101  
   Spanning tree enabled protocol rapid-pvst  
   Root ID  Priority  32869  
        Address   021c.7309.791b  
        This bridge is the root  
   Bridge ID Priority  32869 (priority 32768 sys-id-ext 101)  
        Address   021c.7309.791b  
        Hello Time 2.000 sec Max Age 20 sec Forward Delay 15 sec  
  Interface    Role    State   Cost   Prio.Nbr Type  
  ---------------- ---------- ---------- --------- -------- --------------------  
  Po11       designated forwarding 1999   128.100 P2p  

Notice that both switches are acting as spanning-tree root bridge. Also note that the bridge address comes from the negotiated MLAG system-id (show mlag detail).

Caveats/Thoughts/Conclusions
  • Make sure to have a deterministic MLAG primary switch. 
  • Set all STP commands the same on both switches.
  • At least have 2 links between the switches for MLAG communication in a port-channel.
  • If you have a single homed server, traffic might need to traverse the port-channel to the other switch adding an extra hop latency. 
What are your thoughts on MLAG? How have you deployed it?

Follow to my next blogtorial "Configuring Arista VARP aka Virtual ARP" if you are interested in Active/Active L3 forwarding with MLAG.

Many more articles to come so ....

Please subscribe/comment/+1 if you like my posts as it keeps me motivated to write more and spread the knowledge.