Довольно популярным запросом у заказчиков стало наращивание существующей фабрики Cisco коммутаторами Maipu. Недавно нам удалось наконец собрать необходимый пул оборудования и разобраться, что может и чего не может оборудование Maipu в таких кейсах. Ранее, кстати, рассказал, как мы расширяем существующую DMVPN-сеть Cisco с помощью Maipu, и про коммутаторы Maipu для сетей ЦОД.
Ограничение, которое мы увидели сразу - невозможность реализовать схему Multisite, которую предлагает Cisco. В этой схеме существует специальная роль коммутаторов -- Border Gateway. Эти коммутаторы терминируют VXLAN-туннели внутри фабрики и создают новые туннели до других дата-центров вот так:
Такой подход позволяет ограничить распространение информации, связанной с работой Control Plane, что позволяет лучше масштабировать сети датацентров и сузить домены отказа. При этом между датацентрами работает и L2, и L3.
Функционал основан на нескольких драфтах RFC, и пока не очень распространён. Что-то подобное умеет делать ещё Arista, но на Maipu данная схема недоступна. Поэтому мы пробуем классический Multipod - полный обмен маршрутами между двумя наборами Leaf/Spine-коммутаторов через EBGP. Другой вариант - объединение фабрик через L3, но сейчас не об этом.
Схема, которую нам удалось собрать из имеющегося оборудования, выглядит так:
Второе заметное ограничение - отсутствие на Maipu возможности распространять Broadcast/Unknown Unicast-трафик с помощью PIM и Multicast. Используем BGP Ingress Replication на обоих подах.
На этом грустная часть истории заканчивается и начинается интересная.
Фабрику настраиваем так же, как и в предыдущей серии: в Underlay OSPF и IBGP. Такой вариант мы чаще всего встречаем у заказчиков, эксплуатирующих фабрику на Cisco. Хосты сразу подключаем к двум Leaf-коммутаторам, это VPC и MLAG соответственно.
Между Spine-коммутаторами, где у нас, собственно, и происходит соединение двух фабрик, настраиваем EBGP. Для того, чтобы VXLAN-туннели корректно строились от одного Leaf-коммутатора до другого, сохраняем значение атрибута Next-Hop в передаваемых EVPN-маршрутах.
Конфиги Leaf-коммутаторов Cisco Nexus:
N9k-Leaf1 |
N9k-Leaf2 |
|
cfs eth distribute nv overlay evpn feature ospf feature bgp feature pim feature fabric forwarding feature interface-vlan feature vn-segment-vlan-based feature hsrp feature lacp feature vpc feature lldp feature bfd feature nv overlay
fabric forwarding anycast-gateway-mac 0001.0001.0100
vlan 1,41-42,999 vlan 41 vn-segment 100 vlan 42 vn-segment 200 vlan 999 vn-segment 1
route-map DIRECT-BGP-RM permit 10 match tag 12345 vrf context Keepalive
vrf context vxlan-100 vni 1 rd 100:42 address-family ipv4 unicast route-target import 100:2 route-target import 100:2 evpn route-target export 100:2 route-target export 100:2 evpn vpc domain 1 peer-keepalive destination 10.253.1.5 source 10.253.1.4 vrf Keepalive
interface Vlan41 no shutdown vrf member vxlan-100 ip address 172.16.129.1/24 fabric forwarding mode anycast-gateway
interface Vlan42 no shutdown vrf member vxlan-100 ip address 172.16.130.1/24 fabric forwarding mode anycast-gateway
interface Vlan999 no shutdown vrf member vxlan-100 ip forward
interface port-channel30 switchport switchport mode trunk switchport trunk allowed vlan 41-42 vpc 1
interface port-channel54 switchport switchport mode trunk spanning-tree port type network vpc peer-link
interface nve1 no shutdown host-reachability protocol bgp source-interface loopback101 member vni 1 associate-vrf member vni 100 ingress-replication protocol bgp member vni 200 ingress-replication protocol bgp
interface Ethernet1/1 switchport switchport mode trunk switchport trunk allowed vlan 41-42 channel-group 30 mode active no shutdown
interface Ethernet1/48 description -= Peer Keepalive Link =- vrf member Keepalive ip address 10.253.1.4/31 no shutdown
interface Ethernet1/53 description -= Spine =- ip address 10.253.1.1/31 ip ospf network point-to-point ip router ospf 1 area 0.0.0.0 ip pim sparse-mode no shutdown
interface Ethernet1/54 description -= Peer Link =- switchport switchport mode trunk channel-group 54 mode active no shutdown
interface loopback0 ip address 10.253.127.1/32 ip router ospf 1 area 0.0.0.0
interface loopback101 ip address 10.253.127.253/32 ip address 10.253.127.40/32 secondary ip router ospf 1 area 0.0.0.0
router ospf 1 router-id 10.253.127.1 router bgp 200 address-family l2vpn evpn neighbor 10.253.127.3 remote-as 200 update-source loopback0 address-family ipv4 unicast advertisement-interval 1 address-family l2vpn evpn send-community send-community extended vrf vxlan-100 address-family ipv4 unicast network 172.16.129.0/24 network 172.16.130.0/24 advertise l2vpn evpn redistribute direct route-map DIRECT-BGP-RM evpn vni 100 l2 rd 100:40 route-target import 100:1 route-target export 100:1 vni 200 l2 rd 200:40 route-target import 200:1 route-target export 200:1 |
cfs eth distribute nv overlay evpn feature ospf feature bgp feature pim feature fabric forwarding feature interface-vlan feature vn-segment-vlan-based feature hsrp feature lacp feature vpc feature lldp feature bfd feature nv overlay
fabric forwarding anycast-gateway-mac 0001.0001.0100
vlan 1,41-42,999 vlan 41 vn-segment 100 vlan 42 vn-segment 200 vlan 999 vn-segment 1
route-map DIRECT-BGP-RM permit 10 match tag 12345 vrf context Keepalive
vrf context vxlan-100 vni 1 rd 100:42 address-family ipv4 unicast route-target import 100:2 route-target import 100:2 evpn route-target export 100:2 route-target export 100:2 evpn vpc domain 1 peer-keepalive destination 10.253.1.4 source 10.253.1.5 vrf Keepalive
interface Vlan41 no shutdown vrf member vxlan-100 ip address 172.16.129.1/24 fabric forwarding mode anycast-gateway
interface Vlan42 no shutdown vrf member vxlan-100 ip address 172.16.130.1/24 fabric forwarding mode anycast-gateway
interface Vlan999 no shutdown vrf member vxlan-100 ip forward
interface port-channel30 switchport switchport mode trunk switchport trunk allowed vlan 41-42 vpc 1
interface port-channel54 switchport switchport mode trunk spanning-tree port type network vpc peer-link
interface nve1 no shutdown host-reachability protocol bgp source-interface loopback101 member vni 1 associate-vrf member vni 100 ingress-replication protocol bgp member vni 200 ingress-replication protocol bgp
interface Ethernet1/1 switchport switchport mode trunk switchport trunk allowed vlan 41-42 channel-group 30 mode active no shutdown
interface Ethernet1/48 description -= Peer Keepalive Link =- vrf member Keepalive ip address 10.253.1.5/31 no shutdown
interface Ethernet1/53 description -= Spine =- ip address 10.253.1.3/31 ip ospf network point-to-point ip router ospf 1 area 0.0.0.0 ip pim sparse-mode no shutdown
interface Ethernet1/54 description -= Peer Link =- switchport switchport mode trunk channel-group 54 mode active no shutdown
interface loopback0 ip address 10.253.127.2/32 ip router ospf 1 area 0.0.0.0
interface loopback101 ip address 10.253.127.254/32 ip address 10.253.127.40/32 secondary ip router ospf 1 area 0.0.0.0
router ospf 1 router-id 10.253.127.2 router bgp 200 address-family l2vpn evpn neighbor 10.253.127.3 remote-as 200 update-source loopback0 address-family ipv4 unicast advertisement-interval 1 address-family l2vpn evpn send-community send-community extended vrf vxlan-100 address-family ipv4 unicast network 172.16.129.0/24 network 172.16.130.0/24 advertise l2vpn evpn redistribute direct route-map DIRECT-BGP-RM evpn vni 100 l2 rd 100:40 route-target import 100:1 route-target export 100:1 vni 200 l2 rd 200:40 route-target import 200:1 route-target export 200:1 |
Spine Nexus:
N9k-Spine |
|
nv overlay evpn feature ospf feature bgp feature pim feature fabric forwarding feature interface-vlan feature vn-segment-vlan-based feature hsrp feature lacp feature lldp feature bfd feature nv overlay
vlan 1
route-map BGP-OUT-RM permit 10 set ip next-hop unchanged
vrf context vxlan-100 vni 1 rd 100:4 address-family ipv4 unicast route-target import 100:2 route-target export 100:2
interface Ethernet1/48 ip address 11.0.0.2/24 ip router ospf 1 area 0.0.0.0 no shutdown
interface Ethernet1/49 description -= External Port =- ip address 10.254.100.3/24 ip ospf mtu-ignore ip router ospf 1 area 0.0.0.0 no shutdown
interface Ethernet1/53 description -= Leaf 1 =- ip address 10.253.1.0/31 ip ospf network point-to-point ip router ospf 1 area 0.0.0.0 ip pim sparse-mode no shutdown interface Ethernet1/54 description -= Leaf 2 =- ip address 10.253.1.2/31 ip ospf network point-to-point ip router ospf 1 area 0.0.0.0 ip pim sparse-mode no shutdown
interface loopback0 ip address 10.253.127.3/32 ip router ospf 1 area 0.0.0.0
router ospf 1 router-id 10.253.127.3
router bgp 200 neighbor 10.253.127.1 remote-as 200 update-source loopback0 address-family ipv4 unicast advertisement-interval 1 address-family l2vpn evpn send-community send-community extended neighbor 10.253.127.2 remote-as 200 update-source loopback0 address-family ipv4 unicast advertisement-interval 1 address-family l2vpn evpn send-community send-community extended neighbor 10.254.127.1 remote-as 100 update-source loopback0 ebgp-multihop 5 address-family ipv4 unicast advertisement-interval 1 address-family l2vpn evpn send-community send-community extended route-map BGP-OUT-RM out neighbor 10.254.127.2 remote-as 100 update-source loopback0 ebgp-multihop 5 address-family ipv4 unicast advertisement-interval 1 address-family l2vpn evpn send-community send-community extended route-map BGP-OUT-RM out vrf vxlan-100 address-family ipv4 unicast advertise l2vpn evpn evpn vni 100 l2 rd 100:3 route-target import 100:1 route-target export 100:1 vni 200 l2 rd 200:3 route-target import 200:1 route-target export 200:1 |
Leaf Maipu:
Maipu-L11 |
Maipu-L12 |
|
ip vrf 100 rd 100:2 l3vnid 1 address-family evpn route-target import 100:2 ipv4 route-target export 100:2 ipv4 exit exit
link-aggregation 64 mode lacp link-aggregation 250 mode lacp
vlan 1 exit
vlan 11-13
mlag domain 1 node id 1 node role-priority 50 role preempt system-mac 0001.7a95.000b keepalive ip destination 192.168.168.41 source 192.168.168.40 exit
vxlan 100 vxlan vnid 100 address-family evpn rd 100:10 route-target import 100:1 route-target export 100:1 exit exit vxlan 200 vxlan vnid 200 address-family evpn rd 200:10 route-target import 200:1 route-target export 200:1 exit exit
interface tengigabitethernet0/48 link-aggregation 64 active exit interface 100ge0/49 no switchport ip address 10.254.1.1 255.255.255.254 no ip redirects ip ospf network point-to-point ip ospf bfd ip mtu exception forward exit interface 100ge0/50 no switchport ip address 10.254.2.1 255.255.255.254 no ip redirects ip ospf network point-to-point ip ospf bfd ip mtu exception forward exit interface 100ge0/55 fec mode rs auto link-aggregation 250 active exit interface 100ge0/56 fec mode rs auto link-aggregation 250 active exit !end
interface link-aggregation64 switchport mode trunk switchport trunk allowed vlan add 1,11-12 switchport trunk pvid vlan 1 vxlan 100 encapsulation vlan 11 vxlan 200 encapsulation vlan 12 mlag group 1 exit interface link-aggregation250 switchport mode trunk switchport trunk allowed vlan add 11-13 switchport trunk pvid vlan 1 mlag peer-link exit
interface dc0 ip address 192.168.168.40 255.255.0.0 exit
interface loopback1 description Router-ID ip address 10.254.127.11 255.255.255.255 exit
interface loopback2 description VTEP ip address 10.254.128.11 255.255.255.255 ip address 10.254.128.10 255.255.255.255 secondary exit
interface loopback100 ip vrf forwarding 100 ip address 172.16.250.11 255.255.255.255 exit
interface vxlan100 ip vrf forwarding 100 vxlan distribute-gateway ip address 172.16.129.1 255.255.255.0 mac-address 0001.0001.0100 exit
interface vxlan200 ip vrf forwarding 100 vxlan distribute-gateway ip address 172.16.130.1 255.255.255.0 mac-address 0001.0001.0100 exit
interface nve1 source 10.254.128.10 mac-address 0088.9999.1112 vxlan 100,200 ingress-replication protocol bgp exit
router ospf 100 router-id 10.254.127.11 bfd all-interfaces network 10.254.1.1 0.0.0.0 area 0 network 10.254.2.1 0.0.0.0 area 0 network 10.254.127.11 0.0.0.0 area 0 network 10.254.128.10 0.0.0.0 area 0 maximum-paths 8 exit
router bgp 100 no auto-summary no synchronization bgp router-id 10.254.127.11 neighbor spine peer-group neighbor spine remote-as 100 neighbor spine update-source loopback1 neighbor 10.254.127.1 peer-group spine neighbor 10.254.127.2 peer-group spine maximum-paths ibgp 8 address-family l2vpn evpn neighbor spine activate neighbor spine send-community both neighbor 10.254.127.1 activate neighbor 10.254.127.1 send-community both neighbor 10.254.127.2 activate neighbor 10.254.127.2 send-community both exit-address-family address-family ipv4 vrf 100 advertise-l2vpn-evpn network 172.16.129.0 255.255.255.0 network 172.16.130.0 255.255.255.0 network 172.16.250.11 255.255.255.255 network 172.16.250.12 255.255.255.255 exit-address-family exit |
ip vrf 100 rd 100:2 l3vnid 1 address-family evpn route-target import 100:2 ipv4 route-target export 100:2 ipv4 exit exit
link-aggregation 64 mode lacp link-aggregation 250 mode lacp
vlan 1 exit
vlan 11-13
mlag domain 1 node id 2 system-mac 0001.7a95.000b keepalive ip destination 192.168.168.40 source 192.168.168.41 exit
vxlan 100 vxlan vnid 100 address-family evpn rd 100:10 route-target import 100:1 route-target export 100:1 exit exit vxlan 200 vxlan vnid 200 address-family evpn rd 200:10 route-target import 200:1 route-target export 200:1 exit exit
interface tengigabitethernet0/48 link-aggregation 64 active exit interface 100ge0/49 no switchport ip address 10.254.1.3 255.255.255.254 no ip redirects ip ospf network point-to-point ip ospf bfd ip mtu exception forward exit interface 100ge0/50 no switchport ip address 10.254.2.3 255.255.255.254 no ip redirects ip ospf network point-to-point ip ospf bfd ip mtu exception forward exit interface 100ge0/55 fec mode rs auto link-aggregation 250 active exit interface 100ge0/56 fec mode rs auto link-aggregation 250 active exit !end
interface link-aggregation64 switchport mode trunk switchport trunk allowed vlan add 1,11-12 switchport trunk pvid vlan 1 vxlan 100 encapsulation vlan 11 vxlan 200 encapsulation vlan 12 mlag group 1 exit interface link-aggregation250 switchport mode trunk switchport trunk allowed vlan add 11-13 switchport trunk pvid vlan 1 mlag peer-link exit
interface dc0 ip address 192.168.168.41 255.255.0.0 exit
interface loopback1 description Router-ID ip address 10.254.127.12 255.255.255.255 exit
interface loopback2 description VTEP ip address 10.254.128.12 255.255.255.255 ip address 10.254.128.10 255.255.255.255 secondary exit
interface loopback100 ip vrf forwarding 100 ip address 172.16.250.12 255.255.255.255 exit
interface vxlan100 ip vrf forwarding 100 vxlan distribute-gateway ip address 172.16.129.1 255.255.255.0 mac-address 0001.0001.0100 exit
interface vxlan200 ip vrf forwarding 100 vxlan distribute-gateway ip address 172.16.130.1 255.255.255.0 mac-address 0001.0001.0100 exit
interface nve1 source 10.254.128.10 mac-address 0088.9999.1112 vxlan 100,200 ingress-replication protocol bgp exit
router ospf 100 router-id 10.254.127.12 bfd all-interfaces network 10.254.1.3 0.0.0.0 area 0 network 10.254.2.3 0.0.0.0 area 0 network 10.254.127.12 0.0.0.0 area 0 network 10.254.128.10 0.0.0.0 area 0 maximum-paths 8 exit
router bgp 100 no auto-summary no synchronization bgp router-id 10.254.127.12 neighbor spine peer-group neighbor spine remote-as 100 neighbor spine update-source loopback1 neighbor 10.254.127.1 peer-group spine neighbor 10.254.127.2 peer-group spine maximum-paths ibgp 8 address-family l2vpn evpn neighbor spine activate neighbor spine send-community both neighbor 10.254.127.1 activate neighbor 10.254.127.1 send-community both neighbor 10.254.127.2 activate neighbor 10.254.127.2 send-community both exit-address-family address-family ipv4 vrf 100 advertise-l2vpn-evpn network 172.16.129.0 255.255.255.0 network 172.16.130.0 255.255.255.0 network 172.16.250.11 255.255.255.255 network 172.16.250.12 255.255.255.255 exit-address-family exit |
Spine Maipu:
Maipu-S1 |
Maipu-S2 |
|
ip vrf 100 rd 100:3 l3vnid 1 address-family evpn route-target import 100:2 ipv4 route-target export 100:2 ipv4 exit exit
vxlan 100 vxlan vnid 100 address-family evpn rd 100:1 route-target import 100:1 route-target export 100:1 exit exit vxlan 200 vxlan vnid 200 address-family evpn rd 200:1 route-target import 200:1 route-target export 200:1 exit exit
interface 100ge0/1 no switchport ip address 10.254.1.0 255.255.255.254 no ip redirects ip ospf network point-to-point ip ospf bfd ip mtu exception forward exit
interface 100ge0/2 no switchport ip address 10.254.1.2 255.255.255.254 no ip redirects ip ospf network point-to-point ip ospf bfd ip mtu exception forward exit
interface 100ge0/6 no switchport ip address 10.254.1.4 255.255.255.254 no ip redirects ip ospf network point-to-point ip ospf bfd ip mtu exception forward exit
interface 100ge0/10 no switchport ip address 10.254.1.6 255.255.255.254 no ip redirects ip ospf network point-to-point ip ospf bfd ip mtu exception forward exit
interface 100ge0/14 no switchport ip address 10.254.100.1 255.255.255.0 ip ospf mtu-ignore exit !end
interface loopback1 description Router-ID ip address 10.254.127.1 255.255.255.255 exit
router ospf 100 router-id 10.254.127.1 bfd all-interfaces network 10.254.1.0 0.0.0.0 area 0 network 10.254.1.2 0.0.0.0 area 0 network 10.254.1.4 0.0.0.0 area 0 network 10.254.1.6 0.0.0.0 area 0 network 10.254.100.0 0.0.0.255 area 0 network 10.254.127.1 0.0.0.0 area 0 exit
router bgp 100 no auto-summary no synchronization neighbor leaf peer-group neighbor leaf remote-as 100 neighbor leaf update-source loopback1 neighbor leaf advertisement-interval 0 neighbor 10.253.127.3 remote-as 200 neighbor 10.253.127.3 ebgp-multihop 10 neighbor 10.253.127.3 update-source loopback1 neighbor 10.253.127.3 advertisement-interval 0 neighbor 10.253.127.3 attribute-unchanged neighbor 10.254.127.11 peer-group leaf neighbor 10.254.127.12 peer-group leaf neighbor 10.254.127.21 peer-group leaf address-family l2vpn evpn neighbor 10.253.127.3 activate neighbor 10.253.127.3 send-community both neighbor 10.253.127.3 attribute-unchanged neighbor 10.254.127.11 activate neighbor 10.254.127.11 route-reflector-client neighbor 10.254.127.11 send-community both neighbor 10.254.127.12 activate neighbor 10.254.127.12 route-reflector-client neighbor 10.254.127.12 send-community both neighbor 10.254.127.21 activate neighbor 10.254.127.21 route-reflector-client neighbor 10.254.127.21 send-community both exit-address-family address-family ipv4 vrf 100 advertise-l2vpn-evpn exit-address-family exit |
ip vrf 100 rd 100:3 l3vnid 1 address-family evpn route-target import 100:2 ipv4 route-target export 100:2 ipv4 exit exit
vxlan 100 vxlan vnid 100 address-family evpn rd 100:2 route-target import 100:1 route-target export 100:1 exit exit vxlan 200 vxlan vnid 200 address-family evpn rd 200:2 route-target import 200:1 route-target export 200:1 exit exit
interface 100ge0/1 no switchport ip address 10.254.2.0 255.255.255.254 no ip redirects ip ospf network point-to-point ip ospf bfd ip mtu exception forward exit
interface 100ge0/2 no switchport ip address 10.254.2.2 255.255.255.254 no ip redirects ip ospf network point-to-point ip ospf bfd ip mtu exception forward exit
interface 100ge0/6 no switchport ip address 10.254.2.4 255.255.255.254 no ip redirects ip ospf network point-to-point ip ospf bfd ip mtu exception forward exit
interface 100ge0/10 no switchport ip address 10.254.2.6 255.255.255.254 no ip redirects ip ospf network point-to-point ip ospf bfd ip mtu exception forward exit
interface 100ge0/14 no switchport ip address 10.254.100.2 255.255.255.0 ip ospf mtu-ignore exit !end
interface loopback1 description Router-ID ip address 10.254.127.2 255.255.255.255 exit
router ospf 100 router-id 10.254.127.2 bfd all-interfaces network 2.2.2.2 0.0.0.0 area 0 network 10.254.2.0 0.0.0.0 area 0 network 10.254.2.2 0.0.0.0 area 0 network 10.254.2.4 0.0.0.0 area 0 network 10.254.100.0 0.0.0.255 area 0 network 10.254.127.2 0.0.0.0 area 0 exit
router bgp 100 no auto-summary no synchronization neighbor leaf peer-group neighbor leaf remote-as 100 neighbor leaf update-source loopback1 neighbor leaf advertisement-interval 0 neighbor 10.253.127.3 remote-as 200 neighbor 10.253.127.3 ebgp-multihop 10 neighbor 10.253.127.3 update-source loopback1 neighbor 10.253.127.3 advertisement-interval 0 neighbor 10.253.127.3 attribute-unchanged neighbor 10.254.127.11 peer-group leaf neighbor 10.254.127.12 peer-group leaf neighbor 10.254.127.21 peer-group leaf address-family l2vpn evpn neighbor 10.253.127.3 activate neighbor 10.253.127.3 send-community both neighbor 10.253.127.3 attribute-unchanged neighbor 10.254.127.11 activate neighbor 10.254.127.11 route-reflector-client neighbor 10.254.127.11 send-community both neighbor 10.254.127.12 activate neighbor 10.254.127.12 route-reflector-client neighbor 10.254.127.12 send-community both neighbor 10.254.127.21 activate neighbor 10.254.127.21 route-reflector-client neighbor 10.254.127.21 send-community both exit-address-family address-family ipv4 vrf 100 advertise-l2vpn-evpn exit-address-family exit |
Результат
Хосты друг друга пингуют, прочий трафик тоже ходит. Честно говоря, в этом мы особенно и не сомневались. Опасения вызывало то, как поймут друг друга фабрики, в которых присутствует с одной стороны VPC, а с другой Multi-Chassis LAG. Подёргав интерфейсы с каждой из сторон, убедились в том, что всё работает корректно. Впрочем, на случай проблем с MLAG, на коммутаторах Maipu серии ЦОД доступно стекирование.
Вот так выглядят маршруты на Nexus:
|
N9k-Leaf1# sh bgp l2vpn evpn BGP routing table information for VRF default, address family L2VPN EVPN BGP table version is 1286, Local Router ID is 10.253.127.1 Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - best2
Network Next Hop Metric LocPrf Weight Path Route Distinguisher: 100:1 *>i[2]:[0]:[0]:[48]:[0001.0001.0100]:[0]:[0.0.0.0]/216 10.254.128.20 0 100 0 100 i *>i[2]:[0]:[0]:[48]:[0001.0001.0100]:[32]:[172.16.129.1]/272 10.254.128.20 0 100 0 100 i *>i[3]:[0]:[32]:[10.254.128.20]/88 10.254.128.20 0 100 0 100 i
Route Distinguisher: 100:10 *>i[2]:[0]:[0]:[48]:[0001.0001.0100]:[0]:[0.0.0.0]/216 10.254.128.10 0 100 0 100 i *>i[2]:[0]:[0]:[48]:[dc4a.3e3b.cbf8]:[0]:[0.0.0.0]/216 10.254.128.10 0 100 0 100 i *>i[2]:[0]:[0]:[48]:[0001.0001.0100]:[32]:[172.16.129.1]/272 10.254.128.10 0 100 0 100 i *>i[2]:[0]:[0]:[48]:[dc4a.3e3b.cbf8]:[32]:[172.16.129.11]/272 10.254.128.10 0 100 0 100 i *>i[3]:[0]:[32]:[10.254.128.10]/88 10.254.128.10 0 100 0 100 i
Route Distinguisher: 100:40 (L2VNI 100) *>i[2]:[0]:[0]:[48]:[0001.0001.0100]:[0]:[0.0.0.0]/216 10.254.128.10 0 100 0 100 i * i 10.254.128.20 0 100 0 100 i *>i[2]:[0]:[0]:[48]:[dc4a.3e3b.cbf8]:[0]:[0.0.0.0]/216 10.254.128.10 0 100 0 100 i *>i[2]:[0]:[0]:[48]:[0001.0001.0100]:[32]:[172.16.129.1]/272 10.254.128.10 0 100 0 100 i * i 10.254.128.20 0 100 0 100 i *>i[2]:[0]:[0]:[48]:[dc4a.3e3b.cbf8]:[32]:[172.16.129.11]/272 10.254.128.10 0 100 0 100 i *>l[3]:[0]:[32]:[10.253.127.40]/88 10.253.127.40 100 32768 i *>i[3]:[0]:[32]:[10.254.128.10]/88 10.254.128.10 0 100 0 100 i *>i[3]:[0]:[32]:[10.254.128.20]/88 10.254.128.20 0 100 0 100 i
Route Distinguisher: 200:10 *>i[2]:[0]:[0]:[48]:[0001.0001.0100]:[0]:[0.0.0.0]/216 10.254.128.10 0 100 0 100 i *>i[2]:[0]:[0]:[48]:[0001.0001.0100]:[32]:[172.16.130.1]/272 10.254.128.10 0 100 0 100 i *>i[3]:[0]:[32]:[10.254.128.10]/88 10.254.128.10 0 100 0 100 i
Route Distinguisher: 200:40 (L2VNI 200) *>i[2]:[0]:[0]:[48]:[0001.0001.0100]:[0]:[0.0.0.0]/216 10.254.128.10 0 100 0 100 i *>l[2]:[0]:[0]:[48]:[dc4a.3e3b.cbe9]:[0]:[0.0.0.0]/216 10.253.127.40 100 32768 i *>i[2]:[0]:[0]:[48]:[0001.0001.0100]:[32]:[172.16.130.1]/272 10.254.128.10 0 100 0 100 i *>l[2]:[0]:[0]:[48]:[dc4a.3e3b.cbe9]:[32]:[172.16.130.11]/272 10.253.127.40 100 32768 i *>l[3]:[0]:[32]:[10.253.127.40]/88 10.253.127.40 100 32768 i *>i[3]:[0]:[32]:[10.254.128.10]/88 10.254.128.10 0 100 0 100 i
Route Distinguisher: 100:42 (L3VNI 1) *>i[2]:[0]:[0]:[48]:[0001.0001.0100]:[0]:[0.0.0.0]/216 10.254.128.10 0 100 0 100 i *>i[2]:[0]:[0]:[48]:[0001.0001.0100]:[32]:[172.16.129.1]/272 10.254.128.10 0 100 0 100 i * i 10.254.128.20 0 100 0 100 i *>i[2]:[0]:[0]:[48]:[0001.0001.0100]:[32]:[172.16.130.1]/272 10.254.128.10 0 100 0 100 i *>i[2]:[0]:[0]:[48]:[dc4a.3e3b.cbf8]:[32]:[172.16.129.11]/272 10.254.128.10 0 100 0 100 i *>l[5]:[0]:[0]:[24]:[172.16.129.0]/224 10.253.127.40 100 32768 i *>l[5]:[0]:[0]:[24]:[172.16.130.0]/224 10.253.127.40 100 32768 i |
А вот вывод с Maipu:
|
L11#show bgp l2vpn evpn all all-type BGP local router ID is 10.254.127.11
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, S Stale Origin codes: i - IGP, e - EGP, ? - incomplete EVPN Information for Route Distinguisher:100:1 MAC/IP Advertisement Routes: Network(ETID:MAC:IP) Next Hop Metric LocPrf Weight Path [B]*>i0:48:0001.0001.0100:0:0.0.0.0/96 10.254.128.20 0 100 0 i [B]* i 10.254.128.20 0 100 0 i [B]*>i0:48:0001.0001.0100:32:172.16.129.1/128 10.254.128.20 0 100 0 i [B]* i 10.254.128.20 0 100 0 i Inclusive Multicast Ethernet Tag Routes: Network(Originating IP Addr) Next Hop Metric LocPrf Weight Path [B]*>i0:32:10.254.128.20/72 10.254.128.20 0 100 0 i [B]* i 10.254.128.20 0 100 0 i EVPN Information for Route Distinguisher:100:2 MAC/IP Advertisement Routes: Network(ETID:MAC:IP) Next Hop Metric LocPrf Weight Path [B]*> 0:48:0001.0001.0100:32:172.16.129.1/128 0.0.0.0 0 32768 i [B]* i 10.254.128.20 0 100 0 i [B]*> 0:48:dc4a.3e3b.cbf8:32:172.16.129.11/128 0.0.0.0 0 32768 i [B]*> 0:48:0001.0001.0100:32:172.16.130.1/128 0.0.0.0 0 32768 i [B]*>i0:48:dc4a.3e3b.cbe9:32:172.16.130.11/128 10.253.127.40 0 100 0 200 i IP Prefix Routes: Network(ETID:IP) Next Hop Metric LocPrf Weight Path [B]*> 0:24:172.16.129.0/72 0.0.0.0 0 32768 i [B]*> 0:24:172.16.130.0/72 0.0.0.0 0 32768 i EVPN Information for Route Distinguisher:100:10 MAC/IP Advertisement Routes: Network(ETID:MAC:IP) Next Hop Metric LocPrf Weight Path [B]*> 0:48:0001.0001.0100:0:0.0.0.0/96 0.0.0.0 0 32768 i [B]* i 10.254.128.20 0 100 0 i [B]*> 0:48:dc4a.3e3b.cbf8:0:0.0.0.0/96 0.0.0.0 0 32768 i [B]*> 0:48:0001.0001.0100:32:172.16.129.1/128 0.0.0.0 0 32768 i [B]* i 10.254.128.20 0 100 0 i [B]*> 0:48:dc4a.3e3b.cbf8:32:172.16.129.11/128 0.0.0.0 0 32768 i Inclusive Multicast Ethernet Tag Routes: Network(Originating IP Addr) Next Hop Metric LocPrf Weight Path [B]*>i0:32:10.253.127.40/72 10.253.127.40 0 100 0 200 i [B]*> 0:32:10.254.128.10/72 0.0.0.0 0 32768 i [B]*>i0:32:10.254.128.20/72 10.254.128.20 0 100 0 i EVPN Information for Route Distinguisher:100:40 Inclusive Multicast Ethernet Tag Routes: Network(Originating IP Addr) Next Hop Metric LocPrf Weight Path [B]* i0:32:10.253.127.40/72 10.253.127.40 0 100 0 200 i [B]*>i 10.253.127.40 0 100 0 200 i EVPN Information for Route Distinguisher:200:10 MAC/IP Advertisement Routes: Network(ETID:MAC:IP) Next Hop Metric LocPrf Weight Path [B]*> 0:48:0001.0001.0100:0:0.0.0.0/96 0.0.0.0 0 32768 i [B]*>i0:48:dc4a.3e3b.cbe9:0:0.0.0.0/96 10.253.127.40 0 100 0 200 i [B]* 0.0.0.0 0 32768 i [B]*>i0:48:dc4a.3e3b.cbf8:0:0.0.0.0/96 10.253.127.40 0 100 0 200 i [B]*> 0:48:0001.0001.0100:32:172.16.130.1/128 0.0.0.0 0 32768 i [B]*>i0:48:dc4a.3e3b.cbe9:32:172.16.130.11/128 10.253.127.40 0 100 0 200 i Inclusive Multicast Ethernet Tag Routes: Network(Originating IP Addr) Next Hop Metric LocPrf Weight Path [B]*>i0:32:10.253.127.40/72 10.253.127.40 0 100 0 200 i [B]*> 0:32:10.254.128.10/72 0.0.0.0 0 32768 i EVPN Information for Route Distinguisher:200:40 MAC/IP Advertisement Routes: Network(ETID:MAC:IP) Next Hop Metric LocPrf Weight Path [B]*>i0:48:dc4a.3e3b.cbe9:0:0.0.0.0/96 10.253.127.40 0 100 0 200 i [B]* i 10.253.127.40 0 100 0 200 i [B]* i0:48:dc4a.3e3b.cbf8:0:0.0.0.0/96 10.253.127.40 0 100 0 200 i [B]*>i 10.253.127.40 0 100 0 200 i [B]*>i0:48:dc4a.3e3b.cbe9:32:172.16.130.11/128 10.253.127.40 0 100 0 200 i [B]* i 10.253.127.40 0 100 0 200 i Inclusive Multicast Ethernet Tag Routes: Network(Originating IP Addr) Next Hop Metric LocPrf Weight Path [B]* i0:32:10.253.127.40/72 10.253.127.40 0 100 0 200 i [B]*>i 10.253.127.40 0 100 0 200 i |
Проверили доступность хостов и по L2, при помещении их в один VXLAN-сегмент.
Выводы
Главный вывод - мы не напоролись ни на какой подводный камень. Получили работающий Asymmetric Routing & Bridging. Фабрику Cisco можно нарастить подами Maipu, пусть и при определённых ограничениях.
Хорошим продолжением было бы тестирование такой схемы на пилотном сегменте реальной сети так, чтобы подключить побольше хостов разных типов, получить много MAC-адресов, маршрутов и реальный трафик.
На момент написания поста мы уже получили в лабораторию набор оборудования Maipu, включающий коммутаторы линейки ЦОД и готовы тестировать более конкретные, жизненные кейсы для наших заказчиков.
Кроме того, 18 октября у нас пройдёт онлайн-митап, на котором мы расскажем о доступных сегодня в России альтернативах и на примере Maipu продемонстрируем, как они справляются с типовыми кейсами заказчиков.