This is a feature that seems to exist already for a while in Comware 7 (it works even in Simware, which is pretty old by now), but I just discovered it recently: IP unnumbered interfaces on Ethernet with OSPF point-to-point links.The key advantage is that in the IPv4 routed backbone, no more /30 subnets need to be configured (so no more separate addressing plan either :)) on all those ethernet routed connections between the routers. You just borrow the Loopback IP and go!
LLDP – ARP Rule
The challenge with ethernet is of course that it is a multi-access network, so how can we discover the IP/MAC relation of the peer (ARP)?
This is solved with LLDP on the link. As part of an outgoing LLDP frame, the switch will include the Management Address TLV. This can be set to the Loopback address of the device. The peer device will now get an LLDP frame with the IP address announcement of its neighbor, and it will use the source MAC address of the LLDP frame to make a static ARP record for this MAC/IP of the peer.
This ARP entry allows already for a direct peer ping from loopback to loopback. One problem solved.
Next you need to configure the routed link as an OSPF point-to-point link, so there is no OSPF DR election. When the OSPF mcast is sent out, the source IP will be included, so with the ARP entry, the peer now knows how to reach the peer and establish the OSPF adjacency. Easy and done!
Define a Loopback on sw1
<sw1>dis cur int loop 0 # interface LoopBack0 ip address 18.104.22.168 255.255.255.255 # return
Define a Loopback on sw2
<sw2>dis cur int loop 0 # interface LoopBack0 ip address 22.214.171.124 255.255.255.255 # return
Configure OSPF on both devices, this example just makes a single area 0.
<sw1>dis cur conf ospf # ospf 1 area 0.0.0.0 network 0.0.0.0 255.255.255.255 # return
<sw2>dis cur conf ospf # ospf 1 area 0.0.0.0 network 0.0.0.0 255.255.255.255 # return
Configure the link between the switches:
- as a routed link
- set the IP to the Loopback IP
- configure the Loopback 0 IP as the LLDP management address (outbound)
- enable the programming of static ARP rule for the LLDP management IP/MAC (inbound)
- set as OSPF P2P
<sw1>dis cur int g1/0/1 # interface GigabitEthernet1/0/1 port link-mode route ip address unnumbered interface LoopBack0 ospf network-type p2p lldp management-address arp-learning lldp tlv-enable basic-tlv management-address-tlv interface LoopBack0 # return <sw1>
Repeat this on the peer device and all other routed links in the backbone
<sw2>dis cur int g1/0/1 # interface GigabitEthernet1/0/1 port link-mode route ip address unnumbered interface LoopBack0 ospf network-type p2p lldp management-address arp-learning lldp tlv-enable basic-tlv management-address-tlv interface LoopBack0 # return
Once the links come up, LLDP will do its work, so a new LLDP NEIGHBOR is created. This results in a static (rule based) ARP entry that will be generated for the neighbor. Next OSPF P2P link will establish the adjacency and transition to FULL.
Check out the timing (I know this is just a single link, but still nice to have OSPF FULL fairly quickly after the link is UP)
%Oct 6 13:02:12:939 2016 sw1 IFNET/3/PHY_UPDOWN: Physical state on the interface GigabitEthernet1/0/1 changed to up. %Oct 6 13:02:12:939 2016 sw1 IFNET/5/LINK_UPDOWN: Line protocol state on the interface GigabitEthernet1/0/1 changed to up. %Oct 6 13:02:13:233 2016 sw1 LLDP/6/LLDP_CREATE_NEIGHBOR: Nearest bridge agent neighbor created on port GigabitEthernet1/0/1 (IfIndex 2), neighbor's chassis ID is 24a5-5c86-0200, port ID is GigabitEthernet1/0/1. %Oct 6 13:02:13:242 2016 sw1 OSPF/5/OSPF_NBR_CHG: OSPF 1 Neighbor 126.96.36.199(GigabitEthernet1/0/1) changed from LOADING to FULL.
Some review output screens, this shows the LLDP neighbor details (as you can see, this was done with Simware). Note the Management IPv4 that was announced by the peer:
<sw1>dis ll ne list Chassis ID : * -- -- Nearest nontpmr bridge neighbor # -- -- Nearest customer bridge neighbor Default -- -- Nearest bridge neighbor System Name Local Interface Chassis ID Port ID sw2 GE1/0/1 24a5-5c86-0200 GigabitEthernet1/0/1 <sw1>dis ll ne int g1/0/1 ver <sw1>dis ll ne int g1/0/1 verbose LLDP neighbor-information of port 2[GigabitEthernet1/0/1]: LLDP agent nearest-bridge: LLDP neighbor index : 1 Update time : 0 days, 2 hours, 40 minutes, 30 seconds Chassis type : MAC address Chassis ID : 24a5-5c86-0200 Port ID type : Interface name Port ID : GigabitEthernet1/0/1 Time to live : 121 Port description : GigabitEthernet1/0/1 Interface System name : sw2 System description : H3C Comware Platform Software, Software Version 7.1.059, Alpha 7159 H3C S5820V2-54QS-GE Copyright (c) 2004-2014 Hangzhou H3C Tech. Co., Ltd. All rights reserved. System capabilities supported : Bridge, Router, Customer Bridge, Service Bridge System capabilities enabled : Bridge, Router, Customer Bridge Management address type : IPv4 Management address : 188.8.131.52 Management address interface type : IfIndex Management address interface ID : 1413 Management address OID : 0 Link aggregation supported : Yes Link aggregation enabled : No Aggregation port ID : 0 Auto-negotiation supported : No Auto-negotiation enabled : No OperMau : Speed(0)/Duplex(Unknown) Maximum frame size : 1500 <sw1>
Next the device will you the source MAC of the LLDP frame. Since the L3 interface MAC may be (will be) different from the base system MAC (as announced in the LLDP frame Chassis ID), it would not make sense to use the LLDP Chassis ID MAC. Also keep in mind that the routed port interface MAC may be changed manually to some other MAC, so that scenario also needs to be handled.
This is why the ARP rule will use the source MAC of the LLDP frame. This can be seen with a ‘display interface’ command, but to really verify it, just activate some LLDP and ethernet packet debug on the routed port:
<sw1>debug lldp packet <sw1>debug ethernet packet int g1/0/1 *Oct 6 13:13:16:114 2016 sw1 ETH/7/Eth_rcv: Received an ethernet packet, interface: GigabitEthernet1/0/1, format: 0, src_addr: 24a5-5c86-0206, dst_addr: 0180-c200-000e, payload: 88 cc *Oct 6 13:13:16:114 2016 sw1 LLDP/7/Packet received: Interface GigabitEthernet1/0/1 nearest-bridge; Length is 299. Chassis type : MAC address Chassis ID : 24a5-5c86-0200 Port ID type : Interface name Port ID : GigabitEthernet1/0/1 Time to live : 121
Next review the ARP entry that was created by LLDP. Note the type ‘R’ = Rule. This refers to LLDP that has created this entry for you.
<sw1>dis arp Type: S-Static D-Dynamic O-Openflow R-Rule M-Multiport I-Invalid IP address MAC address VLAN Interface Aging Type 184.108.40.206 24a5-5c86-0206 N/A GE1/0/1 N/A R <sw1>
Then you can verify the OSPF peers:
[sw1]dis ospf peer OSPF Process 1 with Router ID 220.127.116.11 Neighbor Brief Information Area: 0.0.0.0 Router ID Address Pri Dead-Time State Interface 18.104.22.168 22.214.171.124 1 33 Full/ - GE1/0/1 [sw1]
And check out the IP routing table:
[sw1]dis ip routing Destinations : 10 Routes : 10 Destination/Mask Proto Pre Cost NextHop Interface 0.0.0.0/32 Direct 0 0 127.0.0.1 InLoop0 126.96.36.199/32 Direct 0 0 127.0.0.1 InLoop0 188.8.131.52/32 O_INTRA 10 1 184.108.40.206 GE1/0/1 127.0.0.0/8 Direct 0 0 127.0.0.1 InLoop0 127.0.0.0/32 Direct 0 0 127.0.0.1 InLoop0 127.0.0.1/32 Direct 0 0 127.0.0.1 InLoop0 127.255.255.255/32 Direct 0 0 127.0.0.1 InLoop0 220.127.116.11/4 Direct 0 0 0.0.0.0 NULL0 18.104.22.168/24 Direct 0 0 0.0.0.0 NULL0 255.255.255.255/32 Direct 0 0 127.0.0.1 InLoop0 [sw1]
Note on BGP
A similar feature has been added to support BGP next hop handling with this IP Unnumbered system, so the ARP record will be added as a local, direct route. This ensures that the basic BGP next-hop route recursion lookup rule (is next-hop available through regular routing table?) will now be success and BGP routing will also work on the unnumbered link.
You still need to manually define the BGP peer in your BGP configuration (at least on 1 side), but that is just a minor step. This is done with the ‘arp direct-route advertise’ command. (this feature is not available in Simware)
interface GigabitEthernet1/0/1 port link-mode route ip address unnumbered interface LoopBack0 lldp management-address arp-learning lldp tlv-enable basic-tlv management-address-tlv interface LoopBack0 arp route-direct advertise