In our previous blog post, we have introduced you to a Border Gateway Protocol Route-Reflector (BGP-RR) function in SDN controller based on lighty.io. In this article, we’re going to extend the BGP function of an SDN controller with an EVPN extension in the BGP control plane.

Functionality

This article will discuss BGP-EVPN functions in an SDN controller and how the lighty.io BGP function can replace existing legacy route-reflectors running in the service provider’s WAN/DC networks. BGP-EVPN provides:

• Advanced Layer 2 MAC and Layer 3 IP reachability information capabilities in control-plane
• Route-Type 2: advertising MAC/IP address, instead of traditional MAC learning mechanisms
• Route-Type 5: advertising the IP prefix subnet prefix route

We’re going to show you a BGP-EVPN IP subnet routing use-case

A BGP-EVPN control plane can also co-exist with various data-planes, such as MPLS, VXLAN, and PBB.

Use-case: Telecom Data-Center

In this blog, we’re going to show you the BGP-EVPN control plane working together with the VXLAN data plane. The perfect use case for this combination would be a telecom data center.

Virtual Extensible LAN (VXLAN) is an overlay technology for network virtualization. It provides Layer 2 extension over a shared Layer 3 underlay infrastructure network, by using the MAC address in an IP/User Datagram Protocol (MAC in IP/UDP) tunneling encapsulation. The initial IETF VXLAN standards defined a multicast-based flood-and-learn VXLAN without a control plane.

It relies on data-based flood-and-learn behavior for remote VXLAN tunnel endpoint (VTEP) peer-discovery and remote end-host learning. BGP-EVPN, as the control plane for VXLAN, overcomes the limitations of the flood-and-learn mechanism.

Test Bed

In this demo, we will use:

• five Docker containers & three Docker networks.
• Docker auto-generated user-defined bridge networks with mask /16
• Arista’s cEOS software, as we did in our previous demo

Remember, that an Arista cEOS switch creates an EtX port when starting up in the container, which is bridged to the EthX port in Docker.

These auto-generated EtX ports are accessible and configurable from cEOS Cli and on start are in default L2 switching mode. This means they don’t have an IP address assigned.

Well, let’s expand our previous demo topology with few more network elements. Here is a list of Docker containers used in this demo:

• leaf1 & leaf2: WAN switch & access/node
• host1 & host2: Ubuntu VM
• BGP-RR: BGP-EVPN Route Reflector

Here is a list of Docker user-defined networks used in this demo:

• net1 (172.18.0.0/16): connects leaf1 & host1
• net2 (172.19.0.0/16): connects leaf2 & host2
• net3 (172.20.0.0/16: connects leaf1, leaf2 & bgp-rr

Our Goal: Routing

At the end of this blog, we want to be able to reach IP connectivity between virtual machine host1 and host2. For that, we need BGP to advertise loopback networks and VLAN information between nodes.

In this example, we are using one AS-50.

To demonstrate route-reflector EVPN functionality leaf1 & leaf2 doesn’t make an IBGP pair but creates a pair with lighty-BGP instead. This will act as a route reflector. In VxLAN configuration we don’t set up flood vtep. This information should redistribute Route Routing to peers.

The container with lighty-BGP MUST NOT be used as a forwarding node since it doesn’t know the routing table.

Configuration

This demo configuration is prepared and tested on Ubuntu 18.04.2.

Docker Configuration

Before you start, please make sure that you have Docker (download instructions, use version 18.09.6 or higher) & Postman downloaded and installed.

1. Download lighty-BGP Docker image. PANTHEON.tech has its own

https://hub.docker.com/u/pantheontech

sudo docker pull pantheontech/lighty-rr:9.2.0-dev

2. Download the Docker image for Arista cEOS (v. 4.26.1F)

sudo docker import cEOS-lab.tar.xz ceosimage:4.26.1F

3. Download the Ubuntu image from DockerHub

sudo docker pull ubuntu:latest

4. Check the Docker images, successfully installed in the repository

sudo docker images

Preparing the Docker Environment

1. Create Docker networks

sudo docker network create net1
sudo docker network create net2
sudo docker network create net3

2. Check all Docker networks, that have been created

sudo docker network ls

3. Create containers in Docker

sudo docker create --name=bgp-rr --privileged -e INTFTYPE=eth -it pantheontech/lighty-rr:9.2.0-dev
sudo docker create --name=leaf1 --privileged -e INTFTYPE=eth -e ETBA=1 -e SKIP_ZEROTOUCH_BARRIER_IN_SYSDBINIT=1 -e CEOS=1 -e EOS_PLATFORM=ceoslab -e container=docker -i -t ceosimage:4.26.1F /sbin/init systemd.setenv=INTFTYPE=eth systemd.setenv=ETBA=1 systemd.setenv=SKIP_ZEROTOUCH_BARRIER_IN_SYSDBINIT=1 systemd.setenv=CEOS=1 systemd.setenv=EOS_PLATFORM=ceoslab systemd.setenv=container=docker
sudo docker create --name=leaf2 --privileged -e INTFTYPE=eth -e ETBA=1 -e SKIP_ZEROTOUCH_BARRIER_IN_SYSDBINIT=1 -e CEOS=2 -e EOS_PLATFORM=ceoslab -e container=docker -i -t ceosimage:4.26.1F /sbin/init systemd.setenv=INTFTYPE=eth systemd.setenv=ETBA=1 systemd.setenv=SKIP_ZEROTOUCH_BARRIER_IN_SYSDBINIT=1 systemd.setenv=CEOS=2 systemd.setenv=EOS_PLATFORM=ceoslab systemd.setenv=container=docker 
sudo docker create --privileged --name host1 -i -t ubuntu:latest /bin/bash
sudo docker create --privileged --name host2 -i -t ubuntu:latest /bin/bash

4. Connect containers to Docker networks

sudo docker network connect net1 leaf1
sudo docker network connect net1 host1
sudo docker network connect net2 leaf2
sudo docker network connect net2 host2
sudo docker network connect net3 bgp-rr
sudo docker network connect net3 leaf1
sudo docker network connect net3 leaf2

5. Start all containers

sudo docker start leaf1
sudo docker start leaf2
sudo docker start host1
sudo docker start host2
sudo docker start bgp-rr

6. Enable permanent IPv4 forwarding in cEOS containers

sudo docker exec -it leaf1 /sbin/sysctl net.ipv4.conf.all.forwarding=1
sudo docker exec -it leaf2 /sbin/sysctl net.ipv4.conf.all.forwarding=1

7. Check, whether all Docker containers have started successfully

sudo docker container ls

Optional: Use this, if you’re looking for detailed information about running Docker containers (X is replaced by device/host number)

sudo docker container inspect [leaf[X] | bgp-rr | host[X]]

Preparing Ubuntu Environment

1. Get into the machine (X is replaced by device/host number)

sudo docker exec -it host[X] bash

2. Update the machine

apt-get update

3. Install the required packages

apt-get install iproute2
apt-get install iputils-ping

4. Exit the Docker Container (CTRL+D). Repeat steps 2 & 3.

Arista cEOS Switch configuration

Now, we will configure Arista cEOS switches. We will split the configuration of Arista cEOS Switches into several steps.

Click here for full configurations of Arista switches ‘leaf1‘ & ‘leaf2‘.

Ethernet interfaces & connectivity check

1. Go into the Arista switch leaf1

sudo docker exec -it leaf1 Cli

2. Set Privilege, and go to configure-mode

enable
configure terminal

3. Setup the switch’s name

hostname leaf1

4. Setup Ethernet interface. If you use more devices than your devices could be connected to another Ethernet

interface ethernet 2
no switchport
ip address 172.20.0.2/16

5. Check if BGP-RR is reachable from the configured interface.

• When you can’t ping ‘BGP-RR’, check if ‘leaf1′ and ‘BGP-RR’ are located in the same Docker network, or delete the previous step and try it on another Ethernet interface.

ping 172.20.0.4 source ethernet2

6. Repeat Step 5 for ‘leaf2′ & go into the Arista switch leaf2

sudo docker exec -it leaf2 Cli
enable
config t
hostname leaf2
interface ethernet 2 
no switchport
ip address 172.20.0.3/16
ping 172.20.0.4 source ethernet2

Configuring the Border Gateway Protocol

We will have identical configurations for ‘leaf1′ & ‘leaf2′. Exceptions will be highlighted in the instructions below.

1. Enable BGP in Arista switch

• If you are still in the previous settings interface, go to the root of the Arista configuration by repeating the “exit” command.

service routing protocols model multi-agent
ip routing

2. Setup

• For ‘leaf2’, use the Router-ID ‘router-id 172.20.0.3‘

router bgp 50
router-id 172.20.0.2
neighbor 172.20.0.4 remote-as 50
neighbor 172.20.0.4 next-hop-self
neighbor 172.20.0.4 send-community extended
redistribute connected
redistribute attached-host

3. Setup EVPN in BGP

address-family evpn
neighbor 172.20.0.4 activate

Configuring VxLAN Interface & VLAN

We will have identical configurations for leaf1 & leaf2. Exceptions will be highlighted in the instructions below.

1. Enable VLAN with ID 10.

• Make sure that this command is typed in the root of Arista and not in BGP
• If you are still in the BGP configuration, use the command ‘exit’

vlan 10

2. Configure loopback 0, which will be used as a VTEP (VxLAN tunnel endpoint) for VxLAN.

• In ‘leaf2’, use IP ‘10.10.10.2/32’, instead of IP ‘10.10.10.1/32’

interface loopback 0
ip address 10.10.10.1/32

3. Configure VxLAN Interface

• Here we’ll set up loopback 0 as a VTEP and configure VNI (VXLAN Network Identifier) to 3322.

interface Vxlan1
vxlan source-interface Loopback0
vxlan udp-port 4789
vxlan vlan 10 vni 3322
vxlan learn-restrict any

4. Assign Ethernet interface to VLAN

interface Ethernet 1
switchport mode access
switchport access vlan 10

5. Share loopback 0 to BGP-RR

• In ‘leaf2‘, use IP ‘10.10.10.2/32’ instead of ‘10.10.10.1/32’

router bgp 50
address-family ipv4
network 10.10.10.1/32

6. Configure VLAN in BGP

• Here we share the information about VLAN to BGP-RR

router bgp 50
vlan 10
rd 50:3322
route-target both 10:3322
redistribute learned

7. Save your configuration with the ‘wr‘ command in both Arista devices and restart them with the command:

sudo docker restart leaf1 leaf2

lighty.io & BGP Route Reflector

In this part, we will add the Border Gateway Protocol configuration into the lighty.io BGP.

There is a lot to configure, so crucial parts are commented to break it down a little.

If we want to see the logs from lighty.io, we can attach them to the started container:

sudo docker attach bgp-rr

We can start the BGP-RR container with the command:

sudo docker start bgp-rr --attach

to see logs from the beginning. Afterward, send a PUT request to BGP-RR. We should see the following messages in the logs.

More RESTCONF commands can be found here.

Verify device state

Now, we will check if all configurations were set up successfully. We will also check if VxLAN is created and the Virtual PCs can ‘ping’ each other.

1. Check if EVPN BGP peering is established

leaf1(config)#sh bgp evpn summary
BGP summary information for VRF default
Router identifier 172.20.0.2, local AS number 50
Neighbor Status Codes: m - Under maintenance
  Neighbor         V  AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State  PfxRcd PfxAcc
  172.20.0.4       4  50                 3         6    0    0 00:00:09 Estab  0      0

leaf2(config)#sh bgp evpn summary
BGP summary information for VRF default
Router identifier 172.20.0.3, local AS number 50
Neighbor Status Codes: m - Under maintenance
  Neighbor         V  AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State  PfxRcd PfxAcc
  172.20.0.4       4  50               267       315    0    0 00:01:16 Estab  1      1

If your devices are in the state ‘Connected‘ or ‘Active‘, then you have checked this right after you sent a request to lighty.io. Usually, it takes, at most, one minute to establish a connection.

If you still see this state, then there could be something wrong with the BGP configuration. Please check your configuration in Arista CLI, by typing the command ‘show running-config‘ and compare it with the full Arista configuration above.

After you verify the Arista configuration, then there could be a problem in the BGP-RR container. This can be fixed by restarting the BGP-RR container.

2. Check ip route for available loopbacks from other devices

leaf1(config)#sh ip route
VRF: default
Codes: C - connected, S - static, K - kernel,
       O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type2, B I - iBGP, B E - eBGP,
       R - RIP, I L1 - IS-IS level 1, I L2 - IS-IS level 2,
       O3 - OSPFv3, A B - BGP Aggregate, A O - OSPF Summary,
       NG - Nexthop Group Static Route, V - VXLAN Control Service,
       DH - DHCP client installed default route, M - Martian,
       DP - Dynamic Policy Route
 
Gateway of last resort is not set
 
 C      10.10.10.1/32 is directly connected, Loopback0
 B I    10.10.10.2/32 [200/0] via 172.20.0.3, Ethernet2
 C      172.20.0.0/16 is directly connected, Ethernet2

leaf2(config)#sh ip route
VRF: default
Codes: C - connected, S - static, K - kernel,
       O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type2, B I - iBGP, B E - eBGP,
       R - RIP, I L1 - IS-IS level 1, I L2 - IS-IS level 2,
       O3 - OSPFv3, A B - BGP Aggregate, A O - OSPF Summary,
       NG - Nexthop Group Static Route, V - VXLAN Control Service,
       DH - DHCP client installed default route, M - Martian,
       DP - Dynamic Policy Route
 
Gateway of last resort is not set
 
 B I    10.10.10.1/32 [200/0] via 172.20.0.2, Ethernet2
 C      10.10.10.2/32 is directly connected, Loopback0
 C      172.20.0.0/16 is directly connected, Ethernet2

3. Check the VxLAN interface, if it creates and contains VTEP

leaf1#sh interfaces vxlan 1
Vxlan1 is up, line protocol is up (connected)
  Hardware is Vxlan
  Source interface is Loopback0 and is active with 10.10.10.1
  Replication/Flood Mode is headend with Flood List Source: EVPN
 Remote MAC learning via EVPN
  VNI mapping to VLANs
  Static VLAN to VNI mapping is
    [10, 3322]      
  Note: All Dynamic VLANs used by VCS are internal VLANs.
        Use 'show vxlan vni' for details.
  Static VRF to VNI mapping is not configured
  Headend replication flood vtep list is:
    10 10.10.10.2    
  VTEP address mask is None

leaf2(config)#sh interfaces vxlan 1
Vxlan1 is up, line protocol is up (connected)
  Hardware is Vxlan
  Source interface is Loopback0 and is active with 10.10.10.2
  Replication/Flood Mode is headend with Flood List Source: EVPN
 Remote MAC learning via EVPN
  VNI mapping to VLANs
  Static VLAN to VNI mapping is
    [10, 3322]      
  Note: All Dynamic VLANs used by VCS are internal VLANs.
        Use 'show vxlan vni' for details.
  Static VRF to VNI mapping is not configured
  Headend replication flood vtep list is:
    10 10.10.10.1    
  VTEP address mask is None

If you don’t see IP in the section ‘Headend replication flood vtep list is:‘, then the BGP-RR container is not started correctly. This problem can be fixed by removing the BGP-RR container and starting it again.

Restarting BGP-RR container

1. Stop the container

sudo docker stop bgp-rr

2. Remove BGP-RR container

sudo docker rm bgp-rr

3. Create a new container

sudo docker create --name=bgp-rr --privileged -e INTFTYPE=eth -it pantheontech/lighty-rr:9.2.0-dev

4. Connect BGP-RR to docker network

sudo docker network connect net3 bgp-rr

5. Start the container again

sudo docker start bgp-rr

Optional: If you want to see logs from light.io, attached them to the container:

sudo docker attach bgp-rr

Testing IP Connectivity

If everything worked out, we can test IP connectivity in a virtual PC.

1. Open Virtual PC host1

sudo docker exec -it host1 bash

2. Setup IP address for this device

ip addr add 31.1.1.1/24 dev eth1

3. Perform the same configuration at host2

sudo docker exec -it host1 bash
ip addr add 31.1.1.2/24 dev eth1

4. Try to ping host2 to host1

ping 31.1.1.1

root@e344ec43c089:/# ip route
default via 172.17.0.1 dev eth0
31.1.1.0/24 dev eth1 proto kernel scope link src 31.1.1.2
172.17.0.0/16 dev eth0 proto kernel scope link src 172.17.0.5
172.19.0.0/16 dev eth1 proto kernel scope link src 172.19.0.3
 
root@e344ec43c089:/# hostname -I
172.17.0.5 172.19.0.3 31.1.1.2
 
root@e344ec43c089:/# ping 31.1.1.1
PING 31.1.1.1 (31.1.1.1) 56(84) bytes of data.
64 bytes from 31.1.1.1: icmp_seq=1 ttl=64 time=114 ms
64 bytes from 31.1.1.1: icmp_seq=2 ttl=64 time=55.5 ms
64 bytes from 31.1.1.1: icmp_seq=3 ttl=64 time=53.0 ms
64 bytes from 31.1.1.1: icmp_seq=4 ttl=64 time=56.1 ms
^C
--- 31.1.1.1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3005ms
rtt min/avg/max/mdev = 53.082/69.892/114.757/25.929 ms

When we go back to the Arista switch, we can check routed MAC address information.

leaf1#sh mac address-table
          Mac Address Table
------------------------------------------------------------------
 
Vlan    Mac Address       Type        Ports      Moves   Last Move
----    -----------       ----        -----      -----   ---------
  10    0242.211d.8954    DYNAMIC     Et1        1       0:00:54 ago
  10    0242.8b29.b7ea    DYNAMIC     Vx1        1       0:00:40 ago
  10    0242.ac12.0003    DYNAMIC     Et1        1       0:00:14 ago
  10    0242.ac13.0003    DYNAMIC     Vx1        1       0:00:13 ago
  10    ce9a.ca0c.88a1    DYNAMIC     Et1        1       0:00:54 ago
Total Mac Addresses for this criterion: 5
 
          Multicast Mac Address Table
------------------------------------------------------------------
 
Vlan    Mac Address       Type        Ports
----    -----------       ----        -----
Total Mac Addresses for this criterion: 0

leaf2#sh mac address-table
          Mac Address Table
------------------------------------------------------------------
 
Vlan    Mac Address       Type        Ports      Moves   Last Move
----    -----------       ----        -----      -----   ---------
  10    0242.211d.8954    DYNAMIC     Vx1        1       0:00:48 ago
  10    0242.8b29.b7ea    DYNAMIC     Et1        1       0:01:03 ago
  10    0242.ac12.0003    DYNAMIC     Vx1        1       0:00:22 ago
  10    0242.ac13.0003    DYNAMIC     Et1        1       0:00:22 ago
  10    ce9a.ca0c.88a1    DYNAMIC     Vx1        1       0:00:48 ago
Total Mac Addresses for this criterion: 5
 
          Multicast Mac Address Table
------------------------------------------------------------------
 
Vlan    Mac Address       Type        Ports
----    -----------       ----        -----
Total Mac Addresses for this criterion: 0

Conclusion

We have successfully shown the lighty.io BGP functionality, which can replace legacy Route-Reflectors. This situation can be applied to telecom data centers and other use-cases. It demonstrates lighty.io’s versatility and usability. Contact us for more information!

Peter Šuňa & Peter Lučanský

---

You can contact us at https://pantheon.tech/

Explore our Pantheon GitHub.

Watch our YouTube Channel.

lighty YANG Validator

Customers can create, validate and visualize the YANG data model of their application, without the need to call any other external tool – just by using the lighty.io framework.

YANG Tools helps to parse YANG modules, represent the YANG model in Java, and serialize/deserialize YANG model data. However, a custom YANG module can contain improper data that would result in an application failure. To avoid such annoying situations, PANTHEON.tech engineers created the lighty YANG Validator.

lighty.io, a Software Development Kit powered by OpenDaylight and developed by PANTHEON.tech, is a very useful tool to accelerate the development of Software-Defined Networking (SDN) solutions in Java.

Its LightyController component utilizes OpenDaylight’s core components, including YANGtools, that provides a set of tools and libraries for YANG modeling of the network topology, configuration, and state data as defined by YANG 1.0 and YANG 1.1 models.

Prerequisites

1. Download the distribution from this page.
2. Make sure to run the tool in Linux and with Java installed.
3. Unzip the folder and read through the README.md file

What does the lighty YANG Validator offer?

The lighty YANG Validator (lighty-yang-validator) was inspired by pyang – python YANG validation tool. It checks the YANG module using the YANG Parser module. In case of any problem during parsing, the corresponding stack trace is returned to let you know what’s wrong and where.

In addition to the pyang implementation, the lighty YANG Validator, built on top of OpenDaylight’s YANG engine, checks not only the standard YANG compatibility but it validates the given module as a module compatible with lighty.io or OpenDaylight framework.

Users can choose to validate only one module or all modules within the given directory.

It’s not necessary to have all the imported and included modules of a validating module in the same path. It is possible to use -p, –path option with a path or colon-separated paths to needed module(s). YANG Validator can search for modules recursively within the file structure.

Of course, the customer can decide to search for the file just by module name instead of specifying the whole path!

Backwards Compatibility

The lighty YANG Validator allows checking the backward compatibility of the updated YANG module via –check-update-from option. Customers can select to validate backward compatibility according to RFC6020 or RFC7950.

The lighty YANG Validator can be further used for:

• Verification of backward-compatibility for a module
• Notification of users about module status change (removal/deprecation)

Simplify the YANG file

The YANG file is possible to simplify based on the XML payload. The resulting data model can be reduced by removing all nodes that are defined with an “if-feature”. This functionality is very useful with huge YANG files, that are tested with some basic configuration and not all schema nodes are used.

Utilization of such trimmed YANG files can significantly speed uploading of customer’s application in the development phase when the application is started repetitively. Thus, it saves overall development time. A simplified YANG file is printed out to standard output unless the output directory is defined.

User can choose between the following output types:

• Tree in format \<status>–\<flags> \<name>\<opts> \<type> <if-features>
• Name-Revision in format \<module_name>@\<revision>
• List of all modules that validated module depends on
• JSON Tree with all the node information
• HTML Page with javascript for the visualization of the yang tree
• YANG File / simplified YANG file

Goal: Create a stable and reliable custom application

lighty.io was developed to provide a lightweight implementation of core OpenDaylight components so customers are able to run their applications in a plain Java SE environment. PANTHEON.tech keeps working on the improvements for that framework to make its usage as easy as possible to the customers to create stable and reliable applications.

One step forward in this journey is the lighty YANG Validator – customers can create, validate and visualize the YANG data model of their application just by using the lighty.io framework without the need to call any other external tool.