[Hands-On] Network Automation with ServiceNow® & OpenDaylight

by Miroslav Kováč | Leave us your feedback on this post!

PANTHEON.tech s.r.o., its products or services, are not affiliated with ServiceNow®, neither is this post an advertisement of ServiceNow® or its products.

ServiceNow® is a complex cloud application, used to manage companies, their employees, and customers. It was designed to help you automate the IT aspects of your business – service, operations, and business management. It creates incidents where using flows, you can automate part of the work that is very often done manually. All this can be easily set up by any person, even if you are not a developer.

An Example

If a new employee is hired in the company, he will need access to several things, based on his position. An incident will be created in ServiceNow® by HR. This will trigger a pre-created, generic flow, which might, for example, notify his direct supervisor (probably manager) and he would be asked to approve this request of access.

Once approved, the flow may continue and set everything up for this employee. It may notify the network engineer, to provision the required network services like (VPN, static IPs, firewall rules, and more), in order to give a new employee a computer. Once done, he will just update the status of this task to done, which may trigger another action. It can automatically give him access to the company intranet. Once everything is done, it will notify everyone it needs to, about a successful job done, with an email or any other communication resource the company is using.

Showing the ServiceNow® Flow Designer

 

Setting Up the Flow

Let’s take it a step further, and try to replace the network engineer, who has to manually configure the services needed for the device.

In a simple environment with a few network devices, we could set up the ServiceNow® Workflow, so that it can access them directly and edit the configuration, according to the required parameters.

In a complex, multi-tenant environment we could leverage a network controller, that can serve the required service and maintain the configuration of several devices. This will make the required service functional. In that case, we will need ServiceNow® to communicate with the controller, which secures this required network service.

The ServiceNow® orchestration understands and reads REST, OpenDaylight & lighty.io – in our case, the controller. It provides us with the RESTCONF interface, with which we can easily integrate ServiceNow®, OpenDaylight, or lighty.io, thanks to the support of both these technologies.

Now, we look at how to simplify this integration. For this purpose, we used OpenAPI.

This is one of the features, thanks to which we can generate a JSON according to the OpenAPI specification for every OpenDaylight/lighty.io application with RESTCONF, which we can then import into ServiceNow®.

If your question is, whether it is possible to integrate a network controller, for example, OpenDaylight or lighty.io, the answer is yes. Yes, it is.

Example of Network Automation

Let’s say we have an application with a UI, that will let us manage the network with a control station. We want to connect a new device to it and set up its interfaces. Manually, you would have to make sure that the device is running. If not, we have to contact IT support to plug it in, create a request to connect to it. Once done, we have to create another request to set up the interfaces and verify the setup.

Using flows in ServiceNow® will let you do all that automatically. All your application needs to do, is create an incident in ServiceNow ®. This incident would be set up as a trigger, for a flow to start. It would try to create a connection using a REST request, that would be chosen from API operations, which we have from our OpenAPI JSON. This was automatically generated from YANG files, that are used in the project.

If a connection fails, then it would automatically send an email to IT support, creating a new, separate incident, that would have to be marked as done before this flow can continue. Once done, we can try to connect again using the same REST. When the connection is successful, we can choose a new API operation again, that would process the interfaces.

After that, we can choose another API operation that would get all the created settings and send that to the person, that created this incident using an email and mark this incident as done.

OpenAPI & oneOf

Showing the ServiceNow® API Operation

Since the “New York” release of ServiceNow®, the import of OpenAPI is a new feature, it has some limitations.

During usage, we noticed a few inconsistencies, which we would like to share with you. Here are some tips, what you should look out for when using this feature.

OpenAPI & ServiceNow®

OpenAPI supports the oneOf feature, which is something that is needed for choice keywords in YANG. You can choose, which nodes you want to use. Currently, the workaround for this is to use the Swagger 2.0 implementation, which does not support the oneOf feature and will list all the cases that exist in a choice statement. If you go to input variables, you may delete any input variables that you don’t want yourself.

JSONs & identical item names

Another issue is when we have a JSON that contains the same item names in different objects or levels. So if I need the following JSON:

{
    "username": "foo",
    "password": "bar":,
    "another-log-in": {
        "username": "foo",
        "password": "bar"
    }
}

The workaround is, to add other input variables manually, that will have the same name, like the one that is missing. Suddenly, it may appear twice in input variables, but during testing, it appears only once – where it’s supposed to. Therefore, you need to manually fill in all the missing variables using the “+” button in the input variables tab.we have the username and password twice. However, it would appear in the input variables just once. When testing the action, I was unable to fill it in like the above JSON.

showing the ServiceNow® inputs

Input Variables in ServiceNow®

The last issue that we have, is with ServiceNow® not requiring input variables. Imagine you create an action with REST Step. If there are some variables that you don’t need to set up, you would normally not assign any value to that variable and it would not be set.

Here, it would automatically set it to a default value or an empty string if there is no default value, which can cause some problems with decimals as well – since you should not put strings into a decimal variable.

Again, the workaround is to remove all the input variables, that you are not going to use.

This concludes our network automation with the ServiceNow guide. Leave us your feedback on this post!


You can contact us at https://pantheon.tech/

Explore our Pantheon GitHub.

Watch our YouTube Channel.

PANTHEON.tech & OpenDaylight

PANTHEON.tech & OpenDaylight – dedication and continuous support

The year was 2001, when PANTHEON.tech, the software research, and development company was established in Bratislava, Slovakia. It had a focus on computer network technologies with love and care for open-source software development.

12 years into the life of PANTHEON.tech, on 22 March 2013 to be exact, OpenDaylight (ODL) project has seen the first daylight with the initial code drop. It had the ambition to be the biggest and most successful, open-source, Software Defined Networking (SDN) controller in the world.

ODL is a collaborative open-source project aimed to speed up the adoption of SDN and create a solid foundation for Network Functions Virtualization (NFV). ODL was founded by the global industry leaders such as Cisco, Ericsson, Intel, IBM, Dell, HP, Red Hat, Microsoft and PANTHEON.tech and open to all.

PANTHEON.tech’s nurture for OpenDaylight goes back, to when it was forming. In a sense, PANTHEON.tech had led the way to define how an SDN controller is and should be. This requires dedication, which was proven over the years with an extensive amount of contribution thanks to its expert developers.

In ODL’s lifespan, PANTHEON.tech had always been within the top 5 contributors.

Considering the committer’s view on the left chart, the top committer proved his dedication, by having over 20% of the total commits single-handedly – Robert Varga.

State of Commits

PANTHEON.tech had made 12,098 commits with sharp developers in over 50 ODL projects throughout OpenDaylight’s lifespan. That is 740,919 added and 664,588 removed lines of code.

That translates roughly around 20 great novels written in.

ODL continues to be a great example of what open-source software is and how international contributors can collaborate beautifully to create the next great thing.

In the last week of November 2018, Bitergia, a software development analytics company, published a report on the past and current status of the OpenDaylight project, which plays a significant role in PANTHEON.tech’s offerings and solutions.

PANTHEON.tech’s CTO Robert Varga is leading the list of per-user-contributions to the source code of OpenDaylight, with over 980 commits to the source code in Q3 of 2018. This achievement further establishes PANTHEON.tech’s position as one of the largest contributors to the OpenDaylight project.

As for the list of companies that contribute to the source code of OpenDaylight, PANTHEON.tech is the 2nd largest contributor for Q3/2018, with 1034 commits. We were just 34 commits shy of the top contributor position, which belongs to Red Hat.

Inside OpenDaylight

Now, let’s touch on the OpenDaylight’s internal governance, TSCs, and PTLs, etc.

ODL is now a founding member of LF Networking (LFN), an entity that integrates the governance of participating projects to enhance operational excellence, simplify member engagement, and increase collaboration across open source networking projects and standards bodies.

OpenDaylight Technical Steering Committee (TSC) provides leadership regarding the technical direction of the ODL platform, as well as guidance on collaborative practices. Developers are elected by the ODL community to serve one-year terms. Currently, there are only 13 TSC members on duty and proudly a PANTHEON.tech’s CTO – Robert Varga is one of them.

Project Technical Lead (PTL) is the leader of the OpenDaylight projects, who gets to decide if any of the proposed changes become part of the project or not. They are elected by the committers thanks to their expertise in the field.

Anyone can contribute to ODL which is an open-source project, but PTLs are the ones who lead the project to the prospected direction. If you want to learn how to get started with OpenDaylight, you can read more as a developer or as a user.

ODL had started with just five projects and thrived. Currently, there are 91 active projects and PANTHEON.tech proudly involved, contributed to a very big proportion of it. Just another example of its expertise on the subject.

Apart from PANTHEON.tech’s top contribution to ODL, PANTHEON.tech has always been a strong supporter of the open-source community. This support covers a wide range of developers from local private open-source development groups to international open-source platforms. Such examples PANTHEON.tech continuously supports our Local Open Source Networking User Groups, (OSNUG) OpenDaylight User Groups, OSS weekend meetings to name a few. But, the most generous of PANTHOEN.tech was to open-source one of its best developments, called lighty.io.

Future of OpenDaylight

The development of OpenDaylight Sodium is already producing significant improvements, some of which are finding their way to the upcoming Service Release of Neon.

These are the test results for OpenDaylight Neon SR2. We have recorded significant improvements in the following areas:

  • Datastore snapshot-size: ~49% reduction
  • Processing time: ~58% reduction
  • In-memory size: ~25% reduction
  • Object count: ~25% reduction
  • NodeIdentifier: ~99.9% reduction
  • AugmentationIdentifier: ~99.9% reduction

These enhancements will also be present in the future, the long-awaited release of Sodium.

YANG Tools 2.0.1 integrated in ODL Oxygen

YANG Tools 2.0.1 integrated in OpenDaylight Oxygen

OpenDaylight’s YANG Tools project, forms the bottom-most layer of OpenDaylight as an application platform. It defines and implements interfaces for modeling, storing and transforming data modeled in RFC7950, known as YANG 1.1 — such as a YANG parser and compiler.

What is YANG Tools?

Pantheon engineers started developing yangtools some 5 years ago. It originally supported RFC6020, going through a number of different versions. After releasing yangtools-1.0.0, we introduced semantic versioning as an API contract. Since then, we have retrofitted original RFC6020 meta-model to support RFC7950. We also implemented the corresponding parser bits, which were finalized in yangtools-1.2.0 and shipped with the Nitrogen Simultaneous Release.

This release entered its development phase on August 14th 2017. yangtools-2.0.0 was released on November 27th 2017, which is when the search of an integration window started. Even though we had the most critical downstream integration patches prepared, most of down-streams did not have their patches even started. Integration work and coordination was quickly escalated to the TSC. The integration finally kicked off on January 11, 2018.

Integration was mostly complicated by the fact that odlparent-3.0.x was riding with us, along with the usual Karaf/Jetty/Jersey/Jackson integration mess. It is now sorted out, with  yangtools-2.0.1 being the release to be shipped in the Oxygen simultaneous Release.

What is new in yangtools-2.0.1?

  • 309 commits
  • 2009 files changed
  • 54126 insertions(+)
  • 45014 deletions(-)

The most user-visible change is that in-memory data tree now enforces mandatory leaf node presence for operational store by default. This can be tweaked via the DataTreeConfiguration interface on a per-instance basis, if need be, but we recommend against switching it off.

For downstream users using karaf packaging, we have split our features into stable and experimental ones. Stable features are available from features-yangtools and contain the usual set of functionality, which will only expand in its capabilities. Experimental features are available from features-yangtools-experimental and carry functionality which is not stabilized yet and may get removed — this currently includes ObjectCache, which is slated for removal, as Guava’s Interners are better suited for the job.

Users of yang-maven-plugin will find that YANG files packaged in jars now have their names normalized to RFC7950 guidelines. This includes using the actual module or submodule name as well as capturing the revision in the filename.

API Changes

From API change perspective, there are two changes which stand out. We have pruned all deprecated methods and all YANG 1.1 API hacks marked with ‘FIXME: 2.0.0’ have been cleared up. This results in better ergonomics for both API users and implementors.

yang-model-api has seen some incompatible changes, ranging from renaming of AugmentationNode, TypedSchemaNode and ChoiceCaseNode to some targetted use of Optional instead of nullable returns. Most significant change here is the introduction of EffectiveStatement specializations — I will cover these in detail in a follow-up post, but these have enabled us to do the next significant item.

YANG parser has been refactored into multiple components. Its internal structure changed, in order to hide most of the implementation classes and methods. It is now split into:

  • yang-parser-reactor (language-independent inference pipeline)
  • yang-parser-rfc7950 (hosting baseline RFC6020/RFC7950 parser)
  • yang-parser-impl (being the default-configured parser instance)
  • and a slew of parser extensions (RFC6536, RFC7952, RFC8040)

There is an yang-parser-spi artifact, too, which hosts common namespaces and utility classes, but its layout is far from stabilized. Overall the parser has become a lot more efficient, better at detecting and reporting model issues. Implementing new semantic extensions has become really a breeze.

YANG Codecs

YANG codecs have seen a major shift, with the old XML parser in yang-data-impl removed in favor of yang-data-codec-xml. yang-data-codec-gson gains the ability to parse and emit RFC7951 documents. This allows RFC8040 NETCONF module to come closer to full compliance. Since the SchemaContext is much more usable now, with Modules being indexed by their  NameModule, the codec operations have become significantly faster.

Overall, we are in a much better and cleaner shape. We are currently not looking at a 3.0.0 release anytime soon and can actually deliver incremental improvements to YANG Tools in a much more rapid cadence than previously possible with the entire OpenDaylight simultaneous release cycle being in the way.

We already have another round of changes ready for yangtools-2.0.2 and are looking forward to publishing them.

Robert Varga