GraphDBs and Network Automation

Today I posted this picture below on LinkedIn saying how cool http://neo4j.com/ is.
Some of my connections asked me to elaborate so here I am.Screenshot 2018-10-02 at 13.55.32

The requirement

You know how I love ‘requirements’, right? What’s mine?

I am building for a project at work a Python/Ansible automation framework for a DC Network Fabric.

I have done this a few times in my career, and each and every time I bumped into the same problem/realisation.

Automation is (kinda) easy. Managing data models is NOT. The complex thing about automation is not how you push commands/configuration to your devices, but rather how you model the intent of the object you’re trying to automate.

Often times I hear people saying “Ansible is SDN” or “Ansible is an orchestration framework”. Nothing can be more wrong IMHO.
The one and only Peter Sprygada explains it way better than anyone else could do: Ansible is a wrench, a tool in your toolchain.

Ansible requires an “inventory” and some “data” to run. From that standpoint Ansible is quite dumb.

“Give me a structured JSON file, and I will change world” it keeps saying to us!

So, how do we go about building that inventory?
You can certainly compile a YAML/JSON that contains all the possible data you need to “hydrate” configuration templates, but if you tried at least once, you’ll know it’s probably not the best idea.

So, how do you do it?

The Solution

Ansible has this amazing – yet simple and powerful – thing called “dynamic inventory”, that is basically the capability of delegating the creation of the data at execution time to a script. Only thing that the script needs to do is to output the JSON-formatted data to stdout.

How is that important?
The data you might require to configure a fabric may well have repeated values (i.e. BGP ASNs, OSPF AREA IDs, MD5 passwords, VLANs, RTs, etc..) that you don’t want to write an awful lot of times in your “seed files”.

It’s ok if it’s a script does it for you, as long as you type in that value in just one place (doing the same thing twice is EVIL. we are automating!)

But that’s not enough to fix all our problems! How do you model the network you need to configure? How can you efficiently model all your resources? Be them physical (ports/devices/cables) or logical (VRFs/VLANs/IPs/etc..).

A few years ago I had this lightbulb moment with an ex colleague of mine… “Graph DBs!!” Those would be a match made in heaven with the problem we were trying to fix (similar to mine).

That initiative never went far because there wasn’t enough business buy-in to make it a success, but – never mind – a few months later I saw this video that made me say: “Crap! They got there first!”

I then had the immense privilege of meeting Sasha and all the crew at Apstra. Their AOS has a graph DB under the hood!

Implementation

But let me explain to you what I did.
First of, I started describing my physical topology in a .dot syntax file.

graph fabric {
    pos_spine_1       [group=spine, rack=2]
    pos_spine_2       [group=spine, rack=3]
    pos_spine_3       [group=spine, rack=2]
    pos_spine_4       [group=spine, rack=3]

    pos_1_leaf_1      [group=leaf, rack=1]
    pos_1_leaf_2      [group=leaf, rack=1]
    pos_2_leaf_1      [group=leaf, rack=4]
    pos_2_leaf_2      [group=leaf, rack=4]
    [.PARTS OMITTED.]
    pos_1_leaf_1:swp29 -- pos_spine_1:swp1 [speed=40, type=optical]
    pos_1_leaf_1:swp30 -- pos_spine_2:swp1 [speed=40, type=optical]
    pos_1_leaf_1:swp31 -- pos_spine_3:swp1 [speed=40, type=optical]
    pos_1_leaf_1:swp32 -- pos_spine_4:swp1 [speed=40, type=optical]
}

The next step was to put together a simple python script to load the topology file into a Neo4J database.

A GrapDB is made of nodes (or vertexes) and edges (or relationships). Every node or relationship carries “attributes” that can carry and store any property you may want to store. Is the cable blue, is the port 10G, is the switch running NOS_A or NOS_B ?

The script that loads the topology file into Neo4J leverages two very important python libraries:

  1. pydot (https://pypi.org/project/pydot/) to parse the topology.dot file
  2. py2neo (https://py2neo.org/v4/) that one of the Neo4J SDKs available for python

Uh, how do you get Neo4J running? Easy as a walk in the park.

docker run \
--publish=7474:7474 \
--volume=$HOME/neo4j/data:/data \
neo4j:2.3

After the Neo4J DB is up and running, I just launch my python script and boom, my topology is nicely loaded and I can finally run some queries going to http://localhost:7474 on my browser.

MATCH (d:Device) WHERE d.group = 'leaf' RETURN d

Screenshot 2018-10-02 at 20.36.01Not very interesting right? I already knew I had 4 leaf switches. But look what happens if I click on one of them…Screenshot 2018-10-02 at 20.36.41All the graph nodes with ‘relationships’ to the node I clicked on popped up, including that mysterious ‘4’.
What’s that? That is the graph representation of a “rack”. A rack bears relationships from other nodes of the type “LOCATED_IN”.
I can keep on going and click on the other leaf located in Rack 4.Screenshot 2018-10-02 at 20.36.57But none of this is very programmable is it? Nice to the eyes, but not nice to the code.
My dynamic inventory actually performs queries on that DB via the Python API and extracts data out of the topology itself.

With all your data structured this way you could do some very clever things (i.e. give me just the switches that have “subport” connected (a subport is a breakout port – example 40G->10G breakout).

MATCH (d:Device)-[:HAS_PORT]-(p:Port)-[:HAS_SUBPORT]-(sp:Subport) RETURN d,p,sp

Screenshot 2018-10-02 at 20.44.19.png

Conclusion

My use case and example are (for now) very basic and not particularly exciting, but I hope this made you want go and try something out.

It already turned out to be very useful to me to automate a simple DC fabric (next step is to model overlays with it) and to extract some simple data such as “how many 40G->10G breakout cables do I need to buy?” or “how many MTP/MPO cables do I need?”

Think about how powerful this can be when you start modelling VRFs, subnets and routes.
You could run a shortest path calculation from any two network segments and figure out what devices sit in the path, etc, etc…

One thought on “GraphDBs and Network Automation

Add yours

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at WordPress.com.

Up ↑

%d bloggers like this: