Introduction

Before using Nornir, I was using netmiko, netmiko is a steady module which makes configuring, getting information from cisco based devices easily. Of course netmiko is not limited to just Cisco, it is a multi-vendor module. Napalm is another network module which does the same thing as netmiko, however when dealing with Cisco ios based devices Napalm relies on netmiko as the IOS devices do not have REST APIs, and not all cisco devices support the REST API package.

Nornir framework provides a one stop tool for programming infrastructure automation, the framework attempts to gel netmiko, napalm, netbox, ansible, paramiko together, and comes with jinja2 template engine.

Nornir uses inventory just like ansible to push commands or command sets over to multiple devices defined in the inventory file(s).

On this post I am demonstrating the usage example of:

1. Device secrets management with Hashicorp vault.
2. Generating inventory yaml file with dictionary (simulating json request send to api.
3. Manipulating Nornir inventory yaml.
4. Simplest way to initialize Nornir object.
5. Sending show command with netmiko through Nornir.
6. Capturing response after show command was sent.

Prerequisite

1. pip install nonir, the version I am currently using is 2.30.
2. pyvault2, which is a script written to use kv2 engine in Hashicorp vault

create_host_file(payloads=None)

This function helps to create inventory file – cisco_ios.yaml, and save the credential to the hashicorp vault. In this cisco_ios.yaml it is very simple, there is no group, only hostnames of the cisco routers as keys, and each router has username, password, hostname (ip address), platform (cisco_ios, driver name for netmiko) and port as its attributes.

Nornir uses the inventory file and pushes configuration or commands over to the hostnames specified in the yaml file, this works really well if configurations are almost the same, in this situation pushing configuration to all inventory is ideal, however some configurations are unique to one device then special logic has to be programmed to make it work as expected.

Host file path

host_file = join(Path(__file__).parent.parent.absolute(), "network", "inventory", "cisco_ios.yaml"), I am using the Pathlib.Path library to find its parent directory, on each parent is a top of the current directory, two parent will go up two times.

File operation mode

There are three modes, append, read and write, if a file does not exists a write operation will create a file and write data to the file, if the file exists and the mode is write then the current content of the existing file is overwritten. Hence to change mode between “w” and “a” with the below code; if the file exists append the content otherwise create and write to file.
mode = "a" if exists(host_file) else "w"

Payloads handling

The payload is to send a dictionary of the inventory, if more than one host then one dictionary is sufficient, if more than one host then a list of dictionaries is required, so to check if payloads is type list use this code:
if type(payloads) is list

Handling the payloads if it is a list

        for payload in payloads:
            """
            Prepare for data to be sent to hashicorp vault.
            """
            vault_data = {
                "username": payload["username"],
                "password": payload["password"],
                "ip": payload["ip"]
            }

Consolidate each host’s username, password and ip, then prepare a separate dictionary to be sent over to Hashicorp Vault for secret storage.

            if not is_secret_path_exists(mount_path=payload["platform"]):
                # If the mount point does not exist enable the mount_point
                enable_kv2_engine(mount_path=payload["platform"])

This code block checks if the mount point and mount path exist, the pyvault2 package I created uses only the kv2 engine of Hashicorp vault, there are more engines in this Hashicorp vault but for my personal usage kv2 is more than enough. If the mount point does not exist, enable the mount point. In Hashicorp vault’s context, you create an engine which in this case is kv2 engine, and you name your newly created engine which is known as mount point.

To illustrate what is mount point and mount path read below’s screenshot:

After you have logged on to Hashicorp vault, you will be presented with a page with the mount points, in this screenshot there are two mount points – cisco_asa and cubbyhole. cubbyhole is a ready mount point after the vault is installed.

These are the paths within the cisco_asa mount point.

On each path a dictionary of data of your choice is stored, for my case I stored the username, password and ip. If you are using netmiko and not nornir, then you can store your inventory’s netmiko config files here.

create_update_kv2_secrets(mount_path=payload["platform"],
                                      path=payload["hostname"],
                                      **vault_data)

This code block creates new path and stores the vault_data, if the data is already exist no change will be updated, in Hashicorp vault version of updated data is checked i.e. if the data is version 1, the new data has to indicate the data you need to change is version 1.

payload["username"] = ""
payload["password"] = ""

This block clears the username and password, the username and password are not needed anymore for the inventory file, but the keys are required.

hostname = payload.pop("hostname")
payload["hostname"] = payload.pop("ip")

This block transfer the hostname value over to the hostname variable, this is used for the key of the inventory. In the inventory yaml file the attribute for ip address or fqdn is hostname hence the second line is to transfer the ip address over to the payload["hostname"].

_payload = {
                hostname: payload
            }
collect_yaml.append(yaml.safe_dump(_payload))

This block is to prepare a new dictionary to be converted from dictionary to yaml, collect_yaml is a list to collect the yaml objects.

        with open(host_file, mode) as file:
            for yd in collect_yaml:
                file.write(yd)

This block writes the yaml object to the file.

    else:
        vault_data = {
            "username": payloads["username"],
            "password": payloads["password"],
            "ip": payloads["ip"]
        }
        create_update_kv2_secrets(mount_path=payloads["platform"],
                                  path=payloads["hostname"],
                                  **vault_data)
        with open(host_file, mode) as file:
            file.write(yaml.safe_dump(payloads))

If the payload is not a list then it is a dictionary, then directly prepare a vault_data from the dictionary, then send over the username, password and ip address to Hashicorp vault, convert the dictionary to yaml object then write the yaml object to file – cisco_ios.yaml.

show_command_to_all_hosts(cmd=None)

This function takes the cisco show command, and send over to the router.

with InitNornir(
            inventory={
                "plugin": "nornir.plugins.inventory.simple.SimpleInventory",
                "options":
                    {
                        "host_file": host_file
                    }
            }
    ) as nr:

This is the initialization of a nornir object, according to the documentation this is the easiest way to create a nornir object, this nornir object is used to run task, and task is a keyword to accept function such as netmiko_send_command, nornir object can run several task such as jinja2 template, naplam, ansible and netmiko.

        for host in nr.inventory.hosts.keys():
            """
            Collect host information from hashicorp vault.
            """
            vault_data = get_kv2_secret(mount_path="cisco_ios",
                                        path=host,
                                        find="data")
            """
            Modify the username and password of each key in yaml file.
            """
            nr.inventory.hosts[host].username = vault_data["username"]
            nr.inventory.hosts[host].password = vault_data["password"]

This block gets the username, password and ip from the hashicorp vault for each hosts in the inventory file, then update the data over to the existing cisco_ios.yaml inventory file. get_kv2_secret is a function of pyvault2, you can check the code here. In order to update the value over to the cisco_ios.yaml file, the keys username and password must be present in the file.

routers = nr.run(task=netmiko_send_command,
                         command_string=cmd,
                         use_textfsm=True)
    return routers

This block runs the netmiko_send_command function with the nornir object, command_string and use_textfsm are arguments of netmiko_send_command, do note that task accepts functions hence you can also create function and pass your customized function to task. routers is the result after the command is sent.

show_assigned_interface(routers_results=None)

This function is used to process the return result of nornir object, the nornir object returns AggregatedResult object which is a dictionary, and within the dictionary is a Result object which is a list object.

collect_info = []
    for router in routers_results.keys():
        # each router has a list of dictionaries of interface information.
        for router_attr in routers_results[router][0].result:
            # Only interested in assigned interface.
            if "unassigned" not in router_attr["ipaddr"]:
                collect_info.append(
                    {
                        router: router_attr
                    }
                )
    return collect_info

Because AggregatedResult object is a dictionary-like object, for router in routers_results.keys() is to go over the hosts in the AggregatedResult object the key is the hostname of the router.

To explain this routers_results[router][0].result, routers_results is the AggregatedResult object, routers_results[router] filters the hostname, routers_results[router][0] this is where the failure status and result are located, netmiko_send_command only has one index hence [0].

from pyvault2.vault.hvault2 import create_update_kv2_secrets, is_secret_path_exists, enable_kv2_engine, get_kv2_secret
from nornir.plugins.tasks.networking import netmiko_send_command
from nornir import InitNornir
from pathlib import Path
from os.path import join, exists
import yaml
from pprint import pprint

host_file = join(Path(__file__).parent.parent.absolute(), "network", "inventory", "cisco_ios.yaml")


def create_host_file(payloads=None):
    """
    Create host_file and store credentials to hashicorp vault
    :param payloads: dictionary contains the inventory information including username and password.
    :return:
    """
    # file mode: if exists append otherwise create one and write to file.
    mode = "a" if exists(host_file) else "w"

    if type(payloads) is list:
        """
        If the payload is a list of dictionaries, process the payload.
        """
        _payloads = []
        collect_yaml = []
        for payload in payloads:
            """
            Prepare for data to be sent to hashicorp vault.
            """
            vault_data = {
                "username": payload["username"],
                "password": payload["password"],
                "ip": payload["ip"]
            }
            if not is_secret_path_exists(mount_path=payload["platform"]):
                # If the mount point does not exist enable the mount_point
                enable_kv2_engine(mount_path=payload["platform"])

            # Store the prepared vault_data to hashicorp vault
            create_update_kv2_secrets(mount_path=payload["platform"],
                                      path=payload["hostname"],
                                      **vault_data)
            # Clear the username and password after stored information to hashicorp vault
            payload["username"] = ""
            payload["password"] = ""

            hostname = payload.pop("hostname")
            payload["hostname"] = payload.pop("ip")
            """
            To make hostname into keys, hence pop the key "hostname" from payload.
            Once popped, the key - "hostname" - is removed from payload.
            On each hostname (key) is a dictionary which looks like this:
            "mgmt": {
                "username": "username",
                "password": "password",
                "port": "22",
                "platform": "cisco_ios",
                "hostname": "192.168.100.101"
            }
            The hostname key is then replaced with the ip address. This format is required by nornir.
            """
            _payload = {
                hostname: payload
            }
            collect_yaml.append(yaml.safe_dump(_payload))
        with open(host_file, mode) as file:
            for yd in collect_yaml:
                file.write(yd)

    else:
        vault_data = {
            "username": payloads["username"],
            "password": payloads["password"],
            "ip": payloads["ip"]
        }
        create_update_kv2_secrets(mount_path=payloads["platform"],
                                  path=payloads["hostname"],
                                  **vault_data)
        with open(host_file, mode) as file:
            file.write(yaml.safe_dump(payloads))


def show_assigned_interface(routers_results=None):
    """
    Processing routers' result by checking for assigned interfaces.
    :param routers_results:
    :return:
    """
    collect_info = []
    for router in routers_results.keys():
        # each router has a list of dictionaries of interface information.
        for router_attr in routers_results[router][0].result:
            # Only interested in assigned interface.
            if "unassigned" not in router_attr["ipaddr"]:
                collect_info.append(
                    {
                        router: router_attr
                    }
                )
    return collect_info


def show_command_to_all_hosts(cmd=None):
    with InitNornir(
            inventory={
                "plugin": "nornir.plugins.inventory.simple.SimpleInventory",
                "options":
                    {
                        "host_file": host_file
                    }
            }
    ) as nr:
        for host in nr.inventory.hosts.keys():
            """
            Collect host information from hashicorp vault.
            """
            vault_data = get_kv2_secret(mount_path="cisco_ios",
                                        path=host,
                                        find="data")
            """
            Modify the username and password of each key in yaml file.
            """
            nr.inventory.hosts[host].username = vault_data["username"]
            nr.inventory.hosts[host].password = vault_data["password"]

        # routers is the AggregatedResult object of all hosts in inventory
        routers = nr.run(task=netmiko_send_command,
                         command_string=cmd,
                         use_textfsm=True)
    return routers


if __name__ == "__main__":
    from test_env.test_payload import payloads

    # Ensure inventory file in place and credentials in hashicorp vault.
    if not exists(host_file):
        create_host_file(payloads=payloads)

    # Print out the dictionary of show ip int brief.
    # the dictionary is processed by the textfsm with ntc-templates.
    pprint(show_assigned_interface(show_command_to_all_hosts(cmd="show ip int brief")))