[python]Comparing missing ip addresses between original inventory with object group

An inventory list is kept to record hostnames and ip addresses, this inventory list (inventory.xlsx) is currently maintained by automation.

For every virtual machine created, the hostname and its corresponding ip address are recorded in the list, automation will update the ip address into the existing object group in the firewall.

However there is a report that there are certain ip addresses not included in the interesting_group, you need to write a script to leverage Tufin SecureTrack to find out what are the missing ip addresses in the existing group as compared with the inventory list.

In the inventory list there are ip addresses which belong to subnet 192.168.1.0/24, this whole subnet is not the network protected by firewall hence skip during collection.

The script needs to be cautious to skip recording items from interesting_group.

This is a code written in haste, hence not structured, there are repeated items which worth to create functions.

import xml.etree.ElementTree as ET
import requests
from openpyxl import load_workbook
from netaddr import IPNetwork, IPAddress


NSMAP = {'xsi': 'http://www.w3.org/2001/XMLSchema-instance'}
tufin_st_xml = requests.session()
tufin_st_xml.verify = False
tufin_st_xml.headers.update({'Content-type': 'application/xml'})
tufin_st_xml.auth = requests.auth.HTTPBasicAuth('admin', 'password')

ip_collections = []
group_collections = []
base_list = []
row = 2
column = 7
ip_from_inventory = []
filtered_ip = []
filtered_ip_collections = []
missing_ip = []
try:
    response = tufin_st_xml.get('https://secure_track_address/securetrack/api/devices/1/network_objects?type=group&show_members=true&name=interesting_group')
    xml_response = ET.fromstring(response.text)
    for xml_item in xml_response.findall('.//ip'):
        ip_collections.append(xml_item.text)
    for xml_type in xml_response.findall("./network_object[@xsi:type='networkObjectGroupDTO']", NSMAP):
        if 'interesting_group' not in xml_type.find('display_name').text:
            group_collections.append(xml_type.find('display_name').text)
except Exception as e:
    print(e)
    exit(1)

if group_collections:
    for group in group_collections:
        print(group)
        try:
            response = tufin_st_xml.get('https://secure_track_address/securetrack/api/devices/1/network_objects?type=group&show_members=true&name={}'.format(group))
            xml_response = ET.fromstring(response.text)
            for xml_item in xml_response.findall('.//ip'):
                ip_collections.append(xml_item.text)
        except Exception as e:
            print(e)
            exit(1)

for ip_item in ip_collections:
    if IPAddress(ip_item) not in IPNetwork('192.168.1.0/24').iter_hosts():
        filtered_ip_collections.append(ip_item)
wb = load_workbook('inventory.xlsx')
while wb['Sheet1'].cell(row, column).value is not None:
    ip_from_inventory.append(wb['Sheet1'].cell(row, column).value)
    row += 1

for ip_item in ip_from_inventory:
    if IPAddress(ip_item) not in IPNetwork('192.168.1.0/24').iter_hosts():
        filtered_ip.append(ip_item)

missing_ip = set(filtered_ip).difference(filtered_ip_collections)
print(missing_ip)
Advertisements
Posted in Python, Scripting | Tagged , , , , , | Leave a comment

[python]Excluding ip addresses belonging a subnet

suppose if you need to enumerate ip addresses from a list, but want to exclude ip addresses belonging to a subnet here’s a solution which you can do with netaddr module.
Supposed ip_from_inventory is the list of ip addresses you have obtained from a file.

from netaddr import IPNetwork, IPAddress
for ip_item in ip_from_inventory:
    if IPAddress(ip_item) not in IPNetwork('192.168.1.0/24').iter_hosts():
        print(ip_item)

This example exclude ip address that belongs to 192.168.1.0/24, and will list other ip addresses that are not.

Posted in Python, Scripting | Tagged , | Leave a comment

[python]Writing to temporary file

Supposed you have collected a list item known as ip_collections, and you want to write the list to a temp file.

import tempfile
with tempfile.TemporaryFile() as tf:
    for ip in ip_collections:
        tf.write(bytes(ip + '\n', 'utf-8'))
    tf.seek(0)

Need to convert the string in byte, the tf.seek(0) is to re-wind the pointer back to the start of the file.

After the code exits the with context the temp file is removed.

Supposed you want to store the values of temp file into a variable for some other purpose before removal.

import tempfile
base_list = []
with tempfile.TemporaryFile() as tf:
    for ip in ip_collections:
        tf.write(bytes(ip + '\n', 'utf-8'))
    tf.seek(0)
    base_list = tf.read().decode().split()
    tf.seek(0)

print(base_list)

Need to rewind the start of file with tf.seek(0) so that all items in the temp file will be transferred to base_list. Need to take note that tf.seek(0) has to be put outside for loop otherwise the values will be overwritten on each iteration.
the print statement is for testing if the list is stored.

Posted in Python, Scripting | Tagged , | Leave a comment

[python]DNS resolution webapp with flask

Very simple webapp, I still need to do some styling with CSS… this is a simple concept.
here’s an example:
Screen Shot 2018-05-20 at 11.11.59 PM

The output will be:
Screen Shot 2018-05-20 at 11.12.55 PM

Here are the jinja2 templates:
Screen Shot 2018-05-20 at 11.13.42 PM.png

Here’s the code.

from flask import Flask, request, render_template
import socket


def host_processing(host):
    answer = ""
    try:
        answer = socket.gethostbyname(host)
    except Exception:
        # If not resolvable, dun care.
        pass

    if answer:
        return answer
    else:
        return host

app = Flask(__name__)

@app.route('/')
def main_form():
    return render_template('tool.html')


@app.route('/', methods=['POST'])
def post_form():
    answers = []
    hosts = request.form['text'].split(',')
    for host in hosts:
        answers.append(host_processing(host))
    return render_template('answers.html', answers=answers)


if __name__ == '__main__':
    app.run(debug=True)
Posted in Python, Scripting | Tagged , , | 2 Comments

[python] resolve a list of items from file

The list can be ip address, ip address range, ip subnet, firewall objects or hostname.
The objective:

  • If item is resolvable, resolve it and collect the ip address.
  • If item is unresolvable, will collect it as it is.

The list of items must be separated by commas else it will not work.

import socket


def host_processing(host):
    # Initialize to nothing, if hostname is not resolvable answer will at least contain a null string.
    answer = ""
    try:
        answer = socket.gethostbyname(host)
    except Exception:
        # If not resolvable, dun care.
        pass

    # if answer is not null string, or host is an ip address
    # gethostbyname will return the same ip address you put in.
    if answer:
        return answer
    else:  # if there is no answer, or if host is a subnet or range
        return host


if __name__ == '__main__':
    processed_data = []
    with open('host.txt', 'r') as file:
        datas = file.read().split(',')

    for data in datas:
        processed_data.append(host_processing(data))

    print(processed_data)
Posted in Python, Scripting | Tagged , , | Leave a comment

[python]Resolve dns if hostname is resolvable else send as is

There are times which you need to try to resolve valid hostnames, however on user’s input there could be a possibility that the input contains ip addres, firewall objects or firewall object groups that are not resolvable.

here’s a solution which could possibly return ip address if host is resolvable if not resolvable return the value as it is.

import socket

# Input the data to host, can be ip, hostname or just a name that is not resolvable
host = ""
# Initialize to nothing, if hostname is not resolvable answer will at least contain a null string.
answer = ""
try:
    answer = socket.gethostbyname(host)
except Exception as e:
    # If not resolvable, dun care.
    pass

# if answer is not null string, or host is an ip address
# gethostbyname will return the same ip address you put in.
if answer:
    print(answer)
else: # if there is no answer, or if host is a subnet or range
    print(host)
Posted in Python, Scripting | Tagged , | Leave a comment

[python]Remove duplicates from a list

I have learned something interesting… removing duplicates in a list is actually quite easily achieved without using iteration…

from collections import OrderedDict

a = ["1", 1, 2, 3, "3", "3", "4"]
b = list(set(a))

c = list(OrderedDict.fromkeys(a))
print("Unordered non-duplicated list {}\n".format(b))
print("Ordered non-duplicated list {}, preserved the list of the original a".format(c))

The output is self explanatory, you use set if you do not bother about the order, you use Ordereddict if you need to preserve the original order.

Unordered non-duplicated list [1, 2, 3, '4', '3', '1']

Ordered non-duplicated list ['1', 1, 2, 3, '3', '4'], preserved the list of the original a

Posted in Python, Scripting | Tagged , , | Leave a comment