One of the features in Python that draws in and retains many developers and infrastructure automation engineers is the Python ecosystem of packages. Rather than trying to reinvent the wheel, you can reuse other people’s code (or your own old code) to build something new with a few import statements and a few lines of your own to extend the implementation in the direction you need.
What is a Python module? A Python module can be a few different things:
We will go into the details shortly, but here is a quick example using the JSON module to help format some nested objects to be printed nicely:
$ python
Python 3.10.4 (main, Oct 18 2022, 14:16:58) [Clang 14.0.0 (clang-1400.0.29.102)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> INTERFACES = {'GigabitEthernet1': {'ipaddr': '10.1.1.1', 'mask': '30', 'admin': 'up'}, 'GigabitEthernet2': {'ipaddr': '10.2.1.1', 'mask': '255.255.255.252', 'admin': 'down'}, 'GigabitEthernet3': {'ipaddr': '10.3.1.1', 'mask': '255.255.255.252', 'admin': 'up'}, 'GigabitEthernet4': {'ipaddr': '10.4.1.1', 'mask': '255.255.255.252', 'admin': 'down'}}
>>>
>>> print(INTERFACES)
{'GigabitEthernet1': {'ipaddr': '10.1.1.1', 'mask': '30', 'admin': 'up'}, 'GigabitEthernet2': {'ipaddr': '10.2.1.1', 'mask': '255.255.255.252', 'admin': 'down'}, 'GigabitEthernet3': {'ipaddr': '10.3.1.1', 'mask': '255.255.255.252', 'admin': 'up'}, 'GigabitEthernet4': {'ipaddr': '10.4.1.1', 'mask': '255.255.255.252', 'admin': 'down'}}
>>>
>>> import json
>>>
>>> print(json.dumps(INTERFACES, indent=4))
{
"GigabitEthernet1": {
"ipaddr": "10.1.1.1",
"mask": "30",
"admin": "up"
},
"GigabitEthernet2": {
"ipaddr": "10.2.1.1",
"mask": "255.255.255.252",
"admin": "down"
},
"GigabitEthernet3": {
"ipaddr": "10.3.1.1",
"mask": "255.255.255.252",
"admin": "up"
},
"GigabitEthernet4": {
"ipaddr": "10.4.1.1",
"mask": "255.255.255.252",
"admin": "down"
}
}
>>>
Python includes a large library of built-in modules. This means that without any further installation, you have access to more than 200 modules covering a wide range of situations, from CSV file manipulation to manipulating HTML.
A major benefit of using the built-in modules is that if you share your code publicly or with another person, you can be fairly certain the code will work as expected. Python installations can vary wildly when it comes to custom package versions from places like PyPI, so using only built-in modules is an easy way to make sure your code will be very portable.
In order to import a built-in module, first you need to know the module name and the available functions, classes, and variables. There are a few different ways to do this:
The first two ways above are pretty self-explanatory, but if you want to see the docs within Python itself, you can either use the dir() function to see the available attributes, classes, and functions or use the help() function to see the docstring documentation:
>>> dir(json)
['JSONDecodeError', 'JSONDecoder', 'JSONEncoder', '__all__', '__author__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__version__', '_default_decoder', '_default_encoder', 'codecs', 'decoder', 'detect_encoding', 'dump', 'dumps', 'encoder', 'load', 'loads', 'scanner']
>>>
>>>
>>> help(json)
Help on package json:
NAME
json
MODULE REFERENCE
https://docs.python.org/3.10/library/json.html
The following documentation is automatically generated from the Python
source files. It may be incomplete, incorrect or include features that
are considered implementation detail and may vary between Python
implementations. When in doubt, consult the module reference at the
location listed above.
DESCRIPTION
JSON (JavaScript Object Notation) <http://json.org> is a subset of
JavaScript syntax (ECMA-262 3rd edition) used as a lightweight data
interchange format.
:mod:`json` exposes an API familiar to users of the standard library
:mod:`marshal` and :mod:`pickle` modules. It is derived from a
version of the externally maintained simplejson library.
Encoding basic Python object hierarchies::
>>> import json
>>> json.dumps(['foo', {'bar': ('baz', None, 1.0, 2)}])
'["foo", {"bar": ["baz", null, 1.0, 2]}]'
>>> print(json.dumps("\"foo\bar"))
"\"foo\bar"
>>> print(json.dumps('\u1234'))
"\u1234"
>>> print(json.dumps('\\'))
"\\"
>>> print(json.dumps({"c": 0, "b": 0, "a": 0}, sort_keys=True))
{"a": 0, "b": 0, "c": 0}
>>> from io import StringIO
>>> io = StringIO()
...
To import a built-in module, you need to use the syntax import MODULE_NAME. You can include the import statement technically anywhere in your code, though for readability, they are almost always included at the top of your Python file.
Two other examples of a built-in modules are argparse, which allows you to make an interactive CLI tool in Python, and getpass, which allows you to prompt a user for a password at run time and store it securely (not in plaintext).
When you are developing code, it is good to build things in stages and not try and do it all at once and hope it works. In this example, we will make a CLI tool that prompts the user for a username, password, device type, IP address, and a command to send to the network device. Even though we are not sending actual commands, it is good to get the incremental practice of using these new modules.
When importing a module, you can either use the syntax import MODULE_NAME, such as import argparse, or you can use the sytnax from MODULE_NAME import item. In this case, we will use both styles. The script will start with the import statement import argparse and then from getpass import getpass. The getpass module has a function that is also called getpass, so the name appears twice.
import argparse
from getpass import getpass
Open a new file called net-auto-prototype.py with the following contents:
#!/usr/bin/python3
import argparse
from getpass import getpass
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Network Automation command line tool",
usage="python3 net-auto-prototype.py -u cisco -d cisco_ios_telnet -i 10.10.20.175 -c 'show ip interface brief'",
)
parser.add_argument(
"-u",
"--user",
help="Supply a username to login to the device",
)
parser.add_argument(
"-p",
"--password",
help="Supply a password to login to the device, if none is given, you will be prompted for one",
)
parser.add_argument(
"-d",
"--device_type",
choices=[
"cisco_ios_telnet",
"cisco_xr_telnet",
"cisco_ios",
"cisco_nxos",
"cisco_asa",
"cisco_xr",
],
help="Choose a device type for the connection",
required=True,
)
parser.add_argument(
"-i",
"--ipaddr",
choices=["10.10.20.175", "10.10.20.176"],
help="Supply an IP address, choices from the DevNet Sandbox",
required=True,
)
parser.add_argument(
"-c",
"--cmd",
choices=[
"show version",
"show run",
"show ip interface brief",
],
help="Supply a command to execute to the device from the choices",
required=True,
)
args = parser.parse_args()
# if the user doesn't give a password, prompt for one
if not args.password:
# this is where getpass comes in
args.password = getpass()
device_dict = {
"device_type": args.device_type,
"host": args.ipaddr,
"username": args.user,
"password": args.password,
}
print(device_dict)
command_output = "TBD Implement Command Output"
print(command_output)
Try running the script now without putting a password in to prompt the getpass, using the syntax python net-auto-prototype.py -u cisco -d cisco_ios_telnet -i 10.10.20.175 -c 'show ip interface brief':
$ python net-auto-prototype.py -u cisco -d cisco_ios_telnet -i 10.10.20.175 -c 'show ip interface brief'
Password:
{'device_type': 'cisco_ios_telnet', 'host': '10.10.20.175', 'username': 'cisco', 'password': 'PASSWORDINPUT'}
TBD Implement Command Output
$
$
$ python net-auto-prototype.py -h
usage: python3 net-auto-prototype.py -u cisco -d cisco_ios_telnet -i 10.10.20.175 -c 'show ip interface brief'
Network Automation command line tool
options:
-h, --help show this help message and exit
-u USER, --user USER Supply a username to login to the device
-p PASSWORD, --password PASSWORD
Supply a password to login to the device, if none
is given, you will be prompted for one
-d {cisco_ios_telnet,cisco_xr_telnet,cisco_ios,cisco_nxos,cisco_asa,cisco_xr}, --device_type {cisco_ios_telnet,cisco_xr_telnet,cisco_ios,cisco_nxos,cisco_asa,cisco_xr}
Choose a device type for the Netmiko connection
-i {10.10.20.175,10.10.20.176}, --ipaddr {10.10.20.175,10.10.20.176}
Supply an IP address, choices from the DevNet
Sandbox
-c {show version,show run,show ip interface brief}, --cmd {show version,show run,show ip interface brief}
Supply a command to execute to the device from the
choices
Argparse is super useful because it has built-in features like error handling if arguments aren’t given and help text that you can give the user.
Python modules can simply be Python text files that you have locally in your development environment. You can reference your own code and pull it into another Python script to reuse it or extend it. You can import something as small as a single variable, a function, or just import everything to use it in your Python script.
Let’s say you build a quarterly report that gathers CSAT data from your IT Service Management System API, using a list of case numbers given to you. Quarter by quarter, the case numbers will change, but the same logic of getting the CSAT score and plugging that into a CSV file for management to review will stay the same. You can use the CSV built-in library to build the report and abstract the functions that gather the CSAT score and build the report into a separate module.
Create a Python file called shared_resources.py with the following contents:
import random, csv
def lookup_itsm_csat(case_number):
print("making API call to IT Service Management System for case {}".format(str(case_number)))
print("Gathering CSAT Score")
return random.randint(0, 5)
def build_csv_report(case_numbers, csat_list, region, year, quarter):
csv_file = "csat_report_fy{}q{}_{}.csv".format(str(year), str(quarter), region)
try:
with open(csv_file, "w") as csv_file:
writer = csv.DictWriter(
csv_file, fieldnames=["case_number", "csat"]
)
writer.writeheader()
# this could be written in a loop, but writing this way for simplicity
writer.writerow({'case_number': case_numbers[0], 'csat': csat_list[0]})
writer.writerow({'case_number': case_numbers[1], 'csat': csat_list[1]})
writer.writerow({'case_number': case_numbers[2], 'csat': csat_list[2]})
writer.writerow({'case_number': case_numbers[3], 'csat': csat_list[3]})
except IOError:
print("I/O error")
This Python module uses the random standard library to make a fake CSAT score and has a function to write out the CSAT with the case numbers into a CSV file. You could do a bit more massaging of the code to have it loop over the rows, but I am keeping it explicit and simple here because we are just teaching about importing modules.
Now create another file to import that module, calling it FYXX_Q4_BUILD_REPORT.py in the same working directory with the following contents:
#!/usr/bin/python3
from shared_resources import lookup_itsm_csat, build_csv_report
if __name__ == "__main__":
csat_list = []
case_numbers = [1241, 1234, 6453, 3453]
print("building report")
print("looking up CSAT scores using API for case numbers in IT Service Management System")
for case_num in case_numbers:
# add csat to dictionary for case number
csat = lookup_itsm_csat(case_num)
csat_list.append(csat)
print("csat for {} is {}".format(str(case_num), str(csat)))
build_csv_report(case_numbers=case_numbers, csat_list=csat_list, region="emea", year="2023", quarter="4")
This script imports the two functions from the shared_resources Python module, plugging in case numbers and other minor details that are used to generate the CSV filename.
The output should look something like this (with other numbers for CSAT because they are randomly generated):
$ python FYXX_Q4_BUILD_REPORT.py
building report
looking up CSAT scores using API for case numbers in IT Service Management System
making API call to IT Service Management System for case 1241
Gathering CSAT Score
csat for 1241 is 3
making API call to IT Service Management System for case 1234
Gathering CSAT Score
csat for 1234 is 2
making API call to IT Service Management System for case 6453
Gathering CSAT Score
csat for 6453 is 5
making API call to IT Service Management System for case 3453
Gathering CSAT Score
csat for 3453 is 1
$
$ cat csat_report_fy2023q4_emea.csv
case_number,csat
1241,3
1234,2
6453,5
3453,1
$
In Python, it searches for new modules first in your current working directory, and then in your PYTHONPATH environmental variable, and then in any other places you might have configured Python to look. Most of the time, you are going to have your Python modules in your current working directory, or in a subdirectory, or installed from pip. This tutorial was focused on getting you started with some practical examples of importing modules, but there is much more to learn when it comes to Python pathing and managing your Python modules. For most beginners, though, you will not have to worry about it.