Python For Scripting – Guide Towards Automation

Introduction

In this article we will talk about Python for scripting which is the use of Python programming language to automate certain tasks that would otherwise be performed individually by a human operator. We will explain what is scripting, why we will be using Python, and what are the commonly used Python packages for scripting.

Introduction to scripting

Scripting is the process of automating certain tasks and workflows using a program. It can be a short process like manipulating a bunch of files, or a one that operates on a large scale that include servers and may users and so on.

No matter the size of those programs, they all start with the basics of scripting.

Scripting languages are known for being interpreted, meaning that the code will be executed at runtime line by line, rather than compiled like the case of JAVA where the code is first turned into a class to finally be run entirely. One advantage is that it is less code intensive.

Note that Python is an interpreted language but it can also be compiled.

Introduction to python

Python is one of the most famous programming languages ever. It is currently used in a lot of projects and companies like Google and YouTube. For many years it has been know for being one of the most in demand languages.

Features

• Easy to use since it resembles the English language and uses indents rather than semi-colons
• Hight-level interpreted language with dynamic semantics: meaning you don’t need to care about low level details
• Open-source: anyone can use for free and with no limitations.
• Portable, it is cross platform. Meaning we can use it on all Linux, mac, and windows.
• Large community: you will always get help.
• Large open-source modules and frameworks: you don’t need to reinvent the wheel. In most times, you will find a package already made that does what you want for free!
• Extensible: Can integrate all sorts of other languages with it.

Python Versions

You will often hear about two Python version, version 2, and version 3. In this section we will explore each version, and why you should learn one over the other.

Python Version 2x:

The old version of Python. It has old syntaxes and less features than the current version. Many projects are still operations with this languages for multiple reasons like a very large source code which is hard to update, or that the developers are more familiar with Python 2.  

Python version 3x:

The present and the future of Python! It is the only version that is supported by the Python foundation. It provides more features and a better, clear way to write Python code. Moreover is that just by being the latest version, you get better features in general like fixed old bugs and glitches, a better Autocomplete, and a better Interpreter. All those improvements make Python 3 faster and more stable in general.

Recommendation

Always use versions 3 in your new projects. Therefore you should learn it. The only reason you may need to learn version 2 is if, for example, you are applying for a job at a company that still uses Python 2.

Note that Python 3 and Python 2 are the same when it comes to the core. If you learn any of the versions, the other will be very easy to pick up.

Python Modules for scripting

In this section we will explore what are Python modules, and what modules are most used for scripting.

What are Python modules?

To put it simply, a Python module is a bunch of Python code that does specific tasks. Modules either comes with Python or can be downloaded for free from the internet using PIP.  

For reference, in other programming languages, Modules are called Libraries or Packages.

The core modules for scripting purposes come with Python when we installing it since they are used in some core functionalities. Therefore we won’t need to install any additional packages in this article. However, you should keep in mind that you will need to download packages for more advanced automation purposes like Web Scraping and API Integration.

Most used built-in python modules for scripting – core Python modules

In this section we will be exploring the most used Python modules for scripting. As mentioned before, all of the following modules are built-in with Python, meaning that you can use them directly when installing Python on your local computer and you don’t need to download them separately. You simply need to import them to your script using the import.

OS Module – Python OS scripting

The os module is used to interact with the operation system through a set of functions. some of the core features this module gives us is the ability to interact directly through the file system like creating, moving, and deleting files and folders, changing directory paths, getting current working directory and many more functionalities.

Here are few examples to get you started with the os module:

Get current directory using the os module

# import os module
import os 

# get current working directory
os.getcwd()

Changing the working directory. we do that by calling os.chdir. After that, we pass a path as an argument, in my case the path is “Documents/folder1”

# change working directory
os.chdir("Documents/folder1")

Creating a new directory using the os module

We create a new directory using the os.mkdir() method. After that, passing the name of the directory. In my case the new directory name is “New Folder”

# make a new directory (folder)
os.mkdir("New Folder")

Creating multiple directories in a single path

We can create a chain of directories in a single path which is handy when it comes to Python scripting. We do that using the os.makedirs() method, and we pass it the chain of path we want to create. In my case the path is “/folder1/folder2/folder3/folder4”, meaning that Python will create all these directories from first to last.

# create multiple directories in single path
os.makedirs("/folder1/folder2/folder3/folder4")

Changing the working directory

We change the working directory using the os.chdir() method. After that, passing the path of the directory we want to change to.

# change directory
os.chdir("/home/admin/Bureau")

Remove a directory using the os module

We can remove any directory in Python using os.rmdir(). We then pass the path of that directory.

# remove a directory
os.rmdir("repertoire2")

Remove a file using the os module

Besides folders, we can also remove files using os.remove(). Then passing it the name of the file we want to delete.

# remove a txt file
os.remove("file.txt")

Renaming files and directories/folders using the os module

By applying the os.rename() method in Python, we can rename any file or directory/folder. We only need to pass it two arguments, the name of the original file or folder, and the name we want to change it to.

In this example the original file name is “file1.csv”, and we want to change it to “file2.csv”.

# rename a file or a directory/folder
os.rename("file1.csv", "file2.csv")

SYS Module – Python interpreter scripting

Used to interact with the interpreter using specific functions, and well ass accessing variables used by the interpreter. The goal of this module is to manipulate different parts of the Python runtime environment. One of the common usages of the SYS module is the sys.argv. This method allow us to get command line variables. other usages are sys.version to get Python version, sys.exist to stop the code from running, input and output related operations using sys.stdin and sys.stdout and more.

Here are some code examples of the sys module usages:

Getting Python version using the sys module

To get the Python version we are using in our system, we simple call sys.version.

# import sys module
import sys

# get python version
print(sys.version)

Get command line arguments using the sys module

To get the command line arguments we use the sys.argv method. the arguments are always returned in the form of a ordered list from the first to last.

# import sys module
import sys

# get the command line arguments 
arguments = sys.argv

Without addition arguments we only have one by default which is the name of the file to run. in the bellow example it’ is “test.py”

python test.py

Additional arguments would look like:

python test.py argument1 argument2 argument3

In this case the arguments are [‘test.py’, ‘argument1’, ‘argument2’, ‘argument3’]

Getting all paths the interpreted searches in

The interpreter gives back a list of all relative paths of the modules we imported. Look at it as a list of the directories the interpreter searched in to get all required modules.

# import sys module
import sys

print(sys.path)

Getting imported modules using the sys module

We can know all the imported modules in our shell using the sys.modules which returns a list.

# import sys module
import sys

print(sys.modules)

Get the platform using the the sys module

To know what operating system we are using, we can use the sys.platform method.

# import sys module
import sys

# print the platform we are working on 
print(sys.platform)

Exist a program in Python using the sys module

In order to stop a program from running and exiting, we use the sys.exit method. It can be used anywhere in the code. It is commonly used to exist a program when a validation or a condition is not met. We can also pass it a string that it will print when existing.

# import sys module
import sys

# quit the program 
sys.exit("Program Stopped By Exist Function")

Subprocess Module – Python processes scripting

This module is used to create new process. Meaning it interacts with the operating system to run commands and return code. meaning we use it in order to run other external programs and also reading their outputs using Python code.

We use subprocess.run() to run external processes

import subprocess
import sys
result = subprocess.run([sys.executable, "-c", "print('Hello')"])

>>Hello

Random Module

This module is used for random numbers generation. It allows us to generate random integer numbers, random float numbers, a random element from a given list, a random letter from a given word (string).

Generating random integers

We can generate a random integer using the random.randrange() method. If we pass one argument to the method, it will generate a number less than the given one. like follows:

>>import random

# passing one integer
>>random.randrange(10)
>>4

If we pass two integers it will return a random integer located between the.

Here’s a code example:

>>import random

# passing two integers
>>random.randrange(1, 10)
>>4

If we pass three integers, it will use the third for stepping. Meaning, it will create a list of ordered integers where the small value is equal to the first number, and the large value is less than the second number, stepping between each integer by the third value.

Code example:

>>import random

# passing an integer with a step
>>random.randrange(1, 10, 2)
>>9

# pasing integer with a step 
>>random.randrange(0, 100, 10)
>>60

In the first example we have random.randrange(1, 10, 2), meaning that the random number is locate in the following list [1, 3, 5, 7, 9].

Generate random elements from a sequence

The random.choice() method returns a randomly selected element from a sequence that is non-empty. If the sequence is empty, you will get an IndexError.

>>import random

>>random.choice((10,03,4,93,13,87))
>>93

>>random.choice('hello'))
>>l

Reordering a list

The random.shuffle() method is used in order to randomly reorder (shuffle) all elements in a given list.

>>import random

>>list=[1,2,3,4,5,6]

>>random.shuffle(numbers)
>>list 
[4,1,5,6,2,5]

Getting a random integers using randint

Like randrange, randint outputs a random integer between two random integers. However, unlike randrange, you must specify the two integers.

>>import random
>>random.randint(1, 70)
61
>>> random.randint(1, 70)
39

Generating a random float

random.random() output value random float value between 0.0 and 1.0. Note that no arguments are needed.

>> import random
>> random.random()
0.786123893272145

JSON Module

JSON is a text that is written in JavaScript object notation format. This module is used for JSON serialization and deserialization. Meaning that Python can encode and decode JSON data out of the box which is very useful since JSON is the main format for backend and frontend communication (server-side and client side communication). Therefore, it is easy to set up an API.

We commonly used this module when our code uses some kind of a third-party API Integration. That is why this module is powerful when it comes to scripting with Python.

Convert format from JSON to Python

In order to convert from JSON format to a Python dictionary format, we use the json.loads() method.

import json

# JSON:
input =  '{ "name":"alex", "age":27, "city":"Boston"}'

# parse output:
output = json.loads(input)

# the result is a Python dictionary/object:
print(output["age"])

Convert from Python format to JSON format

To convert a Python dictionary/object to a JSON object, we use the json.dumps() method.

Here’s a code example:

import json

# a Python object/dictionary:
input= {
  "name": "Max",
  "age": 42,
  "city": "London"
}

# convert into JSON:
output= json.dumps(input)

# the result is a JSON string:
print(output)

Math Module – Python mathematical scripting

The math module is used to perform pure mathematical operations from calculus to converting degrees to working with functions and a lot of other complex operations. What’s really nice about this is that you can perform a lot of mathematical operations directly with Python alone without downloading any additional modules or packages.

Note that math is Python is devised into two:

• The first is a set of methods that don’t require the math module. This set of method are used for simple operations.
• The second set of methods do more advanced operations and require the math module to be imported.

Mathematical operations that don’t require the math module

we can do mathematical operations out of the box without importing the math module. Here are few code examples:

# get the small value 
>> min(5, 10, 25)
>> 5

# get the biggest value
>> max(5, 10, 25)
>> 25

# get the absolute positive
>> abs(-7.25)
>> 7.25

# get the power to a number of a value 
>> pow(3, 2)
>> 9

Mathematical operations that require the math module

We can do advanced mathematical operations using the math module as mentioned before. Note that we are proving just few example in this article since there is a lot of them. A great place to check them is by visiting the official documentation.

Here are few code examples:

the square root of a number

import math

value = math.sqrt(16)

print(value)

>> 4

Getting the value of Pi

import math

pi = math.pi

print(pi)

>> 3.141592653589793

Getting the round number of a float

The math.ceil() method is used to round a giving float number upwards to its nearest integer, while the math.floor() method rounds a giving float number downwards to its nearest integer, and returns the result:

import math

ceil = math.ceil(5.4)
floor = math.floor(5.4)

print(ceil) # returns 6
print(floor) # returns 5

Converting angles units

The following code is for converting an angle from radian to degree:

import math

# radian to degree 
math.degrees(0.1)

Trigonometric operations with Python

We can do a sorts of trigonometric calculation from sine and cosine to arc tangent and cotangent. Here few examples to give you an idea on how to use them:

import math

# calculate the cos of an angle
print(math.cos(25))

# calcule the arc sin of a nuber 
print(math.asin(0.4))

Datetime Module – Python temporal scripting

It gives us a set of tools to facilitate the work with date and time and their combination. Time in Python has its own placeholder, it is a time object. These object make our life very easy when it comes to anything time related. For example getting the current time, calculating a specific past or near future time, or just making calculations between two of multiple time objects.

The datetime module allow us to return or print the value in many different format.

import datetime

# get today's date and hour
now = datetime.date.today()

# get the year
year = now.year

Creating a date object using datetime

import datetime

# create a date
date = datetime.datetime(2021, 7, 7)

For more on datetime go to the w3chools website.

Conclusion

With the knowledge you acquired in this article, we have a better understanding of Python for scripting and automation using difference built-in modules that facilitate the automation of many tasks otherwise operated by a human. Note that this article was just an entry to the work of Python scripting and didn’t cover everything since scripting and automating is a vast sea!

From here you need to do your own research on the topics and modules covered in this article and pursue more advanced implementations and topics in general.

Check out my other automation related topics with python.

Thank you for reading my post.