How to use subprocess.popen to connect multiple processes by pipes with Python?

The subprocess module in Python provides the Popen class, which allows you to create and manage child processes and interact with them using pipes for input, output, and error streams. Here’s an example of how you can use Popen to connect multiple processes together by pipes:

import subprocess

# Command 1: Echo "Hello" to stdout
command1 = ["echo", "Hello"]

# Command 2: Convert text to uppercase using 'tr' command
command2 = ["tr", "[:lower:]", "[:upper:]"]

# Create Popen objects for command 1 and command 2
p1 = subprocess.Popen(command1, stdout=subprocess.PIPE)
p2 = subprocess.Popen(command2, stdin=p1.stdout, stdout=subprocess.PIPE)
p1.stdout.close()  # Close stdout of p1 to prevent deadlocks

# Execute the commands and capture the output
output, _ = p2.communicate()

# Decode the output and print it
print(output.decode())

Output:

HELLO

In this example, we have two commands to be executed in sequence. command1 echoes “Hello” to stdout, and command2 uses the tr command to convert the text to uppercase.

We create two Popen objects p1 and p2 for command1 and command2, respectively. The stdout of p1 is set to subprocess.PIPE to capture the output of command1. The stdin of p2 is set to p1.stdout, which connects the output of command1 to the input of command2 using a pipe.

After executing the commands with p2.communicate(), we capture the output of command2 in the output variable, and then decode it using the decode() method to convert it from bytes to a string. Finally, we print the decoded output.

Note that in order to prevent deadlocks, we need to close the stdout of p1 using the close() method after creating p2. This ensures that the stdout of p1 is closed and data can flow freely from command1 to command2 through the pipe.