How Can I Run Multiple Python Functions in Parallel Using Multiprocessing?

Run Functions in Parallel: A Guide to Multiprocessing

Concept

Multitasking involves executing multiple tasks concurrently, a feat that can be achieved through either threading or multiprocessing.

Multiprocessing vs. Threading

In Python, threading is primarily useful for I/O-bound operations. For CPU-intensive tasks, multiprocessing offers better performance as it leverages multiple processor cores.

Running Functions in Parallel

Consider the following scenario:

import common

dir1 = 'C:\folder1'
dir2 = 'C:\folder2'
filename = 'test.txt'
addFiles = [25, 5, 15, 35, 45, 25, 5, 15, 35, 45]

def func1():
   c = common.Common()
   for i in range(len(addFiles)):
       c.createFiles(addFiles[i], filename, dir1)
       c.getFiles(dir1)
       time.sleep(10)
       c.removeFiles(addFiles[i], dir1)
       c.getFiles(dir1)

def func2():
   c = common.Common()
   for i in range(len(addFiles)):
       c.createFiles(addFiles[i], filename, dir2)
       c.getFiles(dir2)
       time.sleep(10)
       c.removeFiles(addFiles[i], dir2)
       c.getFiles(dir2)

To execute func1 and func2 concurrently, multiprocessing should be employed. Below is an example:

from multiprocessing import Process

if __name__ == "__main__":
    p1 = Process(target=func1)
    p1.start()
    p2 = Process(target=func2)
    p2.start()
    p1.join()
    p2.join()

Encapsulating Parallel Execution

The code above can be simplified into a function that facilitates parallel execution of multiple functions:

def runInParallel(*fns):
  proc = []
  for fn in fns:
    p = Process(target=fn)
    p.start()
    proc.append(p)
  for p in proc:
    p.join()

runInParallel(func1, func2)

The above is the detailed content of How Can I Run Multiple Python Functions in Parallel Using Multiprocessing?. For more information, please follow other related articles on the PHP Chinese website!