감동, 마음이 움직이는 것

[python] parallel computing (multiprocessing) 본문

Tips (Utility, Computer Language, and etc.)

[python] parallel computing (multiprocessing)

Struggler J. 2018. 11. 8. 01:29

Using "multiprocessing" library in python, we can make parallel computing process (https://m.blog.naver.com/townpharm/220951524843).
This works when the parallel calculations do not exchange their results value but each calculation is independent.


Simple code (example):
==========================
from multiprocessing import Pool
import time
import os
import math

def f(x):
    print("for input: ", x, ", Pid = ",os.getpid()) #Pid seems the id of cpu
    return x*x

if __name__ == '__main__':
    p = Pool(3) #3 means the number of cpus which we will use.
    print(p.map(f, range(0,10)))  #mapping the input and allocate jobs to each cpus.
    endTime = int(time.time())
    print("total time", (endTime - startTime))
==========================


results:
==========================
for input: 0, Pid =  2340
for input: 1, Pid =  2341
for input: 2, Pid =  2342
--------------------------
for input: 3, Pid =  2342
for input: 4, Pid =  2341
for input: 5, Pid =  2340
--------------------------
for input: 6, Pid =  2340
for input: 7, Pid =  2341
for input: 8, Pid =  2342
--------------------------
for input: 9, Pid =  2342
==========================


But it seems that in this case (independent calculations), bash files are better for parallelizing.