在 Windows 上导入和使用使用多处理而不会导致无限循环的模块

importing and using a module that uses multiprocessing without causing infinite loop on Windows(在 Windows 上导入和使用使用多处理而不会导致无限循环的模块)
本文介绍了在 Windows 上导入和使用使用多处理而不会导致无限循环的模块的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着跟版网的小编来一起学习吧!

问题描述

我有一个名为 multi.py 的模块.如果我只是想将 multi.py 作为脚本执行,那么避免在 Windows 上崩溃(产生无限数量的进程)的解决方法是将多处理代码放在:

I have a module named multi.py. If I simply wanted to execute multi.py as a script, then the workaround to avoid crashing on Windows (spawning an infinite number of processes) is to put the multiprocessing code under:

if __name__ == '__main__':

但是,我尝试将它作为模块从另一个脚本导入并调用 multi.start().这如何实现?

However, I am trying to import it as a module from another script and call multi.start(). How can this be accomplished?

# multi.py
import multiprocessing

def test(x):
    x**=2

def start():
    pool = multiprocessing.Pool(processes=multiprocessing.cpu_count()-2)
    pool.map(test, (i for i in range(1000*1000)))
    pool.terminate()
    print('done.')

if __name__ == '__main__':
    print('runs as a script,',__name__)
else:
    print('runs as imported module,',__name__)

这是我运行的 test.py:

# test.py
import multi
multi.start()

推荐答案

我不太明白你在问什么.你不需要做任何事情来防止它产生无限多的进程.我只是在 Windows XP 上运行它 --- 导入文件并运行 multi.start() --- 它在几秒钟内完成.

I don't quite get what you're asking. You don't need to do anything to prevent this from spawning infinitely many processes. I just ran it on Windows XP --- imported the file and ran multi.start() --- and it completed fine in a couple seconds.

您必须执行 if __name__=="__main__" 保护的原因是,在 Windows 上,多处理必须导入主脚本才能运行目标函数,这意味着顶部-该文件中的级别模块代码将被执行.仅当顶级模块代码本身尝试生成新进程时,才会出现问题.在您的示例中,顶级模块代码不使用多处理,因此没有无限的进程链.

The reason you have to do the if __name__=="__main__" protection is that, on Windows, multiprocessing has to import the main script in order to run the target function, which means top-level module code in that file will be executed. The problem only arises if that top-level module code itself tries to spawn a new process. In your example, the top level module code doesn't use multiprocessing, so there's no infinite process chain.

现在我明白你的要求了.您不需要保护 multi.py.你需要保护你的主脚本,不管它是什么.如果您遇到崩溃,那是因为在您的主脚本中,您在顶级模块代码中执行 multi.start().您的脚本需要如下所示:

Now I get what you're asking. You don't need to protect multi.py. You need to protect your main script, whatever it is. If you're getting a crash, it's because in your main script you are doing multi.start() in the top level module code. Your script needs to look like this:

import multi
if __name__=="__main__":
    multi.start()

main 脚本中始终需要保护".

The "protection" is always needed in the main script.

这篇关于在 Windows 上导入和使用使用多处理而不会导致无限循环的模块的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持跟版网!

本站部分内容来源互联网,如果有图片或者内容侵犯了您的权益,请联系我们,我们会在确认后第一时间进行删除!

相关文档推荐

Multiprocessing on Windows breaks(Windows 上的多处理中断)
How to use a generator as an iterable with Multiprocessing map function(如何将生成器用作具有多处理映射功能的可迭代对象)
read multiple files using multiprocessing(使用多处理读取多个文件)
Why does importing module in #39;__main__#39; not allow multiprocessig to use module?(为什么在__main__中导入模块不允许multiprocessig使用模块?)
Trouble using a lock with multiprocessing.Pool: pickling error(使用带有 multiprocessing.Pool 的锁时遇到问题:酸洗错误)
Python sharing a dictionary between parallel processes(Python 在并行进程之间共享字典)