在多处理过程之间传递数据

因为在两个线程之间处理时数据是敏感的(认为并发读取和并发写入可能相互冲突,导致竞争条件),所以制作了一组唯一对象,以便于在线程之间来回传递数据。任何真正的原子操作都可以在线程之间使用,但坚持使用 Queue 总是安全的。

import multiprocessing
import queue
my_Queue=multiprocessing.Queue() 
#Creates a queue with an undefined maximum size
#this can be dangerous as the queue becomes increasingly large
#it will take a long time to copy data to/from each read/write thread

大多数人会建议在使用队列时,始终将队列数据放在 try:except:block 而不是使用 empty。但是,对于跳过扫描周期无关紧要的应用程序(数据可以在将状态从 queue.Empty==True 翻转到 queue.Empty==False 时放入队列中),通常最好在我称之为 Iftry 块的地方放置读写访问权限,因为’if’语句在技术上比捕获异常更高效。

import multiprocessing
import queue
'''Import necessary Python standard libraries, multiprocessing for classes and queue for the queue exceptions it provides'''
def Queue_Iftry_Get(get_queue, default=None, use_default=False, func=None, use_func=False):
    '''This global method for the Iftry block is provided for it's reuse and 
standard functionality, the if also saves on performance as opposed to catching
 the exception, which is expencive.
        It also allows the user to specify a function for the outgoing data to use,
 and a default value to return if the function cannot return the value from the queue'''
        if get_queue.empty():
            if use_default:
                return default
        else:
            try:
                value = get_queue.get_nowait()
            except queue.Empty:
                if use_default:
                    return default
            else:
                if use_func:
                    return func(value)
                else:
                    return value
    def Queue_Iftry_Put(put_queue, value):
        '''This global method for the Iftry block is provided because of its reuse 
and 
standard functionality, the If also saves on performance as opposed to catching
 the exception, which is expensive.
        Return True if placing value in the queue was successful. Otherwise, false'''
        if put_queue.full():
            return False
        else:
            try:
                put_queue.put_nowait(value)
            except queue.Full:
                return False
            else:
                return True