使用具有多个线程的rabbitmq消息队列(Python Kombu)

Consuming a rabbitmq message queue with multiple threads (Python Kombu)(使用具有多个线程的rabbitmq消息队列(Python Kombu))
本文介绍了使用具有多个线程的rabbitmq消息队列(Python Kombu)的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着跟版网的小编来一起学习吧!

问题描述

我有一个带有单个队列的 RabbitMQ 交换.我希望创建一个运行多个线程并尽快通过该队列工作的守护进程.

I have a single RabbitMQ exchange with a single queue. I wish to create a daemon that runs multiple threads and works through this queue as quickly as possible.

工作"涉及与外部服务的通信,因此每个消费者内部都会发生相当多的阻塞.因此,我希望多个线程都处理来自同一个队列的消息.

The "work" involves communicating with external services, so there will be a fair amount of blocking going on within each consumer. As such, I want to have multiple threads all dealing with messages from the same queue.

我可以通过使用主线程上的队列,然后将传入的工作分配给其他线程池来实现这一点,但是有没有办法在各自的线程上下文中启动多个消费者?

I can achieve this by consuming the queue on my primary thread, and then farming the incoming work off to a pool of other threads, but is there a way to launch multiple consumers, each within their own threaded context?

推荐答案

看看 celery - 它是专门设计的排队和处理来自 AMPQ 代理的任务(但也适用于其他排队后端).它处理多进程或多线程并发,并使得创建和使用任务变得非常容易.

Take a look at celery - it is designed to queue and process tasks from an AMPQ broker (but also works with other queuing backends). It handles multiprocess or multithreaded concurrency and makes it very easy to create and consume tasks.

这篇关于使用具有多个线程的rabbitmq消息队列(Python Kombu)的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持跟版网!

本站部分内容来源互联网,如果有图片或者内容侵犯了您的权益,请联系我们,我们会在确认后第一时间进行删除!

相关文档推荐

Seasonal Decomposition of Time Series by Loess with Python(Loess 用 Python 对时间序列进行季节性分解)
Resample a time series with the index of another time series(使用另一个时间序列的索引重新采样一个时间序列)
How can I simply calculate the rolling/moving variance of a time series in python?(如何在 python 中简单地计算时间序列的滚动/移动方差?)
How to use Dynamic Time warping with kNN in python(如何在python中使用动态时间扭曲和kNN)
Keras LSTM: a time-series multi-step multi-features forecasting - poor results(Keras LSTM:时间序列多步多特征预测 - 结果不佳)
Python pandas time series interpolation and regularization(Python pandas 时间序列插值和正则化)