python - celery+rabbitmq asynchronous task, queue messages keep increasing
高洛峰
高洛峰 2017-06-22 11:51:49
0
1
1157

When celery cooperates with rabbitmq to perform asynchronous tasks, it is found that the number of messages in rabbitmq is constantly increasing, but these messages have actually been processed by the task.

Looking at the backend that comes with rabbimtq, we found that in Queued messages, the numbers of ready and total have reached more than 5,000, and the value of unacknowledged is 0. However, when actually working, the value of unacknowledged will change, but it will eventually become 0
There is nothing special about the configuration of celery, it only sets the following content
CELERY_IMPORTS = ('testtasks',)
BROKER_URL = 'amqp://guest:guest@localhost:5672//'
CELERY_RESULT_BACKEND = 'amqp://'

Then look at the system resources. The erl process memory is relatively large, and it probably uses more than 300 M
Is there something wrong with my celery configuration?

高洛峰
高洛峰

拥有18年软件开发和IT教学经验。曾任多家上市公司技术总监、架构师、项目经理、高级软件工程师等职务。 网络人气名人讲师,...

reply all(1)
曾经蜡笔没有小新

It has been solved. Just upgrade rabbitmq to version 3.3 or above.
I used rabbitmq version 3.1 before, and then celery is version 3.1


The real way to solve the problem is to add an ignore_result=True attribute to the task, as follows
@app.task(ignore_result=True)

Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template