5750fddf28
this patches does multiple things: - it ignores batching when pulling from main queue to maintain as much ordering as possible. this avoids related messages split across multiple batches and one batch starting much farther along than another - it sets pipeline processing listeners to single thread. at this stage, the pipeline queue contains less messages and therefore very likely, thread1 and thread2 will grab related messages and race to set cache - adds sorting to pipeline queue so if batching is enabled, we can further ensure that messages are in order before processing. - enables batching by default. (one thread per listener grabbing one message at a time will be slow.) - offers better batching for direct to db dispatcher as resources are grouped Change-Id: Iac5b552bae1e73f93cbfc830b1e83510b1aceb9e
12 lines
568 B
YAML
12 lines
568 B
YAML
---
|
|
upgrade:
|
|
- Batching is enabled by default now when coordinated workers are enabled.
|
|
Depending on load, it is recommended to scale out the number of
|
|
`pipeline_processing_queues` to improve distribution. `batch_size` should
|
|
also be configured accordingly.
|
|
fixes:
|
|
- Fix to improve handling messages in environments heavily backed up.
|
|
Previously, notification handlers greedily grabbed messages from queues
|
|
which could cause ordering issues. A fix was applied to sequentially
|
|
process messages in a single thread to prevent ordering issues.
|