We can't find the internet
Attempting to reconnect
Something went wrong!
Hang in there while we get back on track
Adam Hendel: Blazingly Fast Message Queue on Postgres with Rust (PGConf.EU 2023)
Discover how PGMQ, a fast and simple message queue on Postgres written in Rust, can enhance your data processing pipeline with at-least-once delivery and efficient batching.
- A message queue on Postgres written in Rust, called PGMQ, is a fast and simple solution.
- PGMQ uses a visibility timeout to manage message visibility and ensure at-least-once delivery.
- The queue is implemented as a table in Postgres, with columns for message ID, visibility timeout, and read count.
- PGMQ supports batching and has a simple API for creating and consuming messages.
- The visibility timeout is inspired by SQS and allows messages to be made invisible after a certain time period.
- PGMQ has a link to its documentation and source code.
- The talk mentioned other projects, such as Dagster, River, and PGRX, and discussed the trade-offs between different message queue solutions.
- PGMQ is primarily maintained by Tembo, the company the speaker works for.
- The speaker recommended keeping message sizes small and using batching to improve performance.
- PGMQ supports JSON-B as a data type and is designed to be simple and easy to use.
- The speaker discussed the idea of adding support for other data types, such as binary and Avro.
- The talk also mentioned the importance of using a connection pooler and a visibility timeout to manage message visibility.
- The speaker encouraged attendees to try out PGMQ and provide feedback.
- PGMQ is an open-source project and is available on GitHub.