How to stream PostgreSQL (pgx.Rows) query results into a pager (less) without buffering the entire result set?

3 days ago 4
ARTICLE AD BOX

Im building a PostgreSQL CLI client in Go (pgxcli), inspired by pgcli. I am using the pgx driver, and the app features an REPL, autocompletion, and syntax highlighting.

Currently, the code buffers the entire result set in memory, but this becomes inefficient with large tables (millions of rows). Im considering using cursors to stream results, but I run into two problems:

I won’t know the number of rows in advance.

The row widths vary, making it difficult to render an ASCII table.

I’m streaming to pagers like less and more. How can I manage this without buffering the entire results, especially with dynamic row sizes? Also, I noticed pgcli has a query limit—could that be part of the solution, or is there a better approach?

Any guidance or tips would be thankful!

Code link: https://github.com/Balaji01-4D/pgxcli/blob/2c6ec77db9a54358e6310ac67e927f5085c8c32a/internal/database/executor.go#L72

Read Entire Article