Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dataframes: Demonstrate pandas.read_sql() with both urllib3 vs. psycopg3 #651

Merged
merged 1 commit into from
Jan 10, 2025

Conversation

amotl
Copy link
Member

@amotl amotl commented Oct 1, 2024

About

People asked about PostgreSQL wire protocol support for reading data from CrateDB into dataframes.

References

@cla-bot cla-bot bot added the cla-signed label Oct 1, 2024
@amotl amotl marked this pull request as ready for review October 1, 2024 21:05
@amotl amotl assigned amotl and ckurze Oct 8, 2024
@amotl amotl removed the request for review from ckurze October 8, 2024 17:31
=======
To watch the HTTP traffic to your local CrateDB instance, invoke::

sudo ngrep -d lo0 -Wbyline port 4200
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or 5432 if they use the psycopg example

Copy link
Member Author

@amotl amotl Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, but sniffing the PostgreSQL wire protocol, you will probably not be able to see sensible things as a human. This is different to the HTTP protocol, because it is marshalling from/to JSON.

Do you think anything should be improved here, e.g. by adding such a comment that it only makes sense with HTTP/JSON?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for info, I regularly use this with the pgsql protocol, not everything is human-readable but SQL statements are visible in clear text which is very helpful.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah. Nice to learn about. Thanks! ✨

@amotl amotl force-pushed the pandas-postgresql branch from 991e5da to d6644cb Compare January 10, 2025 18:04
@amotl amotl merged commit f62c263 into main Jan 10, 2025
4 checks passed
@amotl amotl deleted the pandas-postgresql branch January 10, 2025 18:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants