Skip to content
This repository has been archived by the owner on Mar 11, 2024. It is now read-only.

Add a CSV IO manager #16

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Conversation

mjkanji
Copy link

@mjkanji mjkanji commented Jul 7, 2023

This PR adds a basic CSV I/O manager. Since most of the logic is handled by BasePolarsUPathIOManager, the only substantial changes compared to PolarsParquetIOManager are the change to write_csv and using "csv" as the default format for ds.dataset.

There are no additions to the docs or tests. @danielgafni Please let me know what you'd like to see on that front to accept the PR.

@danielgafni
Copy link
Owner

danielgafni commented Jul 7, 2023

Hey!
Thank you for the contribution.
I would like to see the following:

  • add an entry in README.md
  • add convenience import in dagster_polars/__init__.py
  • test the IOManager. Since there are no CSV-specific features to test, you can just add it to this list used in the general tests.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants