Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pl.DataFrame(nan_to_null=True) not work for tuple #20042

Open
2 tasks done
wukan1986 opened this issue Nov 28, 2024 · 0 comments
Open
2 tasks done

pl.DataFrame(nan_to_null=True) not work for tuple #20042

wukan1986 opened this issue Nov 28, 2024 · 0 comments
Labels
A-input-parsing Area: parsing input arguments accepted Ready for implementation bug Something isn't working P-medium Priority: medium python Related to Python Polars

Comments

@wukan1986
Copy link

Checks

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of Polars.

Reproducible example

import numpy as np
import polars as pl
import talib as ta

a = np.array([np.nan, 1, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, ])

df = pl.DataFrame(a, nan_to_null=True)
print(df)
"""
shape: (12, 1)
┌──────────┐
│ column_0 │
│ ---      │
│ f64      │
╞══════════╡
│ null     │
│ 1.0      │
│ 1.0      │
│ 2.0      │
│ 3.0      │
│ …        │
│ 6.0      │
│ 7.0      │
│ 8.0      │
│ 9.0      │
│ 10.0     │
└──────────┘
"""

df = pl.DataFrame((a, a), nan_to_null=True)
print(df)
"""
shape: (12, 2)
┌──────────┬──────────┐
│ column_0 ┆ column_1 │
│ ---      ┆ ---      │
│ f64      ┆ f64      │
╞══════════╪══════════╡
│ NaN      ┆ NaN      │
│ 1.0      ┆ 1.0      │
│ 1.0      ┆ 1.0      │
│ 2.0      ┆ 2.0      │
│ 3.0      ┆ 3.0      │
│ …        ┆ …        │
│ 6.0      ┆ 6.0      │
│ 7.0      ┆ 7.0      │
│ 8.0      ┆ 8.0      │
│ 9.0      ┆ 9.0      │
│ 10.0     ┆ 10.0     │
└──────────┴──────────┘
"""
df = pl.DataFrame({"a": [1, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, ]}).lazy()
a = df.with_columns([
    pl.col('a').map_batches(lambda xx: pl.DataFrame(
        ta.MACD(xx.to_numpy().astype(float), 3, 4, 5),
        nan_to_null=True).to_struct())
])
print(a.collect())
"""
┌──────────────────────────────┐
│ a                            │
│ ---                          │
│ struct[3]                    │
╞══════════════════════════════╡
│ {NaN,NaN,NaN}                │
│ {NaN,NaN,NaN}                │
│ {NaN,NaN,NaN}                │
│ {NaN,NaN,NaN}                │
│ {NaN,NaN,NaN}                │
│ …                            │
│ {NaN,NaN,NaN}                │
│ {0.4676,0.38472,0.08288}     │
│ {0.48056,0.416667,0.063893}  │
│ {0.488336,0.440556,0.04778}  │
│ {0.493002,0.458038,0.034963} │
└──────────────────────────────┘
"""

Log output

No response

Issue description

pl.DataFrame(nan_to_null=True) not work for tuple

Expected behavior

replace nan to null

Installed versions

--------Version info---------
Polars: 1.15.0
Index type: UInt32
Platform: Windows-10-10.0.22631-SP0
Python: 3.11.3 | packaged by Anaconda, Inc. | (main, Apr 19 2023, 23:46:34) [MSC v.1916 64 bit (AMD64)]
LTS CPU: False

----Optional dependencies----
adbc_driver_manager
altair 5.0.1
boto3
cloudpickle
connectorx
deltalake
fastexcel
fsspec
gevent
google.auth 2.26.1
great_tables
matplotlib 3.7.1
nest_asyncio 1.5.6
numpy 1.26.4
openpyxl 3.1.2
pandas 2.2.3
pyarrow 12.0.0
pydantic 1.10.7
pyiceberg
sqlalchemy 2.0.13
torch 2.0.1+cu118
xlsx2csv
xlsxwriter 3.2.0

@wukan1986 wukan1986 added bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars labels Nov 28, 2024
@wukan1986 wukan1986 changed the title nan_to_null=True not work for tuple pl.DataFrame(nan_to_null=True) not work for tuple Nov 28, 2024
@nameexhaustion nameexhaustion added accepted Ready for implementation P-medium Priority: medium A-input-parsing Area: parsing input arguments and removed needs triage Awaiting prioritization by a maintainer labels Nov 28, 2024
@github-project-automation github-project-automation bot moved this to Ready in Backlog Nov 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-input-parsing Area: parsing input arguments accepted Ready for implementation bug Something isn't working P-medium Priority: medium python Related to Python Polars
Projects
Status: Ready
Development

No branches or pull requests

2 participants