Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LazyFrame sort and slice with nulls_last causes panic #20079

Open
2 tasks done
daviskirk opened this issue Nov 30, 2024 · 0 comments · May be fixed by #20084
Open
2 tasks done

LazyFrame sort and slice with nulls_last causes panic #20079

daviskirk opened this issue Nov 30, 2024 · 0 comments · May be fixed by #20084
Labels
bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars

Comments

@daviskirk
Copy link
Contributor

daviskirk commented Nov 30, 2024

Checks

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of Polars.

Reproducible example

pl.LazyFrame({"a": [None, 1.2]}).sort("a", nulls_last=True).slice(0, 10).collect()

Log output

thread '<unnamed>' panicked at crates/polars-core/src/chunked_array/ops/sort/arg_sort.rs:83:23:
range end index 9 out of range for slice of length 1
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
---------------------------------------------------------------------------
PanicException                            Traceback (most recent call last)
Cell In[2], line 1
----> 1 pl.LazyFrame({"a": [None, 1.2]}).sort("a", nulls_last=True).slice(0, 10).collect()

File polars/lazyframe/frame.py:2029, in LazyFrame.collect(self, type_coercion, predicate_pushdown, projection_pushdown, simplify_expression, slice_pushdown, comm_subplan_elim, comm_subexpr_elim, cluster_with_columns, collapse_joins, no_optimization, streaming, engine, background, _eager, **_kwargs)
   2027 # Only for testing purposes
   2028 callback = _kwargs.get("post_opt_callback", callback)
-> 2029 return wrap_df(ldf.collect(callback))

PanicException: range end index 9 out of range for slice of length 1

Issue description

Sorting and then slicing when using nulls_last, causes polars to panic.

The panic does not occur when:

  • nulls_last is not set
  • no nulls are in the sort column
  • we collect before slicing
  • setting maintain_order=True flag

Expected behavior

No panic should occur and the result should be the same as for collected dataframes.

Installed versions

--------Version info---------
Polars:              1.15.0
Index type:          UInt32
Platform:            Linux-6.8.0-49-generic-x86_64-with-glibc2.39
Python:              3.11.9 (main, Apr 19 2024, 16:48:06) [GCC 11.2.0]
LTS CPU:             False

----Optional dependencies----
adbc_driver_manager  <not installed>
altair               <not installed>
boto3                <not installed>
cloudpickle          <not installed>
connectorx           <not installed>
deltalake            <not installed>
fastexcel            <not installed>
fsspec               2024.3.1
gevent               <not installed>
google.auth          2.29.0
great_tables         <not installed>
matplotlib           <not installed>
nest_asyncio         <not installed>
numpy                1.24.3
openpyxl             3.1.2
pandas               2.1.4
pyarrow              16.0.0
pydantic             1.10.13
pyiceberg            <not installed>
sqlalchemy           1.4.39
torch                <not installed>
xlsx2csv             <not installed>
xlsxwriter           <not installed>
@daviskirk daviskirk added bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars labels Nov 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant