@pandera.check_types + pandera.typing.Series[...] type hints = *whole lotta nuthin'*

[leycec]

This is your friendly neighbourhood @beartype developer @leycec. Hi, everybody! Pandera makes even my cats happy. 😸

@beartype has officially supported pandera for a year or so. Because of that, I've been fielding increasingly many @beartype issues, feature requests, and discussions pertaining to pandera. The problem is that I have no idea what I'm doing. Pandera is a vast supernova of typing I only dimly pretend to understand.

In this discussion, I'd like to upstream issue beartype/beartype#337 directly to the wonderful pandera community. @beartype user @komodovaran intends to type-check a variety of pandas objects via the standard decorator pattern. Pandera trivially supports type-checking pandas.DataFrame objects via well-documented integration between the @pandera.check_types decorator and pandera.typing.DataFrame[pandera.DataFrameModel] type hints annotating parameters and returns decorated by that decorator.

So far, so good.

pandera.typing.Series: It's Series Business

But what about typing-checking pandas.Series objects (i.e., one-dimensional ndarray columns)? Similar attempts to integrate the @pandera.check_types decorator with pandera.typing.Series[dtype] type hints... silently does nothing at all. That's not great. Even a warning that this is a bad approach would've been helpful. I'll take anything at this point.

@beartype's official FAQ entry promoting pandera cites a similar example. So, I'd really like to get this right:

from pandas import Series
from pandera import check_types as pandera_check_types
from pandera.dtypes import Int64
from pandera.typing import Series as SeriesBusiness  # <-- get it, "series business"? ...i'll stop now

@pandera_check_types
def some_func(dataframe: SeriesBusiness[Int64]) -> None:
    print(dataframe)

some_func(Series([1, 2, 3]))  # <-- this passes. *GOOD*
some_func(Series(['ok', 'ko', 'yo']))  # <-- this also passes. that can't be right? *BAD*
some_func(Series([1.2, 2.3, 3.4]))  # <----- this also passes. that can't be right? *BAD*

...which prints:

0    1
1    2
2    3
dtype: int64
0    ok
1    ko
2    yo
dtype: object
0    1.2
1    2.3
2    3.4
dtype: float64

Basically, everything passes – but I only expected the first call to some_func() to pass. Clearly, I'm confused.

There's a truckload of different pandera decorators. That's not a bad thing per say, but I'm unclear which one @beartype users should be using. @pandera.check_types seems right, but maybe users want @pandera.check_input or @pandera.check instead or whatevahs? Also, how actually do you type-check pandas.Series objects? I couldn't quite tell from pandera's otherwise awesome documentation. I assumed pandera.typing.Series[int] or whatevahs would do the trick, but now I'm getting no treats.

Clearly, I have no idea what I'm doing here. Help a @beartype bro help pandera, @cosmicBboy.

^{Post-mortem: I'm on the recent pandera 0.18.3 release. Incidentally, this might be a useful time to mention that I personally package pandera for Gentoo Linux. It's a pleasure. 😄}

[cosmicBboy]

So pandera.typing.Series[int] is only meant as purely metadata for DataFrameModel. Now it's not even necessary when defining DataFrameModel.

from pandera import DataFrameModel
from pandera.typing import Series

# original syntax
class Model(DataFrameModel):
    x: Series[int]

# newer syntax
class Model(DataFrameModel):
    x: int

Not against proper support for this, but I was going more for "if it's not documented, the feature doesn't exist" approach to this. In hindsight, it's probably a good idea to add some kind of note or warning in the docs saying that @pa.check_types only works for DataFrameModel and not for pandera.typing.Series[TYPE].

See #688 for previous discussion on this topic.

[leycec]

Gotcha! Thanks a heap for your detailed writeup. That... totally makes sense. Is there any other means of type-checking pandas.Series objects that you know of – or is that currently out-of-scope for pandera?

No worries either way. I'd imagine you have significantly bigger fish to fry. There's always a bigger fish. 😄