Skip to content

Prefer "own" type vars (owned by current function) when building "any of" constraints #18986

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

sterliakov
Copy link
Collaborator

@sterliakov sterliakov commented Apr 28, 2025

Sometimes our constraints builder needs to express "at least one of these constraints must be true". Sometimes it's trivial, but sometimes they have nothing in common - in that case we fall back to forgetting all of them and (usually) inferring Never.

This PR extends the fallback handling logic by one more rule: before giving up, we try restricting the constraints to type variables "owned" by the target function. This means that when we try to express T :> str || U :> str in the following setup:

from typing import TypeVar

T = TypeVar("T")
U = TypeVar("U")

class Foo(Generic[T]):
    def func(self, arg: T | U) -> None: ...

we won't ignore both and fall back to Never but will pick U :> str instead. The reason for this heuristic is that function-scoped typevars are definitely something we're trying to infer, while everything else is usually out of our control, any other type variable should stay intact.

This isn't safe in general case - it might be that another typevar satisfies the constraints but the chosen one doesn't. However, this shouldn't make any existing inference worse: if we used to infer Never and it worked, then anything else should almost definitely work as well.

See the added testcase for motivation: currently mypy fails to handle Mapping.get with default without return type context when Mapping has a type variable as the second argument. https://mypy-play.net/?mypy=1.15.0&python=3.12&flags=strict&gist=2f9493548082e66b77750655d3a90218

This is a prerequisite of #18976 - that inference change makes the problem solved here occur more often.

This definitely should be a function parameter, but I don't feel like adjusting all 60 infer_constraints calls in constraints.py so just put a stack in typestate.

This comment has been minimized.

This comment has been minimized.

@sterliakov sterliakov changed the title [experiment] Prefer "own" type vars (owned by current function) when building constraints Prefer "own" type vars (owned by current function) when building "any of" constraints Apr 28, 2025
@sterliakov sterliakov marked this pull request as ready for review April 28, 2025 02:31
@sterliakov sterliakov requested a review from ilevkivskyi April 28, 2025 16:25
@sterliakov
Copy link
Collaborator Author

@ilevkivskyi I understand no one wants to touch this inference logic to avoid regressions, but I think I found a very nice improvement in #18976 that closes a bunch of issues with a commonly used idiom, and this is the only prerequisite to fix one major regression caused by that. Could you have a look and say whether this idea makes sense at all?

@ilevkivskyi
Copy link
Member

Hm, although the general idea is correct (and moreover unfortunately there are places scattered around the code where we treat unrelated type variables incorrectly), I think the implementation may be simplified. Instead of what you are doing, can you try to filter away type variables that are not meta vars? I.e, try simply

[c for c in option if c.type_var.id.is_meta_var()]

as a fallback (you may need to handle cases when this returns all and/or none of the constraints).

@sterliakov
Copy link
Collaborator Author

...I'm feeling dumb now, I swear that was my very first attempt! This works and is definitely saner - perhaps I missed the empty result case initially.

Yes, I do see that non-meta typevars are mishandled here, but probably don't have enough energy to fix that.

Copy link
Contributor

github-actions bot commented May 5, 2025

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants