PRAW unable to handle x-ratelimit-remaining header

[dkozinn]

Describe the Bug

Calls to subreddit.mod.stream.modqueue are failing with error message similar to this:

File "/home/ubuntu/src/src/prawcore/prawcore/rate_limit.py", line 79, in update
    self.remaining = float(response_headers["x-ratelimit-remaining"])
ValueError: could not convert string to float: '991, 98'

Desired Result

Ratelimit handling works without errors.

Code to reproduce the bug

#!/usr/bin/python3

import praw

SUB = "nasa"

def main():
    """Main loop"""

    reddit = praw.Reddit("nasamodqbot")
    subreddit = reddit.subreddit(SUB)

    for submission in subreddit.mod.stream.modqueue():
        print("Made it here")

if __name__ == "__main__":
    main()

My code does not include sensitive credentials

• Yes, I have removed sensitive credentials from my code.

Relevant Logs

Exception has occurred: ValueError
could not convert string to float: '962, 99'
  File "/home/david/src/r-nasabot/src/test99.py", line 16, in main
    for submission in subreddit.mod.stream.modqueue():
  File "/home/david/src/r-nasabot/src/test99.py", line 20, in <module>
    main()
ValueError: could not convert string to float: '962, 99'

This code has previously worked as intended

Yes

Operating System/Environment

Ubuntu 20.04, 22.04

Python Version

3.8.10, 3.10.12

prawcore Version

2.3.0

Links, references, and/or additional comments?

No response

[dkozinn]

Update: As of 21:00 UTC the issue seems to be resolved. The sample code will no longer reproduce the error at this time.

[snarfed]

My guess is that this was a transient bug on Reddit's part, since their docs on https://support.reddithelp.com/hc/en-us/articles/16160319875092-Reddit-Data-API-Wiki say that this header value is a single number:

X-Ratelimit-Remaining: Approximate number of requests left to use

Regardless, ideally prawcore should just ignore bad header values here, not crash.

[oussama-gourari]

Kept getting this today at around 14:35 UTC:

ValueError: could not convert string to float: '544.0, 944'
ValueError: could not convert string to float: '543.0, 943'
ValueError: could not convert string to float: '542.0, 942'
ValueError: could not convert string to float: '540.0, 940'
ValueError: could not convert string to float: '539.0, 939'
ValueError: could not convert string to float: '537.0, 937'
ValueError: could not convert string to float: '536.0, 936'
ValueError: could not convert string to float: '534.0, 934'
ValueError: could not convert string to float: '533.0, 933'
ValueError: could not convert string to float: '531.0, 931'
ValueError: could not convert string to float: '530.0, 930'
ValueError: could not convert string to float: '529.0, 929'
ValueError: could not convert string to float: '528.0, 928'
ValueError: could not convert string to float: '527.0, 927'
ValueError: could not convert string to float: '526.0, 926'
ValueError: could not convert string to float: '525.0, 925'
ValueError: could not convert string to float: '524.0, 924'
ValueError: could not convert string to float: '523.0, 923'
ValueError: could not convert string to float: '522.0, 922'
ValueError: could not convert string to float: '521.0, 921'
ValueError: could not convert string to float: '520.0, 920'
ValueError: could not convert string to float: '519.0, 919'
ValueError: could not convert string to float: '518.0, 918'
ValueError: could not convert string to float: '517.0, 917'
ValueError: could not convert string to float: '516.0, 916'
ValueError: could not convert string to float: '515.0, 915'
ValueError: could not convert string to float: '514.0, 914'

[0xj7r]

+1 here - now running into this error

[ERROR] ValueError: could not convert string to float: '599.0, 999'
Traceback (most recent call last):
...

[0xj7r]

@oussama-gourari @the1daBread any thoughts on the origins of this error? Do you suspect it is an issue with prawcore iteself or the Reddit API?

[oussama-gourari]

@0xj7r Seems to be a Reddit issue, a fix is on the way: https://www.reddit.com/r/redditdev/comments/1df1bmg/xratelimitremaining_header_value_issue/l8fu2lk/

[LilSpazJoekp]

This is an error with Reddit's API. It should be returning floats for headers.

[snarfed]

This is an error with Reddit's API. It should be returning floats for headers.

True! ...but also ideally prawcore should probably handle it better than crashing. 😆 #173 (comment)

[LilSpazJoekp]

True, but this is rate limit code and handling it would likely involve discarding it and could cause your app to exceed the rate limits. This isn't a good thing for an API wrapper than handles rate limits for you.

[snarfed]

True! That seems like a policy choice though, right? You could easily fail closed, ie interpret it as over the rate limit, instead of failing open and interpreting as under the rate limit. Or something else entirely.

I'm sure you all will know the options and policy reasoning better than me! It just seems like surfacing an unexpected ValueError here, which most developers are unlikely to catch and will therefore crash them too, seems like a bad status quo.

[bboe]

I'm going to close this issue as it seems like a one off bug on Reddit's end that was resolved. For the PRAW project, it is customary for unexpected data to result in unexpected exceptions occurring.