Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IndexError: list index out of range while getting collections #34

Closed
rajcanius opened this issue Feb 28, 2019 · 2 comments
Closed

IndexError: list index out of range while getting collections #34

rajcanius opened this issue Feb 28, 2019 · 2 comments
Labels

Comments

@rajcanius
Copy link

Describe the bug
While requesting NEW_FREE collection I get IndexError exception:

File ".../python2.7/site-packages/play_scraper/api.py", line 41, in collection
return s.collection(collection, category, **kwargs)
File ".../python2.7/site-packages/play_scraper/scraper.py", line 134, in collection
for app_card in soup.select('div[data-uitype="500"]')]
File ".../python2.7/site-packages/play_scraper/utils.py", line 358, in parse_card_info
developer_id = dev_soup.attrs['href'].split('=')[1]
IndexError: list index out of range

To Reproduce
run play_scraper.collection(collection='NEW_FREE',results=100,page=0,gl='us')

Expected behavior
Should receive list of app metadata from specified Chart in Google Play

Screenshots
Not applicable

Desktop (please complete the following information):

  • OS: [Linux Ubuntu 12.04.5 LTS, Precise Pangolin, MacOS Mojave 10.14.3]
  • Python Version: [2.7.3, 2.7.10]
  • play_scraper Version: [0.5.2: 2019-01-19]

Additional context
It seems, like the scraper is getting wrong response from Google Play and cannot parse it correctly.

@rajcanius rajcanius added the bug label Feb 28, 2019
@rajcanius
Copy link
Author

Upon better inspection I have realised, there is this exception because (at least) one app in the list has no developer name (not sure how is this possible).

Namely, it is app https://play.google.com/store/apps/details?id=org.selfie.beauty.camera.pro
(notice, that there is no developer name, 'Photography' is the category of the app)

Same thing is noticeable in the list view - https://play.google.com/store/apps/collection/topselling_new_free?gl=us

I'd suggest adding checks, wether fields are actually splittable.

@danieliu
Copy link
Owner

danieliu commented Mar 2, 2019

Should be fixed by 27bca88.

@danieliu danieliu closed this as completed Mar 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants