Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to execute jobs in parallel potential Stackoverflow ? #292

Open
CharlesCCC opened this issue Mar 11, 2019 · 0 comments
Open

How to execute jobs in parallel potential Stackoverflow ? #292

CharlesCCC opened this issue Mar 11, 2019 · 0 comments

Comments

@CharlesCCC
Copy link

Hi @dbader ,

First of all thank you for creating this also package, it is very useful and versatile.

I have been using the following feature for little over a year now,until recently I have noticed some issue:
https://schedule.readthedocs.io/en/stable/faq.html#how-to-execute-jobs-in-parallel

Recently, the number of thread that I'm running has been increased(it was about 10-20 pending thread, now it about 40+ pending thread), then the schedule.run_pending() will not kick off any task afterwards, more strangely, there is no error or exception has been through, it felt that the whole method call been silenced/skipped.

I did some initial investigation, noticed the thread limit might be the issue, not the number of thread but the stack size. I'm running it on MacOS, the default stack size is about 8Mb, the max size the OS allowed is 65532K, which is what I set for now, after change the Stack Size to the Max. the issue seems went away at the moment.

So, my question goes, is there a potential stack overflow issue while doing multi-threading with many task running in parallel ? If is, why it didn't blow up... with limited knowledge of system programming, I'm hoping someone know about it can help answer it.

System: Mac OS 10.14.3
Python: 3.7.2
Schedule Version: 0.6.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant