Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the FormSum memory leak #3897

Merged
merged 12 commits into from
Dec 6, 2024
Merged

Fix the FormSum memory leak #3897

merged 12 commits into from
Dec 6, 2024

Conversation

Ig-dolci
Copy link
Contributor

@Ig-dolci Ig-dolci commented Nov 28, 2024

Description

We are having memory leaks and more expensive computation of solvers involving FormSum (See more details on this discussion). That is caused by the sum operation involving dat. My solution here is to make a sum operation through the numpy arrays. Below is a comparison (using the example added at the discussion) of time and memory computation differences between the master branch and the current PR.

result

Copy link

github-actions bot commented Nov 28, 2024

TestsPassed ✅Skipped ⏭️Failed ❌
Firedrake complex8125 ran6540 passed1585 skipped0 failed

Copy link

github-actions bot commented Nov 28, 2024

TestsPassed ✅Skipped ⏭️Failed ❌
Firedrake real8131 ran7345 passed786 skipped0 failed

@JHopeCollins
Copy link
Member

Oh dear, if this fixes the issue then that interface could do with updating!

I think that the traversal method is taking advantage of the behaviour described here: #3348 (comment)

This behaviour is not widely known and is quite unintuitive - it's usually considered a bug. It might be best to change this method signature (and the preorder traversal method) to use visited=None as the kwarg, and use a self._visited dict with visited = visited if visited else self._visited

@Ig-dolci
Copy link
Contributor Author

Oh dear, if this fixes the issue then that interface could do with updating!

I think that the traversal method is taking advantage of the behaviour described here: #3348 (comment)

This behaviour is not widely known and is quite unintuitive - it's usually considered a bug. It might be best to change this method signature (and the preorder traversal method) to use visited=None as the kwarg, and use a self._visited dict with visited = visited if visited else self._visited

No. This does not fix it. I am still debugging.

@JHopeCollins
Copy link
Member

No. This does not fix it. I am still debugging.

Shame it wasn't so simple! It may still be good to change the signatures to avoid that behaviour though

@Ig-dolci
Copy link
Contributor Author

Oh dear, if this fixes the issue then that interface could do with updating!

I think that the traversal method is taking advantage of the behaviour described here: #3348 (comment)

This behaviour is not widely known and is quite unintuitive - it's usually considered a bug. It might be best to change this method signature (and the preorder traversal method) to use visited=None as the kwarg, and use a self._visited dict with visited = visited if visited else self._visited

I see. I gonna try visited=None.

firedrake/assemble.py Outdated Show resolved Hide resolved
@Ig-dolci Ig-dolci changed the title Fix the FormSum!? Fix the FormSum memory leak Dec 2, 2024
firedrake/assemble.py Outdated Show resolved Hide resolved
@Ig-dolci Ig-dolci marked this pull request as ready for review December 2, 2024 14:14
firedrake/assemble.py Outdated Show resolved Hide resolved
Copy link
Member

@JHopeCollins JHopeCollins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just a couple of small suggestions on the code.

It would be good to have a test for the axpy methods too!

pyop2/types/dat.py Show resolved Hide resolved
pyop2/types/dat.py Outdated Show resolved Hide resolved
pyop2/types/dat.py Show resolved Hide resolved
pyop2/types/dat.py Outdated Show resolved Hide resolved
pyop2/types/glob.py Outdated Show resolved Hide resolved
pyop2/types/glob.py Show resolved Hide resolved
Copy link
Contributor

@connorjward connorjward left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I would also like to see some tests. Including for the failure conditions like mismatching dats. These tests could go in https://github.com/firedrakeproject/firedrake/blob/master/tests/pyop2/test_dats.py (which contains some examples you could work off)

pyop2/types/dat.py Show resolved Hide resolved
pyop2/types/dat.py Show resolved Hide resolved
pyop2/types/dat.py Outdated Show resolved Hide resolved
Copy link
Contributor

@connorjward connorjward left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just docstring nitpicks from me. Looks fine otherwise. Thanks!

pyop2/types/dat.py Outdated Show resolved Hide resolved
pyop2/types/dat.py Outdated Show resolved Hide resolved
pyop2/types/dat.py Outdated Show resolved Hide resolved
pyop2/types/dat.py Outdated Show resolved Hide resolved
pyop2/types/glob.py Outdated Show resolved Hide resolved
pyop2/types/glob.py Outdated Show resolved Hide resolved
pyop2/types/glob.py Outdated Show resolved Hide resolved
pyop2/types/glob.py Outdated Show resolved Hide resolved
pyop2/types/dat.py Outdated Show resolved Hide resolved
Copy link
Member

@JHopeCollins JHopeCollins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks @Ig-dolci!

@Ig-dolci Ig-dolci enabled auto-merge (squash) December 6, 2024 09:35
@Ig-dolci Ig-dolci dismissed connorjward’s stale review December 6, 2024 09:37

Connor reviewed.

@Ig-dolci Ig-dolci merged commit ded8f14 into master Dec 6, 2024
18 checks passed
@Ig-dolci Ig-dolci deleted the dolci/fix_leaking_formsum branch December 6, 2024 09:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants