Skip to content

assign back to self.l2 in reset_parameters in RGATConv #10203

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

bedwards
Copy link

I can't quite explain this, and I do not (yet) have the minimal code to reproduce this, but I was randomly getting all nan's for the l2 parameter of RGATConv. Randomly meaning I would start up my script multiple times and initialize 13 models with RGATConv layers and I would end up with different numbers of ones with l2 being all nan. This is my workaround

      for _ in range(100):

        layer = RGATConv(
          inp,
          out,
          num_relations=edges["Type"].unique().shape[0],
          heads=heads,
          edge_dim=len(["Direction", "Delta"]),
        )

        if not torch.isnan(layer.get_parameter("l2")).any():
          break

        print("Found nans in l2 parameter during layer initialization, retrying.")

      else:
        raise ValueError("Always got nans in l2 parameter during layer initialization")

So I don't know why my fix would fix the randomness, but in investigating it I found this bug where the result of torch.full is dropped and not assigned back to self.l2.

diff --git a/torch_geometric/nn/conv/rgat_conv.py b/torch_geometric/nn/conv/rgat_conv.py
index 4c3a9937..b6f87b87 100644
--- a/torch_geometric/nn/conv/rgat_conv.py
+++ b/torch_geometric/nn/conv/rgat_conv.py
@@ -307,7 +307,7 @@ class RGATConv(MessagePassing):
         zeros(self.bias)
         ones(self.l1)
         zeros(self.b1)
-        torch.full(self.l2.size(), 1 / self.out_channels)
+        self.l2 = torch.full(self.l2.size(), 1 / self.out_channels)
         zeros(self.b2)
         if self.lin_edge is not None:
             glorot(self.lin_edge)

@bedwards bedwards requested a review from EdisonLeeeee as a code owner April 16, 2025 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant