Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bottleneck for large networks #70

Closed
KevCaz opened this issue Jun 29, 2019 · 2 comments
Closed

bottleneck for large networks #70

KevCaz opened this issue Jun 29, 2019 · 2 comments
Assignees
Labels
enhancement New feature or request Performance Performance related issue

Comments

@KevCaz
Copy link
Member

KevCaz commented Jun 29, 2019

reprex: get_network_by_id(19)

I've been waiting for >15min, still not done, all data are in, it's a list of >15,000 elements, not that big.

The culprit is the do.call(rbind,) approach for large list of sf objects see r-spatial/sf#798. I'm working on a workaround, basically I'll delay the creation of sf object as much as possible.

@KevCaz KevCaz self-assigned this Jun 29, 2019
@KevCaz KevCaz added the enhancement New feature or request label Jun 29, 2019
@KevCaz KevCaz changed the title bottleneck for for large networks bottleneck for large networks Jun 29, 2019
@KevCaz KevCaz added the Performance Performance related issue label Jun 29, 2019
@KevCaz
Copy link
Member Author

KevCaz commented Jun 29, 2019

Ok, in this case, actually we can avoid creating a sf object for nodes.
Not even sure it makes sense to have several sf in mgNetwork, better having a sf object for network and a dataframe for the nodes associated. What do you think @SteveViss ?

@KevCaz
Copy link
Member Author

KevCaz commented Jun 29, 2019

So even if we may not need it, new solution:

R> system.time(
+...   {gg = resp_to_spatial(dat)}
+... )

^C
Timing stopped at: 12.5 0 12.5 # did not want to wait
R> system.time(
+...   {gg = resp_to_spatial2(dat)}
+... )
   user  system elapsed 
  1.536   0.000   1.537 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Performance Performance related issue
Projects
None yet
Development

No branches or pull requests

1 participant