Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes foreign key statement bugs in postgres manager #3256

Merged
merged 8 commits into from
Feb 13, 2025

Conversation

alishakawaguchi
Copy link
Contributor

@alishakawaguchi alishakawaguchi commented Feb 13, 2025

  1. Updates postgres manager to create foreign key statements instead of using constraint definition from the database. These definitions are missing the schema in the definition causing problems.

  2. Separates SQL to get foreign key statements and non foreign key statements. This makes it clear which columns are for which. Reduces complexity.

  3. Updates non foreign key statements SQL to use higher level tables. Makes it easier to understand the query. Was unable to use higher level tables for the foreign key statements SQL because FK constraint names are not unique and caused duplicate issues with the reference columns.

  4. Deprecates dead code. The first version of init statements. which were full create statements that included constraints.

@alishakawaguchi alishakawaguchi added the bug Something isn't working label Feb 13, 2025
@alishakawaguchi alishakawaguchi self-assigned this Feb 13, 2025
Copy link

vercel bot commented Feb 13, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
neosync-docs ⬜️ Ignored (Inspect) Visit Preview Feb 13, 2025 8:53pm

Copy link

github-actions bot commented Feb 13, 2025

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedFeb 13, 2025, 8:53 PM

Copy link

codecov bot commented Feb 13, 2025

Codecov Report

Attention: Patch coverage is 5.37634% with 88 lines in your changes missing coverage. Please review.

Project coverage is 23.42%. Comparing base (93f2f37) to head (5f00685).
Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
...ackend/pkg/sqlmanager/postgres/postgres-manager.go 0.00% 85 Missing ⚠️
backend/internal/connectiondata/sql.go 0.00% 2 Missing ⚠️
cli/internal/cmds/neosync/sync/sync.go 83.33% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3256      +/-   ##
==========================================
- Coverage   23.48%   23.42%   -0.07%     
==========================================
  Files         387      387              
  Lines       44474    44418      -56     
==========================================
- Hits        10445    10404      -41     
+ Misses      32970    32961       -9     
+ Partials     1059     1053       -6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

github-actions bot commented Feb 13, 2025

Benchstat Geomean Results

0.95% sec/op, -0.04% B/op, 0.00% allocs/op

Benchstat results
                                   │ main-benchmark.txt │       branch-benchmark.txt        │
                                   │       sec/op       │   sec/op     vs base              │
CleanPostgresType-4                        42.75n ± 12%   42.82n ± 1%       ~ (p=1.000 n=6)
Runner_Single-4                            7.685µ ±  2%   7.844µ ± 3%  +2.07% (p=0.041 n=6)
GenerateBool-4                             11.39n ±  0%   11.41n ± 1%       ~ (p=0.216 n=6)
GenerateBusinessName-4                     288.1n ± 28%   309.2n ± 9%       ~ (p=0.310 n=6)
GenerateCardNumber-4                       28.41n ±  4%   28.48n ± 1%       ~ (p=0.619 n=6)
GenerateCategorical-4                      104.0n ±  1%   104.6n ± 1%       ~ (p=0.128 n=6)
GenerateCity-4                             173.8n ±  0%   175.3n ± 1%  +0.89% (p=0.006 n=6)
GenerateCountry-4                          105.6n ±  1%   105.6n ± 2%       ~ (p=0.662 n=6)
GenerateEmail-4                            1.349µ ±  0%   1.350µ ± 0%       ~ (p=0.677 n=6)
GenerateFirstName-4                        204.2n ±  1%   205.8n ± 1%       ~ (p=0.065 n=6)
GenerateFloat64-4                          28.79n ±  0%   28.98n ± 1%  +0.68% (p=0.002 n=6)
GenerateFullAddress-4                      1.425µ ±  0%   1.437µ ± 1%  +0.81% (p=0.002 n=6)
GenerateFullName-4                         1.224µ ±  2%   1.273µ ± 1%  +3.92% (p=0.002 n=6)
GenerateGender-4                           41.47n ±  1%   41.62n ± 0%  +0.36% (p=0.026 n=6)
GenerateInt64-4                            28.12n ±  0%   28.22n ± 0%  +0.34% (p=0.019 n=6)
GenerateInt64PhoneNumber-4                 51.46n ±  1%   51.39n ± 1%       ~ (p=0.610 n=6)
GenerateInternationalPhoneNumber-4         175.2n ±  0%   178.4n ± 5%  +1.83% (p=0.002 n=6)
GenerateIpAddress-4                        578.6n ±  1%   567.5n ± 1%  -1.93% (p=0.002 n=6)
GenerateLastName-4                         189.8n ±  1%   191.8n ± 1%  +1.05% (p=0.002 n=6)
GenerateRandomString-4                     983.4n ±  1%   987.5n ± 3%  +0.42% (p=0.022 n=6)
GenerateSHA256Hash-4                       1.012µ ±  1%   1.016µ ± 3%       ~ (p=0.095 n=6)
GenerateSSN-4                              305.1n ±  0%   306.8n ± 1%  +0.56% (p=0.013 n=6)
GenerateState-4                            104.6n ±  1%   105.9n ± 1%  +1.20% (p=0.004 n=6)
GenerateStreetAddress-4                    203.7n ±  1%   205.6n ± 0%  +0.93% (p=0.002 n=6)
GenerateStringPhoneNumber-4                178.3n ±  1%   179.6n ± 3%       ~ (p=0.102 n=6)
GenerateUnixTimestamp-4                    77.44n ±  1%   77.75n ± 1%       ~ (p=0.457 n=6)
GenerateUsername-4                         352.5n ±  4%   360.3n ± 1%       ~ (p=0.065 n=6)
GenerateUTCTimestamp-4                     103.8n ±  1%   104.3n ± 2%  +0.48% (p=0.019 n=6)
GenerateUUID-4                             735.0n ±  2%   733.7n ± 1%       ~ (p=0.240 n=6)
GenerateZipcode-4                          105.7n ±  1%   105.8n ± 1%       ~ (p=1.000 n=6)
TransformCharacterScramble-4               171.4n ±  1%   172.3n ± 1%  +0.55% (p=0.041 n=6)
TransformE164PhoneNumber-4                 179.2n ±  1%   181.3n ± 0%  +1.23% (p=0.017 n=6)
TransformEmail-4                           1.798µ ±  0%   1.809µ ± 0%  +0.61% (p=0.024 n=6)
TransformFirstName-4                       233.6n ±  0%   237.9n ± 1%  +1.84% (p=0.002 n=6)
TransformFloat64-4                         109.7n ±  1%   111.6n ± 1%  +1.73% (p=0.017 n=6)
TransformFullName-4                        1.233µ ±  0%   1.288µ ± 2%  +4.50% (p=0.002 n=6)
TransformInt64-4                           33.52n ±  1%   33.60n ± 2%       ~ (p=0.197 n=6)
TransformInt64PhoneNumber-4                57.92n ±  0%   58.00n ± 1%       ~ (p=0.069 n=6)
TransformLastName-4                        191.8n ±  0%   195.1n ± 1%  +1.75% (p=0.002 n=6)
TransformString-4                          1.022µ ±  1%   1.024µ ± 2%       ~ (p=0.253 n=6)
TransformStringPhoneNumber-4               210.2n ±  0%   211.9n ± 0%  +0.83% (p=0.002 n=6)
TransformUuid-4                            36.51n ±  1%   36.65n ± 0%       ~ (p=0.065 n=6)
geomean                                    193.9n         195.8n       +0.95%

                                   │ main-benchmark.txt │         branch-benchmark.txt         │
                                   │        B/op        │     B/op      vs base                │
CleanPostgresType-4                        0.000 ± 0%       0.000 ± 0%       ~ (p=1.000 n=6) ¹
Runner_Single-4                          2.296Ki ± 1%     2.290Ki ± 3%       ~ (p=0.589 n=6)
GenerateBool-4                             0.000 ± 0%       0.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateBusinessName-4                     304.0 ± 0%       304.0 ± 0%       ~ (p=1.000 n=6) ¹
GenerateCardNumber-4                       8.000 ± 0%       8.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateCategorical-4                      64.00 ± 0%       64.00 ± 0%       ~ (p=1.000 n=6) ¹
GenerateCity-4                             160.0 ± 0%       160.0 ± 0%       ~ (p=1.000 n=6) ¹
GenerateCountry-4                          24.00 ± 0%       24.00 ± 0%       ~ (p=1.000 n=6) ¹
GenerateEmail-4                            385.0 ± 0%       385.0 ± 0%       ~ (p=1.000 n=6) ¹
GenerateFirstName-4                        208.0 ± 0%       208.0 ± 0%       ~ (p=1.000 n=6) ¹
GenerateFloat64-4                          8.000 ± 0%       8.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateFullAddress-4                      512.0 ± 0%       512.0 ± 0%       ~ (p=1.000 n=6) ¹
GenerateFullName-4                         418.0 ± 0%       418.0 ± 0%       ~ (p=1.000 n=6) ¹
GenerateGender-4                           16.00 ± 0%       16.00 ± 0%       ~ (p=1.000 n=6) ¹
GenerateInt64-4                            7.000 ± 0%       7.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateInt64PhoneNumber-4                 8.000 ± 0%       8.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateInternationalPhoneNumber-4         40.00 ± 0%       40.00 ± 0%       ~ (p=1.000 n=6) ¹
GenerateIpAddress-4                        31.00 ± 0%       31.00 ± 0%       ~ (p=1.000 n=6) ¹
GenerateLastName-4                         176.0 ± 0%       176.0 ± 0%       ~ (p=1.000 n=6) ¹
GenerateRandomString-4                     186.0 ± 0%       186.0 ± 0%       ~ (p=1.000 n=6) ¹
GenerateSHA256Hash-4                       288.0 ± 0%       288.0 ± 0%       ~ (p=1.000 n=6) ¹
GenerateSSN-4                              46.00 ± 0%       46.00 ± 0%       ~ (p=1.000 n=6) ¹
GenerateState-4                            24.00 ± 0%       24.00 ± 0%       ~ (p=1.000 n=6) ¹
GenerateStreetAddress-4                    240.0 ± 0%       240.0 ± 0%       ~ (p=1.000 n=6) ¹
GenerateStringPhoneNumber-4                40.00 ± 0%       40.00 ± 0%       ~ (p=1.000 n=6) ¹
GenerateUnixTimestamp-4                    8.000 ± 0%       8.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateUsername-4                         205.0 ± 0%       205.0 ± 0%       ~ (p=1.000 n=6) ¹
GenerateUTCTimestamp-4                     24.00 ± 0%       24.00 ± 0%       ~ (p=1.000 n=6) ¹
GenerateUUID-4                             80.00 ± 0%       80.00 ± 0%       ~ (p=1.000 n=6) ¹
GenerateZipcode-4                          24.00 ± 0%       24.00 ± 0%       ~ (p=1.000 n=6) ¹
TransformCharacterScramble-4               40.00 ± 3%       39.50 ± 1%       ~ (p=1.000 n=6)
TransformE164PhoneNumber-4                 40.00 ± 0%       40.00 ± 0%       ~ (p=1.000 n=6) ¹
TransformEmail-4                           489.0 ± 0%       489.0 ± 0%       ~ (p=1.000 n=6) ¹
TransformFirstName-4                       224.0 ± 0%       224.0 ± 0%       ~ (p=1.000 n=6) ¹
TransformFloat64-4                         88.00 ± 0%       88.00 ± 0%       ~ (p=1.000 n=6) ¹
TransformFullName-4                        418.0 ± 0%       418.0 ± 0%       ~ (p=1.000 n=6) ¹
TransformInt64-4                           8.000 ± 0%       8.000 ± 0%       ~ (p=1.000 n=6) ¹
TransformInt64PhoneNumber-4                8.000 ± 0%       8.000 ± 0%       ~ (p=1.000 n=6) ¹
TransformLastName-4                        176.0 ± 0%       176.0 ± 0%       ~ (p=1.000 n=6) ¹
TransformString-4                          202.0 ± 0%       202.0 ± 0%       ~ (p=1.000 n=6) ¹
TransformStringPhoneNumber-4               56.00 ± 0%       56.00 ± 0%       ~ (p=1.000 n=6) ¹
TransformUuid-4                            16.00 ± 0%       16.00 ± 0%       ~ (p=1.000 n=6) ¹
geomean                                               ²                 -0.04%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                   │ main-benchmark.txt │        branch-benchmark.txt        │
                                   │     allocs/op      │ allocs/op   vs base                │
CleanPostgresType-4                        0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
Runner_Single-4                            24.00 ± 0%     24.00 ± 0%       ~ (p=1.000 n=6) ¹
GenerateBool-4                             0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateBusinessName-4                     2.000 ± 0%     2.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateCardNumber-4                       1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateCategorical-4                      2.000 ± 0%     2.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateCity-4                             2.000 ± 0%     2.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateCountry-4                          2.000 ± 0%     2.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateEmail-4                            8.000 ± 0%     8.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateFirstName-4                        2.000 ± 0%     2.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateFloat64-4                          1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateFullAddress-4                      10.00 ± 0%     10.00 ± 0%       ~ (p=1.000 n=6) ¹
GenerateFullName-4                         6.000 ± 0%     6.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateGender-4                           1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateInt64-4                            0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateInt64PhoneNumber-4                 1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateInternationalPhoneNumber-4         3.000 ± 0%     3.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateIpAddress-4                        2.000 ± 0%     2.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateLastName-4                         2.000 ± 0%     2.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateRandomString-4                     3.000 ± 0%     3.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateSHA256Hash-4                       7.000 ± 0%     7.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateSSN-4                              3.000 ± 0%     3.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateState-4                            2.000 ± 0%     2.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateStreetAddress-4                    2.000 ± 0%     2.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateStringPhoneNumber-4                3.000 ± 0%     3.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateUnixTimestamp-4                    1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateUsername-4                         6.000 ± 0%     6.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateUTCTimestamp-4                     1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateUUID-4                             3.000 ± 0%     3.000 ± 0%       ~ (p=1.000 n=6) ¹
GenerateZipcode-4                          2.000 ± 0%     2.000 ± 0%       ~ (p=1.000 n=6) ¹
TransformCharacterScramble-4               2.000 ± 0%     2.000 ± 0%       ~ (p=1.000 n=6) ¹
TransformE164PhoneNumber-4                 3.000 ± 0%     3.000 ± 0%       ~ (p=1.000 n=6) ¹
TransformEmail-4                           14.00 ± 0%     14.00 ± 0%       ~ (p=1.000 n=6) ¹
TransformFirstName-4                       3.000 ± 0%     3.000 ± 0%       ~ (p=1.000 n=6) ¹
TransformFloat64-4                         3.000 ± 0%     3.000 ± 0%       ~ (p=1.000 n=6) ¹
TransformFullName-4                        6.000 ± 0%     6.000 ± 0%       ~ (p=1.000 n=6) ¹
TransformInt64-4                           1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=6) ¹
TransformInt64PhoneNumber-4                1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=6) ¹
TransformLastName-4                        2.000 ± 0%     2.000 ± 0%       ~ (p=1.000 n=6) ¹
TransformString-4                          4.000 ± 0%     4.000 ± 0%       ~ (p=1.000 n=6) ¹
TransformStringPhoneNumber-4               4.000 ± 0%     4.000 ± 0%       ~ (p=1.000 n=6) ¹
TransformUuid-4                            1.000 ± 0%     1.000 ± 0%       ~ (p=1.000 n=6) ¹
geomean                                               ²               +0.00%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

@alishakawaguchi alishakawaguchi merged commit 69a2437 into main Feb 13, 2025
24 of 26 checks passed
@alishakawaguchi alishakawaguchi deleted the alisha/pg-fk-create branch February 13, 2025 21:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants