Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: generate showcase using docker image #3568

Open
wants to merge 50 commits into
base: main
Choose a base branch
from

Conversation

diegomarquezp
Copy link
Contributor

@diegomarquezp diegomarquezp commented Jan 14, 2025

Fixes #2470

Approach

In showcase/

  • Revamp the generation and verification scripts used by -Penable-golden-tests and -Pupdate to build and use the Hermetic Build Docker image

Effects

  • New samples/ folder added in showcase
  • Reformatted all of grpc, proto and gapic libraries in showcase/

@product-auto-label product-auto-label bot added the size: m Pull request size is medium. label Jan 14, 2025
@diegomarquezp diegomarquezp self-assigned this Jan 14, 2025
@diegomarquezp diegomarquezp marked this pull request as draft January 14, 2025 23:36
@product-auto-label product-auto-label bot added size: l Pull request size is large. and removed size: m Pull request size is medium. labels Jan 15, 2025
@diegomarquezp diegomarquezp marked this pull request as ready for review January 15, 2025 18:25
@diegomarquezp diegomarquezp marked this pull request as draft January 22, 2025 05:25
@@ -125,6 +117,14 @@ RUN owl-bot copy-code --version
RUN chmod o+rx $(which owl-bot)
RUN apk del -r npm && apk cache clean

# Here we transfer gapic-generator-java from the previous stage.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved the generator jar copy later in the final stage so the previous steps can run in parallel with mvn install. It saves dozens of seconds.

@product-auto-label product-auto-label bot added size: m Pull request size is medium. and removed size: l Pull request size is large. labels Jan 22, 2025
@diegomarquezp diegomarquezp marked this pull request as ready for review January 22, 2025 17:19
@@ -89,6 +89,10 @@ pushd "${api_def_dir}"
git checkout "${googleapis_commitish}"
popd

# we also setup showcase
source java-showcase/scripts/showcase_utilities.sh
append_showcase_to_api_defs "${api_def_dir}"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this function be called from generate_showcase.sh instead of here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's called both from the automatic generation script (this one) and the manual update + golden test scripts (generate_showcase.sh). Since both scripts need to append the showcase protos to googleapis, I extracted this function into a utility file.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My concern is that this hermetic_library_generation.sh is reused by google-cloud-java and all handwritten libraries, they don't need to copy showcase protos to googleapis. So can we move this step to a sdk-platform-java specific place? For example, maybe in the workflow?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, since the workflow also ends up using the common-usage action.yaml, I modified the scripts to accept a new flag showcase_mode that defaults to false. When true, it will download the api definitions. The download operation is still done in hermetic_library_generation.sh, but we prevent it from happening in repos that don't need these definitions.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think only action.yaml is reused. The hermetic_library_generation.yaml though is different in each repo. This is the one in sdk-platform-java and this is the one in google-cloud-java. So we could add the logic that appends showcase protos to the hermetic_library_generation.yaml in sdk-platform-java only, and it would not affect other repos.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

@diegomarquezp
Copy link
Contributor Author

Failing sonar checks are due to reformatting of showcase files

Copy link

Quality Gate Failed Quality Gate failed for 'gapic-generator-java-root'

Failed conditions
16.3% Coverage on New Code (required ≥ 80%)
31.2% Duplication on New Code (required ≤ 3%)
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Copy link

Quality Gate Failed Quality Gate failed for 'java_showcase_integration_tests'

Failed conditions
36.8% Coverage on New Code (required ≥ 80%)
18.6% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size: m Pull request size is medium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

showcase samples should be part of golden tests
3 participants