Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Greenbids fix geolookup: host and fetch GCS + mock dbReader UT #3626

Open
wants to merge 22 commits into
base: master
Choose a base branch
from

Conversation

EvgeniiMunin
Copy link
Contributor

@EvgeniiMunin EvgeniiMunin commented Dec 13, 2024

🔧 Type of changes

  • new bid adapter
  • bid adapter update
  • new feature
  • new analytics adapter
  • new module
  • module update
  • bugfix
  • documentation
  • configuration
  • dependency update
  • tech debt (test coverage, refactorings, etc.)

✨ What's the context?

🧠 Rationale behind the change

  • We host the GeoLite2-Country.mmdb in Greenbids GCS bucket instead of open source URL: gs://greenbids-europe-west1-prebid-server-staging/GeoLite2-Country.mmdb
  • We mock the dbReader in unit test instead of loading real mmdb
  • We add the logic to fallback to getCountry(ip) only in case if Device::getGeo is null (geoLookupEnabled is false in GeoLocationServiceWrapper)

🔎 New Bid Adapter Checklist

  • verify email contact works
  • NO fully dynamic hostnames
  • geographic host parameters are NOT required
  • direct use of HTTP is prohibited - implement an existing Bidder interface that will do all the job
  • if the ORTB is just forwarded to the endpoint, use the generic adapter - define the new adapter as the alias of the generic adapter
  • cover an adapter configuration with an integration test

🧪 Test plan

How do you know the changes are safe to ship to production?

🏎 Quality check

  • Are your changes following our code style guidelines?
  • Are there any breaking changes in your code?
  • Does your test coverage exceed 90%?
  • Are there any erroneous console logs, debuggers or leftover code in your changes?

@EvgeniiMunin EvgeniiMunin changed the title Greenbids fix geolookup: mock dbReader TU + host GCS Greenbids fix geolookup: host and fetch GCS + mock dbReader TU Dec 13, 2024
@EvgeniiMunin EvgeniiMunin changed the title Greenbids fix geolookup: host and fetch GCS + mock dbReader TU Greenbids fix geolookup: host and fetch GCS + mock dbReader UT Dec 13, 2024
@EvgeniiMunin EvgeniiMunin force-pushed the greenbids-fix-geolookup branch from e5faad5 to ee640d3 Compare December 13, 2024 18:56
@EvgeniiMunin EvgeniiMunin marked this pull request as ready for review December 13, 2024 18:59
AntoxaAntoxic
AntoxaAntoxic previously approved these changes Dec 16, 2024
@EvgeniiMunin
Copy link
Contributor Author

Groovy IT is failing

Error:  Tests run: 38, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 21.95 s <<< FAILURE! -- in org.prebid.server.functional.tests.module.responsecorrenction.ResponseCorrectionSpec
Error:  org.prebid.server.functional.tests.module.responsecorrenction.ResponseCorrectionSpec.PBS should modify meta.mediaType and type for original response and also emit logs when response contains native meta.mediaType and adm without asset -- Time elapsed: 0.286 s <<< FAILURE!

I will push an empty commit to see if it is a flaky one

@AntoxaAntoxic AntoxaAntoxic added the do not merge Not the time for merging yet label Dec 16, 2024
Comment on lines +91 to +96
final String country = Optional.ofNullable(bidRequest.getDevice())
.map(Device::getGeo)
.map(Geo::getCountry)
.map(GreenbidsInferenceDataService::getCountryNameFromAlpha3)
.orElseGet(() -> getCountry(ip));

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this change is not covered by the unit tests

Copy link
Contributor Author

@EvgeniiMunin EvgeniiMunin Dec 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After checking Locale docs, the alpha3 code is not convertable to countryName via getCountry (only works for alpha2).
image

So need to use CountryCodeMapper mapToAlpha2 to convert alpha3 -> alpha2 and then via Locale alpha2 -> countryName.

private String resolveAlpha2CountryCode(Device device) {
        return Optional.ofNullable(device)
                .map(Device::getGeo)
                .map(Geo::getCountry)
                .map(countryCodeMapper::mapToAlpha2)
                .orElse(null);
    }

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, if you need it

@EvgeniiMunin
Copy link
Contributor Author

Also need to not forget to remove .eventually(v -> tmpFile.close()) downloadFile and find workaround for ensuring file closing

As we discussed on Slack on the current state the usage of .eventually(v -> tmpFile.close()) leads to exception on file closing
java.lang.IllegalStateException: File handle is closed as it looks like pipeTo closes file automatically. Manual closing is recommended but leads to exception.

As the current objective of this PR is to fix the PBS codebase I propose to find the solution in the PR #3596

Comment on lines +51 to +52
final String downloadUrl = properties.geoLiteCountryPath;
final String tmpPath = properties.tmpPath;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please use getters

Comment on lines +91 to +96
final String country = Optional.ofNullable(bidRequest.getDevice())
.map(Device::getGeo)
.map(Geo::getCountry)
.map(GreenbidsInferenceDataService::getCountryNameFromAlpha3)
.orElseGet(() -> getCountry(ip));

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, if you need it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
do not merge Not the time for merging yet do not port
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants