Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add postgis geocoder as a postcodes.io replacement #211

Merged
merged 10 commits into from
Feb 10, 2025

Conversation

joaquimds
Copy link

Description

Adds a new geocoding dataloader that uses PostGIS queries on our database to provide postcodes.io functionality.

Motivation and Context

Improves import performance.

How Can It Be Tested?

Import an area_stats dataset and check the worker logs to make sure the postgis_geocoder is running.

@commonknowledge-bot commonknowledge-bot deployed to feat/postgis-geocoding - meep-database PR #211 February 7, 2025 13:13 — with Render Active
@commonknowledge-bot commonknowledge-bot temporarily deployed to feat/postgis-geocoding - meep-intelligence-hub-backend PR #211 February 7, 2025 13:14 — with Render Destroyed
@commonknowledge-bot commonknowledge-bot temporarily deployed to feat/postgis-geocoding - meep-intelligence-hub-backend PR #211 February 7, 2025 13:14 — with Render Destroyed
@commonknowledge-bot commonknowledge-bot temporarily deployed to feat/postgis-geocoding - meep-intelligence-hub-frontend PR #211 February 7, 2025 13:14 — with Render Destroyed
@commonknowledge-bot commonknowledge-bot had a problem deploying to feat/postgis-geocoding - meep-intelligence-hub-worker PR #211 February 7, 2025 13:14 — with Render Failure
hub/data_imports/geocoding_config.py Outdated Show resolved Hide resolved
Comment on lines 262 to 265
if literal_lih_area_type__code:
area_filters["area_type__code"] = literal_lih_area_type__code
if literal_mapit_type:
area_filters["mapit_type"] = literal_mapit_type
area_filters["mapit_type__in"] = literal_mapit_type
Copy link
Member

@janbaykara janbaykara Feb 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to ensure_list() on mapit_type__in and also listify the area_type__code conditional, just above?

utils/postgis_geocoder.py Outdated Show resolved Hide resolved
utils/postgis_geocoder.py Show resolved Hide resolved
utils/postgis_geocoder.py Show resolved Hide resolved
@commonknowledge-bot commonknowledge-bot temporarily deployed to feat/postgis-geocoding - meep-intelligence-hub-frontend PR #211 February 10, 2025 10:55 — with Render Destroyed
@commonknowledge-bot commonknowledge-bot had a problem deploying to feat/postgis-geocoding - meep-intelligence-hub-worker PR #211 February 10, 2025 10:55 — with Render Failure
@commonknowledge-bot commonknowledge-bot temporarily deployed to feat/postgis-geocoding - meep-intelligence-hub-backend PR #211 February 10, 2025 11:21 — with Render Destroyed
@commonknowledge-bot commonknowledge-bot temporarily deployed to feat/postgis-geocoding - meep-intelligence-hub-worker PR #211 February 10, 2025 11:21 — with Render Destroyed
@joaquimds joaquimds merged commit 15895ad into main Feb 10, 2025
4 checks passed
@joaquimds joaquimds deleted the feat/postgis-geocoding branch February 10, 2025 11:33
Copy link

sentry-io bot commented Feb 10, 2025

Suspect Issues

This pull request was deployed and Sentry observed the following issues:

  • ‼️ SyntaxError: f-string expression part cannot include a backslash (postgis_geocoder.py, line 104) hub.models in <module> View Issue
  • ‼️ TypeError: GeocodeResult.init() got an unexpected keyword argument 'title' hub.data_imports.geocoding_config in import_are... View Issue
  • ‼️ TypeError: hub.data_imports.geocoding_config.GeocodeResult() got multiple values for keyword argument 'geocode_data' hub.data_imports.geocoding_config in geocode_re... View Issue

Did you find this useful? React with a 👍 or 👎

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants