[Datahub] Feature catalog - Load attributes description from a linked record if present #1137

AlitaBernachot · 2025-02-26T17:01:48Z

Description

This PR introduces feature catalog. It is a first technical step to load a feature catalog in the record before displaying it in a record page.
Attributes from a feature catalog can be retrieve in two different ways:

Case 1: the feature catalog is already embedded in the record definition returned by Elastic Search, it will appear in the featureTypes property
Case 2: the feature catalog is present, but in a linked record, in this case the linked record will appear in the related.fcats property: in this case we will store the linked record uuid as property (featureCatalogIdentifier) and will load (getRecord()) its definition to get its own featureTypes

Thanks @Guillaume-d-o for your work on this PR.

💡 Example of a record with a linked feature catalog (for Case 2)

curl -X POST "https://geonetwork-ui.labs.camptocamp.com/geonetwork/srv/api/search/records/_search?bucket=bucket&relatedType=fcats" -H "accept: application/json" -H "Content-Type: application/json" -H "X-XSRF-TOKEN: b8a9b3ae-fe1c-466a-bb9f-9aff3a959b7e" -d "{\"query\":{\"ids\":{\"values\":[\"IGNF_BD-TOPO\"]}}}"

Architectural changes

New mappers were added to load featureTypes and get the featureCatalogIdentifier.

Screenshots

No UI

Quality Assurance Checklist

Commit history is devoid of any merge commits and readable to facilitate reviews
If new logic ⚙️ is introduced: unit tests were added
If new user stories 🤏 are introduced: E2E tests were added
If new UI components 🕹️ are introduced: corresponding stories in Storybook were created
If breaking changes 🪚 are introduced: add the breaking change label
If bugs 🐞 are fixed: add the backport <release branch> label
The documentation website 📚 has received the love it deserves

github-actions · 2025-02-26T17:03:01Z

Affected libs:
api-metadata-converter,api-repository,feature-catalog,feature-record,feature-router,feature-editor,feature-search,feature-map,feature-auth,ui-search,common-domain,common-fixtures,feature-dataviz,ui-elements,feature-notifications,ui-catalog,util-shared,ui-widgets,ui-dataviz,ui-inputs,ui-layout,ui-map

Affected apps:
metadata-converter,metadata-editor,datahub,demo,webcomponents,map-viewer,search,datafeeder,data-platform

🚀 Build and deploy storybook and demo on GitHub Pages
📦 Build and push affected docker images

github-actions · 2025-02-27T14:31:11Z

📷 Screenshots are here!

Guillaume-d-o · 2025-03-03T14:09:58Z

libs/feature/record/src/lib/state/mdview.actions.ts

@@ -28,6 +29,21 @@ export const loadFullMetadataFailure = createAction(
  props<{ otherError?: string; notFound?: boolean }>()
 )

+export const loadFeatureCatalog = createAction(


@jahow says : need to be tested (not action but effects, facade, reducer .. )

test on action done previously

libs/feature/record/src/lib/state/mdview.effects.ts

libs/feature/record/src/lib/state/mdview.reducer.ts

Guillaume-d-o · 2025-03-03T14:11:53Z

libs/api/metadata-converter/src/lib/gn4/gn4.converter.spec.ts

@@ -2642,6 +2643,7 @@ describe('Gn4Converter', () => {
            extras: {
              catalogUuid: 'metawal.wallonie.be',
              favoriteCount: 0,
+              featureTypes: [],


@jahow says : is there a way to get a test to show that this takes in the correct value? e.g. by changing the mock values used as input?

If i understand well we tested this in the gn4 repository by checking featureType has some elements no ?

coveralls · 2025-03-04T08:47:53Z

coverage: 83.817% (+0.9%) from 82.949%
when pulling 75aaeb4 on GSIGNGPF-38-feat-cat-linked-record
into da7bc31 on main.

Guillaume-d-o · 2025-03-04T08:01:34Z

libs/api/metadata-converter/src/lib/gn4/gn4.converter.spec.ts

@@ -2642,6 +2643,7 @@ describe('Gn4Converter', () => {
            extras: {
              catalogUuid: 'metawal.wallonie.be',
              favoriteCount: 0,
+              featureTypes: [],


If i understand well we tested this in the gn4 repository by checking featureType has some elements no ?

libs/api/repository/src/lib/gn4/gn4-repository.ts

Guillaume-d-o · 2025-03-06T09:57:54Z

libs/api/repository/src/lib/gn4/gn4-repository.ts

+      )
+    }
+
+    return of(null)


@jahow comments : keep the "happy path" on the left by checking the null return condition first , then the non null condition after

FYI can't apply comment suggestion as this function has evolved, there is no more test on decodeMap.

Guillaume-d-o · 2025-03-06T10:01:13Z

libs/feature/record/src/lib/state/mdview.facade.ts

+  /**
+   * FeatureCatalog
+   */
+  loadFeatureCatalog(metadata: CatalogRecord) {


@jahow cmments : we should not need this on the facade because loading the attributes will not be required by the components; instead, an effect in the state should do it

Guillaume-d-o · 2025-03-06T10:02:18Z

libs/feature/record/src/lib/state/mdview.reducer.ts

@@ -17,13 +18,18 @@ export interface MetadataViewState {
  allUserFeedbacksLoading: boolean
  addUserFeedbackLoading: boolean
  chartConfig?: DatavizConfigurationModel
+  featureCatalog?: DatasetFeatureCatalog
+  featureCatalogLoading: boolean
+  featureCatalogError: { notFound?: boolean; otherError?: string } | null


@jahow comments : I think it'd make sense to have a separate error for the feature catalog loading; this error can only be a string because in the case of feature catalogs it's OK to not find any

Alita, does the notFound necessary ?

Guillaume-d-o · 2025-03-06T10:04:25Z

libs/feature/record/src/lib/state/mdview.facade.ts

@@ -52,6 +52,11 @@ export class MdViewFacade {
    filter((md) => !!md)
  )

+  featureCatalog$ = this.store.pipe(
+    select(MdViewSelectors.getFeatureCatalog),
+    filter((fc) => !!fc)


@jahow comments : I would remove the filter here as it can be misleading for a consumer of this stream; e.g. if no feature catalog was found, this stream would never emit a value

Guillaume-d-o · 2025-03-06T10:06:31Z

libs/feature/record/src/lib/state/mdview.effects.ts

@@ -33,6 +32,31 @@ export class MdViewEffects {
    )
  )

+  loadFeatureCatalog$ = createEffect(() =>


@jahow comments :
Here I would add an additional effect that emits a loadFeatureCatalogAttributes action when a loadFullMetadataSuccess action is received; this way the loading of the feature catalog will be done automatically when a full record is available, and then repository.getFeatureCatalog(record) can be called

@jahow finally we keept only the loadFeatureCatalog associated to the full metadata loading action as long as there might be a associated effects to feature loading. Are you ok with this behavior ?

Guillaume-d-o · 2025-03-06T10:14:39Z

libs/feature/record/src/lib/state/mdview.effects.ts

+      )
+    )
+  )
+


@jahow says : need tests

AlitaBernachot force-pushed the GSIGNGPF-38-feat-cat-linked-record branch from 0451715 to e54e85e Compare February 27, 2025 14:16

AlitaBernachot force-pushed the GSIGNGPF-38-feat-cat-linked-record branch 8 times, most recently from 97caf72 to 6909b57 Compare February 27, 2025 17:45

AlitaBernachot mentioned this pull request Feb 27, 2025

load attribute descriptions embedded ( feature catalog) #1135

Closed

7 tasks

Guillaume-d-o reviewed Mar 3, 2025

View reviewed changes

AlitaBernachot marked this pull request as ready for review March 4, 2025 08:35

Guillaume-d-o and others added 14 commits March 4, 2025 15:45

GSIGNGPF-39-load attribute descriptions embedded ( feature catalog)

279879c

mapping des featureTypes dans extras

cf19431

UT adaptation

8d0ae28

Commit intermédiaire

dcbadc4

feat: feature catalog

31b919d

feat: feature catalog from related in repo and mapper

57cf729

init xtra test + code review feedbacks

50037e6

fixtures test and adjustments

b32c400

fix tests and function

a504ca0

suppression related

61be185

reducer facade and effects tests

a483769

feat: feature catalog from linked record

7291367

fix: eclude fcats identifier from keys

35ce1f8

fix: remove use of recordLink_fcats_uuid

f98e22f

AlitaBernachot force-pushed the GSIGNGPF-38-feat-cat-linked-record branch from d9bdc0a to f98e22f Compare March 4, 2025 15:13

Guillaume-d-o reviewed Mar 6, 2025

View reviewed changes

AlitaBernachot added 2 commits March 6, 2025 13:08

fix: featureCatalogError is now a string instead of object

6647179

fix: remove filter on featureCatalog$ stream

b25a6d9

AlitaBernachot force-pushed the GSIGNGPF-38-feat-cat-linked-record branch from 9e34c0d to b25a6d9 Compare March 6, 2025 12:20

code review feedbacks

75aaeb4

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Datahub] Feature catalog - Load attributes description from a linked record if present #1137

[Datahub] Feature catalog - Load attributes description from a linked record if present #1137

AlitaBernachot commented Feb 26, 2025 •

edited

Loading

github-actions bot commented Feb 26, 2025 •

edited

Loading

github-actions bot commented Feb 27, 2025 •

edited

Loading

Guillaume-d-o Mar 3, 2025 •

edited

Loading

Guillaume-d-o Mar 6, 2025

Guillaume-d-o Mar 3, 2025

Guillaume-d-o Mar 4, 2025

coveralls commented Mar 4, 2025 •

edited

Loading

Guillaume-d-o Mar 4, 2025

Guillaume-d-o Mar 6, 2025

AlitaBernachot Mar 6, 2025 •

edited

Loading

Guillaume-d-o Mar 6, 2025

Guillaume-d-o Mar 6, 2025

Guillaume-d-o Mar 6, 2025

AlitaBernachot Mar 6, 2025

Guillaume-d-o Mar 6, 2025

Guillaume-d-o Mar 6, 2025

Guillaume-d-o Mar 6, 2025

Guillaume-d-o Mar 6, 2025

[Datahub] Feature catalog - Load attributes description from a linked record if present #1137

Are you sure you want to change the base?

[Datahub] Feature catalog - Load attributes description from a linked record if present #1137

Conversation

AlitaBernachot commented Feb 26, 2025 • edited Loading

Description

Architectural changes

Screenshots

Quality Assurance Checklist

github-actions bot commented Feb 26, 2025 • edited Loading

github-actions bot commented Feb 27, 2025 • edited Loading

Guillaume-d-o Mar 3, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

coveralls commented Mar 4, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AlitaBernachot Mar 6, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AlitaBernachot commented Feb 26, 2025 •

edited

Loading

github-actions bot commented Feb 26, 2025 •

edited

Loading

github-actions bot commented Feb 27, 2025 •

edited

Loading

Guillaume-d-o Mar 3, 2025 •

edited

Loading

coveralls commented Mar 4, 2025 •

edited

Loading

AlitaBernachot Mar 6, 2025 •

edited

Loading