datahubs per sub-portal #8644

Guillaume-d-o · 2025-02-06T14:56:04Z

This is the continuation of the proof-of-concept for the proposal #8021

This work aim to create and manage a datahub in a plugin. The Datahub application, once compiled, is a collection of static files (HTML, JS, CSS) that can be served as is. If you want more information, I strongly encourage you to read the proposal #8021 that explain all the developement in details.
We tried to maximise all the code on the plugin side.

Checklist

CLAassistant · 2025-02-06T14:56:11Z

All committers have signed the CLA.

fgravin · 2025-02-18T14:13:43Z

Awesome !! 🤩
Thanks @Guillaume-d-o

fgravin · 2025-02-21T08:17:07Z

@Guillaume-d-o could you check the CI please, seems there is an issue during the build.

jahow

Thank you! Haven't tested yet but I've seen it work. We should address my comments first and then I'll do another final review.

web/src/main/webapp/WEB-INF/classes/setup/sql/migrate/v447/migrate-default.sql

web/pom.xml

jahow · 2025-02-21T09:29:03Z

docs/manual/docs/administrator-guide/configuring-the-catalog/system-configuration.md

+## Geonetwork-ui configuration
+
+*Enable datahub*:
+You can adapt your GeoNetwork UI by enabling Datahub. First, download the specific plugin datahub-integration. Then, on the settings page, enable Datahub . you can provide an other specific Datahub configuration than the default one. You can also configure it for each portal. Ensure that Datahub is enabled in the settings to make it work on portals.


Thanks for documenting your work. IMO this should go in a separate section of the documentation. The things that need to be documented are:

what the plugin does (with a link to GeoNetwork-UI project)

how to build the plugin

how to enable it

how to configure the datahub instances (link to the relevant documentation in GN-UI)

We might start a new "plugins" section in the documentation for that, I think that'd be appropriate. @jodygarnett do you have any opinion on how/where to put this? thanks!

jahow

Thank you! It's looking good, I made several suggestions to the code though, could you please look into them? As for the documentation I'll add a commit to your branch to make it clearer if that's OK.

jahow · 2025-03-05T09:32:54Z

plugins/datahub-integration/src/main/java/org/fao/geonet/datahub/DatahubController.java

+    @Autowired
+    SettingManager settingManager;


jahow · 2025-03-05T09:39:45Z

plugins/datahub-integration/src/main/java/org/fao/geonet/datahub/DatahubController.java

+        handleDatahubRequest(request, response, locale);
+    }
+
+    void handleDatahubRequest(HttpServletRequest request, HttpServletResponse response,String locale) throws IOException {


Many lines have inconsistent code formatting; could you run a code formatting task with your IDE?

jahow · 2025-03-05T09:40:09Z

plugins/datahub-integration/src/main/java/org/fao/geonet/datahub/DatahubController.java

+    }
+
+    void handleDatahubRequest(HttpServletRequest request, HttpServletResponse response,String locale) throws IOException {
+        Log.debug(LOGGER_NAME, "enter in datahub");


I don't think this is necessary

jahow · 2025-03-05T09:43:03Z

plugins/datahub-integration/src/main/java/org/fao/geonet/datahub/DatahubController.java

+        int pathPartToSkip = 3;// "/srv/datahub/bla/bla"
+        if (locale != null) {
+            pathPartToSkip = 4;// /srv/fre/datahub/bla/bla
+        }


This is a little bit hacky; ideally the isolation of the path in the request should have been done earlier; to keep in mind if we have to rework this

jahow · 2025-03-05T09:44:02Z

plugins/datahub-integration/src/main/java/org/fao/geonet/datahub/DatahubController.java

+
+        String filePath = Stream.of(reqPath.split("/")).skip(pathPartToSkip).collect(Collectors.joining("/"));
+        if (!FileUtils.fileExistsInJar("/datahub/" + filePath)) {
+            return new File(filePath);


The intention here was to cause an explicit exception of type "File ... not found" that can bubble up to the log I think? if yes, we should leave a comment because otherwise this line makes no sense

I need it during the test phase but i can throw an exception directly

jahow · 2025-03-05T09:44:23Z

plugins/datahub-integration/src/main/java/org/fao/geonet/datahub/DatahubController.java

+            return FileUtils.getFileFromJar("/datahub/" + filePath);
+        } catch (IOException e) {
+            Log.error(LOGGER_NAME, e.getMessage());
+            return new File(filePath);


same, see above, leave a comment to explain

jahow · 2025-03-05T09:45:47Z

plugins/datahub-integration/src/main/java/org/fao/geonet/datahub/DatahubController.java

+        }
+        String configuration = "";
+        if (portal != null && !portal.getDatahubConfiguration().isEmpty()) {
+            configuration = portal.getDatahubConfiguration();


Here we also need to implement a fallback; if the configuration of the subportal is empty, we should use the one from the main portal.

jahow · 2025-03-05T09:47:40Z

web-ui/src/main/resources/catalog/templates/admin/settings/sources.html

+              id="datahubEnabled"
+              data-ng-model="source.datahubEnabled"
+            />
+            <p class="help-block" data-translate="">sourceDatahubEnabled-help</p>


All the translation keys need to be added to the translation file here: https://github.com/geonetwork/core-geonetwork/blob/main/web-ui/src/main/resources/catalog/locales/en-admin.json

jahow · 2025-03-05T09:48:37Z

web/src/main/webapp/WEB-INF/classes/setup/sql/migrate/v447/migrate-default.sql

@@ -3,3 +3,6 @@ UPDATE Settings SET value='SNAPSHOT' WHERE name='system/platform/subVersion';

 INSERT INTO Settings (name, value, datatype, position, internal) VALUES ('system/banner/enable', 'false', 2, 1920, 'n');
 INSERT INTO Settings (name, value, datatype, position, internal) VALUES ('system/auditable/enable', 'false', 2, 12010, 'n');
+
+ALTER TABLE Sources ADD COLUMN datahubEnabled BOOLEAN DEFAULT FALSE;
+ALTER TABLE Sources ADD COLUMN datahubConfiguration TEXT DEFAULT '';


A default configuration should be provided for this column since with an empty configuration the datahub app will crash.

done below :)

plugins/datahub-integration/src/main/java/org/fao/geonet/datahub/DatahubController.java

+        }
+
+        File actualFile = getRequestedFile(request);
+        if (!actualFile.exists()) {


plugins/datahub-integration/src/main/java/org/fao/geonet/datahub/DatahubController.java

+        response.setStatus(HttpServletResponse.SC_OK);
+        String extension = actualFile.getName().toLowerCase();
+        String contentType = extension.equals("js") ? "text/javascript; charset=UTF-8"
+                : Files.probeContentType(actualFile.toPath());


plugins/datahub-integration/src/main/java/org/fao/geonet/datahub/DatahubController.java

+    void writeResponseContent(HttpServletRequest request, HttpServletResponse response, File actualFile,
+            String portalName) throws IOException {
+        InputStream inStream = actualFile.getName().equals("default.toml") ? readConfiguration(portalName)
+                : new FileInputStream(actualFile);


Guillaume-d-o force-pushed the GSIRD-3-temp branch 3 times, most recently from 2fbf024 to 30d9db8 Compare February 17, 2025 15:50

Guillaume-d-o changed the title ~~Gsird 3 temp~~ datahubs per sub-portal Feb 17, 2025

Guillaume-d-o marked this pull request as ready for review February 18, 2025 13:43

Guillaume-d-o marked this pull request as draft February 18, 2025 13:46

fgravin requested a review from josegar74 February 18, 2025 14:12

Guillaume-d-o force-pushed the GSIRD-3-temp branch from 4b09816 to a351411 Compare February 18, 2025 14:17

Guillaume-d-o marked this pull request as ready for review February 18, 2025 14:20

feat(geonetwork/web): Add datahub integration.

dd54831

Guillaume-d-o force-pushed the GSIRD-3-temp branch from a351411 to dd54831 Compare February 18, 2025 14:22

fgravin requested a review from fxprunayre February 20, 2025 10:50

jahow reviewed Feb 21, 2025

View reviewed changes

Guillaume-d-o added 2 commits February 24, 2025 10:20

module and settings params

33ab08b

documentation changes

903669b

jahow requested changes Mar 5, 2025

View reviewed changes

jahow added the backport 4.2.x label Mar 5, 2025

code review feedback

4ec09de

github-advanced-security bot found potential problems Mar 6, 2025

View reviewed changes

Guillaume-d-o added 2 commits March 6, 2025 15:58

add plugin to nav tree

34c9358

remove todo

448c17d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

datahubs per sub-portal #8644

datahubs per sub-portal #8644

Guillaume-d-o commented Feb 6, 2025 •

edited

Loading

CLAassistant commented Feb 6, 2025 •

edited

Loading

fgravin commented Feb 18, 2025

fgravin commented Feb 21, 2025

jahow left a comment

jahow Feb 21, 2025

jahow left a comment

jahow Mar 5, 2025

jahow Mar 5, 2025

jahow Mar 5, 2025

jahow Mar 5, 2025

jahow Mar 5, 2025

Guillaume-d-o Mar 5, 2025

jahow Mar 5, 2025

jahow Mar 5, 2025

jahow Mar 5, 2025

jahow Mar 5, 2025

Guillaume-d-o Mar 6, 2025

datahubs per sub-portal #8644

Are you sure you want to change the base?

datahubs per sub-portal #8644

Conversation

Guillaume-d-o commented Feb 6, 2025 • edited Loading

Checklist

CLAassistant commented Feb 6, 2025 • edited Loading

fgravin commented Feb 18, 2025

fgravin commented Feb 21, 2025

jahow left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jahow left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Guillaume-d-o commented Feb 6, 2025 •

edited

Loading

CLAassistant commented Feb 6, 2025 •

edited

Loading