Plan / implement solution for handling multiple versions of products in search results

[jordanpadams]

💡 Description

Current implementation attempts to use Solr collapse functionality.

https://github.com/NASA-PDS/registry-mgr-legacy/blob/main/src/main/resources/collections/data/solrconfig.xml#L789
https://github.com/NASA-PDS/registry-mgr-legacy/blob/main/src/main/resources/collections/data/solrconfig.xml#L848

A few issues with this method:

1. It attempts to collapse on version_id as a float, which won't work since PDS4 uses semantic versioning (e.g. in PDS4 1.100 > 1.2). We need to figure out some sort of normalized numeric version we can use for this.

2. For collapse to work it requires all versions of that product to be on the same shard, which is unlikely. We would need to change the installation to 1 shard for this to work.

3. The search faceting overwrites the fq value. So as soon as someone facets on something, we no longer have the latest versions only.

Alternative options to collapse:

1. Remove previous versions and only load the latest versions.
2. Create separate index to support latest-only, and write tool to query data core to get only the latest products.
3. Create separate field for maintaining latest version, add that to search and archive-filter request handlers. (see provenance in registry-sweepers for possible implementation)
4. Switch to use 1 shard, create new some version_id_normalized field for handling numeric comparison to support Solr collapse, update add-hierarchy.xsl to calculate and add version_id_normalized to metadata onload.

Sub-tasks

• As a user, I want to only search amongst the latest versions of products #83

[nutjob4life]

Or maybe look to registry-sweepers 🤔