source product version and slicing checks

[johntruckenbrodt]

Currently the processor does not do anything to filter duplicate source products. It often happens that source products (GRD/SLC) are reprocessed and, if the data archive is not optimally curated, multiple versions of the same acquisition may exist. In some cases the reprocessed products are also resliced so that both versions may not easily be compared by the acquisition times.

Consider the following case. A data take of products was first acquired in NRT slicing mode (manifest.safe metadata attribute sliceNumber=0), reprocessed in NRT slicing mode using a different slicing, and later reprocessed again, this time in regular slicing mode (sliceNumber>0) and using the same slicing as first reprocessed version. Hence, three versions of the same data take may exist. At the time of writing, ASF only offers the older NRT version and CDSE offers all three versions. In case CDSE is used as reference archive, s1ard has no way of determining which are the right products to use.

product ID	sliceNumber	best	ASF	CDSE
S1A_IW_GRDH_1SDV_20231207T064644_20231207T064709_051545_0638F1_9569	0		x	x
S1A_IW_GRDH_1SDV_20231207T064649_20231207T064714_051545_0638F1_4B74	6	x		x
S1A_IW_GRDH_1SDV_20231207T064649_20231207T064714_051545_0638F1_95E7	0			x
S1A_IW_GRDH_1SDV_20231207T064709_20231207T064734_051545_0638F1_440B	0		x	x
S1A_IW_GRDH_1SDV_20231207T064714_20231207T064739_051545_0638F1_4799	7	x		x
S1A_IW_GRDH_1SDV_20231207T064714_20231207T064739_051545_0638F1_AC9A	0			x
S1A_IW_GRDH_1SDV_20231207T064734_20231207T064759_051545_0638F1_9500	0		x	x
S1A_IW_GRDH_1SDV_20231207T064739_20231207T064804_051545_0638F1_C7A2	8	x		x
S1A_IW_GRDH_1SDV_20231207T064739_20231207T064804_051545_0638F1_FECD	0			x
S1A_IW_GRDH_1SDV_20231207T064759_20231207T064824_051545_0638F1_8191	0		x	x

[johntruckenbrodt]

The situation is even more complicated. The reprocessed NRT product is the one that was processed last:

grd	slice_num	grd_ptime
S1A_IW_GRDH_1SDV_20231207T064739_20231207T064804_051545_0638F1_FECD	0	2023-12-07 08:52:07.933950
S1A_IW_GRDH_1SDV_20231207T064649_20231207T064714_051545_0638F1_95E7	0	2023-12-07 08:52:14.515021
S1A_IW_GRDH_1SDV_20231207T064714_20231207T064739_051545_0638F1_AC9A	0	2023-12-07 08:52:28.772885
S1A_IW_GRDH_1SDV_20231207T064649_20231207T064714_051545_0638F1_4B74	6	2023-12-07 15:51:00.176554
S1A_IW_GRDH_1SDV_20231207T064739_20231207T064804_051545_0638F1_C7A2	8	2023-12-07 15:51:40.226163
S1A_IW_GRDH_1SDV_20231207T064714_20231207T064739_051545_0638F1_4799	7	2023-12-07 15:51:51.035157
S1A_IW_GRDH_1SDV_20231207T064644_20231207T064709_051545_0638F1_9569	0	2023-12-07 17:11:37.576017
S1A_IW_GRDH_1SDV_20231207T064734_20231207T064759_051545_0638F1_9500	0	2023-12-07 17:11:39.886424
S1A_IW_GRDH_1SDV_20231207T064709_20231207T064734_051545_0638F1_440B	0	2023-12-07 17:11:45.360334
S1A_IW_GRDH_1SDV_20231207T064759_20231207T064824_051545_0638F1_8191	0	2023-12-07 17:11:49.801547

So, ASF just keeps the products that were processed last. In this case this might not suffice. I have opened an issue at ASF about this.

[johntruckenbrodt]

@avalentino may I ask for your opinion?

[johntruckenbrodt]

#236 is only a temporary fix for GRD neighbor selection. the scene search functionality still does not differentiate between different versions.