Min/Max value check does not handle ASCII_Numeric_Base16 values

[jstone-psi]

Checked for duplicates

Yes - I've already checked

🐛 Describe the bug

When I validated previously archived data from the Hayabusa2 LIDAR bundle, I noticed that the validator was throwing errors related to casting hexadecimal values to integers.

Examples:

 Begin Content Validation: file:/Users/jessestone/Desktop/neese_lidar_hex_20250211/hyb2_ldr_l0_aocsm_range_ts_20151219_v01.csv
      ERROR  [error.table.field_value_not_a_number]   data object Hayabusa2 LIDAR Raw Time Series Range Data or index 1, record 1, field 2: Cannot cast field value '3EE9746F' to a Number data type to validate against the min/max values defined in the label.
      ERROR  [error.field.invalid_special_constant]   data object Hayabusa2 LIDAR Raw Time Series Range Data or index 1, record 1, field 4: One of the special constants could not be converted to the numeric data type of the table cell
      ERROR  [error.table.field_value_not_a_number]   data object Hayabusa2 LIDAR Raw Time Series Range Data or index 1, record 2, field 2: Cannot cast field value '3EE974AF' to a Number data type to validate against the min/max values defined in the label.
      ERROR  [error.table.field_value_not_a_number]   data object Hayabusa2 LIDAR Raw Time Series Range Data or index 1, record 2, field 4: Cannot cast field value '74A0' to a Number data type to validate against the min/max values defined in the label.

The LIDAR data contains data values that are in ASCII_Numeric_Base16 format, and special constants (min/max ) defined in the same format.

🕵️ Expected behavior

I expected min/max validation to convert the hexadecimal values in the data file and label to integers as described in https://pds.nasa.gov/datastandards/documents/im/v1/index_1N00.html#class_pds_ascii_numeric_base16.

This issue seems to only occur when performing a min/max check. Removing the special constants from the labels causes the files to pass. So, the values themselves are not invalid.

Validate 3.1.1 passes these data files, but it may not be performing a min/max check on them at all.

📜 To Reproduce

1. Run validate on the supplied label (hyb2_ldr_l0_aocsm_range_ts_20151219_v01.xml)
2. Observe the errors
3. Run validate on the modfied label that has the special constants section removed (hyb2_ldr_l0_aocsm_range_ts_20151219_v01.no_special.xml)
4. Observe the lack of errors.

🖥 Environment Info

• Version of this software 3.6.3
• Operating System: MacOS 14.6.1

📚 Version of Software Used

Validate 3.6.3

🩺 Test Data / Additional context

Archive.zip

🦄 Related requirements

#611

There is discussion of parsing hex strings in this issue, but it is focused on comparing binary hex strings to ascii hex strings.

Other notes:

This is retroactively causing previously archived data to fail, and is causing issues to incorporating the next delivery of data.

Additionally, we anticipate that other missions (such as MMX) will use the same data type later this year.

Acceptance Criteria

Given
When I perform
Then I expect

⚙️ Engineering Details

No response

🎉 Integration & Test

No response

[al-niessner]

@jstone-psi @jordanpadams

As the discussion in #611 states, there was some "how to do it" on defining hex, dec, or oct. There was a traditional way 16#a# that we discussed to move to 0xa like the rest of the world. In our usual way we decided to not decide and support all of them.

Since special constants are not required to be of the same type as the value they compare against, one must prefix hex with 0x or 0X or 16#...#. The not being the same type as the array came up because people want to use bit patterns for floats - like nan or inf or other special patterns - and representing them as hex means they are not float - aka not same type.

If they need to match, then #611 has to be unimplemented. Choose wisely @jordanpadams .

[al-niessner]

Yes, I just looked up the special constants here:
https://pds.nasa.gov/datastandards/documents/im/v1/index_1N00.html#attribute_pds_special_constants_pds_valid_maximum

and it stays it just has to be an ASCII_Short_String_Collapsed or something like that. It is not assumed to be the same as the cell.