[New Feature]: Stage-in input to support "list of files"

[mike-gangl]

Some feedback we're getting on STAC is that it's very complicated to put it together for what's essentially a list of input files. A request was made for inputs to stage-in to be a simple list of files to process:

s3://my-bucket/my-prefix/file.txt
https://github.com/raw/file.nc
https://archive.podaac.earthdatacloud.nasa.gov/my-collection/my-awsome-datafile.nc

this would be "much" easier on a user so they don't need to wrap everything in STAC.

The stage-in process would then download these files and create a stac catalog or featureCollection as an output with these assets. In this instance, the collection ID wouldn't matter, and all the files listed would be of type 'data'.

Questions/comments:

• Can we support this sort of 'return delimited' data? should it be comma-delimited? both?
• While i've show the above to include all types of files (s3, https, EDL https) that more for illustrative purposes. Support for multiple domains/types is a different effort.

[rtapella]

Do we expect inputs to all be positional? Based on a keyword? Or look at the input parameter and figure out what it's for?

In other words:

• do we expect someone's code to inspect the value of each input to figure out how to use it?
• does the order of the inputs matter?
• should we have key:value sets that get passed in like text_file: s3://mybucket/blah.txt

I thought I wrote this somewhere but I guess not...

[ngachung]

We can extend unity_sds_client to take in a list of files and output a FeatureCollection that can be passed to stage-in