Labcas Backend Incorporation of Aspera

[yuliujpl]

Hi Sean,

Here is the the how I use javascript to download from aspera!

#This function seems to init the client for the download
function initAsperaConnect() {
this.client = new AW4.Connect();
this.client.initSession();
}

async function download() {

#This part creates the transfer_spec that is used to get the token
const transfer_spec = {
"transfer_requests": [
{
"transfer_request": {
"paths": [
{
"source": "/workspaces/107571/home/[email protected] (1197343)/nist/test"
}
],
"remote_host": "ats-aws-us-west-2.aspera.io",
"remote_user": "xfer",
"authentication": "token",
"direction": "receive",
"ssh_port" : 33001
}
}
]
}

try {
#This step uses a fetch to get the token using the transfer_spec from above! Please make sure to base64 encode accesskey:secretkey and put it below next to Authorization!

    const response = await fetch('https://ats-aws-us-west-2.aspera.io/files/download_setup', {
      method: 'POST',
      headers: {
        Authorization: 'Basic {Base64 of accesskey:secretkey}',
        'Content-Type': 'application/json',
        Accept: 'application/json'
      },
      body: JSON.stringify(transfer_spec)
    });

    if (!response.ok) {
      throw new Error(`HTTP error! Status: ${response.status}`);
    }

    const data = await response.json();

#The data returned from the token request returns a list of transfer_specs, you need to grab each individual to submit for download, here I'm just grabbing the first one!
spec = data.transfer_specs[0].transfer_spec;

#For some reason, the returned transfer_spec is missing "authentication" = "token" key/value pair, must add it or it will not download properly! Remember, not the actual token, that's already in the transfer_spec, just simply the key/value pair saying you need to make this token based auth!
spec['authentication'] = "token";
#This triggers the client to start the transfer!
this.client.startTransfer(spec);
} catch (error) {
console.error('Error:', error);
}
}

[nutjob4life]

@yuliujpl fantastic, thanks.

Okay so the backend should provide a new API, such as:

https:/labcas-dev…/data-access-api/transfer-spec?param1=x&param2=y&…

So far so good? Are there any parameters the UI will pass in?

It should return the "transfer spec" which is an application/json payload that looks like this?

{
    "transfer_requests": [
        {
            "transfer_request": {
                "paths": [
                    {
                        "source": "/workspaces/107571/home/[email protected] (1197343)/nist/test"
                    }
                ],
                "remote_host": "ats-aws-us-west-2.aspera.io",
                "remote_user": "xfer",
                "authentication": "token",
                "direction": "receive",
                "ssh_port": 33001
            }
        }
    ]
}

Are there any special response headers needed? Is the token in authentication just the word "token"?

The "source" in the "paths" is going to take some thinking!

Lastly, is there a place where I can try out your demo from November 19? It might be helpful to use the Web Inspector to see what happens (I don't speak fluent JavaScript).

[yuliujpl]

Yes, the UI should be able to pass a list of the filepaths that's recorded in the labcas file metadata and labcas-backend would construct the url to point to the s3 node:

Input into the labcas-backend api:
POST ["nist/test1.txt","nist/test2.txt"...]

Then labcas-backend internally would construct the following transfer_request:
"transfer_requests": [
{
"transfer_request": {
"paths": [
{
"source": "/workspaces/107571/home/[email protected] (1197343)/nist/test1.txt"
},
{
"source": "/workspaces/107571/home/[email protected] (1197343)/nist/test2.txt"
},
...
],

It should then post the transfer request to aspera's api and then return the "aspera returned transfer request". The "aspera returned transfer request" should look something like the attached screenshot:

Please note the token that's returned from aspera, I need that as part of the transfer_spec to initiate an authorized download from apsera!

Sean: "Are there any special response headers needed? Is the token in authentication just the word "token"?"
In the screenshot above, you can see that there's an added token: "..." that aspera gives me that labcas-backend needs to pass back into the UI to enable authorized download. "authentication":"token" ALSO needs to be added in there as well, for some reason that gets stripped away after authorizing via aspera api, so please make sure to re-inject "authentication":"token" back into the dictionary, and yes, I do mean simply the string "token".

Sean: "The "source" in the "paths" is going to take some thinking!"
Agreed! I think as long as the relative path stays the same as the ones in labcas file metadata, we can append whatever aspera node prefix to those labcas paths?

Sean: "Lastly, is there a place where I can try out your demo from November 19? It might be helpful to use the Web Inspector to see what happens (I don't speak fluent JavaScript)."
Yes! http://52.1.109.209/aspera/index.html