Show readme, changelog and authors in run form for dockstore imported IWC workflows

[ahmedhamidawan]

show_dockstore_readme_in_run_form.mp4

simplified_wf_run_READMEs.mp4

How to test the changes?

(Select all options that apply)

• I've included appropriate automated tests.
• This is a refactoring of components with existing test coverage.
• Instructions for manual testing are as follows:
  1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

• I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

[mvdbeek]

Fetching the changelog would also be pretty nice

[ahmedhamidawan]

It is fine to throw this, right? Instead of just returning from the function.
One way we could get to this point is if a workflow has source_metadata but the trs_id is not from iwc for example.
I made sure we rethrow the error (simply does a console.debug), and then in WorkflowAnnotation I also catch the error, but do nothing.

[mvdbeek]

It's ok, but you might be hiding real errors. I would use a regular return with empty values in the expected case of it not being an iwc workflow, and then handle the case that something went wrong with the request and show that to the user ? Or log at an error level at least. Always good to discriminate expected things that won't work from unexpected failures that might require a fix.

[bernt-matthias]

Help on top. Nice :)

[lldelisle]

Wouah! Thank you so much

[jmchilton]

The feature is great and needed and I love it. The implementation is not a good idea IMO - the readme shouldn't be assumed to be the "Help" for the workflow - the workflow should have help/instructions metadata that can optionally just reference a workflow description if the workflow author doesn't want to write separate help instructions. Also the correct place to store this information should be in the workflow not in our Github pipeline. Requiring features of our distribution pipeline to properly render an artifact was such a self own with the toolshed that we are still not recovered from and we should not replicate it here.

Add the metadata you want to the workflow, update the IWC tooling to extract and reuse the metadata how it wants, use it properly in Galaxy based only on the target artifact. The project is 20 years old and this is the most important feature under active develop - we don't need to take shortcuts and we really shouldn't. If we're happy with the UI here - I'd be willing to write the backend pieces to add these artifacts to the workflow itself and to assist with updating the IWC tooling to avoid going down this road.

[ahmedhamidawan]

I made sure that we restricted this to the case where the workflow is most definitely from the IWC (as far as I understand those are the most well maintained ones), and again, as far as I understand, the readme originally comes from that repository, no?

If we have that, why not add an additional (expandable/not in your way) help section which shows it? It's just additional helpful information for the user.

Of course renaming it from Help to README is no big deal.

When you say

Add the metadata you want to the workflow, update the IWC tooling to extract and reuse the metadata how it wants, use it properly in Galaxy based only on the target artifact.

You mean we add the metadata to the workflow object which then references the readme ; but the readme is still stored where?
I do not understand what that means...

I would also love to write the backend pieces which would help do this properly, as opposed to taking this shortcut, and systematically add this information to the workflows in the DB (for the workflows that have TRS ids)

[jmchilton]

I made sure that we restricted this to the case where the workflow is most definitely from the IWC

This is my concern, I get that this was the feature request and you did an excellent job I just don't think it was the right request.

If we have that, why not add an additional (expandable/not in your way) help section which shows it? It's just additional helpful information for the user.

I am happy with the readme being available - it just should be part of the workflow and not dependent on the IWC tooling or publication.

You mean we add the metadata to the workflow object which then references the readme ; but the readme is still stored where? I do not understand what that means...

I've added this #19559 to explain it in detail and recommend an implementation - let me know if it is too vague.

add this information to the workflows in the DB (for the workflows that have TRS ids)

This is I think the mistake we made with tool shed - is is backwards to collect this data from the TRS ID. We implemented features for things that had been published when those features should have been available to tools before publication. You cannot have an image in your tool unless you've published the tool to the tool shed. How do you develop the tool help then? How do you know the publication is going to be correct? What if the tool shed is unavailable for installing the tool in the future?

Having the TRS ID is a link and maybe you can use it for information about how you got the workflow - but the workflow's intrinsic metadata should be editable and available without the TRS ID. Analogous to the tool issues - if I'm developing a workflow and the readme or instructions are going to be included and presented to the user - I think I should be able to see how that is rendered without the publication step so I know I'm publishing the right thing. I also think workflow developers who want to publish their own collections of workflows should have these features available.

It is a lot more work for sure - but I think it is the right thing to do.

[mvdbeek]

for this PR then I think we could go ahead with showing the resolved ORCIDs and the changelog if we have one ?

[ahmedhamidawan]

for this PR then I think we could go ahead with showing the resolved ORCIDs and the changelog if we have one ?

But the changelog is being retrieved the same way the readmes are? Are we ok with keeping that as is then?

(And if yes, maybe we can also show the readmes for now until we implement builtin readmes (or whatever we wanna call them) in Galaxy?)

[mvdbeek]

Are we ok with keeping that as is then?

As @jmchilton said:

The CHANGELOG should absolutely be included but we have other issues (xref #9166, #6410) for this and the problem is hard enough I'm happy to have external links to this data for now (i.e. #19536) as long as everyone is on board that we should have a structured way to store and reference this data from the workflow itself longer term.

[ahmedhamidawan]

#19591 should give us what we were looking for...