Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[24.2] Add bwa_mem2_index directory datatype, framework enhancements for testing directories #19694

Open
wants to merge 11 commits into
base: release_24.2
Choose a base branch
from

Conversation

mvdbeek
Copy link
Member

@mvdbeek mvdbeek commented Feb 25, 2025

This is essentially a one-liner with a lot of testing work 😅.

The basic need is for a directory datatype, subclasses like bwa_mem2_index can represent most if not all reference data.
Here I've switched our primary tool testing function over to target __FETCH_DATA__ (workflow tests are already using this), and I fixed up cwl-style directory uploads which (at least currently) are not working.
It is now possible to upload directories in xml tool tests and workflow tests (via planemo and our internal framework).
Also adds format into the expression input object, so we can use format to decide if a workflow step needs to run.

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

  • I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

@mvdbeek mvdbeek changed the title Add bwa_mem2_index directory datatype, framework enhancements for testing directories [24.2] Add bwa_mem2_index directory datatype, framework enhancements for testing directories Feb 25, 2025
@github-actions github-actions bot added this to the 25.0 milestone Feb 25, 2025
@mvdbeek mvdbeek force-pushed the bwa_index_directory_datatype branch from 1374fff to 84e3350 Compare February 25, 2025 13:45
@@ -304,6 +304,7 @@
<converter file="archive_to_directory.xml" target_datatype="directory"/>
</datatype>
<datatype extension="directory" type="galaxy.datatypes.data:Directory"/>
<datatype extension="bwa_mem2_index" display_in_upload="true" type="galaxy.datatypes.data:Directory"/>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if we should subclass and have some basic sanity checks on the contents. At the moment users can upload anything, or?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

subclass yes, checks no. you can already upload anything, I don't want to touch that.

@mvdbeek mvdbeek force-pushed the bwa_index_directory_datatype branch 2 times, most recently from bc1dcaa to a43e655 Compare February 25, 2025 14:43
@jmchilton
Copy link
Member

Why 24.2 😅 - I mean I think switching the upload API for the tool tests is a long standing issue that I'm so excited to see resolved but it is a big thing that should probably be against dev right?

@mvdbeek mvdbeek force-pushed the bwa_index_directory_datatype branch from 5a0e23b to 23a623d Compare February 25, 2025 17:10
</outputs>
<tests>
<test>
<param name="reference" value="bwa_mem2_index" class="Directory"></param>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ftype in addition?

@@ -0,0 +1,17 @@
<tool id="use_directory_index" name="Uses directory index" version="1.0" profile="24.2">
<command><![CDATA[
cp '$reference.extra_files_path'/1.fasta '$output'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
cp '$reference.extra_files_path'/1.fasta '$output'
cp '$reference.extra_files_path/1.fasta' '$output'

@@ -60,6 +60,7 @@
</datatype>
<datatype extension="directory" type="galaxy.datatypes.data:Directory">
</datatype>
<datatype extension="bwa_mem2_index" display_in_upload="true" type="galaxy.datatypes.data:Directory"/>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

subclass=true ?

<tests>
<test>
<param name="reference" value="1.fasta"></param>
<output name="index">
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<output name="index">
<output name="index" ftype="bwa_mem2_index">

<tool id="create_directory_index" name="Create directory index" version="1.0" profile="24.2">
<command><![CDATA[
mkdir '$index.extra_files_path' &&
cp '$reference' '$index.extra_files_path'/1.fasta
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
cp '$reference' '$index.extra_files_path'/1.fasta
cp '$reference' '$index.extra_files_path/1.fasta'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants