science sequence not doing the post processing (apero for Spirou)

[clairem789]

I am trying to run a set of new nights with Spirou data and using the science_seq.
It doesn't do the postprocessing while it seems to me it is included in the sequence (http://apero.exoplanets.ca/auto/recipe_sequences/spirou/sequence_science_seq.html)
I ran it and didn't get the outputs, so I re-ran in test mode and didn't get the commands, it stops at POL recipes.
Here is the ini file (renamed .txt to be included in GitHub)

new_run.txt

and the end of the apero_processing.py --test=True new_run.ini

11:36:12.597-|PROC| ***************************************************************************
11:36:12.604-|PROC| * GROUP 95/97 (FTTEMP2)
11:36:12.611-|PROC| ***************************************************************************
11:36:12.619-|PROC|
11:36:15.205-|PROC| TID01316|C01/05| apero_mk_template_spirou.py V1298TAU --crunfile=new_run.ini --program=FTTEMP2[01316] --recipe_kind=tellu-science --shortname=FTTEMP2 --parallel=True
11:36:17.102-|PROC| TID01317|C02/05| apero_mk_template_spirou.py V_AP_COL --crunfile=new_run.ini --program=FTTEMP2[01317] --recipe_kind=tellu-science --shortname=FTTEMP2 --parallel=True
11:36:17.404-|PROC|
11:36:17.412-|PROC| ***************************************************************************
11:36:17.419-|PROC| * GROUP 96/97 (CCF)
11:36:17.426-|PROC| ***************************************************************************
11:36:17.433-|PROC|
11:36:20.045-|PROC| TID01376|C01/05| apero_ccf_spirou.py 2024-12-23 3101106o_pp_e2dsff_tcorr_AB.fits --crunfile=new_run.ini --program=CCF[01376] --recipe_kind=rv-tcorr --shortname=CCF --parallel=True
11:36:21.993-|PROC| TID01400|C02/05| apero_ccf_spirou.py 2024-12-23 3101082o_pp_e2dsff_tcorr_AB.fits --crunfile=new_run.ini --program=CCF[01400] --recipe_kind=rv-tcorr --shortname=CCF --parallel=True
11:36:23.896-|PROC| TID01685|C03/05| apero_ccf_spirou.py 2024-12-18 3100348o_pp_e2dsff_tcorr_AB.fits --crunfile=new_run.ini --program=CCF[01685] --recipe_kind=rv-tcorr --shortname=CCF --parallel=True
11:36:24.199-|PROC|
11:36:24.207-|PROC| ***************************************************************************
11:36:24.214-|PROC| * GROUP 97/97 (POLAR)
11:36:24.221-|PROC| ***************************************************************************
11:36:24.228-|PROC|
11:36:26.789-|PROC| TID01780|C01/05| apero_pol_spirou.py 2024-12-23 --exposures 3101088o_pp_e2dsff_tcorr_AB.fits 3101089o_pp_e2dsff_tcorr_AB.fits 3101090o_pp_e2dsff_tcorr_AB.fits 3101091o_pp_e2dsff_tcorr_AB.fits --crunfile=new_run.ini --program=POLAR[01780] --recipe_kind=polar-tcorr --shortname=POLAR --parallel=True
11:36:27.368- |PROC|
11:36:27.375- |PROC| ***************************************************************************
11:36:27.383- |PROC| Timings:
11:36:27.390- |PROC| ***************************************************************************
11:36:27.397- |PROC|
11:36:27.404- |PROC| ***************************************************************************
11:36:27.411-|PROC| Cumulative time taken = 0.000
11:36:27.418-|PROC| Actual time taken = 987.598
11:36:27.425-|PROC| Speed up: 0.000 (Number of cores = 5)
11:36:27.432- |PROC| ***************************************************************************
11:36:27.439- |PROC|
11:36:27.836-|PROC| ***************************************************************************
11:36:27.843-|PROC| Recipe apero_processing has been successfully completed (7225.988 seconds)
11:36:27.850-|PROC| ***************************************************************************

Any clue?

[njcuk9999]

So I'm not sure how the run.ini file you have there was generated - its not for a science_run.ini

If you look at the page you linked to (http://apero.exoplanets.ca/auto/recipe_sequences/spirou/sequence_science_seq.html) there is a table (section 3)

If you used only "science_seq" in the ids at the bottom, e.g. this

#id00000 = pp_seq
#id00001 = calib_seq
# id00002 = tellu_seq
id00003 = science_seq

Then the only RUN_XXX you can use are the short name in that table.

E.g.
EXTOBJ, FTFIT1, FTTEMP1, FTFIT2, FTTEMP2, CCF, POLAR, SCIPOST

Your run.ini file has things like EXTALL and POSTALL which wont work with science_seq (and aren't in any of your other sequences)

You'll note that you do have RUN_POSTALL which isn't in the table, so in fact I expect apero_processing not to run the post process steps.

So I can only assume this run.ini was taken from the complete_run.ini example (for full_seq).

I could write a apero tool to verify these files and say "this, this and this will be run, this and this wont" - maybe that would useful and avoid the confusion these sequences are giving.

The reason there is a POSTALL and a SCIPOST (and a EXTOBJ and a EXTALL) is the distinction between using SCIENCE_TARGETS - the POSTALL and EXTALL ones will try to post process or extract everything possible, where as the "SCIPOST" and "EXTOBJ" will only do science targets (dprtype=OBJ_DARK, OBJ_FP and drsobjname in SCIENCE TARGETS)

[njcuk9999]

In short, remove RUN_EXTALL and RUN_POSTALL and replace with RUN_EXTOBJ (which you have) and RUN_SCIPOST (which you don't have).

I'll write something to verify these run files to avoid problems in the future.