missing pytrec_eval measures

[cmacdonald]

This list is not exhaustive, but just the other measures more likely to be used.

• iprec_at_recall
• num_ret
• num_rel_ret
• num_q
• num_rel
• success
• set_P
• infAP

[cmacdonald]

The following nicknames might be handy:

• 'official'
• 'set'
• 'all_trec'
• 'iprec_at_recall'

[seanmacavaney]

For a pytrec_eval (nick)name -> irm mapping function, would something like this meet your needs?

ir_measures.convert_trec_name(name: str) -> List[BaseMeasure]

# Examples
ir_measures.convert_trec_name('map') -> [AP]
ir_measures.convert_trec_name('P_5') -> [P@5]
ir_measures.convert_trec_name('P_5,10') -> [P@5, P@10]
ir_measures.convert_trec_name('P.5') -> [P@5]
ir_measures.convert_trec_name('P.5,10') -> [P@5, P@10]
ir_measures.convert_trec_name('P') -> [P@5, P@10, P@15, P@20, P@30, P@100, P@200, P@500, P@1000] # expando
ir_measures.convert_trec_name('ndcg') -> [nDCG]
ir_measures.convert_trec_name('ndcg_cut') -> [nDCG@5, nDCG@10, ...] # expando
ir_measures.convert_trec_name('official') -> [NumRet, ...] # all those from the family

[cmacdonald]

Also:

ir_measures.convert_trec_name('ndcg_cut_5') -> [nDCG@5]

Your implementation might come from:
https://github.com/terrier-org/pyterrier/blob/ir_measures/pyterrier/pipelines.py#L64

[seanmacavaney]

Remaining measures (identified via supported_measures in pytrec_eval):

• Rndcg
• num_nonrel_judged_ret
• 11pt_avg
• relative_P
• gm_map
• binG
• ndcg_rel
• Rprec_mult
• gm_bpref
• G
• utility
• set_recall
• set_F
• set_relative_P
• set_map
• runid
• relstring