Multiple reportportal plugin improvements and fixes

[seberm]

This PR should fix the problem with a possible reportportal error when using result: restraint or result: custom:

Received non-ok status code 406 from ReportPortal: {"errorCode":40025,"message":"Start time of child ['Tue Nov 12 11:10:19 UTC 2024'] item should be same or later than start time ['2024-11-12T11:36:35.007'] of the parent item/launch '
11810'"}

The PR fixes multiple bugs, for more info see:

• Not possible to use reportportal report plugin with custom results #3360
• Multiple reportportal plugin improvements and fixes #3356 (comment)

This error is probably caused by the badly sorted result start times. It can happen the datetime formats in the list are not in the same format. The Result.start_time is in ISO format but the self.time() returns str(int(time() * 1000)).

The basic reproducer

$ cat main.fmf
test: |
    tmt-report-result /test/good PASS
    tmt-report-result /test/fail FAIL
    tmt-report-result /test/weird WARN

result: restraint

duration: 5m

$ python -m tmt -r ~/tmt-learning/20 run -a report --how reportportal -v --force

Other improvements

There are also small code improvements like the possibility to ignore the SSL verification, adding the usage of Python properties instead of class methods, or adding handling of ResultOutcome.SKIP outcome which I believe is missing from the mapping.

TODOs:

• Link the issue with the problem description as soon as it's created.

Pull Request Checklist

• implement the feature
• update reportportal schema for ssl-verify option
• extend the test coverage - Tests will be extended and fixed (only for local testing) as a part of Add subresults support for reportportal report plugin #3331
• include a release note

[seberm]

The problem with start-time/end-time timestamps should be also resolved by existing PR by @kkaarreell :

• Fix timestamp processing #3345

[seberm]

Hello @mildas, could you please test if the changes in this PR resolve your issue (#3360) and if the data are imported to ReportPortal as expected? Thanks!

[kkaarreell]

I have just tested a plain import of a test plan with passing and failing tests and it worked well.
I also tried to import test results that originally revealed the timestamp issue and it went well too.
I do not plan any particular testing ATM, from my POV it works well.

[seberm]

@kkaarreell

I have just tested a plain import of a test plan with passing and failing tests and it worked well.
I also tried to import test results that originally revealed the timestamp issue and it went well too.
I do not plan any particular testing ATM, from my POV it works well.

Thanks for helping me with the testing! Could you please add a check to this PR that you did a review?

Also, do you know anyone else who uses the ReportPortal and could us help with the testing of these changes? What about @KwisatzHaderach?

[KwisatzHaderach]

Also, do you know anyone else who uses the ReportPortal and could us help with the testing of these changes? What about @KwisatzHaderach?

not actually using the rp plugin, we do all our exports through data router using polarion xml

[The-Mule]

@seberm Hi Otto, we found one more issues in the RP plugin, in method def _filter_log_per_size there is the following line:

return f"{header}{data[:settings.size.to('bytes').magnitude]}"

But settings.size.to('bytes').magnitude is float actually (e.g. 1000000.0) and it causes:

The exception was caused by 1 earlier exceptions

Cause number 1:

    slice indices must be integers or None or have an __index__ method

        Traceback (most recent call last):

        File /usr/lib/python3.12/site-packages/tmt/base.py, line 3749, in go
          plan.go()
        File /usr/lib/python3.12/site-packages/tmt/base.py, line 2439, in go
          step.go()
        File /usr/lib/python3.12/site-packages/tmt/steps/report/__init__.py, line 121, in go
          cast(Union[Action, ReportPlugin[ReportStepData]], phase).go()
        File /usr/lib/python3.12/site-packages/tmt/steps/report/reportportal.py, line 670, in go
          message = _filter_log(log,
        File /usr/lib/python3.12/site-packages/tmt/steps/report/reportportal.py, line 94, in _filter_log
          log = log_filter(log, settings=settings)
        File /usr/lib/python3.12/site-packages/tmt/steps/report/reportportal.py, line 81, in _filter_log_per_size
          return f"{header}{data[:settings.size.to('bytes').magnitude]}"

The fix is easy - int that, ie. replace that with:

return f"{header}{data[:int(settings.size.to('bytes').magnitude)]}"

(consulted with Karel and Milos).

Since this is rather generic PR for RP plugin fixes, would it be possible to squeeze this in?

[psss]

Nice! Thanks much for improving this.

[psss]

Red jobs are irrelevant failures. I've reported the epel-next-release download problem here:

• https://pagure.io/fedora-infrastructure/issue/12326

The remaining failure seems to be a network glitch. Merging.