How to fail a task without logging an exception

[davidolrik]

I want to set a task to failed from within the task it self.

I can do this by throwing an Exception, but that will also output the full exception to the log which I would like to avoid.

From reading the code, it looks like there is no way to avoid this as it is now, or am I missing something?

[ewjoachim]

I think there's currently no way to do that. What would you think would be a good API for this ?

[davidolrik]

Raising exceptions seems like the best way to do it, like the JobRetry exception.

A simple JobForcedFail exception would do the trick, but we could also do a more elaborate thing and create a FailureStrategy object similar to RetryStrategy - or simply extend RetryStrategy with a forced_failure_exceptions.

On that same note, if I have a "sequence" of jobs with the same lock value and the first job fails, the rest of the jobs in the sequence will happily continue to run - sometimes I would like the chain to stop running, maybe even cancel the rest of the jobs in the "sequence".

So, a FailureStrategy seems to be the most flexible way to go.

[kbakk]

We have a similar need, where we start to run a task, to find out that it need to be retried in a few minutes. This happens quite often, and because of this we're currently getting errors in logs and Sentry (We've silenced these, but they're still causing noise when sifting through logs).

Another idea is to raise a specific exception that has some arguments to control retrying:

if not (something_is_ready):
  raise ForcedFailureException(retry_in="15m")

In case one knows that something_is_ready in 5 min, one postpone it to that exact time (upon reading https://procrastinate.readthedocs.io/en/stable/howto/retry.html#implementing-your-own-strategy it already seems possible).

[ewjoachim]

You could write a log Filter and install it on the logger. The filter would let you examine the message and choose to omit it on arbitrary conditions.

Code from the top of my head, not tested:

class OmitErrorFilter(logging.Filter):
    def filter(self, record: logging.LogRecord) -> bool:
        return getattr(record, "action", "") == "job_error" and replace_this_with_your_custom_conditions

logging.getLogger("procrastinate").addFilter(OmitErrorFilter())