Pytest and structured logs

[ashemedai]

So I am currently using pytest for building my unit tests. And while I can use caplog with POWERTOOLS_LOG_DEDUPLICATION_DISABLED to test the text of simple log.info() calls, I am currently breaking my head to get the structured log output that's printed to stdout. Using capsys and such does not seem to work as I only get empty strings. I am probably missing something super obvious, so any help will be useful to get me unstuck.

[leandrodamascena]

Hi @ashemedai, can you share a snippet of what you are trying to test?

BTW, we don't use caplog to capture stdout, instead we output this to the memory (using StringIO) and then capture this. You can check an example here: https://github.com/aws-powertools/powertools-lambda-python/blob/develop/tests/functional/logger/required_dependencies/test_logger.py#L64

[ashemedai]

Ola Leandro,

I have a Lambda that I want to output some structured logging towards CloudWatch for others to easily do operational monitoring/troubleshooting from.

As such I want to ensure the structured logging does the right thing.

So something like this in my Lambda:

def process_body(data):
    # code to get the right data from events and put in a dict called data_keys
    log.info("Triggered X", data=data_keys)

And then with pytest I wanted to use caplog to get the structured logging data and assert on that. E.g. a log like this:

{"level":"INFO","location":"process_body:93","message":"Triggered X","timestamp":"2024-08-22 11:51:29,865+0200","service":"lambdafunc","data":{...}}

But using caplog, I can only get something like this, since the structured log is output to stdout and not the logging that pytest can access:

INFO     lambdafunc:processor.py:93 Triggered X

So I figured that capsys would be helpful here, but nope, using capsys.readouterr() only yields an empty string. A bit curious why.

After posting my question, I look at the logger tests if you folks did something like this. So I will give it a try.

[leandrodamascena]

Ola Leandro,

Greetings in portuguese? Eu amo isso! 😃

After posting my question, I look at the logger tests if you folks did something like this. So I will give it a try.

Yep, based on your explanation we do the same in our tests. So, please test and let me know if you were able to fix the problem. If not, I can help with some additional tips.

Thanks!