You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Split core functionality and support orjson and msgspec (#9)
## Summary of changes
### Refactor common functionality into base module
This allows support multiple JSON encoders by having common
functionality in `pythonjsonlogger.core` and then specialist formatters
for each encoder.
This is useful / needed, as not all JSON encoders support the
`json.dumps` or `json.JSONEncoder` interfaces exactly. This enables us
to support other JSON encoders like orjson and msgspec. In the future we
may add support for other encoders.
### Better handling for custom styles
Achieved by mimicking `logging.Formatter.__init__` without actually
calling it.
A code snippet is worth `2**10` words:
```python
from pythonjsonlogger.core import BaseJsonLogger
class CommaSupport(BaseJsonFormatter):
def parse(self) -> list[str]:
if isinstance(self._style, str) and self._style == ",":
return self._fmt.split(",")
return super().parse()
f = CommaSupport("message,asctime", style=",", validate=False)
```
### Rename `jsonlogger` module to `json` module
Compatibility is maintained for the moment using `__getattr__` in
`__init__`.
This is to enable more consistent naming of implementation specific
module names. It also stops throwing around the word "logger" when this
module only contains formatters.
### Add support for orjson
[orjson](https://github.com/ijl/orjson) is a high performance (and more
JSON spec correct) encoder. Given how many logging calls may occur -
having a performant formatter available is important.
This includes ensuring it is covered in tests on appropriate platforms.
Note: orjson is not supported on pypy, and currently does not build for
py313.
### Add support for msgspec
[msgspec](https://jcristharif.com/msgspec/index.html) is another library
containing a high performance JSON encoder.
Note: msgspec is not supported on pypy, and currently does not build for
py313.
### Drops python 3.7 support
This is primary due do making use of the
[`validate`](https://docs.python.org/3/library/logging.html#formatter-objects)
argument. I was also having issues with CI because python 3.7 is not support on most "latest"
This splits common funcitonality out to allow supporting other JSON encoders. Although this is a large refactor, backwards compatibility has been maintained.
10
+
11
+
### Added
12
+
-`.core` - more details below.
13
+
- Orjson encoder support via `.orjson.OrjsonFormatter`.
14
+
- MsgSpec encoder support via `.msgspec.MsgspecFormatter`.
15
+
16
+
### Changed
17
+
-`.jsonlogger` has been moved to `.json` with core functionality moved to `.core`.
18
+
-`.core.BaseJsonFormatter` properly supports all `logging.Formatter` arguments:
19
+
-`fmt` is unchanged.
20
+
-`datefmt` is unchanged.
21
+
-`style` can now support non-standard arguments by setting `validate` to `False`
22
+
-`validate` allows non-standard `style` arguments or prevents calling `validate` on standard `style` arguments.
23
+
-`default` is ignored.
24
+
25
+
### Deprecated
26
+
-`.jsonlogger` is now `.json`
27
+
-`.jsonlogger.RESERVED_ATTRS` is now `.core.RESERVED_ATTRS`.
28
+
-`.jsonlogger.merge_record_extra` is now `.core.merge_record_extra`.
29
+
30
+
### Removed
31
+
- Python 3.7 support dropped
32
+
-`.jsonlogger.JsonFormatter._str_to_fn` replaced with `.core.str_to_object`.
Python JSON Logger provides `logging.Formatter`s that encode the logged message into JSON. Although a variety of JSON encoders are supported, in the following examples we will use the `pythonjsonlogger.json.JsonFormatter` which uses the the `json` module from the standard library.
57
+
56
58
### Integrating with Python's logging framework
57
59
58
-
Json outputs are provided by the JsonFormatter logging formatter. You can add the custom formatter like below:
60
+
To produce JSON output, attach the formatter to a logging handler:
59
61
60
62
```python
61
63
import logging
62
-
from pythonjsonlogger importjsonlogger
64
+
from pythonjsonlogger.jsonimportJsonFormatter
63
65
64
66
logger = logging.getLogger()
65
67
66
68
logHandler = logging.StreamHandler()
67
-
formatter =jsonlogger.JsonFormatter()
69
+
formatter = JsonFormatter()
68
70
logHandler.setFormatter(formatter)
69
71
logger.addHandler(logHandler)
70
72
```
71
73
72
-
### Customizing fields
73
-
74
-
The fmt parser can also be overidden if you want to have required fields that differ from the default of just `message`.
74
+
### Output fields
75
75
76
-
These two invocations are equivalent:
76
+
You can control the logged fields by setting the `fmt` argument when creating the formatter. By default formatters will follow the same `style` of `fmt` as the `logging` module: `%`, `$`, and `{`. All [`LogRecord` attributes](https://docs.python.org/3/library/logging.html#logrecord-attributes) can be output using their name.
You can also add extra fields to your json output by specifying a dict in place of message, as well as by specifying an `extra={}` argument.
@@ -94,9 +86,9 @@ Contents of these dictionaries will be added at the root level of the entry and
94
86
You can also use the `add_fields` method to add to or generally normalize the set of default set of fields, it is called for every log event. For example, to unify default fields with those provided by [structlog](http://www.structlog.org/) you could do something like this:
To use the module with a config file using the [`fileConfig` function](https://docs.python.org/3/library/logging.config.html#logging.config.fileConfig), use the class `pythonjsonlogger.jsonlogger.JsonFormatter`. Here is a sample config file.
148
+
To use the module with a config file using the [`fileConfig` function](https://docs.python.org/3/library/logging.config.html#logging.config.fileConfig), use the class `pythonjsonlogger.json.JsonFormatter`. Here is a sample config file.
0 commit comments