How-to Guide

Encoding and decoding protocol-based objects

Added in version 2.0.

Encoding protocol-based objects

Changed in version 2.4: Made frozendict serializable by default.

By default, protocol types are encoded from instances of the corresponding Python types shown below. Additional types can be registered with types.

Type	Default	Required methods
"array"	list and tuple	__iter__() or __len__() and __getitem__()
"bool"	True and False	__bool__(), __len__() or neither (true)
"float"	float	__str__() or __repr__()
"int"	int	__str__() or __repr__()
"null"	None	none (not extensible)
"object"	dict and frozendict	values() and items()
"str"	str	__str__() or __repr__()

Example with numpy:

>>> import jsonyx as json
>>> import numpy as np
>>> obj = np.array([
...     np.bool_(), np.int8(), np.uint8(), np.int16(), np.uint16(), np.int32(),
...     np.uint32(), np.intp(), np.uintp(), np.int64(), np.uint64(), np.float16(),
...     np.float32(), np.float64()
... ], dtype="O")
>>> types = {
...     "array": np.ndarray,
...     "bool": np.bool_,
...     "float": np.floating,
...     "int": np.integer
... }
>>> json.dump(obj, types=types)
[false, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0, 0.0, 0.0, 0.0]

Note

Custom types must be registered manually, jsonyx does not infer serializability based on method presence.

Warning

Avoid specifying ABCs for types, that is very slow.

Decoding protocol-based objects

By default, decoded protocol types are converted to the corresponding Python types shown below. The conversion can be customized with hooks.

Hook	Default	Called with
"array"	list	list
"bool"	bool	True or False
"float"	float	str
"int"	int	str
"null"	types.NoneType	nothing (not customizable)
"object"	dict	list[tuple[Any, Any]]
"str"	str	str

Example with numpy:

>>> import jsonyx as json
>>> from functools import partial
>>> import numpy as np
>>> hooks = {
...     "array": partial(np.array, dtype="O"),
...     "bool": np.bool_,
...     "float": np.float64,
...     "int": np.int64
... }
>>> json.loads("[false, 0.0, 0]", hooks=hooks)
array([np.False_, np.float64(0.0), np.int64(0)], dtype=object)

Using decimal.Decimal instead of float

>>> import jsonyx as json
>>> from decimal import Decimal
>>> json.loads("1.1", hooks={"float": Decimal})
Decimal('1.1')
>>> json.dump(Decimal('1.1'), types={"float": Decimal})
1.1

Using multidict.MultiDict instead of dict

>>> import jsonyx as json
>>> from multidict import MultiDict
>>> json.loads('{"a": 1, "a": 2, "a": 3}', hooks={"object": MultiDict})
<MultiDict('a': 1, 'a': 2, 'a': 3)>
>>> obj = MultiDict([('a', 1), ('a', 2), ('a', 3)])
>>> json.dump(obj, types={"object": MultiDict})
{"a": 1, "a": 2, "a": 3}

Encoding and decoding arbitrary objects

Added in version 2.1.

Encoding arbitrary objects

>>> import jsonyx as json
>>> def complex_hook(obj):
...     if isinstance(obj, complex):
...         return {"__complex__": True, "real": obj.real, "imag": obj.imag}
...     return obj
...
>>> json.dump(1 + 2j, hook=complex_hook)
{"__complex__": true, "real": 1.0, "imag": 2.0}

Tip

Use functools.singledispatch() to make this extensible.

Warning

This function is called for every object during encoding, even if the object is normally serializable.

See also

The pickle and shelve modules which are better suited for this.

Decoding arbitrary objects

>>> import jsonyx as json
>>> def object_hook(obj):
...     obj = dict(obj)
...     if "__complex__" in obj:
...         return complex(obj["real"], obj["imag"])
...     return obj
...
>>> s = '{"__complex__": true, "real": 1.0, "imag": 2.0}'
>>> json.loads(s, hooks={"object": object_hook})
(1+2j)

Note

hooks["object"] is called with a list of tuples, not a dict.

See also

The pickle and shelve modules which are better suited for this.

Formatting numbers

Added in version 2.4.

By default, numbers are formatted using str. Custom formatters can be registered with formatters. Formatters must return a string containing a valid JSON number.

Formatter	Default	Called with
"float"	str	float or types["float"]
"int"	str	int or types["int"]

Example:

>>> import jsonyx as json
>>> json.dump([1.234, 5.678], formatters={"float": "{:.2f}".format})
[1.23, 5.68]

Encoding enum.ReprEnum

>>> import jsonyx as json
>>> from enum import ReprEnum
>>> class MyEnum(int, ReprEnum):
...     ZERO = 0
...
>>> json.dump(MyEnum.ZERO)
0

Warning

Don’t use enum.Enum, because it overrides the string representation.

Encoding and decoding big integers

Python has a global limit for converting between int and str. If you need to process integers exceeding this limit, use sys.set_int_max_str_digits() to increase it:

>>> import jsonyx as json
>>> from sys import set_int_max_str_digits
>>> set_int_max_str_digits(0)
>>> json.loads("9" * 5_000) == 10 ** 5_000 - 1
True
>>> len(json.dumps(10 ** 5_000))
5002

See Integer string conversion length limitation for more information.

Better error messages

Better error messages for other JSON libraries

>>> import json, jsonyx
>>> try:
...     json.loads("[,]")
... except json.JSONDecodeError as exc:
...     raise jsonyx.JSONSyntaxError(exc.msg, "<string>", exc.doc, exc.pos) from None
...
Traceback (most recent call last):
  File "<string>", line 1, column 2
    [,]
     ^
jsonyx.JSONSyntaxError: Expecting value

Better error messages for encoding strings

>>> import jsonyx as json
>>> try:
...     "café".encode("ascii")
... except UnicodeEncodeError as exc:
...     raise json.TruncatedSyntaxError(
...         f"(unicode error) {exc}", "<string>", exc.object, exc.start, exc.end,
...     ) from None
...
Traceback (most recent call last):
  File "<string>", line 1, column 4-5
    café
       ^
jsonyx.TruncatedSyntaxError: (unicode error) 'ascii' codec can't encode character '\xe9' in position 3: ordinal not in range(128)

Better error messages for decoding bytes

>>> import jsonyx as json
>>> try:
...     b"caf\xe9".decode("ascii")
... except UnicodeDecodeError as exc:
...     doc = exc.object.decode(exc.encoding, "replace")
...     start = exc.object[:exc.start].decode(exc.encoding, "replace")
...     end = exc.object[:exc.end].decode(exc.encoding, "replace")
...     raise json.TruncatedSyntaxError(
...         f"(unicode error) {exc}", "<string>", doc, len(start), len(end),
...     ) from None
...
Traceback (most recent call last):
  File "<string>", line 1, column 4-5
    caf�
       ^
jsonyx.TruncatedSyntaxError: (unicode error) 'ascii' codec can't decode byte 0xe9 in position 3: ordinal not in range(128)

See also

jsonyx.format_syntax_error() for formatting the exception.