Pydantic v2 vs requests POST JSON - 'not serializable' nightmares

MarkBenson
July 5, 2024
231 views
1 vote
2 Answers

I am writing a script to post data from an internal system to a third party web API. Our internal system uses data Models defined in Pydantic. I am trying to send data in one of these models to the API using requests.post but it can’t serialize some types and it’s driving me insane.

Take this data model as an example:

class Product(BaseModel):
    id: int
    group_id: int
    sku: str
    price: Decimal
    special_offer_price: Decimal
    last_update: datetime.date

If I try to send data using this Model using requests.post thus:

product_item = website_api.Product(**product.__dict__)

requests.post(url, headers=jwt_auth.auth_header, json=product_item)

The str and int fields are fine but Decimal and datetime.date fields throw up an exception as they are not serializable, giving the error somewhat like:

TypeError: Object of type date is not JSON serializable

I have tried adding a @field_serializer for the fields to the class definition for pydantic;

    @field_serializer("price", "special_offer_price")
    def serialize_decimal(self, value):
        return str(value)
    
    @field_serializer("last_update")
    def serialize_date(self, value):
        return str(value)

but requests doesn’t seem to use it when serializing.

Answers

Here:

product_item = website_api.Product(**product.__dict__)

if your product_item is an instance of Product model you mentioned, then you should do:

requests.post(url, headers=jwt_auth.auth_header, json=product_item.model_dump())

You shouldn’t need a serializers for datetime.date and Decimal. When I tested this on this Product class I was able to serialize it with product_item.model_dump() without any problem.

Like here:

import datetime
from decimal import Decimal
from pydantic import BaseModel


class Product(BaseModel):
    id: int
    group_id: int
    sku: str
    price: Decimal
    special_offer_price: Decimal
    last_update: datetime.date


product = Product(
    id=1,
    group_id=1,
    sku="123456",
    price=Decimal("100.00"),
    special_offer_price=Decimal("90.00"),
    last_update=datetime.date(2021, 1, 1),
)

print(product.model_dump())

produces:

{'id': 1, 'group_id': 1, 'sku': '123456', 'price': Decimal('100.00'), 'special_offer_price': Decimal('90.00'), 'last_update': datetime.date(2021, 1, 1)}

or if you’s like it as json string, then this:

print(product.model_dump_json())

produces:

{"id":1,"group_id":1,"sku":"123456","price":"100.00","special_offer_price":"90.00","last_update":"2021-01-01"}

The Date object, as the error suggests, is not seriazible by default. For this purpose, you can create an encoder:

import json
from decimal import Decimal
from datetime import date
from pydantic import BaseModel

class CustomJSONEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, Decimal):
            return str(obj)
        if isinstance(obj, date):
            return obj.isoformat()
        return super().default(obj)

class Product(BaseModel):
    id: int
    group_id: int
    sku: str
    price: Decimal
    special_offer_price: Decimal
    last_update: date

product_item = Product(
    id=1,
    group_id=10,
    sku="ABC123",
    price=Decimal('19.99'),
    special_offer_price=Decimal('15.99'),
    last_update=date.today()
)

product_dict = product_item.dict()
data = json.dumps(product_dict, cls=CustomJSONEncoder)

headers = {
    'Content-Type': 'application/json',
    **jwt_auth.auth_header
}

response = requests.post(url, headers=headers, data=data)

From here and on, you will be able to do whatever you want with this JSON serialized data.

Please signup or login to give your own answer.

Click here to cancel reply.

Pydantic v2 vs requests POST JSON – 'not serializable' nightmares

Answers