Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

Pydantic v2 vs requests POST JSON – 'not serializable' nightmares

I am writing a script to post data from an internal system to a third party web API. Our internal system uses data Models defined in Pydantic. I am trying to send data in one of these models to the API using requests.post but it can’t serialize some types and it’s driving me insane.

Take this data model as an example:

class Product(BaseModel):
    id: int
    group_id: int
    sku: str
    price: Decimal
    special_offer_price: Decimal
    last_update: datetime.date

If I try to send data using this Model using requests.post thus:

product_item = website_api.Product(**product.__dict__)

requests.post(url, headers=jwt_auth.auth_header, json=product_item)

The str and int fields are fine but Decimal and datetime.date fields throw up an exception as they are not serializable, giving the error somewhat like:

TypeError: Object of type date is not JSON serializable

I have tried adding a @field_serializer for the fields to the class definition for pydantic;

    @field_serializer("price", "special_offer_price")
    def serialize_decimal(self, value):
        return str(value)
    
    @field_serializer("last_update")
    def serialize_date(self, value):
        return str(value)

but requests doesn’t seem to use it when serializing.

Tags:

2

Answers


  1. 0

    Here:

    product_item = website_api.Product(**product.__dict__)

    if your product_item is an instance of Product model you mentioned, then you should do:

    requests.post(url, headers=jwt_auth.auth_header, json=product_item.model_dump())

    You shouldn’t need a serializers for datetime.date and Decimal. When I tested this on this Product class I was able to serialize it with product_item.model_dump() without any problem.

    Like here:

    import datetime
from decimal import Decimal
from pydantic import BaseModel


class Product(BaseModel):
    id: int
    group_id: int
    sku: str
    price: Decimal
    special_offer_price: Decimal
    last_update: datetime.date


product = Product(
    id=1,
    group_id=1,
    sku="123456",
    price=Decimal("100.00"),
    special_offer_price=Decimal("90.00"),
    last_update=datetime.date(2021, 1, 1),
)

print(product.model_dump())

    produces:

    {'id': 1, 'group_id': 1, 'sku': '123456', 'price': Decimal('100.00'), 'special_offer_price': Decimal('90.00'), 'last_update': datetime.date(2021, 1, 1)}

    or if you’s like it as json string, then this:

    print(product.model_dump_json())

    produces:

    {"id":1,"group_id":1,"sku":"123456","price":"100.00","special_offer_price":"90.00","last_update":"2021-01-01"}
    Login or Signup to reply.
  2. 0

    The Date object, as the error suggests, is not seriazible by default. For this purpose, you can create an encoder:

    import json
from decimal import Decimal
from datetime import date
from pydantic import BaseModel

class CustomJSONEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, Decimal):
            return str(obj)
        if isinstance(obj, date):
            return obj.isoformat()
        return super().default(obj)

class Product(BaseModel):
    id: int
    group_id: int
    sku: str
    price: Decimal
    special_offer_price: Decimal
    last_update: date

product_item = Product(
    id=1,
    group_id=10,
    sku="ABC123",
    price=Decimal('19.99'),
    special_offer_price=Decimal('15.99'),
    last_update=date.today()
)

product_dict = product_item.dict()
data = json.dumps(product_dict, cls=CustomJSONEncoder)

headers = {
    'Content-Type': 'application/json',
    **jwt_auth.auth_header
}

response = requests.post(url, headers=headers, data=data)

    From here and on, you will be able to do whatever you want with this JSON serialized data.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search