Question posted in Json
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

Json – Deserialization of a list in jackson-databind

vatbub
September 19, 2023
223 views
0 votes
2 Answers

In an application I am serializing Kotlin objects to Json, sending them through HTTP and deserializing them on the other end back to Kotlin objects. To do so, I am using the jackson-dataind library. In one particular case, I am having trouble with deserializing a list. Consider the following classes and code:

@JsonTypeInfo(use = JsonTypeInfo.Id.CLASS, include = JsonTypeInfo.As.PROPERTY, property = "type")
interface ServerInteraction

data class GetJobResultResponse<T>(
    @JsonTypeInfo(include = JsonTypeInfo.As.WRAPPER_OBJECT, use = JsonTypeInfo.Id.CLASS)
    val result: T?
) : ServerInteraction

data class TotalStationPoint(val pointName: String, val location: Point3, val timestamp: Instant)

class LoadTotalStationDataSerializationTest {
    @Test
    fun loadTotalStationDataIsSerializedCorrectly() {
        val result: List<TotalStationPoint> = loadTotalStationData()
        val getJobResultResponse: GetJobResultResponse<List<TotalStationPoint>> = GetJobResultResponse(result)
        val serializedResponse: String = JsonSerializer.serialize(getJobResultResponse)

        /*
        Serializes to:
        {
          "type": "de.uni_freiburg.inatech.streem.image_converter.batch_job_executor.common.responses.GetJobResultResponse",
          "result": {
            "java.util.ArrayList": [
              {
                "pointName": "V1_22_0000000956",
                "location": {
                  "x": 10012.21984,
                  "y": 10002.558907,
                  "z": 2.342047000000001
                },
                "timestamp": 1670328663.04
              },
              {
                "pointName": "V1_22_0000000955",
                "location": {
                  "x": 10012.219816,
                  "y": 10002.559031,
                  "z": 2.3418279999999996
                },
                "timestamp": 1670328662.91
              }
              // ...
            ]
          }
        }
         */
        val deserializedResponse: GetJobResultResponse<*> = JsonSerializer.deserialize<GetJobResultResponse<*>>(serializedResponse)

        assertTrue((deserializedResponse.result as List<*>).first() is TotalStationPoint)
        assertEquals(getJobResultResponse, deserializedResponse)
    }

    private fun loadTotalStationData(): List<TotalStationPoint> {
        // Loads and parses a CSV file to List<TotalStationData>
        // ...
    }
}

The issue is that deserializedResponse.result is deserialized as ArrayList<Map<String, String>> where each former TotalStationPoint is now a map of attributes:

Instead, I want the list to be deserialized as List<TotalStationPoint> again, like the input object was:

I know that I can build a custom solution to this problem, but before resorting to a custom solution, I would like to know if Jackson-databind has a more idiomatic way to solve this.

Answers

Chosen as BEST ANSWER

So, I ended up modifying the JSON a little to include more type information about the result object. To explain the solution, I need to explain some more project-related context:

The result value in GetJobResultResponse is actually generated by processing jobs which run over a long time (possibly hours) on the server and eventually return a value, in this case a List<TotalStationPoint>. For different reasons, each Job has to specify the Class<*> of the object it returns and if it returns a List, it also has to specify the Class<*> of the elements contained in the list. So, the server already knows the Class<*> of the elements in the list, it merely needs to be transmitted to the client and taken into account during deserialization. I therefore modified the request and server like so:

@Suppress("MemberVisibilityCanBePrivate")
@JsonTypeInfo(use = JsonTypeInfo.Id.CLASS, include = JsonTypeInfo.As.PROPERTY, property = "type")
sealed class Job<TId : Job.Id<T>, T : Any>(
    val id: TId,
    private val outputClass: Class<T>?,
    private val listOutputClass: Class<*>? = null
) {
    val outputClassName = outputClass?.canonicalName
    val listOutputClassName = listOutputClass?.canonicalName
    
    // ...
    
    var result: T?
        private set
}

data class GetJobResultResponse<T>(
    @JsonTypeInfo(include = JsonTypeInfo.As.WRAPPER_OBJECT, use = JsonTypeInfo.Id.CLASS)
    val result: T?,
    val resultListType: String? = null
) : ServerInteraction

object GetJobResultMapping : RequestMapping<GetJobResultRequest, GetJobResultResponse<*>> {
    override fun invoke(request: GetJobResultRequest): GetJobResultResponse<Any> {
            val job = jobExecutor.completedJobs[request.jobId]
            return GetJobResultResponse(
                job.result,
                job.listOutputClassName
            )
        }
}

GetJobResultResponse therefore now also includes the canonical class name of the list element class. I then modified the deserialization method like so, such that it creates a TypeReference based on the additional information and then uses mapper.convertValue(result, typeReference) to convert the resultproperly:

object JsonSerializer {
    private fun objectMapper(): ObjectMapper = // ...

    inline fun <reified T : Any> deserialize(string: String): T = deserialize(string, T::class.java)

    fun <T : Any> deserialize(string: String, clazz: Class<T>): T {
            val mapper = objectMapper()
            val initialDeserialization = mapper.readValue(string, clazz)
            if (initialDeserialization !is GetJobResultResponse<*>) return initialDeserialization
            val resultListType = initialDeserialization.resultListType ?: return initialDeserialization

            val elementClass = Class.forName(resultListType)

            @Suppress("UNCHECKED_CAST")
            val typeReference = mapper.typeFactory
                .constructCollectionType(
                    initialDeserialization.result?.javaClass as Class<out Collection<*>>? ?: List::class.java,
                    elementClass
                )

            @Suppress("UNCHECKED_CAST")
            return GetJobResultResponse<Any>(
                mapper.convertValue(initialDeserialization.result, typeReference),
                initialDeserialization.resultListType
            ) as T
        }
}

(Edit)

You lose too much type information, that’s the main problem.

However, if you can allow a subclass on GetJobResultResponse then sufficient type information is passed:

class TotalStationPointsResponse(result: List<TotalStationPoint>) :
    GetJobResultResponse<List<TotalStationPoint>>(result)

working solution:

package de.uni_freiburg.inatech.streem.image_converter.batch_job_executor.common.responses

import com.fasterxml.jackson.annotation.JsonTypeInfo
import com.fasterxml.jackson.databind.json.JsonMapper
import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule
import com.fasterxml.jackson.module.kotlin.KotlinFeature
import com.fasterxml.jackson.module.kotlin.KotlinModule
import org.junit.Assert.assertEquals
import org.junit.Assert.assertTrue
import org.junit.jupiter.api.Test
import java.time.Instant

@JsonTypeInfo(use = JsonTypeInfo.Id.CLASS, include = JsonTypeInfo.As.PROPERTY, property = "type")
interface ServerInteraction

// this class changed from data class to open class to permit subclassing
open class GetJobResultResponse<T>(
    @JsonTypeInfo(include = JsonTypeInfo.As.WRAPPER_OBJECT, use = JsonTypeInfo.Id.CLASS)
    val result: T?
) : ServerInteraction

class TotalStationPointsResponse(result: List<TotalStationPoint>) :
    GetJobResultResponse<List<TotalStationPoint>>(result)

data class Point3(val x: Double, val y: Double, val z: Double)

data class TotalStationPoint(val pointName: String, val location: Point3, val timestamp: Instant)

class LoadTotalStationDataSerializationTest {
    @Test
    fun loadTotalStationDataIsSerializedCorrectly() {
        val result: List<TotalStationPoint> = loadTotalStationData()
        //val getJobResultResponse: GetJobResultResponse<List<TotalStationPoint>> = GetJobResultResponse(result)
        val getJobResultResponse: TotalStationPointsResponse = TotalStationPointsResponse(result)
        // this is just a different way to serialize... nothing really different here:
        val mapper = JsonMapper.builder()
            .addModule(KotlinModule.Builder().configure(KotlinFeature.StrictNullChecks, true).build())
            .addModule(JavaTimeModule())
            .build()
        val serializedResponse: String =
            mapper.writerWithDefaultPrettyPrinter().writeValueAsString(getJobResultResponse)
        println(serializedResponse)
        /*
        Serializes to:
            {
              "type" : "de.uni_freiburg.inatech.streem.image_converter.batch_job_executor.common.responses.TotalStationPointsResponse",
              "result" : [ {
                "pointName" : "V1_22_0000000956",
                "location" : {
                  "x" : 10012.21984,
                  "y" : 10002.558907,
                  "z" : 2.342047000000001
                },
                "timestamp" : 1695116403.684844000
              } ]
            }
         */
        val deserializedResponse: GetJobResultResponse<*> =
            mapper.readValue(serializedResponse, GetJobResultResponse::class.java)

        assertTrue((deserializedResponse.result as List<*>).first() is TotalStationPoint)
        // following fails because the change away from data class means there is no proper equals() 
        assert(getJobResultResponse, deserializedResponse)
    }

    private fun loadTotalStationData(): List<TotalStationPoint> {
        return listOf(
            TotalStationPoint(
                "V1_22_0000000956",
                Point3(10012.21984, 10002.558907, 2.342047000000001),
                Instant.now(),
            ),
        )
    }
}

Please signup or login to give your own answer.

Click here to cancel reply.