Json - Deserialization of a list in jackson-databind

vatbub
September 19, 2023
205 views
0 votes
2 Answers

In an application I am serializing Kotlin objects to Json, sending them through HTTP and deserializing them on the other end back to Kotlin objects. To do so, I am using the jackson-dataind library. In one particular case, I am having trouble with deserializing a list. Consider the following classes and code:

@JsonTypeInfo(use = JsonTypeInfo.Id.CLASS, include = JsonTypeInfo.As.PROPERTY, property = "type")
interface ServerInteraction

data class GetJobResultResponse<T>(
    @JsonTypeInfo(include = JsonTypeInfo.As.WRAPPER_OBJECT, use = JsonTypeInfo.Id.CLASS)
    val result: T?
) : ServerInteraction

data class TotalStationPoint(val pointName: String, val location: Point3, val timestamp: Instant)

class LoadTotalStationDataSerializationTest {
    @Test
    fun loadTotalStationDataIsSerializedCorrectly() {
        val result: List<TotalStationPoint> = loadTotalStationData()
        val getJobResultResponse: GetJobResultResponse<List<TotalStationPoint>> = GetJobResultResponse(result)
        val serializedResponse: String = JsonSerializer.serialize(getJobResultResponse)

        /*
        Serializes to:
        {
          "type": "de.uni_freiburg.inatech.streem.image_converter.batch_job_executor.common.responses.GetJobResultResponse",
          "result": {
            "java.util.ArrayList": [
              {
                "pointName": "V1_22_0000000956",
                "location": {
                  "x": 10012.21984,
                  "y": 10002.558907,
                  "z": 2.342047000000001
                },
                "timestamp": 1670328663.04
              },
              {
                "pointName": "V1_22_0000000955",
                "location": {
                  "x": 10012.219816,
                  "y": 10002.559031,
                  "z": 2.3418279999999996
                },
                "timestamp": 1670328662.91
              }
              // ...
            ]
          }
        }
         */
        val deserializedResponse: GetJobResultResponse<*> = JsonSerializer.deserialize<GetJobResultResponse<*>>(serializedResponse)

        assertTrue((deserializedResponse.result as List<*>).first() is TotalStationPoint)
        assertEquals(getJobResultResponse, deserializedResponse)
    }

    private fun loadTotalStationData(): List<TotalStationPoint> {
        // Loads and parses a CSV file to List<TotalStationData>
        // ...
    }
}

The issue is that deserializedResponse.result is deserialized as ArrayList<Map<String, String>> where each former TotalStationPoint is now a map of attributes:

Instead, I want the list to be deserialized as List<TotalStationPoint> again, like the input object was:

I know that I can build a custom solution to this problem, but before resorting to a custom solution, I would like to know if Jackson-databind has a more idiomatic way to solve this.

Answers

Chosen as BEST ANSWER

So, I ended up modifying the JSON a little to include more type information about the result object. To explain the solution, I need to explain some more project-related context:

The result value in GetJobResultResponse is actually generated by processing jobs which run over a long time (possibly hours) on the server and eventually return a value, in this case a List<TotalStationPoint>. For different reasons, each Job has to specify the Class<*> of the object it returns and if it returns a List, it also has to specify the Class<*> of the elements contained in the list. So, the server already knows the Class<*> of the elements in the list, it merely needs to be transmitted to the client and taken into account during deserialization. I therefore modified the request and server like so:

@Suppress("MemberVisibilityCanBePrivate")
@JsonTypeInfo(use = JsonTypeInfo.Id.CLASS, include = JsonTypeInfo.As.PROPERTY, property = "type")
sealed class Job<TId : Job.Id<T>, T : Any>(
    val id: TId,
    private val outputClass: Class<T>?,
    private val listOutputClass: Class<*>? = null
) {
    val outputClassName = outputClass?.canonicalName
    val listOutputClassName = listOutputClass?.canonicalName
    
    // ...
    
    var result: T?
        private set
}

data class GetJobResultResponse<T>(
    @JsonTypeInfo(include = JsonTypeInfo.As.WRAPPER_OBJECT, use = JsonTypeInfo.Id.CLASS)
    val result: T?,
    val resultListType: String? = null
) : ServerInteraction

object GetJobResultMapping : RequestMapping<GetJobResultRequest, GetJobResultResponse<*>> {
    override fun invoke(request: GetJobResultRequest): GetJobResultResponse<Any> {
            val job = jobExecutor.completedJobs[request.jobId]
            return GetJobResultResponse(
                job.result,
                job.listOutputClassName
            )
        }
}

GetJobResultResponse therefore now also includes the canonical class name of the list element class. I then modified the deserialization method like so, such that it creates a TypeReference based on the additional information and then uses mapper.convertValue(result, typeReference) to convert the resultproperly:

object JsonSerializer {
    private fun objectMapper(): ObjectMapper = // ...

    inline fun <reified T : Any> deserialize(string: String): T = deserialize(string, T::class.java)

    fun <T : Any> deserialize(string: String, clazz: Class<T>): T {
            val mapper = objectMapper()
            val initialDeserialization = mapper.readValue(string, clazz)
            if (initialDeserialization !is GetJobResultResponse<*>) return initialDeserialization
            val resultListType = initialDeserialization.resultListType ?: return initialDeserialization

            val elementClass = Class.forName(resultListType)

            @Suppress("UNCHECKED_CAST")
            val typeReference = mapper.typeFactory
                .constructCollectionType(
                    initialDeserialization.result?.javaClass as Class<out Collection<*>>? ?: List::class.java,
                    elementClass
                )

            @Suppress("UNCHECKED_CAST")
            return GetJobResultResponse<Any>(
                mapper.convertValue(initialDeserialization.result, typeReference),
                initialDeserialization.resultListType
            ) as T
        }
}

(Edit)

You lose too much type information, that’s the main problem.

However, if you can allow a subclass on GetJobResultResponse then sufficient type information is passed:

class TotalStationPointsResponse(result: List<TotalStationPoint>) :
    GetJobResultResponse<List<TotalStationPoint>>(result)

working solution:

package de.uni_freiburg.inatech.streem.image_converter.batch_job_executor.common.responses

import com.fasterxml.jackson.annotation.JsonTypeInfo
import com.fasterxml.jackson.databind.json.JsonMapper
import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule
import com.fasterxml.jackson.module.kotlin.KotlinFeature
import com.fasterxml.jackson.module.kotlin.KotlinModule
import org.junit.Assert.assertEquals
import org.junit.Assert.assertTrue
import org.junit.jupiter.api.Test
import java.time.Instant

@JsonTypeInfo(use = JsonTypeInfo.Id.CLASS, include = JsonTypeInfo.As.PROPERTY, property = "type")
interface ServerInteraction

// this class changed from data class to open class to permit subclassing
open class GetJobResultResponse<T>(
    @JsonTypeInfo(include = JsonTypeInfo.As.WRAPPER_OBJECT, use = JsonTypeInfo.Id.CLASS)
    val result: T?
) : ServerInteraction

class TotalStationPointsResponse(result: List<TotalStationPoint>) :
    GetJobResultResponse<List<TotalStationPoint>>(result)

data class Point3(val x: Double, val y: Double, val z: Double)

data class TotalStationPoint(val pointName: String, val location: Point3, val timestamp: Instant)

class LoadTotalStationDataSerializationTest {
    @Test
    fun loadTotalStationDataIsSerializedCorrectly() {
        val result: List<TotalStationPoint> = loadTotalStationData()
        //val getJobResultResponse: GetJobResultResponse<List<TotalStationPoint>> = GetJobResultResponse(result)
        val getJobResultResponse: TotalStationPointsResponse = TotalStationPointsResponse(result)
        // this is just a different way to serialize... nothing really different here:
        val mapper = JsonMapper.builder()
            .addModule(KotlinModule.Builder().configure(KotlinFeature.StrictNullChecks, true).build())
            .addModule(JavaTimeModule())
            .build()
        val serializedResponse: String =
            mapper.writerWithDefaultPrettyPrinter().writeValueAsString(getJobResultResponse)
        println(serializedResponse)
        /*
        Serializes to:
            {
              "type" : "de.uni_freiburg.inatech.streem.image_converter.batch_job_executor.common.responses.TotalStationPointsResponse",
              "result" : [ {
                "pointName" : "V1_22_0000000956",
                "location" : {
                  "x" : 10012.21984,
                  "y" : 10002.558907,
                  "z" : 2.342047000000001
                },
                "timestamp" : 1695116403.684844000
              } ]
            }
         */
        val deserializedResponse: GetJobResultResponse<*> =
            mapper.readValue(serializedResponse, GetJobResultResponse::class.java)

        assertTrue((deserializedResponse.result as List<*>).first() is TotalStationPoint)
        // following fails because the change away from data class means there is no proper equals() 
        assert(getJobResultResponse, deserializedResponse)
    }

    private fun loadTotalStationData(): List<TotalStationPoint> {
        return listOf(
            TotalStationPoint(
                "V1_22_0000000956",
                Point3(10012.21984, 10002.558907, 2.342047000000001),
                Instant.now(),
            ),
        )
    }
}

Please signup or login to give your own answer.

Click here to cancel reply.

Json – Deserialization of a list in jackson-databind

Answers