Json Question posted in
Our archive of expertly curated questions and answers provides insights and solutions to common problems related to this popular data interchange format. From parsing and manipulating JSON data to integrating it with various programming languages and web services, our archive has got you covered. Start exploring today and take your JSON skills to the next level

Json – How to create an escape hatch in swift's Encodable

I’m looking for a way to create an escape hatch in Encodable. There are many JSON APIs that are more flexible than Encodable allows for. As an example, take the Anthropic API which allows you to specify a JSON schema as part of the request body. This schema is not known ahead of time to the programmer, as the end user can craft it. See the input_schema field here: https://docs.anthropic.com/en/docs/build-with-claude/tool-use

My goal is to enforce a strict contract where it makes sense (fixed fields) and allow
for an escape hatch of [String: Any] when the structure if flexible. These aims are hard to achieve. I have been working on it for two days, writing various encoder hacks trying to get the desired output. What I have found is that it’s trivial to stay fully in the untyped world or fully in the strict contract world:

This is very flexible, and works out of the box:

let myTree: [String: Any] = [
    "a": "xyz",
    "b": [
        "c": 2,
        "d": false
    ]
]

do {
    let data = try JSONSerialization.data(withJSONObject: myTree)
    print(String(decoding: data, as: UTF8.self))
} catch {
    print("Could not convert flexible dictionary to JSON: (error)")
}

This is very rigid, and works out of the box:

struct Root: Encodable {
    let a: String
    let b: Child
}

struct Child: Encodable {
    let c: Int
    let d: Bool
}

let myTree = Root(a: "xyz", b: Child(c: 2, d: false))

do {
    let encoder = JSONEncoder()
    encoder.outputFormatting = .sortedKeys
    let data = try encoder.encode(myTree)
    print(String(decoding: data, as: UTF8.self))
} catch {
    print("Could not convert encodable struct to JSON: (error)")
}

If you run both of those, you’ll see that the two print statements produce the same JSON, great! Now assume that the structure of field b isn’t known ahead of time, e.g. in the case of an API where the user specifies the schema. I want to do this:

struct RootWithEscapeHatch: Encodable {
    let a: String
    let b: [String: Any]

    private enum Fields: CodingKey {
        case a
        case b
    }

    func encode(to encoder: any Encoder) throws {
        var container = encoder.container(keyedBy: Fields.self)
        try container.encode(self.a, forKey: .a)
        // try container.encode(self.b, forKey: .b)  <-- What goes here?
    }
}


let myFailingTree = RootWithEscapeHatch(a: "xyz", b: ["c": 2, "d": false])
do {
    let data = try JSONEncoder().encode(myFailingTree)
    print(String(decoding: data, as: UTF8.self))
} catch {
    print("Could not convert encodable with escape hatch to JSON: (error)")
}

You can see that myFailingTree is isomorphic with myTree examples above. I want the print statements to produce the same JSON. If you have an idea for what goes in the "What goes here?" line, please let me know. I’m looking for a general solution, I.e. I don’t want to hard code that the structure will always be ["c": 2, "d": false]. The point is that any [String: Any] field should be serializable just like the first example in this question.

Thanks!

Tags:

3

Answers


  1. Chosen as BEST ANSWER
    0

    I'm going to leave what I have here. It's not complete, and is still relatively easy to break. Making this approach correct is too cumbersome and hard to follow. @xAlien95's approach looks nice, and @mojtaba-hosseini's conclusion is worth heeding.

    The approach below works for the tree in the original challenge, but you can still break it with this tree:

    let myFailingTree = RootWithEscapeHatch(a: "xyz", b: ["c": [1, ["d": [2,false]]]])

    Here are the utility functions:

    func encodeJSONEscapeHatch<T>(
    encoder: Encoder,
    container: KeyedEncodingContainer<T>,
    key: T,
    dictionary: [String: Any]
) throws {
    var container = container
    var dictContainer = container.nestedContainer(keyedBy: MyCodingKey.self, forKey: key)
    try dictionary.forEach { key, value in
        guard let codingKey = MyCodingKey(stringValue: key) else {
            throw DecodingError.typeMismatch(
                type(of: key),
                DecodingError.Context(
                    codingPath: container.codingPath,
                    debugDescription: "Only strings are allowed as nested keys"
                )
            )
        }
        switch value {
        case let value as Encodable:
            try dictContainer.encode(value, forKey: codingKey)
        case let value as [Any]:
            let dummy = DummyArray(arr: value)
            try dictContainer.encode(dummy, forKey: codingKey)
        case let value as [String: Any]:
            try encodeJSONEscapeHatch(
                encoder: encoder,
                container: dictContainer,
                key: MyCodingKey(stringValue: key)!,
                dictionary: value
            )
        default:
            throw DecodingError.typeMismatch(
                type(of: value),
                DecodingError.Context(
                    codingPath: container.codingPath,
                    debugDescription: "Found a value that can't be mapped: (value)"
                )
            )
        }
    }
}


struct AnyEncodable: Encodable {
    private let _encode: (Encoder) throws -> Void
    init<T: Encodable>(_ wrapped: T) {
        _encode = wrapped.encode
    }

    func encode(to encoder: Encoder) throws {
        try _encode(encoder)
    }
}

private struct DummyArray: Encodable {
    var arr: [Any]

    func encode(to encoder: any Encoder) throws {
        var container = encoder.singleValueContainer()
        let arrayEncodable = try self.arr.map { element in
            if let element = element as? Encodable {
                return AnyEncodable(element)
            }
            throw DecodingError.typeMismatch(
                type(of: element),
                DecodingError.Context(
                    codingPath: container.codingPath,
                    debugDescription: "Found a non-encodable element: (element)"
                )
            )
        }
        try container.encode(arrayEncodable)
    }
}

struct MyCodingKey: CodingKey {
    let stringValue: String

    init?(stringValue: String) {
        self.stringValue = stringValue
        self.intValue = nil
    }

    let intValue: Int?
    init?(intValue: Int) {
        self.intValue = intValue
        self.stringValue = ""
    }
}

    Usage:

    struct RootWithEscapeHatch: Encodable {
    let a: String
    let b: [String: Any]

    private enum Fields: CodingKey {
        case a
        case b
    }

    func encode(to encoder: any Encoder) throws {
        var container = encoder.container(keyedBy: Fields.self)
        try container.encode(self.a, forKey: .a)
        try encodeJSONEscapeHatch(
            encoder: encoder,
            container: container,
            key: Fields.b,
            dictionary: self.b
        )
    }
}

let myTree = RootWithEscapeHatch(a: "xyz", b: ["c": 2, "d": false])
do {
    let data = try JSONEncoder().encode(myTree)
    print(String(decoding: data, as: UTF8.self))
} catch {
    print("Could not convert encodable with escape hatch to JSON: (error)")
}

    Prints:

    {"a":"xyz","b":{"c":2,"d":false}}


  2. 0

    🎮 Experimental solution:

    ⚠️ I don’t recommend this in production but it may help you come up with an idea 🙂

    Use JSONSerialization to encode the dictionary to a json string.

    func encode(to encoder: any Encoder) throws {
    var container = encoder.container(keyedBy: Fields.self)
    try container.encode(self.a, forKey: .a)
    let data = try JSONSerialization.data(withJSONObject: b)
    try container.encode(String(data: data, encoding: .utf8), forKey: .b)
}

    This will cause the b value to be escaped again! So remove the escaping from the result (not like this):

    print(String(decoding: data, as: UTF8.self).replacingOccurrences(of: "\"", with: """))

    Result:

    {"b":"{"c":2,"d":false}","a":"xyz"}

    💡 Path to the redemption:

    There are only 2 situations:

    1. You know the schema -> Use Encoder
    2. You don’t know the schema -> Use Serializer

    So maybe you needed something like the following pseudo code:

    func encodedData() throws -> Data {
  return switch wrappedValue {
  case let .knownSchema(myWellKnownObject): try JSONEncoder().encode(myWellKnownObject)
  case let .unknownSchema(myUnknownObject): try JSONSerialization.data(withJSONObject: myUnknownObject)
}

    I hope this gives you some idea

    Login or Signup to reply.
  3. 0

    You can use a custom JSONValue type instead of [String: Any]:

    enum JSONValue {
  case none
  case bool(Bool)
  case int(Int)
  case double(Double)
  case string(String)
  indirect case array([JSONValue])
  indirect case object([String: JSONValue])
}

    make it Encodable:

    extension JSONValue: Encodable {
  private struct CodingKeys: CodingKey {
    var stringValue: String
    init(stringValue: String) {
      self.stringValue = stringValue
    }

    var intValue: Int? = nil
    init?(intValue: Int) { return nil }
  }

  func encode(to encoder: Encoder) throws {
    switch self {
    case .none:
      var container = encoder.singleValueContainer()
      try container.encodeNil()
    case .bool(let value):
      var container = encoder.singleValueContainer()
      try container.encode(value)
    case .int(let value):
      var container = encoder.singleValueContainer()
      try container.encode(value)
    case .double(let value):
      var container = encoder.singleValueContainer()
      try container.encode(value)
    case .string(let value):
      var container = encoder.singleValueContainer()
      try container.encode(value)
    case .array(let values):
      var container = encoder.unkeyedContainer()
      for value in values { try container.encode(value) }
    case .object(let dictionary):
      var container = encoder.container(keyedBy: CodingKeys.self)
      for (key, value) in dictionary {
        try container.encode(value, forKey: .init(stringValue: key))
      }
    }
  }
}

    and, optionally, easier to type by conforming it to the various ExpressibleBy…Literal protocols:

    extension JSONValue: ExpressibleByNilLiteral {
  init(nilLiteral: ()) {
    self = .none
  }
}

extension JSONValue: ExpressibleByBooleanLiteral {
  init(booleanLiteral value: BooleanLiteralType) {
    self = .bool(value)
  }
}

extension JSONValue: ExpressibleByIntegerLiteral {
  init(integerLiteral value: IntegerLiteralType) {
    self = .int(value)
  }
}

extension JSONValue: ExpressibleByFloatLiteral {
  init(floatLiteral value: FloatLiteralType) {
    self = .double(value)
  }
}

extension JSONValue: ExpressibleByStringLiteral {
  init(stringLiteral value: StringLiteralType) {
    self = .string(value)
  }
}

extension JSONValue: ExpressibleByArrayLiteral {
  init(arrayLiteral elements: JSONValue...) {
    self = .array(elements)
  }
}

extension JSONValue: ExpressibleByDictionaryLiteral {
  init(dictionaryLiteral elements: (String, JSONValue)...) {
    self = .object(.init(uniqueKeysWithValues: elements))
  }
}

    With JSONValue you can then get what you’re looking for as follows:

    struct RootWithEscapeHatch: Encodable {
  let a: String
  let b: JSONValue
}

let tree = RootWithEscapeHatch(a: "xyz", b: ["c": 2, "d": false])
try print(String(data: JSONEncoder().encode(tree), encoding: .utf8)!)

    It is more generale than providing [String: Any] since you now can place anything as JSONValue, e.g. nil, scalar types or eterogeneous arrays.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search