Querying JSON

Here is a task that I have commonly had, but never felt happy about; I need to dig into a heterogenous, deeply-nested structure and extract a value, where that value or any of it’s intermediate ancestors might be missing.

Anyone who’s ever written Javascript code like json["config"]["addresses"][0], will know what I’m talking about. That little snippet of code should make you feel uncomfortable. Any one of the values passed to the subscript might result in null being returned and everything breaking.

As an experiment, I’m going to attempt to write some code that’ll help me accomplish the task of safely extracting values from a nested JSON structure. We’ll be doing it in Swift.

Encoding JSON

We could just use a Dictionary<String, Any> as our JSON value, but since JSON only supports a small number of types, we can easily encode them in a custom type.

enum JSON {
  case array([JSON])
  case object([String: JSON])
  case number(Int)
  case string(String)
  case bool(Bool)
}

For now I’m only encoding Int in the JSON just to keep my examples simple. Here is how we might manually construct a value using this type.

let json: JSON = .object(["foo": .array(.bool(false), .bool(true))])

This corresponds to this JSON literal.

{
  "foo": [
    false,
    true
  ]
}

Not very nice to write, but I’m sure you can imagine how this JSON value might be constructed by parsing a String or converting a Dictionary<String, Any>.

Asking Questions

Now we’ve got a nice type for our JSON, we can ask questions of it. Imagine we want to verify that we have an object first. We can define a function which does this for us.

func object(_ input: JSON) -> JSON? {
  switch input {
  case .object: return input
  default: return nil
  }
}

It returns the value if it is a JSON object or otherwise returns nil. We can also define others for array, number, bool and string. Not that interesting so far, but given JSON’s heterogenous nature, verifying the type of values is important for querying.

Next step is to add some functions which allow us to peer inside of structures. Here are two for indexing into objects and arrays.

func key(_ key: String) -> (JSON) -> JSON? {
  return { input in
    switch object(input) {
    case let .some(.object(dict)):
      return dict[key]
    default:
      return nil
    }
  }
}

func index(_ index: Int) -> (JSON) -> JSON? {
  return { input in
    switch array(input) {
      case let .some(.array(array)):
        if index > array.count - 1 {
          return nil
        }
        else {
          return array[index]
        }
      default:
        return nil
    }
  }
}

These take the value to index by and return a new function which tests the JSON type and attempts to extract the value at the index. You might wonder why return a function, but that will be clear later!

Let’s have a go at using these.

// {"foo": {"bar": [1]}}
let json: JSON = .object(["foo": .object(["bar":.array([.number(1)])])])

func extract(_ json: JSON) -> JSON? {
  guard let foo = key("foo")(json) else { return nil }
  guard let bar = key("bar")(foo) else { return nil }
  guard let first = index(0)(bar) else { return nil }
  return number(first)
}

Well. That sucks frankly. It’s not really any better than doing the lookup manually without these supposed helper functions. That’s where the next round comes in!

Composition is Close to Godliness

Since we have had the foresight to construct these helpers as bare functions, we now get to wield the flaming sword of composition. All we need is a way to bind two of these functions into one. It’s time the functional programmer’s second favourite tool after functions; custom operators!

typealias JSONQuery = (JSON) -> JSON?

infix operator >>: AdditionPrecedence
func >> (lhs: @escaping JSONQuery, rhs: @escaping JSONQuery) -> JSONQuery {
  return { input in
    guard let result = lhs(input) else { return nil }
    return rhs(result)
  }
}

The implementation is pretty simple. It takes two of our query functions and returns a new function, ensuring that the second only gets called with non-nil values from the first.

Let’s try it out on our previous example.

let extract = key("foo") >> key("bar") >> index(0) >> number

Heaps better! Now it should also be clear why the key and index functions return functions; it is entirely to enable this kind of composition, since Swift does not support partial-application.

This basic set of functions could also be expanded to include more clever lookups. For example we might want to check for numbers within a range, if a boolean value is actually true or only grab the prefix of an array. No matter how complex a lookup’s implementation is, they still compose in exactly the same way.

Bonus Round. FIGHT!

The full, executable code for this post is available as a gist. As a bonus, it also includes implementations of some ExpressibleBy* protocols for JSON values. These allow a JSON value to be constructed with String, Array, Dictionary, Boolean and Int literals.