Serialization

Jackson serialization

In the Kalix Java SDK, you do not need to create serializers for the messages, events, or the state of Kalix components. The same is true for Kalix endpoints. Kalix exposes the inputs and outputs of endpoints as JSON, but you need to make them serializable with Jackson. There are two ways to do this.

If you are using Java record then no annotation is needed. It just works. It’s as simple as using record instead of class. Kalix leverages Jackson under the hood and makes these records serializable for you.
If you are using Java class then you need to annotate them with the proper Jackson annotation.

Kalix uses a predefined Jackson configuration, for serialization. Use the JsonSupport utility to update the ObjectMapper with your custom requirements. To minimize the number of Jackson annotations, Java classes are compiled with the -parameters flag.

public class Main {
  public static void main(String[] args) {
    JsonSupport.getObjectMapper()
      .configure(FAIL_ON_NULL_CREATOR_PROPERTIES, true); (1)
    SpringApplication.run(Main.class, args);
  }
}

1	Sets custom `ObjectMapper` configuration.

Type name

It’s highly recommended to add a @TypeName annotation to all persistent classes: entity states, events, Workflow step inputs/results. Information about the type, persisted together with the JSON payload, is used to deserialize the payload and to route it to an appropriate Subscription or View handler. By default, a FQCN is used, which requires extra attention in case of renaming or repacking. Therefore, we recommend using a logical type name to simplify refactoring tasks. Migration from the old name is also possible, see renaming class.

Schema evolution

When using Event Sourcing, but also for rolling updates, schema evolution becomes an important aspect of your application development. A production-ready solution should be able to update any persisted models. The requirements as well as our own understanding of the business domain may (and will) change over time.

Removing a field

Removing a field can be done without any migration code. The Jackson serializer will ignore properties that do not exist in the class.

Adding an optional field

Adding an optional field can be done without any migration code. The default value will be Optional.empty or null if the field is not wrapped with an Optional type.

Old class:

record NameChanged(String newName) implements CustomerEvent {
}

New class with optional oldName and nullable reason.

record NameChanged(String newName, Optional<String> oldName, String reason) implements CustomerEvent {
}

Adding a mandatory field

Let’s say we want to have a mandatory reason field. Always set to a some (non-null) value. One solution could be to override the constructor, but with more complex and nested types, this might quickly become a hard to follow solution.

Another approach is to use the JsonMigration extension that allows you to create a complex migration logic based on the payload version number.

public class NameChangedMigration extends JsonMigration { (1)

  @Override
  public int currentVersion() {
    return 1; (2)
  }

  @Override
  public JsonNode transform(int fromVersion, JsonNode json) {
    if (fromVersion < 1) { (3)
      ObjectNode objectNode = ((ObjectNode) json);
      objectNode.set("reason", TextNode.valueOf("default reason")); (4)
    }
    return json; (5)
  }
}

1	Migration must extend `JsonMigration` class.
2	Sets current version number. The first version, when no migration was used, is always 0. Increase this version number whenever you perform a change that is not backwards compatible without migration code.
3	Implements the transformation of the old JSON structure to the new JSON structure.
4	The JsonNode is mutable, so you can add and remove fields, or change values. Note that you have to cast to specific sub-classes such as `ObjectNode` and `ArrayNode` to get access to mutators.
5	Returns updated JSON matching the new class structure.

The migration class must be linked to the updated model with the @Migration annotation.

@Migration(NameChangedMigration.class) (1)
record NameChanged(String newName, Optional<String> oldName, String reason) implements CustomerEvent {
}

1	Links the migration implementation with the updated event.

Renaming a field

Renaming a field is a very similar migration.

Old class:

record AddressChanged(Address address) implements CustomerEvent {
}

New class:

@Migration(AddressChangedMigration.class)
record AddressChanged(Address newAddress) implements CustomerEvent {
}

The migration implementation:

public class AddressChangedMigration extends JsonMigration {

  @Override
  public int currentVersion() {
    return 1;
  }

  @Override
  public JsonNode transform(int fromVersion, JsonNode json) {
    if (fromVersion < 1) {
      ObjectNode objectNode = ((ObjectNode) json);
      JsonNode oldField = json.get("address"); (1)
      objectNode.set("newAddress", oldField); (2)
      objectNode.remove("address"); (3)
    }
    return json;
  }
}

1	Finds the old `address` field.
2	Updates the JSON tree with the `newAddress` field name.
3	Removes the old field.

Changing the structure

Old class:

record CustomerCreated(String email, String name, String street, String city) implements CustomerEvent {
}

New class with the Address type:

record CustomerCreated(String email, String name, Address address) implements CustomerEvent {
}

The migration implementation:

public class CustomerCreatedMigration extends JsonMigration {

  @Override
  public int currentVersion() {
    return 1;
  }

  @Override
  public JsonNode transform(int fromVersion, JsonNode json) {
    if (fromVersion == 0) {
      ObjectNode root = ((ObjectNode) json);
      ObjectNode address = root.with("address"); (1)
      address.set("street", root.get("street"));
      address.set("city", root.get("city"));
      root.remove("city");
      root.remove("street");
    }
    return json;
  }
}

1	Creates a new nested JSON object, with the data from the old schema.

Renaming class

Renaming the class doesn’t require any additional work when @TypeName annotation is used. For other cases, the JsonMigration implementation can specify all supported class names.

public class AddressChangedMigration extends JsonMigration {

  @Override
  public int currentVersion() {
    return 1;
  }


  @Override
  public List<String> supportedClassNames() {
    return List.of("customer.domain.CustomerEvent$CustomerAddressChanged"); (1)
  }

}

1	Specifies the old event name.

Testing

It’s highly recommended to cover all schema changes with unit tests. In most cases it won’t be possible to reuse the same class for serialization and deserialization, since the model is different from version 0 to version N. One solution could be to create a byte array snapshot of each version and save it as a Base64 test variable.

Any serialized = JsonSupport.encodeJson(
    new CustomerCreated("bob@lightbend.com", "bob", "Wall Street", "New York"));

new String(Base64.getEncoder().encode(serialized.toByteArray()));

Test example:

@Test
public void shouldDeserializeCustomerCreated_V0() throws InvalidProtocolBufferException {
  String encodedBytes = "Cktqc29uLmthbGl4LmlvL2N1c3RvbWVyLmRvbWFpbi5zY2hlbWFldm9sdXRpb24uQ3VzdG9tZXJFdmVudCRDdXN0b21lckNyZWF0ZWQSVQpTeyJlbWFpbCI6ImJvYkBsaWdodGJlbmQuY29tIiwibmFtZSI6ImJvYiIsInN0cmVldCI6IldhbGwgU3RyZWV0IiwiY2l0eSI6Ik5ldyBZb3JrIn0=";
  byte[] bytes = Base64.getDecoder().decode(encodedBytes.getBytes()); (1)
  Any serializedAny = Any.parseFrom(ByteString.copyFrom(bytes)); (2)

  CustomerEvent.CustomerCreated deserialized = JsonSupport.decodeJson(CustomerEvent.CustomerCreated.class,
      serializedAny); (3)

  assertEquals("Wall Street", deserialized.address().street());
  assertEquals("New York", deserialized.address().city());
}

1	Decodes Base64 bytes.
2	Parses bytes into `Any` object.
3	Verifies JSON deserialization.