Implementing Views

You can access a single Entity with its Entity key. But sometimes this is not enough. You might want to retrieve multiple Entities, or retrieve them using an attribute other than the key. Kalix Views allow you achieve this. You can create each View, so each one is optimized for an specific query.

Views can be defined from any of the following:

The remainder of this page describes:

Be aware that Views are not updated immediately when Entity state changes. Kalix does update Views as quickly as possible. It is not instant but eventually all changes will become visible in the query results. View updates might also take more time during failure scenarios than during normal operation.

Creating a View from a Value Entity

Consider an example of a Customer Registry service with a customer Value Entity. When customer state changes, the entire state is emitted as a value change. Value changes update any associated Views. To create a View that lists customers by their name.

Define the View for a service that selects customers by name and associates a table name with the View. The table is created and used by Kalix to store the View.

This example assumes the following Customer exists:

src/main/java/customer/api/Customer.java
public record Customer(String customerId, String email, String name, Address address) {

  public Customer withName(String newName){
    return new Customer(customerId, email, newName, address);
  }

  public Customer withAddress(Address newAddress){
    return new Customer(customerId, email, name, newAddress);
  }
}

As well as a Value Entity service CustomerEntity.java that will produce the state changes consumed by the View. You can consult Value Entity documentation on how to create such an entity if in need.

Define the View

You implement a View by extending kalix.javasdk.view.View and subscribing to changes from an entity. You specify how to query it by providing a method annotated with @Query, which is then made accessible via REST annotations.

src/main/java/customer/view/CustomerByNameView.java
import customer.api.Customer;
import customer.api.CustomerEntity;
import kalix.javasdk.view.View;
import kalix.springsdk.annotations.Query;
import kalix.springsdk.annotations.Subscribe;
import kalix.springsdk.annotations.Table;
import kalix.springsdk.annotations.ViewId;
import org.springframework.web.bind.annotation.GetMapping;

@ViewId("view_customers_by_name") (1)
@Table("customers_by_name")  (2)
@Subscribe.ValueEntity(CustomerEntity.class) (3)
public class CustomerByNameView extends View<Customer> { (4)

  @GetMapping("/customer/by_name/{customer_name}")   (5)
  @Query("SELECT * FROM customers_by_name WHERE name = :customer_name") (6)
  public Customer getCustomer(String name) {
    return null; (7)
  }
}
1 defining view ID
2 defining table name
3 subscribing to CustomerEntity
4 extending from View
5 defining endpoint
6 defining the query
7 note that no return is needed
Note that the return value of the method is null. You may ask yourself, how is it that the endpoint respond with any Customer at all?. When you call this endpoint, it first hits the proxy, which calls directly to the database. When the proxy receives the response, it sends it directly to you without any further intervention from the View. Therefore, null is valid as a return value in the endpoint of a View. The choice of null is our way to make clear that the response doesn’t come from the return of this method. But you can choose any response you like as long is compatible with the return type.
Adding a view ID to your View allows you to refactor the name of the class later on without the risk of losing the view. If you don’t define a view ID the class name becomes its ID. Therefore, if you change the name of your class afterwards Kalix will not recognize this new name as the same view and will create a brand-new view. This is resource consuming for a view from an Event Sourced Entity because it will reprocess all the events of that entity to rebuild it. While for a view built from a topic you can lose all the previous events because depending on the topic configuration you may only process events from current time forwards. Last but not least, it’s also a problem for Value Entities because it will need to index again them when grouping them by some value.

Using a transformed model

Often, you will want to transform the entity model to which the view is subscribing into a different representation. To do that, let’s have a look at the example in which we store a summary of the Customer used in the previous section instead of the original one:

src/main/java/com/example/api/CustomerSummary.java
public record CustomerSummary(String id, String name) { }

In this scenario, the view state should be of type CustomerSummary and you will need to handle and transform the incoming state changes into it, as shown below:

src/main/java/com/example/api/CustomerSummaryByName.java
@Table("customers")
public class CustomerSummaryByName extends View<CustomerSummary> { (1)

  @Subscribe.ValueEntity(CustomerEntity.class) (2)
  public UpdateEffect<CustomerSummary> onChange(Customer customer) { (3)
    return effects()
        .updateState(new CustomerSummary(customer.email(), customer.name())); (4)
  }
  @GetMapping("/summary/by_name/{customerName}")   (5)
  @Query("SELECT * FROM customers WHERE name = :customerName") (6)
  public CustomerSummary getCustomer() { (7)
    return null;
  }
}
1 View is of type CustomerSummary.
2 @Subscribe annotation is at method level rather than at class level.
3 Annotated method needs to handle the state changes of the entity being subscribed to.
4 Transform Customer into CustomerSummary.
5 Define route to this view.
6 Define the query matching by name.
7 Query method returns a CustomerSummary.

Handling Value Entity deletes

Value Entities can be deleted. We can update our view model based on that fact with an additional flag handleDeletes for the subscription.

src/main/java/com/example/api/CustomerSummaryByName.java
  @Subscribe.ValueEntity(value = CustomerEntity.class, handleDeletes = true) (1)
  public UpdateEffect<CustomerSummary> onDelete() { (2)
    return effects()
        .deleteState(); (3)
  }
1 @Subscribe annotation with handleDeletes=true.
2 Dedicated (parameter-less) handler for deletion.
3 An effect to delete the view state effects().deleteState(). It could be also an update of a special column, to mark view state as deleted.
When using @Subscribe on a class level, handleDeletes=true will also work. Kalix will automatically delete the View state.

Creating a View from an Event Sourced Entity

You can create a View from an Event Sourced Entity by using events that the Entity emits to build a state representation. Using a Customer Registry service example, to create a View for querying customers by name:

This example assumes a Customer equal to the previous example and an Event Sourced Entity that uses this Customer, and it is in charge of producing the events that update the View. These events are defined as subtypes of the class CustomerEvent following standard Jackson notation like this:

src/main/java/customer/domain/CustomerEvent.java
import kalix.springsdk.annotations.TypeName;
import static customer.domain.CustomerEvent.*;

public sealed interface CustomerEvent {

  @TypeName("customer-created") (1)
  record CustomerCreated(String email, String name, Address address) implements CustomerEvent {}

  @TypeName("name-changed")
  record NameChanged(String newName) implements CustomerEvent {}

  @TypeName("address-changed")
  record AddressChanged(Address address) implements CustomerEvent {}
}
1 Includes the logical type name using @TypeName annotation.
It’s highly recommended to add a @TypeName to your persisted events. Kalix needs to identify each event in order to deliver them to the right event handlers. If no logical type name is specified, Kalix uses the non-qualified class name. Once an event is persisted in Kalix, you won’t be able to rename your class if no logical type name has been specified, as Kalix won’t be able to recognize previously persisted events. Therefore, we recommend all event classes use a logical type name.

Define the View to consume events

The definition of the view for an Event Sourced Entity is the same as for a Value Entity. However, in this example, the subscription is at the method level rather than the type level. The advantage of this approach is that you can create multiple methods to handle different events. It is recommended you add a view ID to your view.

Every time an event is processed by the view, the state of the view can be updated. You can do this with the .updateState method, which is available through the effects() API. Here you can see how the View is updated with a new name:

src/main/java/customer/view/CustomerByNameView.class
import customer.api.CustomerEntity;
import customer.domain.CustomerEvent;
import kalix.javasdk.view.View;
import kalix.springsdk.annotations.Query;
import kalix.springsdk.annotations.Subscribe;
import kalix.springsdk.annotations.Table;
import kalix.springsdk.annotations.ViewId;
import reactor.core.publisher.Flux;

import org.springframework.web.bind.annotation.GetMapping;

@ViewId("view_customers_by_name") (1)
@Table("customers_by_name")
public class CustomerByNameView extends View<CustomerView> {

  @GetMapping("/customer/by_name/{customer_name}")
  @Query("SELECT * FROM customers_by_name WHERE name = :customer_name")
  public Flux<CustomerView> getCustomer(String name) {
    return null;
  }

  @Subscribe.EventSourcedEntity(CustomerEntity.class)
  public UpdateEffect<CustomerView> onEvent(CustomerEvent.CustomerCreated created) {
    return effects().updateState(new CustomerView(created.email(), created.name(), created.address()));
  }

  @Subscribe.EventSourcedEntity(CustomerEntity.class)
  public UpdateEffect<CustomerView> onEvent(CustomerEvent.NameChanged event) {
    return effects().updateState(viewState().withName(event.newName())); (2)
  }

  @Subscribe.EventSourcedEntity(CustomerEntity.class)
  public UpdateEffect<CustomerView> onEvent(CustomerEvent.AddressChanged event) {
    return effects().updateState(viewState().withAddress(event.address()));
  }
}
1 setting view ID
2 updating the state of the view with the new name

An Event Sourced entity can emit many types of events. You need to define a method for each event type. They return an UpdateEffect, which describes next processing actions, such as updating the view state.

See Query syntax reference for more examples of valid query syntax.

Ignoring events

When consuming events, each event must be matched by a View service method. In case your View is only interested in certain events:

  1. You can add event handlers for all of them and return Effect.ignore for those you are not interested.

  2. You can add ignoreUnknown = true to your @Subcribe annotation but only if it is a type level annotation. This works in a View the same way as in an Action. Check out this example in type level subscribing for an action.

If there is no handler for an incoming event and there is no ignoreUnknown = true at type level, the View will fail. Views are designed to restart, but since it can’t process the event, the view will keep failing trying to reprocess it.

Creating a View from a topic

The source of a View can be a topic. It works the same way as shown in Creating a View from an Event Sourced Entity or Creating a View from a Value Entity, but you define it with @Subscribe.Topic instead.

How to transform results

When creating a View, you can transform the results as a relational projection instead of using a SELECT * statement.

Relational projection

Instead of using SELECT * you can define which columns will be used in the response message. So, if you want to use a CustomerSummary used on the previous section.

You will need to define your entity as this:

src/main/java/com/example/api/CustomersStreamByName.java
@Table("customers")
@Subscribe.ValueEntity(CustomerEntity.class)
public class CustomersStreamByName extends View<Customer> { (1)

  @GetMapping("/summary/by_name/{customerName}")   (2)
  @Query(
      value = "SELECT customerId AS id, name FROM customers WHERE name = :customerName", (3)
      streamUpdates = true) (4)
  public Flux<CustomerSummary> getCustomer() { (5)
    return null;
  }
}
1 View state type is the original Customer as shown at the beginning of this section.
2 Query is mapped to an external route as usual receiving a customerName as parameter.
3 Note the renaming from customerId as id on the query, as id and name match the record CustomerSummary.
4 Since this query can return multiple results, mark it as streaming updates.
5 Return type of the query is Flux<CustomerSummary>.
In the example, when not interested in having multiple results, the streamUpdates flag could be removed and the return type would be only CustomerSummary.

In a similar way, you can include values from the request in the response, for example :requestId:

SELECT :requestId, customerId as id, name FROM customers
WHERE name = :customerName

Response message including the result

Instead of streamed results you can include the results in a Collection field in the response object:

src/main/java/com/example/api/CustomersResponse.java
public record CustomersResponse(Collection<Customer> results) { }
src/main/java/com/example/api/CustomersResponseByName.java
@Subscribe.ValueEntity(CustomerEntity.class)
public class CustomersResponseByName extends View<Customer> { (1)

  @GetMapping("/wrapped/by_name/{customerName}")   (2)
  @Query("SELECT * AS results FROM customers_by_name WHERE name = :customerName") (3)
  public CustomersResponse getCustomer() { (4)
    return null;
  }
}
1 View state type is the original Customer as shown at the beginning of this section.
2 Query is mapped to an external route as usual receiving a customerName as parameter.
3 Note the use of * AS results so records are matched to the collection in CustomersResponse.
4 Return type of the query is CustomersResponse.

How to modify a View

Kalix creates indexes for the View based on the query. For example, the following query will result in a View with an index on the name column:

SELECT * FROM customers WHERE name = :customer_name

You may realize after a deployment that you forgot adding some parameters to the query Parameters that aren’t exposed to the endpoint of the View. After adding these parameters the query is changed and therefore Kalix needs to add indexes for these new columns. For example, changing the above query to filter to add users that are active would mean that Kalix needs to build a View with the index on the is-active column.

SELECT * FROM customers WHERE name = :customer_name AND is-active = true

Such changes require you to define a new View. Kalix will then rebuild it from the source event log or value changes.

Views from topics cannot be rebuilt from the source messages, because it might not be possible to consume all events from the topic again. The new View is built from new messages published to the topic.

Rebuilding a new View may take some time if there are many events that have to be processed. The recommended way when changing a View is multi-step, with two deployments:

  1. Define the new View with a new @ViewId, and keep the old View intact.

  2. Deploy the new View, and let it rebuild. Verify that the new query works as expected. The old View can still be used.

  3. Remove the old View but keep its @GetMapping path, and use it in the new View.

  4. Deploy the second change.

The View definitions are stored and validated when a new version is deployed. There will be an error message if the changes are not compatible.

Streaming view updates

A query can provide a near real time stream of results for the query, emitting new entries matching the query as they are added or updated in the view.

This will first list the complete result for the query and then keep the response stream open, emitting new or updated entries matching the query as they are added to the view. The stream does not complete until the client closes it.

This is not intended as transport for service to service propagation of updates and it does not guarantee delivery. For such use cases you should instead publish events to a topic, see Publishing and Subscribing with Actions

Query syntax reference

Define View queries in a language that is similar to SQL. The following examples illustrate the syntax. To retrieve:

  • All customers without any filtering conditions (no WHERE clause):

    SELECT * FROM customers
  • Customers with a name matching the customer_name property of the request message:

    SELECT * FROM customers WHERE name = :customer_name
  • Customers matching the customer_name AND city properties of the request message:

    SELECT * FROM customers WHERE name = :customer_name AND address.city = :city
  • Customers in a city matching a literal value:

    SELECT * FROM customers WHERE address.city = 'New York'

Filter predicates

Use filter predicates in WHERE conditions to further refine results.

Comparison operators

The following comparison operators are supported:

  • = equals

  • != not equals

  • > greater than

  • >= greater than or equals

  • < less than

  • <= less than or equals

Logical operators

Combine filter conditions with the AND operator, and negate using the NOT operator. Group conditions using parentheses.

OR support is currently disabled, until it can be more efficiently indexed.
SELECT * FROM customers WHERE
  name = :customer_name AND NOT (address.city = 'New York' AND age > 65)

Array operators

Use IN or = ANY to check whether a value is contained in a group of values or in an array column or parameter (a repeated field in the Protobuf message).

Use IN with a list of values or parameters:

SELECT * FROM customers WHERE email IN ('bob@example.com', :some_email)

Use = ANY to check against an array column (a repeated field in the Protobuf message):

SELECT * FROM customers WHERE :some_email = ANY(emails)

Or use = ANY with a repeated field in the request parameters:

SELECT * FROM customers WHERE email = ANY(:some_emails)

Pattern matching

Use LIKE to pattern match on strings. The standard SQL LIKE patterns are supported, with _ (underscore) matching a single character, and % (percent sign) matching any sequence of zero or more characters.

SELECT * FROM customers WHERE name LIKE 'Bob%'
For index efficiency, the pattern must have a non-wildcard prefix or suffix. A pattern like '%foo%' is not supported. Given this limitation, only constant patterns with literal strings are supported; patterns in request parameters are not allowed.

Use the text_search function to search text values for words, with automatic tokenization and normalization based on language-specific configuration. The text_search function takes the text column to search, the query (as a parameter or literal string), and an optional language configuration.

text_search(<column>, <query parameter or string>, [<configuration>])

If the query contains multiple words, the text search will find values that contain all of these words (logically combined with AND), with tokenization and normalization automatically applied.

The following text search language configurations are supported: 'danish', 'dutch', 'english', 'finnish', 'french', 'german', 'hungarian', 'italian', 'norwegian', 'portuguese', 'romanian', 'russian', 'simple', 'spanish', 'swedish', 'turkish'. By default, a 'simple' configuration will be used, without language-specific features.

SELECT * FROM customers WHERE text_search(profile, :search_words, 'english')
Text search is currently only available for deployed services, and can’t be used in local testing.

Data types

The following data types are supported, for their corresponding Protobuf types. Arrays are created for a repeated field in a Protobuf message. Timestamps can be stored and compared using the google.protobuf.Timestamp message type.

Data type Protobuf type

Text

string

Integer

int32

Long (Big Integer)

int64

Float (Real)

float

Double

double

Boolean

bool

Byte String

bytes

Array

repeated fields

Timestamp

google.protobuf.Timestamp

Optional fields

Fields in a Protobuf message that were not given a value are handled as [the default value](https://developers.google.com/protocol-buffers/docs/proto3#default) of the field data type.

In some use cases it is important to explicitly express that a value is missing, doing that in a view column can be done in three ways:

  • mark the message field as optional

  • use one of the Protobuf "wrapper" types for the field (messages in the package google.protobuf ending with Value)

  • make the field a part of a nested message and omit that whole nested message, for example address.street where the lack of an address message implies there is no street field.

Optional fields with values present can be queried just like regular view fields:

SELECT * FROM customers WHERE phone_number = :number

Finding results with missing values can be done using IS NULL:

SELECT * FROM customers WHERE phone_number IS NULL

Finding entries with any value present can be queried using IS NOT NULL:

SELECT * FROM customers WHERE phone_number IS NOT NULL

Optional fields in query requests messages are handled like normal fields if they have a value, however missing optional request parameters are seen as an invalid request and lead to a bad request response.

Sorting

Results for a view query can be sorted. Use ORDER BY with view columns to sort results in ascending (ASC, by default) or descending (DESC) order.

If no explicit ordering is specified in a view query, results will be returned in the natural index order, which is based on the filter predicates in the query.

SELECT * FROM customers WHERE name = :name AND age > :min_age ORDER BY age DESC
Some orderings may be rejected, if the view index cannot be efficiently ordered. Generally, to order by a column it should also appear in the WHERE conditions.

Paging

Splitting a query result into one "page" at a time rather than returning the entire result at once is possible in two ways:

  • with a count based offset or

  • a token based offset.

In both cases OFFSET and LIMIT are used.

OFFSET specifies at which offset in the result to start

LIMIT specifies a maximum number of results to return

Count based offset

The values can either be static, defined up front in the query:

SELECT * FROM customers LIMIT 10

Or come from fields in the request message:

SELECT * FROM customers OFFSET :start_from LIMIT :max_customers

Note: Using numeric offsets can lead to missing or duplicated entries in the result if entries are added to or removed from the view between requests for the pages.

Token based offset

The count based offset requires that you keep track of how far you got by adding the page size to the offset for each query.

An alternative to this is to use a string token emitted by Kalix identifying how far into the result set the paging has reached using the functions next_page_token() and page_token_offset().

When reading the first page, an empty token is provided to page_token_offset. For each returned result page a new token that can be used to read the next page is returned by next_page_token(), once the last page has been read, an empty token is returned (see also has_more for determining if the last page was reached).

The size of each page can optionally be specified using LIMIT, if it is not present a default page size of 100 is used.

With the query return type like this:

public record Response(List<Customer> customers, string next_page_token) { }

A query such as the one below will allow for reading through the view in pages, each containing 10 customers:

SELECT * AS customers, next_page_token() AS next_page_token
FROM customers
OFFSET page_token_offset(:page_token)
LIMIT 10

The token value is not meant to be parseable into any meaningful information other than being a token for reading the next page.

Total count of results

To get the total number of results that will be returned over all pages, use COUNT(*) in a query that projects its results into a field. The total count will be returned in the aliased field (using AS) or otherwise into a field named count.

SELECT * AS customers, COUNT(*) AS total, has_more() AS more FROM customers LIMIT 10

Check if there are more pages

To check if there are more pages left, you can use the function has_more() providing a boolean value for the result. This works both for the count and token based offset paging, and also when only using LIMIT without any OFFSET:

SELECT * AS customers, has_more() AS more_customers FROM customers LIMIT 10

This query will return more_customers = true when the view contains more than 10 customers.