After my previous blogpost on client side projections, Pedro Léon made the accurate observation that the direct projection approach is only feasible, performance wise, when working with a single aggregate.
To query across sets of projected view models, we need to add another pattern into the mix: the registry pattern.
Registry Pattern
A registry is a well-known object that other objects can use to find common objects and services.
In order to efficiently find objects, a registry typically needs to maintain a list of pointers to those objects, in other words: it needs to maintain indexes.
For maintaining indexes where the lookup value is well known, such as reference identifiers or date ranges, we can rely on the native indexing capabilities of IndexedDB, the browsers database.
IndexedDB requires exact values though. To perform searches with fuzzy values, such as names, we can use the excellent fuse.js library as an additional indexing mechanism.
Let's dive into some code samples for each of these options.
Indexing with IndexedDB
For the majority of use cases, the indexedDB setup I used for the Cache aside pattern will be sufficient to perform as a registry as well.
All you need to do is ensure that the correct indexes exist on the object store, so that queries can run efficiently.
upgrade(tx){
var rostersStore;
if (!this.db.objectStoreNames.contains("rosters")) {
rostersStore = this.db.createObjectStore("rosters", { keyPath: "rosterId" });
}
else{
rostersStore = tx.objectStore("rosters");
}
if(!rostersStore.indexNames.contains('organization')) {
rostersStore.createIndex("organization", "organizationId");
}
if(!rostersStore.indexNames.contains('group')) {
rostersStore.createIndex("group", "groupId");
}
}
Then run the projection after you've executed any commands and persist the results into the cache.
This will add or override the view models previously downloaded from the api as a result of the cache aside mechanism.
import rosters from "../../clubmanagement.roster.app.js";
var rosterId = guid.generate();
var roster = await app.repository.getRoster(rosterId);
roster.create(name);
roster.associate(this.organizationId, this.groupId, season);
await app.repository.persistRoster(roster);
await app.repository.flush();
// project into the cache/registry
var projected = await app.projection.projectRoster(rosterId);
await rosters.cache.persistRoster(projected);
Note: it is obviously very important that the projection logic on the client and the api are producing the exact same view model structures, so that data downloaded and data locally projected can actually be mixed without undesired consequences.
Querying with IndexedDB
Queries on the projected data can now be run using any of the supported IDBKeyRange key ranges on any of the indexes present. These include bound ranges, upper and lower bound ranges as well as exact matches.
async getRosters(organizationId){
return new Promise((resolve, reject) => {
var tx = this.db.transaction("rosters", "readonly");
var store = tx.objectStore("rosters")
var index = store.index("organization");
var range = IDBKeyRange.only(organizationId);
var rosters = [];
index.openCursor(range).onsuccess = (e) => {
var cursor = e.target.result;
if(cursor) {
rosters.push(cursor.value);
cursor.continue();
}
}
tx.onerror = () => {
reject();
}
tx.oncomplete = () => {
resolve(rosters);
};
});
}
Fuzzy search with Fuse.js
The IndexedDB IDBKeyRange methods get you quite far when it comes to searching for objects based on property values as long as you exactly know what the values are that you are looking for.
Humans however do not always exactly know what they are looking for and could use some fuzzy search logic to help them find entities by similarities (e.g. misspelled names, partial search terms, case insensitive matches etc...).
Secondly, IndexedDB cannot handle nested search scenarios very well either as you can only create indexes on top level properties or on properties of direct children (e.g. properties on children in child arrays are not supported).
For fuzzy, or nested search, I very much like the fuse.js library, which can perform these kinds of searches. Yet it requires an in memory dataset.
As the data set is in memory Fuse will only work well with small to medium sized datasets, so it's best to use it in combination with a result set loaded from specific indexes in IndexedDB.
Note: For large datasets I use server side search solutions, but for data sets up to a few thousand items fuse works like a charm.
Using Fuse.js
First import fuse, there is now a nice ESM module available, even for browsers, to import from.
Then define the configuration options to tell Fuse how fuzzy the matches can be.
Fuse has a wide range of options available to configure it's search behavior, you can find them all here.
Instantiate a fuse instance with the source data, this data is typically loaded from IndexedDB.
Finally perform the search. The search operation will return the items themselves associated with scoring metrics that indicate how well the items matched the search term.
import Fuse from "../fuse.esm.js";
async searchRosters(organizationId, term){
if(!this.fuse){
var all = await rosters.cache.getRosters(organizationId);
var options = {
shouldSort: true,
threshold: 0.3,
location: 0,
distance: 100,
maxPatternLength: 32,
minMatchCharLength: 1,
keys: [
"name"
]
};
this.fuse = new Fuse(all, options);
}
return this.fuse.search(term).map(r => r.item); // item contains the actual object, next to item there is a score in the resultset
}
Whenever you create a new fuse instance, it will build up an index in memory based on the input array that you provided it.
This can be an expensive operation and therefore it's best to cache the fuse instance as a member field in the registry.
To keep it's memory data structure up to date, you can use its remove and add functions in order to replace updated roster instances as part of the registries persist logic.
persistRoster(roster){
return new Promise((resolve, reject) => {
var tx = this.db.transaction("rosters", "readwrite");
var store = tx.objectStore("rosters");
store.put(roster);
tx.onerror = () => {
reject();
}
tx.oncomplete = () => {
resolve();
};
}).then(() => {
if(this.fuse){
this.fuse.remove(r => r.rosterId == roster.rosterId);
this.fuse.add(roster);
}
});
}
Rendering
The search results contain an item and score property. If you map the item property as I did above, rendering the search results works exactly the same as you would render the results from the IndexedDB queries.
Wrapup
With this article I think I've covered all client side aspects of developing offline capable progressive web applications using event sourcing. Don't hesitate to reach out to me if you want more info on a certain aspect.
For this approach to work however, the server side needs to accept replicated events, in addition to the traditional CQRS style api interface it needs to expose for public consumption.
I'll start covering the server side next time, see you then!
