Event Store Projections – Counting events of a specific type

Our first Event Store Projections example simply counts the number of events of a specific type. Pretty mundane, I know, but it’s a good place to start.

Projection-MeasurementRead

In this example, the projection will take all MeasurementRead events from all streams and increase a counter in its global state for each one. The end result is a global state containing the number of MeasurementRead events.

Projection-MeasurementReadCounter-Events

JavaScript implementation

The projection logic is implemented in a self-revealing module. This pattern allows us to decide which methods of a module should be accessible from outside, much like creating a class with public and private methods. The pattern is probably overkill for this projection, since it only has public methods, but it allows us to create unit tests for its logic:

The init method initializes the state for the first time by setting the count property to 0. The increase method increases the count property by 1 when called.

This module is instantiated once globally and then referenced from the projection’s definition:

The fromAll definition tells the engine that the projection’s events come from all streams. Inside, we delegate $init and MeasurementRead to our counter module implementation. The $init field defines the method that initializes the projection’s state, and the MeasurementRead field defines that all MeasurementRead events are to be handled by the counter.increase method.

producesResults is set to true so that the projection’s state is stored in a stream. This allows us to subscribe to projection state updates without polling and use the output of the projection as input for other projections, as I’ll demonstrate later. In the future, we can replace this with the outputState method.

Testing the projection

I used Jasmine to test this logic. Jasmine is a really nice BDD framework for testing JavaScript code that encourages developers to write readable test scenarios:

The test contains two cases: the first one verifies the initial state of the projection, and the second one verifies the count property is increased by 1 for each call of the increase method. The first two lines define the referenced JavaScript files, which enables Intellisense in Visual Studio and allows R# to run the tests from your IDE.

Querying the results with the Client API

To read the value of the projection, we use the getState method of the ProjectionsManager class.

Since we specified that the output of the state should be stored in a stream, we can subscribe to event updates without polling:

Source code

A working project for this example can be found on github: https://github.com/tim-cools/EventStore-Examples

Event Store Projections by Example

This post is part of a series:

  1. EventStore Client API Basics (C#)
  2. Counting events of a specific type
  3. Partition events based on data found in previous events
  4. Calculating an average per day
  5. The irresponsible gambler
  6. Distribute events to other streams
  7. Temporal Projection to generate alarms
  8. Projection in C#

Event Store Client API Basics (C#)

Here I’ll give a overview on how to use the Event Store Client API from C#. I’ll be highlighting the features used to implement the examples described in the following blog posts. If you’re only interested in projections written in JavaScript, you can safely skip this post and go directly to the first example.

Running Event Store

These examples are based on the latest main branch version available at the time of writing (RC3.0). Since you have to run the Client API against the same version of Event Store, I’ve included the binaries in the github repository. This means you can check out the examples and run them without worrying about version compatibility or building Event Store yourself.

To run the examples, run Event Store as an administrator with –run-projections=all. The following command will run Event Store with projections enabled and an in-memory database:

Event Store Client API Connections

Event Store is accessible through two protocols: HTTP and TCP. The default HTTP endpoint is port 1113 and the default TCP endpoint to 2113, though both of these can be modified from the command line using –tcp-port and –http-port:

Creating a connection

The Event Store Client API  connection is accessible through the IEventStoreConnection interface and connects to the TCP endpoint:

Most connection methods accept credentials for connecting to Event Store. These examples always use the default admin account:

Converting events to and from JSON

These examples use static-typed events which are serialized to JSON when they are sent to Event Store. When they are retrieved from the database, they are deserialized from JSON into static-typed events. Two extension methods take care of this process:

These methods use JSON.NET for JSON serialization.

Appending an event to a stream

Now that we’ve created a connection and know how to serialize and deserialize events, we can append an event to a stream. We don’t have to create the stream first in this case: if we try to append an event to a stream that doesn’t exist, the stream is automatically created for us.

Note that concurrency control in Event Store is handled through versioning. In this example, we skip concurrency control by passing ExpectedVersion.Any when an event is appended to a stream.

Getting the last event number from a stream

To read events from a stream in reverse order, we need to know the number of the last event in the stream. To get this number, we ask for the last event of the stream by passing an EventNumber of -1 when calling the ReadEvent method:

Getting events from a stream in reverse order

Now we can read events in reverse order starting from the number of the last event. This example reads events in pages of 10 at a time:

Subscribing to all new events

We can use the connection’s SubscribeToAll method to subscribe to all new events. This method is called each time a new event is stored in Event Store:

Subscribing to new events from a specific stream

Similarly to SubscribeToAll, we can also use SubscribeToStream to retrieve events for a specific stream:

Managing projections

We can manage projections using the ProjectionsManager class of the Client API. This class allows us to retrieve information about projections as well as enable, create, and update them.

Creating a ProjectionsManager

This ProjectionsManager will use HTTP (default port 2113) to connect to Event Store:

In the example, we’re using the ConsoleLogger provided by the Client API. You can replace it with your own logger if necessary.

Manipulating projections

The methods of the ProjectionsManager class are fairly straightforward:

Subscribing to a projection’s state change

We can store the state of an event in a stream. This allows us to subscribe to the stream and retrieve updates when its state changes. If there’s only one state stored for all projected streams, the stream with the state is named “$projections-ProjectionName-result”, where ProjectionName is the name of the stream.

Keep in mind that the projection needs to know that it should store the state in the stream. We can do this by setting the producesResults option to true or by calling the outputState method in the JavaScript projection definition:

Source code

A working project for these examples can be found on github: https://github.com/tim-cools/EventStore-Examples

Event Store Projections by Example

This post is part of a series:

  1. EventStore Client API Basics (C#)
  2. Counting events of a specific type
  3. Partition events based on data found in previous events
  4. Calculating an average per day
  5. The irresponsible gambler
  6. Distribute events to other streams
  7. Temporal Projection to generate alarms
  8. Projection in C#

Event Store Projections By Example

I’ve spent some time recently exploring Event Store Projections written in JavaScript. Since there isn’t a lot of documentation yet, I’ll be publishing my findings in a series of blog posts.

Event Store Projections

Event Store is an open-source functional database that stores data as a series of immutable events. The concept isn’t new: logs have been used to store data in this exact fashion for decades, but it’s still garnered much attention from the DDD community in the past few years, especially from people implementing CQRS+ES. Event Store Projections allow us to process events, filter events, and even produce new events, features we can leverage for event processing in systems based on Event-Driven Architecture.

Why JavaScript?

Although it’s possible to write projections in any language, Event Store supports JavaScript natively. This means we don’t have to rely on external programs to process our projections if we write them in JavaScript, and we don’t have to manage their progress and state manually.

I used to hate having to deal with JavaScript; in most projects where I’ve seen it used, it’s an absolute mess. But the language has matured in the past few years: now we have test frameworks, design patterns, and guidelines to work with. Thanks to these tools, I’ve actually learned to like programming in JavaScript.

Examples

In the following posts, I provide examples of solutions to real-world problems I’ve faced using JavaScript and Event Store Projections. These include developing maintainable and testable code, filtering data, and calculating averages. Here are the topics I intend to write about in the future:

  1. EventStore Client API Basics (C#)
  2. Counting events of a specific type
  3. Partition events based on data found in previous events
  4. Calculating an average per day
  5. The irresponsible gambler
  6. Distribute events to other streams
  7. Temporal Projection to generate alarms
  8. Projection in C#

Source code

The source for these examples can be found on github:

https://github.com/tim-cools/EventStore-Examples

Be sure to read the README file first.

More Event Store Projections documentation

  1. Official Wiki
  2. Greg Young’s Blog
  3. Rob Ashton’s Blog
Welcome to my new blog

Welcome to my new blog

I’ve finally revamped the Soloco website and started blogging again. It was time for a change: the previous version of the site was static and boring. I also have a lot of new ideas and goals I want to talk about.

What you can expect

First of all, I’ll be discussing the technical and nontechnical challenges and solutions I deal with in my daily consultancy work and my position as team leader. This will range from-low level .NET stuff to high-level architectural stuff – in other words, pretty much all the things that qualitative software development involves.

Topics will include the following:

  • General .NET software development
  • Applying TDD and BDD
  • Refactoring and qualitative code design
  • Working safely with legacy code
  • DDD, DCI and Event-Driven Architecture
  • Continuous integration and delivery

Training

In addition to my consultancy work, I intend to teach a course starting in 2014, Qualitative Software with .NET, which will cover things that every .NET developer should know. From my experience, .NET developers desperately need a course like this – too many of them still don’t know or care about the things I’ll be discussing. I plan to keep this blog updated with my progress in this endeavor as well.

Travel

I’ve always been interested in going to new places, meeting new people, and learning about new cultures, and so I’m planning to travel far abroad to give this course, even to places where courses on these topics are normally unavailable. So if I’m lucky, you’ll also get reports about my travels around the world!

Let me know what you think

I’ll have more to say in the coming weeks and months.  Don’t hesitate to leave a comment or contact me directly and let me know what you think!

Take care!

Tim Cools