Event Stream Processing

Read and write events, such as Kafka^® events

When you install the Streaming Data Framework for MATLAB^® Production Server™, you can:

Read and write event streams from an event streaming platform, such as Kafka.
Simulate the production environment for testing your streaming analytic algorithms. (Requires MATLAB Compiler SDK™)
Scale your streaming analytic algorithms by deploying them. (Requires MATLAB Production Server)

Install the Streaming Data Framework for MATLAB Production Server support package from the MATLAB Add-On Explorer. For information about installing add-ons, see Get and Manage Add-Ons (MATLAB).

Functions

expand all

Connect to Event Streams

`kafkaStream`	Create connection to event stream in Kafka topic
`inMemoryStream`	Create connection to event stream hosted by MATLAB without schema processing applied
`testStream`	Create connection to event stream hosted by MATLAB with schema processing applied

Read and Write Events

Basic Import and Export

`readtimetable`	Read timetable from event stream
`writetimetable`	Write timetable to event stream
`seek`	Set read position in event stream
`preview`	Preview subset of events from event stream
`identifyingName`	Event stream name

Import and Export Options

`detectImportOptions`	Create import options based on event stream content
`eventStreamImportOptions`	Create options for importing events from stream into MATLAB
`ImportOptions`	Import options for event stream
`getvartype`	Data types used to export variables to stream
`setvartype`	Set data types used to import and export variables to stream
`detectExportOptions`	Create export options based on event stream content
`ExportOptions`	Export options for event stream

Kafka Stream Operations

`readevents`	Read raw events from Kafka stream without schema processing applied
`flush`	Reset read window boundaries
`stop`	Stop processing event streams from Kafka topic
`createTopic`	Create topic in Kafka cluster
`deleteTopic`	Remove topic from Kafka cluster
`loggederror`	Error information for Kafka stream operation

Kafka Provider Properties

`categoryList`	Kafka stream provider property list
`getProviderProperties`	Kafka stream configuration property data
`setProviderProperties`	Set properties specific to Kafka configuration
`isProperty`	Determine if Kafka stream provider property is set

Simulate Production Environment

`eventStreamProcessor`	Apply stream analytic function to event stream
`execute`	Execute event stream processing function on specific number of event windows
`package`	Package stream processing function into deployable archive configured by `EventStreamProcessor`
`seek`	Set position in event stream to begin processing events
`start`	Start processing event streams using local test server
`startServer`	Start local test server
`stop`	Stop processing event streams using local test server
`stopServer`	Shut down local test server

Deploy Streaming Analytic

streamingDataCompiler Package stream processing function into deployable archive

Topics

Process Event Streams

Streaming Data Framework for MATLAB Production Server Basics
Use Streaming Data Framework for MATLAB Production Server to read from and write to event streaming platforms, such as Kafka.
Process Kafka Events Using MATLAB
Process events from a Kafka stream using a streaming analytic function.
Connect to Secure Kafka Cluster
Set Kafka provider properties during stream object creation.

Test and Deploy Streaming Analytic Function

Test Streaming Analytic Function Using Local Test Server
Use the development version of MATLAB Production Server to test a streaming analytic function before deployment to MATLAB Production Server.
Deploy Streaming Analytic Function to MATLAB Production Server
Package a streaming analytic function and deploy to server.

Troubleshooting

Obtain Kafka Event Stream Log Files

Locate event stream log files to help with debugging issues.