Spring XD Stream – How to start with Stream

I. Technology for Spring XD Stream

– Java 8
– Spring XD 1.3.1.Release

II. Concepts

A Spring XD is define by 3 kinds of modules: SOURCE | PROCESSOR 1 | PROCESSOR 2 | … PROCESSOR n | SINK. 2 modules Source & Sink are mandatory, and Processor is option.

Stream Modules are built with Spring Integration.

1. Source

A source is a message flow, contains a direct channel named output which is fed by an inbound adapter (configured with a poller, or triggered event)

List of sources supported by Spring XD, you can see at spring-xd-1.3.1.RELEASE\xd\modules\source:

File
FTP
GemFire Continuous Query (gemfire-cq)
GemFire Source (gemfire)
HTTP
JDBC Source (jdbc)
JMS
Kafka
Mail
MongoDB Source (mongodb)
MQTT
RabbitMQ
Reactor IP (reactor-ip)
SFTP
Stdout Capture
Syslog
Tail
TCP
TCP Client (tcp-client)
Time
Trigger Source (trigger)
Twitter Search (twittersearch)
Twitter Stream (twitterstream)

We can create a customized source module using Spring Integration.

2. Processor

A processor is a message flow that contains a direct channel: input and a subscribable channel: output. It is use for transformation on the message.

List of Processors supported by Spring XD, you can see at \spring-xd-1.3.1.RELEASE\xd\modules\processor:

Aggregator
Filter
Header Enricher (header-enricher)
HTTP Client (http-client)
JSON to Tuple (json-to-tuple)
Object to JSON (object-to-json)
Script
Shell
Splitter
Transform

3. Sink

A sink is a message flow that contains a direct channel: input and an outbound adapter, or service activator used to provide the message to an external resource.

List of sinks supported by Spring XD, you can see at \spring-xd-1.3.1.RELEASE\xd\modules\sink:

Dynamic Router (router)
File Sink (file)
FTP Sink (ftp)
GemFire Server
GPFDIST
Cassandra
Hadoop (HDFS) (hdfs)
HDFS Dataset (Avro/Parquet) (hdfs-dataset)
JDBC
Kafka Sink (kafka)
Log
Mail
Mongo
MQTT Sink (mqtt)
Null Sink (null)
RabbitMQ
Redis
Shell Sink (shell)
Splunk Server (splunk)
TCP Sink (tcp)

III. Practices

Create a simple stream: http | transform –expression=payload.toUpperCase() | log
Relate Post: Spring XD Batch Job – How to Start

1. Start Spring XD with single-node mode

Open a CMD
Go to: ./spring-xd-1.3.1.RELEASE\xd\bin
Run xd-singlenode

2. Create a Stream

Go to: .\spring-xd-1.3.1.RELEASE\shell\bin
Run xd-shell

Then run: stream create --definition "http | transform --expression=payload.toUpperCase() | log" --name myprocstrem --deploy

Now open Browser, go to http://localhost:9393/admin-ui/#/streams/definitions
Stream is deployed:

You also can view on Containers:

3. Check Result

Create a Post from xd-shell
http post --target http://localhost:9000 --data "hello"
On xd-singlenode console, view log:

For undeploy or destroy the stream, you can use commandlines or web admins:
With command line:

xd:> stream undeploy --name myprocstrem
xd:> stream destroy --name myprocstrem