Striim 3.9.4 / 3.9.5 documentation

MapRStreamWriter

Writes to a stream in MapR Converged Data Platform version 5.1.

property

type

default value

notes

MapRStreamConfig

java.lang.String

Optionally specify Kafka producer properties, separated by semicolons.

Mode

java.lang.String

Sync

See Setting KafkaWriter's mode property: sync versus async for discussion of this property.

Topic

java.lang.String

Specify the fully-qualified topic name. The syntax is /<stream>:<topic>, where <stream> includes its path (which may be displayed using the command hadoop fs -ls /), for example, /tmp/myfirststream:topic1. The stream must already exist. Striim will create the topic if it does not exist.

This adapter has a choice of formatters. See Supported writer-formatter combinations for more information.

Sample code:

CREATE TARGET MapRStreamSample USING MapRStreamWriter (
  MapRStreamConfig: "acks=0;batch.size=250000;max.request.size=40000000",
  Mode: "ASync",
  Topic:'/striim/myfirststream:topic1'
)
FORMAT USING JSONFormatter ()
INPUT FROM TypedCSVStream;

If the MapR topic is partitioned, events will be distributed among the partitions based on the target's input stream's PARTITION BY field. If the input stream is not partitioned, all events will be written to partition 0.

MapRStreamWriter is based on KafkaWriter and the KafkaWriter sample code can be used as a model by replacing the target with the sample code above and creating a MapR stream named striim. See Apache Kafka and MapR Streams: Terms, Techniques and New Designs for more information about MapR's Kafka implementation.