Striim 3.9.4 / 3.9.5 documentation

Filter the data with a CQ SELECT statement

The next step is to filter the data.

In simple.tql, replace these lines:

CREATE TARGET SimpleRawOutput
USING SysOut(name:simpleRaw)
INPUT FROM RawDataStream;

with the following:

CREATE TYPE FilteredDataType(
  merchantId String KEY,
  dateTime DateTime,
  amount double,
  zip String
);
CREATE STREAM FilteredDataStream OF FilteredDataType;

CREATE CQ Raw2FilteredCQ
INSERT INTO FilteredDataStream
SELECT data[1],
  TO_DATEF(data[4],'yyyyMMddHHmmss'),
  TO_DOUBLE(data[7]),
  data[9]
FROM RawDataStream;

CREATE TARGET SimpleFilteredOutput
USING SysOut(name:simpleFiltered)
INPUT FROM FilteredDataStream;

This selects four of the fields from the source data, converts the data types as necessary, and discards the remaining data. Save the file, then stop, undeploy, and drop the old version, load the new version, deploy, and run it (see Loading and reloading TQL applications during development). You should see these results in striim-node.log:

simpleFiltered: FilteredDataType_1_0{
  merchantId: "D6RJPwyuLXoLqQRQcOcouJ26KGxJSf6hgbu"
  dateTime: 1363134730000
  amount: 2.2
  zip: "41363"
};
simpleFiltered: FilteredDataType_1_0{
  merchantId: "OFp6pKTMg26n1iiFY00M9uSqh9ZfMxMBRf1"
  dateTime: 1363134730000
  amount: 22.78
  zip: "16950"
};