Striim 3.9.4 / 3.9.5 documentation

ALTER and RECOMPILE

Use these commands to modify applications that are loaded (status Created) but not deployed or running. Using ALTER instead of dropping and reloading retains persisted WActionStore and Kafka stream data. This could be useful, for example, when there is a DDL change to a source database table.

Note that some changes to the application can be made safely but others may be incompatible with persisted data, causing errors. For example:

should be compatible with persisted data

may be incompatible with persisted data

add components or flows

remove components or flows

add a field to a type

remove a field from a type

change a type from:

  • byte to integer, short, or long

  • integer to short or long

  • short to long

  • float to double

change a type from:

  • long to short, integer, or byte

  • short to integer or byte

  • integer to byte

  • double to float

Using ALTER when recovery is enabled

If recovery is enabled, ALTER also retains recovery checkpoints, so when you restart the altered application, sources will attempt to pick up where they left off. In this case, you may use QUIESCE or STOP. Depending on the source and how long the application is offline, QUIESCE may be better for ensuring that you will have no missing or duplicate events ("exactly-once processing"). See Recovering applications for details.  The following types of changes should not interfere with exactly-once processing:

  • adding components to an unbranched data flow (that is, to a series of components in which the output of each component is the input of only one other component)

  • removing components from an unbranched data flow

  • adding a branch to a data flow

  • removing a branch from a data flow

  • simple modifications to a CQ

Change of the following types may result in duplicate or missing events after the application is restarted, even if exactly-once processing would otherwise be guaranteed:

  • changing a window's KEEP clause

  • changing a CQ's GROUP BY clause

  • changing the number of fields in a CQ's FROM clause

  • changing the size of a CQ's MATCH_PATTERN selection

  • changing the value of a CQ's LIMIT clause

  • changing a KafkaWriter's mode from sync to async

Example

The workflow for ALTER and RECOMPILE is:

  1. If the application is running, STOP or QUIESCE it (see Console commands). Use QUIESCE if recovery is enabled.

  2. Undeploy the application.

  3. Alter the application as described below.

  4. Recompile, deploy, and start the application.

To begin altering an application, use:

USE <application's namespace>;
ALTER APPLICATION <application name>;

At this point, enter CREATE, DROP, or CREATE OR REPLACE statements to modify the application. For example, to add the email subscription described in Sending alerts from applications to the PosApp sample application:

USE Samples;
ALTER APPLICATION PosApp;
CREATE SUBSCRIPTION PosAppEmailAlert
USING EmailAdapter (
  SMTPUSER:'sender@example.com',
  SMTPPASSWORD:'password', 
  smtpurl:'smtp.gmail.com',
  starttls_enable:'true',
  subject:"test subject",
  emailList:"recipient@example.com,recipient2.example.com",
  senderEmail:"alertsender@example.com" 
)
INPUT FROM AlertStream;
ALTER APPLICATION PosApp RECOMPILE;

Then complete the alteration with:

ALTER APPLICATION <application name> RECOMPILE;

At this point you may deploy and start the modified application. If recovery was enabled for the application when it was loaded, when it is restarted, it will pick up source data (subject to the usual limitations detailed in Recovering applications) back to the time it went offline.

Keep in mind that a change made to one component may require changes to multiple downstream components and their types. For example, consider .../Striim/Samples/RetailApp/RetailApp.tql. Its OrderType contains city and state fields:

CREATE TYPE OrderType(
  storeId      String,
  orderId      String,
  sku          String,
  orderAmount  Double,
  dateTime     DateTime,
  hourValue    Integer,
  state        String,
  city         String,
  zip          String
);
CREATE STREAM RetailOrders Of OrderType;

The city and state fields can be retrieved downstream when creating TrackStoreActivity via a SELECT statement that gets the fields ZipCodeLookup. This means that the city and state fields can be excluded from several earlier statements. We will use CREATE OR REPLACE commands for the following portions of the RetailApp TQL:

  • OrderType

  • ParseOrderData

  • StoreOrdersTrackingType

  • GetStoreActivity

  • GetStoreStatus

  • TrackStoreActivity

The following TQL can be used to update the sample:

USE Samples;
ALTER APPLICATION RetailApp;
CREATE OR REPLACE OrderType (
  storeId      String,
  orderId      String,
  sku          String,
  orderAmount  Double,
  dateTime     DateTime,
  hourValue    Integer,
  zip          String
);
CREATE OR REPLACE CQ ParseOrderData
INSERT INTO RetailOrders
SELECT  data[0],
        data[6],
        data[7],
        TO_DOUBLE(SRIGHT(data[8],1)),
        TO_DATE(data[9],'yyyyMMddHHmmss'),
        DHOURS(TO_DATE(data[9],'yyyyMMddHHmmss')),
        data[4]
FROM Orders;
CREATE OR REPLACE TYPE StoreOrdersTrackingType (
  storeId String KEY,
  zip   String,
  startTime DateTime,
  ordersCount Integer,
  salesAmount Double,
  hourlyAvg Integer,
  upperLimit Double,
  lowerLimit Double,
  category String,
  status String
);
CREATE OR REPLACE CQ GetStoreActivity
INSERT INTO StoreOrdersTracking
SELECT rd.storeId, rd.zip, first(rd.dateTime),
       COUNT(rd.storeId), SUM(rd.orderAmount), l.hourlyAvg/6,
       l.hourlyAvg/6 + l.hourlyAvg/8,
       l.hourlyAvg/6 - l.hourlyAvg/10,
       '<NOTSET>', '<NOTSET>'
FROM RetailData_5MIN rd, HourlyStoreSales_Cache l
WHERE rd.storeId = l.storeId AND rd.hourValue = l.hourValue
GROUP BY rd.storeId;
CREATE OR REPLACE CQ GetStoreStatus
INSERT INTO StoreOrdersTracking_Status
SELECT storeId, zip, startTime,
       ordersCount, salesAmount, hourlyAvg, upperLimit, lowerLimit,
       CASE
         WHEN salesAmount > (upperLimit + 2000) THEN 'HOT'
         WHEN salesAmount > upperLimit THEN 'MEDIUM'
         WHEN salesAmount < lowerLimit THEN 'COLD'
         ELSE 'COOL' END,
       CASE
         WHEN salesAmount > upperLimit THEN 'TOOHIGH'
         WHEN salesAmount < lowerLimit THEN 'TOOLOW'
         ELSE 'OK' END
FROM StoreOrdersTracking;
CREATE OR REPLACE CQ TrackStoreActivity
INSERT INTO StoreActivity
SELECT s.storeId,
  s.startTime,
  n.storeName,
  s.category,
  s.status,
  s.ordersCount,
  s.salesAmount,
  s.hourlyAvg,
  s.upperLimit,
  s.lowerLimit,
  z.zip,
  z.city,
  z.state,
  z.latVal,
  z.longVal
FROM StoreOrdersTracking_Status s, StoreNameLookup n, ZipCodeLookup z
WHERE s.storeId = n.storeId AND s.zip = z.zip
LINK SOURCE EVENT;
ALTER APPLICATION RetailApp RECOMPILE;