Striim 3.9.4 / 3.9.5 documentation

Creating sources and caches using Source Preview

Source Preview is a graphical alternative to defining sources and caches using TQL. With it, you:

  1. browse regular or HDFS volumes accessible by the Striim server

  2. select the file you want

  3. select the appropriate parser (Apache, structured text, unstructured text, or XML)

  4. choose settings for the selected parser, previewing the effects on how the data is parsed

  5. generate a new application containing the source or cache, or add it to an existing application

For sources, Source Preview will also create:

  • a CQ to filter the raw data and convert the fields to Striim data types

  • a stream of type WAEvent linking the source and CQ

  • an output stream of a new type based on the parser settings you chose in Source Preview

The following steps create a source from the sample data used by PosApp:

  1. From the main Striim menu, select Source Preview.

    Screen_Shot_2015-09-22_at_4.44.52_PM.png
  2. Click Samples.

    source_preview_samples.png
  3. Select PosDataPreview.csv, then click Preview.

    source_preview_3.png
  4. Check Use first line for column names and set columndelimiter to , (comma).

    source_preview_4.png
  5. PosApp uses only the MERCHANTID, DATETIME, AUTHAMOUNT, and ZIP columns, so uncheck the others.

    250_source_preview_05.png
  6. Set the data types for DATETIME to DateTime (select Unix Timestamp) and for AUTHAMOUNT to Double. Leave MERCHANTID and ZIP set to String.

    312_source_preview_06.png
  7. The data is now parsed correctly, the columns have been selected, and their names and data types have been set, so click Save.

    spsave1.png
  8. Set the Save Data Source application options as shown above, then click Next.

    spsave2.png
  9. Set the source as shown above, then click Save.

    spnewapp.png

The new PosSourceApp application appears in the flow editor. At this point you could add additional components such as a window, CQ, and target to refine the application, or export it to TQL for use in manually coded applications.

The following steps will add a cache to the PosSourceApp application:

  1. From the menu at top left, select Source Preview, click Browse, navigate to Samples/PosApp/appData, select USAddressesPreview.txt, and click Preview.

    zipcache_preview.png
  2. Check Use first line for column names, set columndelimiter to \t (tab), set the data type for latVal and longVal to Double, and click Save.

    cache1.png
    cache2.png
  3. Set the Save Data Source options as shown above, then click Save.

Warning

If you save as a cache and deploy the application, the entire file will be loaded into memory.

Continue with Modifying an application using the Flow Designer.