Using the HDAT Reader Adapter

The HDAT reader adapter resides in dfx-esp-hdatreader-adapter.jar, which bundles the Java publisher SAS Event Stream Processing client. The adapter converts each row in an HDAT file into an ESP event and injects event blocks into a source window of an engine. Each event is built with an INSERT opcode unless you specify the —s parameter.

The number of fields in the target source window schema must match the number of columns in the HDAT row data. Also, all HDAT column types must be numeric, except for columns that correspond to an ESP field of type UTF8STR. In that case, the column must contain character data.

The source Hadoop Distributed File System (HDFS) and the name of the file within the file system are passed as required parameters to the adapter. The client target platform must define the environment variable DFESP_HDFS_JARS. This specifies the location of the Hadoop JAR files.

List the JAR files in this order: hadoop-common-*.jar , hadoop-hdfs-*.jar , common hadoop JARs.

For example, in Linux:

$ export DFESP_HDFS_JARS=/usr/local/hadoop-2.5.0/share/hadoop/common/hadoop-common-2.5.0.jar:
/usr/local/hadoop-2.5.0/share/hadoop/hdfs/hadoop-hdfs-2.5.0.jar:
/usr/local/hadoop-2.5.0/share/hadoop/common/lib/*

Usage:

Parameter	Definition
`—u url`	Specifies the publish standard URL in the form `"dfESP://host:port/project/continuousquery/window"`.
`—f hdfs`	Specifies the target file system, in the form `“hdfs://host:port”`. Specifies the file system that is normally configured in property `fs.defaultFS` in core-site.xml.
`—i inputfile`	Specifies the input CSV file, in the form `“/path/filename.csv”`.
`—b blocksize`	Specifies the number of events per event block.
`-t`	Specifies that event blocks are transactional. The default is normal.
`- g gdconfigfile`	Specifies the guaranteed delivery configuration file for the client.
`—l native \| solace \| tervela \| rabbitmq \| kafka`	Specifies the transport type. When you specify `solace`, `tervela`, `rabbitmq`, or `kafka` transports instead of the default `native` transport, use the required client configuration files specified in Using Alternative Transport Libraries for Java Clients in SAS Event Stream Processing: Publish/Subscribe API.
`-o severe \| warning \| info`	Specifies the application logging level.
`-c [configfilesection]`	Specifies the name of the section in file `/etc/javaadapters.config` to parse for configuration parameters.
`-s`	Specifies to build events with opcode = Upsert instead of Insert.
`-O tokenlocation`	Specifies the location of the file in the local file system that contains the OAuth token required for authentication by the publish/subscribe server.
`-Q`	Specify to quiesce the project after all events are injected into the source window.