1. Filewatcher Python
2. Filewatcher.exe
3. C# Directory Monitor
4. Filewatcher.com

KX Stream contains a File Watcher (FW) component for loading batch data in from files. This data can be in a variety of formats; JSON, CSV, fixed-width binary etc. FW processes are configured with a list of directories and regex file patterns to scan for. If a file is located corresponding to a known pattern, the file will be loaded using custom-defined logic.

• FileWatcher implementation contract; Support and discussions. You can discuss IntelliJ IDEA with its developers and with other users on our forums. The forums are also mirrored as newsgroups if you prefer reading them from an NNTP reader. You can submit bug reports through our issue tracker. Please take note of the Working with issue tracker.
• Using FileSystemWatcher to monitor a directory. I am using a Windows Forms Application to monitor a directory and move the files dropped in it to another directory. At the moment it will copy the file to another directory, but when another file is added it will just end with no error message.

Directory and File Name

This section assumes that historical data will be placed under ${DELTADATA_HOME}/KxWarehouseData/filewatcher directory. CPU data will be saved as CSV, Mem as JSON and Disk as fixed-width binary. Each file should contain a single day's worth of data of a single table. The file names will be comprised of the raw table, date, and the file format. e.g. monCPU_20190910.csv

This is a helper package to monitor files. You can use Cyberinfy-filewatcher to write your content. Check out a usage example with explaination at. Cyberinfy-filewatcher filewatcherusage.py.

Create a FW service class and assign the analytics that will read and load historical data from files.

Analytics¶

• Create an analytic group named fwCore.

• Create an analytic for each of the three raw tables; .mon.fileLoadCPU, .mon.fileLoadMem, and .mon.fileLoadDisk. Add them to the group.

• Each of these analytics contain logic to read from different file types, save to disk, and reload the HDB. They take the same two parameters; directory and file name. This logic could be modified to publish data into the system as opposed to writing it to disk.

.mon.fileLoadCPU¶

.mon.fileLoadCPU

Create the other two analytics by referencing below.

.mon.fileLoadMem

.mon.fileLoadDisk

FW config¶

• Create a filewatcher config parameter.

• In the Attributes Values subtab, specify the following:
  • filename regex
  • directory
  • analytic to be called for this filetype

File watcher processs¶

• Create a file watcher service class.

• Add the override parameter to the Service Params subtab

• Add the fwCore analytic group.

• Run the service class.

Dummy historical data¶

In reality, the historical data would contain information that you want to migrate into the system.

Filewatcher Python

For the purpose of this example, the code below creates and saves dummy data as a CSV, JSON and fixed-width binary text to your filewatcher directory.

Filewatcher.exe

• Call this function from any process.

C# Directory Monitor

Filewatcher.com

• Once the files are placed, the filewatcher process will find them and call the relevant analytics.