

API Docs - v2.0.23¶

Tested Siddhi Core version: 5.1.26

It could also support other Siddhi Core minor versions.

File¶

isDirectory (Function)¶

This function checks for a given file path points to a directory

Syntax

<BOOL> file:isDirectory(<STRING> uri)
<BOOL> file:isDirectory(<STRING> uri, <STRING> file.system.options)

QUERY PARAMETERS

Name	Description	Default Value	Possible Data Types	Optional	Dynamic
uri	The path to be checked for a directory.		STRING	No	Yes
file.system.options	The file options in key:value pairs separated by commas. eg:'USER_DIR_IS_ROOT:false,PASSIVE_MODE:true,AVOID_PERMISSION_CHECK:true,IDENTITY:file://demo/.ssh/id_rsa,IDENTITY_PASS_PHRASE:wso2carbon' Note: when IDENTITY is used, use a RSA PRIVATE KEY		STRING	Yes	No

Examples EXAMPLE 1

file:isDirectory(filePath) as isDirectory

Checks whether the given path is a directory. Result will be returned as an boolean.

isExist (Function)¶

This function checks whether a file or a folder exists in a given path

Syntax

<BOOL> file:isExist(<STRING> uri)
<BOOL> file:isExist(<STRING> uri, <STRING> file.system.options)

QUERY PARAMETERS

Name	Description	Default Value	Possible Data Types	Optional	Dynamic
uri	File path to check for existence.		STRING	No	Yes
file.system.options	The file options in key:value pairs separated by commas. eg:'USER_DIR_IS_ROOT:false,PASSIVE_MODE:true,AVOID_PERMISSION_CHECK:true,IDENTITY:file://demo/.ssh/id_rsa,IDENTITY_PASS_PHRASE:wso2carbon' Note: when IDENTITY is used, use a RSA PRIVATE KEY		STRING	Yes	No

Examples EXAMPLE 1

file:isExist('/User/wso2/source/test.txt') as exists

Checks existence of a file in the given path. Result will be returned as an boolean .

EXAMPLE 2

file:isExist('/User/wso2/source/') as exists

Checks existence of a folder in the given path. Result will be returned as an boolean .

isFile (Function)¶

This function checks for a given file path points to a file

Syntax

<BOOL> file:isFile(<STRING> file.path)
<BOOL> file:isFile(<STRING> file.path, <STRING> file.system.options)

QUERY PARAMETERS

Name	Description	Default Value	Possible Data Types	Optional	Dynamic
file.path	The path to be checked for a file.		STRING	No	Yes
file.system.options	The file options in key:value pairs separated by commas. eg:'USER_DIR_IS_ROOT:false,PASSIVE_MODE:true,AVOID_PERMISSION_CHECK:true,IDENTITY:file://demo/.ssh/id_rsa,IDENTITY_PASS_PHRASE:wso2carbon' Note: when IDENTITY is used, use a RSA PRIVATE KEY		STRING	Yes	No

Examples EXAMPLE 1

file:isFile(filePath) as isFile

Checks whether the given path is a file. Result will be returned as an boolean.

lastModifiedTime (Function)¶

Checks for the last modified time for a given file path

Syntax

<STRING> file:lastModifiedTime(<STRING> uri)
<STRING> file:lastModifiedTime(<STRING> uri, <STRING> datetime.format)
<STRING> file:lastModifiedTime(<STRING> uri, <STRING> datetime.format, <STRING> file.system.options)

QUERY PARAMETERS

Name	Description	Default Value	Possible Data Types	Optional	Dynamic
uri	File path to be checked for te last modified time.		STRING	No	Yes
datetime.format	Format of the last modified datetime to be returned.	MM/dd/yyyy HH:mm:ss	STRING	Yes	No
file.system.options	The file options in key:value pairs separated by commas. eg:'USER_DIR_IS_ROOT:false,PASSIVE_MODE:true,AVOID_PERMISSION_CHECK:true,IDENTITY:file://demo/.ssh/id_rsa,IDENTITY_PASS_PHRASE:wso2carbon' Note: when IDENTITY is used, use a RSA PRIVATE KEY		STRING	Yes	No

Examples EXAMPLE 1

file:lastModifiedTime(filePath) as lastModifiedTime

Last modified datetime of a file will be returned as an string in MM/dd/yyyy HH:mm:ss.

EXAMPLE 2

file:lastModifiedTime(filePath, dd/MM/yyyy HH:mm:ss) as lastModifiedTime

Last modified datetime of a file will be returned as an string in 'dd/MM/yyyy HH:mm:ss' format.

size (Function)¶

This function checks for a given file's size

Syntax

<LONG> file:size(<STRING> uri)
<LONG> file:size(<STRING> uri, <STRING> file.system.options)

QUERY PARAMETERS

Name	Description	Default Value	Possible Data Types	Optional	Dynamic
uri	Absolute path to the file or directory to be checked for the size.		STRING	No	Yes
file.system.options	The file options in key:value pairs separated by commas. eg:'USER_DIR_IS_ROOT:false,PASSIVE_MODE:true,AVOID_PERMISSION_CHECK:true,IDENTITY:file://demo/.ssh/id_rsa,IDENTITY_PASS_PHRASE:wso2carbon' Note: when IDENTITY is used, use a RSA PRIVATE KEY		STRING	Yes	No

Examples EXAMPLE 1

file:size('/User/wso2/source/test.txt') as fileSize

Size of a file in a given path will be returned.

archive (Stream Function)¶

Archives files and folders as a zip or in tar format that are available in the given file uri.

Syntax

file:archive(<STRING> uri, <STRING> destination.dir.uri)
file:archive(<STRING> uri, <STRING> destination.dir.uri, <STRING> archive.type)
file:archive(<STRING> uri, <STRING> destination.dir.uri, <STRING> archive.type, <STRING> include.by.regexp)
file:archive(<STRING> uri, <STRING> destination.dir.uri, <STRING> archive.type, <STRING> include.by.regexp, <BOOL> exclude.subdirectories)
file:archive(<STRING> uri, <STRING> destination.dir.uri, <STRING> archive.type, <STRING> include.by.regexp, <BOOL> exclude.subdirectories, <STRING> file.system.options)

QUERY PARAMETERS

Name	Description	Default Value	Possible Data Types	Optional	Dynamic
uri	Absolute path of the file or the directory		STRING	No	Yes
destination.dir.uri	Absolute directory path of the the archived file.		STRING	No	Yes
archive.type	Archive type can be zip or tar	zip	STRING	Yes	No
include.by.regexp	Only the files matching the patterns will be archived. Note: Add an empty string to match all files		STRING	Yes	No
exclude.subdirectories	This flag is used to exclude the subdirectories and its files without archiving.	false	BOOL	Yes	No
file.system.options	The file options in key:value pairs separated by commas. eg:'USER_DIR_IS_ROOT:false,PASSIVE_MODE:true,AVOID_PERMISSION_CHECK:true,IDENTITY:file://demo/.ssh/id_rsa,IDENTITY_PASS_PHRASE:wso2carbon' Note: when IDENTITY is used, use a RSA PRIVATE KEY		STRING	Yes	No

Examples EXAMPLE 1

InputStream#file:archive('/User/wso2/to_be_archived', '/User/wso2/archive_destination/file.zip')

Archives to_be_archived folder in zip format and stores archive_destination folder as file.zip.

EXAMPLE 2

InputStream#file:archive('/User/wso2/to_be_archived', '/User/wso2/archive_destination/file', 'tar')

Archives to_be_archived folder in tar format and stores in archive_destination folder as file.tar.

EXAMPLE 3

InputStream#file:archive('/User/wso2/to_be_archived', '/User/wso2/archive_destination/file', 'tar', '.*test3.txt$')

Archives files which adheres to '.*test3.txt$' regex in to_be_archived folder in tar format and stores in archive_destination folder as file.tar.

EXAMPLE 4

InputStream#file:archive('/User/wso2/to_be_archived', '/User/wso2/archive_destination/file', '', '', 'false')

Archives to_be_archived folder excluding the sub-folders in zip format and stores in archive_destination folder as file.tar.

copy (Stream Function)¶

This function performs copying file from one directory to another.

Syntax

file:copy(<STRING> uri, <STRING> destination.dir.uri)
file:copy(<STRING> uri, <STRING> destination.dir.uri, <STRING> include.by.regexp)
file:copy(<STRING> uri, <STRING> destination.dir.uri, <STRING> include.by.regexp, <BOOL> exclude.root.dir)
file:copy(<STRING> uri, <STRING> destination.dir.uri, <STRING> include.by.regexp, <BOOL> exclude.root.dir, <STRING> file.system.options)

QUERY PARAMETERS

Name	Description	Default Value	Possible Data Types	Optional	Dynamic
uri	Absolute path of the File or the directory.		STRING	No	Yes
destination.dir.uri	Absolute path of the destination directory. Note: Parent folder structure will be created if it does not exist.		STRING	No	Yes
include.by.regexp	Only the files matching the patterns will be copied. Note: Add an empty string to match all files		STRING	Yes	Yes
exclude.root.dir	This flag is used to exclude parent folder when copying the content.	false	BOOL	Yes	No
file.system.options	The file options in key:value pairs separated by commas. eg:'USER_DIR_IS_ROOT:false,PASSIVE_MODE:true,AVOID_PERMISSION_CHECK:true,IDENTITY:file://demo/.ssh/id_rsa,IDENTITY_PASS_PHRASE:wso2carbon' Note: when IDENTITY is used, use a RSA PRIVATE KEY		STRING	Yes	No

Extra Return Attributes

Name	Description	Possible Types
isSuccess	Status of the file copying operation (true if success)	BOOL

Examples EXAMPLE 1

InputStream#file:copy('/User/wso2/source/test.txt', 'User/wso2/destination/')

Copies 'test.txt' in 'source' folder to the 'destination' folder.

EXAMPLE 2

InputStream#file:copy('/User/wso2/source/', 'User/wso2/destination/')

Copies 'source' folder to the 'destination' folder with all its content

EXAMPLE 3

InputStream#file:copy('/User/wso2/source/', 'User/wso2/destination/', '.*test3.txt$')

Copies 'source' folder to the 'destination' folder ignoring files doesnt adhere to the given regex.

EXAMPLE 4

InputStream#file:copy('/User/wso2/source/', 'User/wso2/destination/', '', true)

Copies only the files resides in 'source' folder to 'destination' folder.

create (Stream Function)¶

Create a file or a folder in the given location

Syntax

file:create(<STRING> uri)
file:create(<STRING> uri, <BOOL> is.directory)
file:create(<STRING> uri, <BOOL> is.directory, <STRING> file.system.options)

QUERY PARAMETERS

Name	Description	Default Value	Possible Data Types	Optional	Dynamic
uri	Absolute file path which needs to be created.		STRING	No	Yes
is.directory	This flag is used when creating file path is a directory	false	BOOL	Yes	Yes
file.system.options	The file options in key:value pairs separated by commas. eg:'USER_DIR_IS_ROOT:false,PASSIVE_MODE:true,AVOID_PERMISSION_CHECK:true,IDENTITY:file://demo/.ssh/id_rsa,IDENTITY_PASS_PHRASE:wso2carbon' Note: when IDENTITY is used, use a RSA PRIVATE KEY		STRING	Yes	No

Examples EXAMPLE 1

from CreateFileStream#file:create('/User/wso2/source/test.txt', false)

Creates a file in the given path with the name of 'test.txt'.

EXAMPLE 2

from CreateFileStream#file:create('/User/wso2/source/', true)

Creates a folder in the given path with the name of 'source'.

move (Stream Function)¶

This function performs copying file from one directory to another.

Syntax

file:move(<STRING> path, <STRING> destination.dir.path)
file:move(<STRING> path, <STRING> destination.dir.path, <STRING> include.by.regexp)
file:move(<STRING> path, <STRING> destination.dir.path, <STRING> include.by.regexp, <BOOL> exclude.root.dir)
file:move(<STRING> path, <STRING> destination.dir.path, <STRING> include.by.regexp, <BOOL> exclude.root.dir, <STRING> file.system.options)

QUERY PARAMETERS

Name	Description	Default Value	Possible Data Types	Optional	Dynamic
path	Absolute file or directory path.		STRING	No	Yes
destination.dir.path	Absolute file path to the destination directory. Note: Parent folder structure will be created if it does not exist.		STRING	No	Yes
include.by.regexp	Only the files matching the patterns will be moved. Note: Add an empty string to match all files		STRING	Yes	Yes
exclude.root.dir	Exclude parent folder when moving the content.	false	BOOL	Yes	Yes
file.system.options	The file options in key:value pairs separated by commas. eg:'USER_DIR_IS_ROOT:false,PASSIVE_MODE:true,AVOID_PERMISSION_CHECK:true,IDENTITY:file://demo/.ssh/id_rsa,IDENTITY_PASS_PHRASE:wso2carbon' Note: when IDENTITY is used, use a RSA PRIVATE KEY		STRING	Yes	No

Extra Return Attributes

Name	Description	Possible Types
isSuccess	Status of the file moving operation (true if success)	BOOL

Examples EXAMPLE 1

InputStream#file:move('/User/wso2/source/test.txt', 'User/wso2/destination/')

Moves 'test.txt' in 'source' folder to the 'destination' folder.

EXAMPLE 2

InputStream#file:move('/User/wso2/source/', 'User/wso2/destination/')

Moves 'source' folder to the 'destination' folder with all its content

EXAMPLE 3

InputStream#file:move('/User/wso2/source/', 'User/wso2/destination/', '.*test3.txt$')

Moves 'source' folder to the 'destination' folder excluding files doesnt adhere to the given regex.

EXAMPLE 4

InputStream#file:move('/User/wso2/source/', 'User/wso2/destination/', '', true)

Moves only the files resides in 'source' folder to 'destination' folder.

This method can be used to rename a file/folder in a particular path, move a file from to a different path.
Ex-
file:rename('/User/wso2/source', 'User/wso2/destination')
file:rename('/User/wso2/source/file.csv', 'User/wso2/source/newFile.csv')
file:rename('/User/wso2/source/file.csv', 'User/wso2/destination/file.csv')

Syntax

file:rename(<STRING> uri, <STRING> new.destination.name)

QUERY PARAMETERS

Name	Description	Possible Data Types	Optional	Dynamic
uri	Absolute path of the file or the directory to be rename.	STRING	No	Yes
new.destination.name	Absolute path of the new file/folder	STRING	No	Yes
file.system.options	The file options in key:value pairs separated by commas. eg:'USER_DIR_IS_ROOT:false,PASSIVE_MODE:true,AVOID_PERMISSION_CHECK:true,IDENTITY:<Realative path from '<Product_Home>/wso2/server/' directory>,IDENTITY_PASS_PHRASE:wso2carbon' Note: when IDENTITY is used, use a RSA PRIVATE KEY	STRING	Yes	No

Extra Return Attributes

Name	Description	Possible Types
isSuccess	Status of the file rename operation (true if success)	BOOL

Examples EXAMPLE 1

InputStream#file:rename('/User/wso2/source/', 'User/wso2/destination/')

Rename the file resides in 'source' folder to 'destination' folder.

EXAMPLE 2

InputStream#file:rename('/User/wso2/folder/old.csv', 'User/wso2/folder/new.txt')

Rename 'old.csv' file resides in folder to 'new.txt'

search (Stream Function)¶

Searches files in a given folder and lists.

Syntax

file:search(<STRING> uri)
file:search(<STRING> uri, <STRING> include.by.regexp)
file:search(<STRING> uri, <STRING> include.by.regexp, <BOOL> exclude.subdirectories)
file:search(<STRING> uri, <STRING> include.by.regexp, <BOOL> exclude.subdirectories, <STRING> file.system.options)

QUERY PARAMETERS

Name	Description	Default Value	Possible Data Types	Optional	Dynamic
uri	Absolute file path of the directory.		STRING	No	Yes
include.by.regexp	Only the files matching the patterns will be searched. Note: Add an empty string to match all files		STRING	Yes	Yes
exclude.subdirectories	This flag is used to exclude the files un subdirectories when listing.	false	BOOL	Yes	No
file.system.options	The file options in key:value pairs separated by commas. eg:'USER_DIR_IS_ROOT:false,PASSIVE_MODE:true,AVOID_PERMISSION_CHECK:true,IDENTITY:file://demo/.ssh/id_rsa,IDENTITY_PASS_PHRASE:wso2carbon' Note: when IDENTITY is used, use a RSA PRIVATE KEY		STRING	Yes	No

Extra Return Attributes

Name	Description	Possible Types
fileNameList	The lit file name matches in the directory.	OBJECT

Examples EXAMPLE 1

ListFileStream#file:search(filePath)

This will list all the files (also in sub-folders) in a given path.

EXAMPLE 2

ListFileStream#file:search(filePath, '.*test3.txt$')

This will list all the files (also in sub-folders) which adheres to a given regex file pattern in a given path.

EXAMPLE 3

ListFileStream#file:search(filePath, '.*test3.txt$', true)

This will list all the files excluding the files in sub-folders which adheres to a given regex file pattern in a given path.

searchInArchive (Stream Function)¶

This.

Syntax

file:searchInArchive(<STRING> uri)
file:searchInArchive(<STRING> uri, <STRING> include.by.regexp)
file:searchInArchive(<STRING> uri, <STRING> include.by.regexp, <STRING> file.system.options)

QUERY PARAMETERS

Name	Description	Possible Data Types	Optional	Dynamic
uri	Absolute file path of the zip or tar file.	STRING	No	Yes
include.by.regexp	Only the files matching the patterns will be searched. Note: Add an empty string to match all files	STRING	Yes	No
file.system.options	The file options in key:value pairs separated by commas. eg:'USER_DIR_IS_ROOT:false,PASSIVE_MODE:true,AVOID_PERMISSION_CHECK:true,IDENTITY:file://demo/.ssh/id_rsa,IDENTITY_PASS_PHRASE:wso2carbon' Note: when IDENTITY is used, use a RSA PRIVATE KEY	STRING	Yes	No

Extra Return Attributes

Name	Description	Possible Types
fileNameList	The list file names in the archived file.	OBJECT

Examples EXAMPLE 1

ListArchivedFileStream#file:listFilesInArchive(filePath)

Lists the files inside the compressed file in the given path.

EXAMPLE 2

ListArchivedFileStream#file:listFilesInArchive(filePath, '.*test3.txt$')

Filters file names adheres to the given regex and lists the files inside the compressed file in the given path.

unarchive (Stream Function)¶

This function decompresses a given file

Syntax

file:unarchive(<STRING> uri, <STRING> destination.dir.uri)
file:unarchive(<STRING> uri, <STRING> destination.dir.uri, <BOOL> exclude.root.dir)
file:unarchive(<STRING> uri, <STRING> destination.dir.uri, <BOOL> exclude.root.dir, <STRING> file.system.options)

QUERY PARAMETERS

Name	Description	Default Value	Possible Data Types	Optional	Dynamic
uri	Absolute path of the file to be decompressed in the format of zip or tar.		STRING	No	Yes
destination.dir.uri	Absolute path of the destination directory. Note: If the folder structure does not exist, it will be created.		STRING	No	Yes
exclude.root.dir	This flag excludes parent folder when extracting the content.	false	BOOL	Yes	No
file.system.options	The file options in key:value pairs separated by commas. eg:'USER_DIR_IS_ROOT:false,PASSIVE_MODE:true,AVOID_PERMISSION_CHECK:true,IDENTITY:file://demo/.ssh/id_rsa,IDENTITY_PASS_PHRASE:wso2carbon' Note: when IDENTITY is used, use a RSA PRIVATE KEY		STRING	Yes	No

Examples EXAMPLE 1

file:unarchive('/User/wso2/source/test.zip', '/User/wso2/destination')

Unarchive a zip file in a given path to a given destination.

EXAMPLE 2

file:unarchive('/User/wso2/source/test.tar', '/User/wso2/destination')

Unarchive a tar file in a given path to a given destination.

EXAMPLE 3

file:unarchive('/User/wso2/source/test.tar', '/User/wso2/destination', true)

Unarchive a tar file in a given path to a given destination excluding the root folder.

Sink¶

file (Sink)¶

The File Sink component of the 'siddhi-io-fie' extension publishes (writes) event data that is processed within Siddhi to files.
Siddhi-io-file sink provides support to write both textual and binary data into files

Syntax

@sink(type="file", file.uri="<STRING>", append="<BOOL>", add.line.separator="<BOOL>", file.system.options="<STRING>", @map(...)))

QUERY PARAMETERS

Name	Description	Default Value	Possible Data Types	Optional	Dynamic
file.uri	The path to thee file in which the data needs to be published.		STRING	No	Yes
append	This specifies whether the data should be appended to the file or not. If this parameter is set to 'true', data is written at the end of the file without changing the existing content. If the parameter is set to 'false', the existing content of the file is deleted and the content you are publishing is added to replace it. If the file does not exist, a new file is created and then the data is written in it. In such a scenario, the value specified for this parameter is not applicable	true	BOOL	Yes	No
add.line.separator	If this parameter is set to 'true', events added to the file are separated by adding each event in a new line.	true. (However, if the 'csv' mapper is used, it is false)	BOOL	Yes	No
file.system.options	The file options in key:value pairs separated by commas. eg:'USER_DIR_IS_ROOT:false,PASSIVE_MODE:true,AVOID_PERMISSION_CHECK:true,IDENTITY:file://demo/.ssh/id_rsa,IDENTITY_PASS_PHRASE:wso2carbon' Note: when IDENTITY is used, use a RSA PRIVATE KEY		STRING	Yes	No

Examples EXAMPLE 1

@sink(type='file', @map(type='json'), append='false', file.uri='/abc/{{symbol}}.txt') define stream BarStream (symbol string, price float, volume long);

In the above configuration, each output event is published in the '/abc/{{symbol}}.txt' file in JSON format.The output looks as follows:
{
    "event":{
        "symbol":"WSO2",
        "price":55.6,
        "volume":100
    }
}
If the file does not exist at the time an output event is generated, the system creates the file and proceeds to publish the output event in it.

Source¶

file (Source)¶

The File Source component of the 'siddhi-io-fie' extension allows you to receive the input data to be processed by Siddhi via files. Both text files and binary files are supported.

Syntax

@source(type="file", dir.uri="<STRING>", file.uri="<STRING>", mode="<STRING>", tailing="<BOOL>", action.after.process="<STRING>", action.after.failure="<STRING>", move.after.process="<STRING>", move.if.exist.mode="<STRING>", move.after.failure="<STRING>", begin.regex="<STRING>", end.regex="<STRING>", file.polling.interval="<STRING>", dir.polling.interval="<STRING>", timeout="<STRING>", file.read.wait.timeout="<STRING>", header.present="<BOOL>", header.line.count="<INT>", read.only.header="<BOOL>", read.only.trailer="<BOOL>", skip.trailer="<BOOL>", buffer.size="<STRING>", cron.expression="<STRING>", file.name.pattern="<STRING>", file.system.options="<STRING>", @map(...)))

QUERY PARAMETERS

Name	Description	Default Value	Possible Data Types	Optional	Dynamic
dir.uri	The path to the directory to be processed. During execution time, Siddhi by default processes all the files within this directory. However, if you have entered specific files to be processed via the 'file.name.list' parameter, only those files are processed. The URI specified must include the file handling protocol to be used for file processing. e.g., If the file handling protocol to be used is 'ftp', the URI must be provided as 'ftp://<DIRECTORY_PATH>>'. At a given time, you should provide a value only for one out of the 'dir.uri' and 'file.uri' parameters. You can provide the directory URI if you have multiple files that you want to process within a directory. You can provide the file URI if you only need to process one file.	file:/var/tmp	STRING	Yes	No
file.uri	The path to the file to be processed. The URI specified must include the file handling protocol to be used for file processing. Only one of 'dir.uri' and 'file.uri' should be provided. e.g., If the file handling protocol to be used is 'ftp', the URI must be provided as 'ftp://<FILE_PATH>>'. At a given time, you should provide a value only for one out of the 'dir.uri' and 'file.uri' parameters. You can provide the directory URI if you have multiple files that you want to process within a directory. You can provide the file URI if you only need to process one file.	file:/var/temp/tmp.text	STRING	Yes	No
mode	This specifies the mode in which the files in given directory must be read.Possible values for this parameter are as follows: - TEXT.FULL : to read a text file completely at once. - BINARY.FULL : to read a binary file completely at once. - BINARY.CHUNKED : to read a binary file chunk by chunk. - LINE : to read a text file line by line. - REGEX : to read a text file and extract data using a regex.	line	STRING	Yes	No
tailing	If this parameter is set to 'true', the file/the first file of the directory is tailed. Do not set the parameter to 'true' and enable tailing if the mode is 'binary.full', 'text.full' or 'binary.chunked'.	true	BOOL	Yes	No
action.after.process	The action to be carried out after processing the file/directory. Possible values are 'DELETE' and 'MOVE'. 'DELETE' is default. If you specify 'MOVE', you need to specify a value for the 'move.after.process' parameter to indicate the location to which the consumed files should be moved.	delete	STRING	Yes	No
action.after.failure	The action to be taken if a failure occurs while the file/directory is being processed. Possible values are 'DELETE' and 'MOVE'. 'DELETE' is default. If you specify 'MOVE', you need to specify a value for the 'move.after.failure' parameter to indicate the location to which the files that could not be read need to be moved	delete	STRING	Yes	No
move.after.process	If you specify 'MOVE' as the value for the 'action.after.process' parameter, use this parameter to specify the location to which the consumed files need to be moved.This should be the absolute path of the file that is going to be created after the moving is done. This URI must include the file handling protocol used for file processing. e.g., If the file handling protocol is 'ftp', the URI must be provided as 'ftp://<FILE_PATH>>'.		STRING	Yes	No
move.if.exist.mode	If you specify 'MOVE' as the value for the 'action.after.process' parameter, use this parameter to specify what happens if a file exist in the same location.Possible values are 'OVERWRITE' and 'KEEP' where KEEP will append a UUID to existing filename and keep both files while OVERWRITE will simply overwrite the existing file.		STRING	Yes	No
move.after.failure	If you specify 'MOVE' as the value for the 'action.after.failure' parameter, use this parameter to specify the location to which the files should be moved after the failure This should be the absolute path of the file that is going to be created after the failure. This URI must include the file handling protocol used for file processing. e.g., If the file handling protocol is 'ftp', the URI must be provided as 'ftp://<FILE_PATH>>'.		STRING	Yes	No
begin.regex	The regex to be matched at the beginning of the retrieved content.	None	STRING	Yes	No
end.regex	The regex to be matched at the end of the retrieved content.	None	STRING	Yes	No
file.polling.interval	The time interval (in milliseconds) of a polling cycle for a file.	1000	STRING	Yes	No
dir.polling.interval	The time period (in milliseconds) of a polling cycle for a directory.	1000	STRING	Yes	No
timeout	The maximum time duration (in milliseconds) that the system should wait until a file is processed.	5000	STRING	Yes	No
file.read.wait.timeout	The maximum time duration (in milliseconds) that the system should wait before retrying to read the full file content.	1000	STRING	Yes	No
header.present	If this parameter is set to 'true', it indicates the file(s) to be processed includes a header line(s). In such a scenario, the header line(s) are not processed. Number of header lines can be configured via 'header.line.count' parameter.	false	BOOL	Yes	No
header.line.count	Number of lines to be considered as the file header. This parameter is applicable only if the parameter 'header.present' is set to 'true'.	1	INT	Yes	No
read.only.header	This parameter is applicable only if the value for the 'mode' parameter is 'LINE'. If this parameter is set to 'true', only the first line (i.e., the header line) of a text file (e.g., CSV) is read. If it is set to 'false', the full content of the file is read line by line.	false	BOOL	Yes	No
read.only.trailer	This parameter is applicable only if the value for the 'mode' parameter is 'LINE'. If this parameter is set to 'true', only the last line (i.e., the trailer line) of a text file (e.g., CSV) is read. If it is set to 'false', the full content of the file is read line by line. This will only work if trailer appears once at the last line of file.	false	BOOL	Yes	No
skip.trailer	This parameter is applicable only if the value for the 'mode' parameter is 'LINE'. If this parameter is set to 'true', only the last line (i.e., the trailer line) of a text file (e.g., CSV) will be skipped. If it is set to 'false', the full content of the file is read line by line.	false	BOOL	Yes	No
buffer.size	This parameter used to get the buffer size for binary.chunked mode.	65536	STRING	Yes	No
cron.expression	This is used to specify a timestamp in cron expression. The file or files in the given dir.uri or file.uri will be processed when the given expression satisfied by the system time.	None	STRING	Yes	No
file.name.pattern	Regex pattern for the filenames that should be read from the directory. Note: This parameter is applicable only if the connector is reading from a directory		STRING	Yes	No
file.system.options	The file options in key:value pairs separated by commas. eg:'USER_DIR_IS_ROOT:false,PASSIVE_MODE:true,AVOID_PERMISSION_CHECK:true,IDENTITY:file://demo/.ssh/id_rsa,IDENTITY_PASS_PHRASE:wso2carbon' Note: when IDENTITY is used, use a RSA PRIVATE KEY		STRING	Yes	No

Examples EXAMPLE 1

@source(type='file',
mode='text.full',
tailing='false'
 dir.uri='file://abc/xyz',
action.after.process='delete',
@map(type='json')) 
define stream FooStream (symbol string, price float, volume long);

In the above configuration, all the files in the given directory are picked and read one by one.
Here, it is assumed that all the files contain valid json strings with 'symbol', 'price', and 'volume' keys.
Once a file is read, its content is converted to events via the 'siddhi-map-json' extension. Those events are then received as input events in the the 'FooStream' stream.
Finally, after the reading is completed, the file is deleted.

EXAMPLE 2

@source(type='file',
mode='files.repo.line',
tailing='true',
dir.uri='file://abc/xyz',
@map(type='json')) 
define stream FooStream (symbol string, price float, volume long);

In the above configuration, the first file in '/abc/xyz' directory is picked and read line by line.
Here, it is assumed that the file contains lines json strings.
For each line, the line content is converted to an event via the 'siddhi-map-json' extension. Those events are then received as input events in the the 'FooStream' stream.
Once the file content is completely read, the system keeps checking for new entries added to the file. If it detects a new entry, it immediately picks it up and processes it.

EXAMPLE 3

@source(type='file',
mode='text.full',
tailing='false'
 dir.uri='file://abc/xyz',
action.after.process='delete',
@map(type='csv' @attributes(eof = 'trp:eof', fp = 'trp:file.path'))) 
define stream FooStream (symbol string, price float, volume long);

In the above configuration, all the files in the given directory are picked and read one by one.
Here, it is assumed that each file contains valid json strings with 'symbol', and 'price' keys.
Once a file is read, its content is converted to an event via the 'siddhi-map-json' extension with the additional 'eof' attribute. Then, that event is received as an input event in the 'FooStream' stream.
Once a file is completely read, it is deleted.

fileeventlistener (Source)¶

The 'fileeventlistener' component of the 'siddhi-io-fie' extension allows you to get the details of files that have been created, modified or deleted during execution time.Supports listening to local folder/file paths.

Syntax

@source(type="fileeventlistener", dir.uri="<STRING>", monitoring.interval="<STRING>", file.name.list="<STRING>", file.system.options="<STRING>", @map(...)))

QUERY PARAMETERS

Name	Description	Default Value	Possible Data Types	Optional	Dynamic
dir.uri	The path to the directory to be processed. During execution time, Siddhi by default processes all the files within this directory. However, if you have entered specific files to be processed via the 'file.name.list' parameter, only those files are processed. The URI specified must include the file handling protocol to be used for file processing.		STRING	No	No
monitoring.interval	The time duration (in milliseconds) for which the system must monitor changes to the files in the specified directory.	100	STRING	Yes	No
file.name.list	If you want to carry out processing for only for one or more specific files in the the given directory URI, you can use this parameter to specify those files as a comma-separated list. e.g., 'abc.txt,xyz.csv'		STRING	Yes	No
file.system.options	The file options in key:value pairs separated by commas. eg:'USER_DIR_IS_ROOT:false,PASSIVE_MODE:true,AVOID_PERMISSION_CHECK:true,IDENTITY:file://demo/.ssh/id_rsa,IDENTITY_PASS_PHRASE:wso2carbon' Note: when IDENTITY is used, use a RSA PRIVATE KEY		STRING	Yes	No

Examples EXAMPLE 1

@source(type='fileeventlistener', dir.uri='file://abc/xyz, file.name.list = 'xyz.txt, test') 
define stream FileListenerStream (filepath string, filename string, status string);
@sink(type='log')
define stream FooStream (filepath string, filename string, status string); 
from FileListenerStream
select *
insert into FooStream;

In the above configuration, the system monitors the given directory URI to check whether any file named either 'xyz.txt' or 'test' gets created, modified or deleted. If any such activity is detected, an input event is generated in the 'FooStream' stream. The information included in the event are the filepath, filename, and the status of the file.

EXAMPLE 2

@source(type='fileeventlistener',dir.uri='file://abc/xyz') 
define stream FileListenerStream (filepath string, filename string, status string);
@sink(type='log')
define stream FooStream (filepath string, filename string, status string); 
from FileListenerStream
select *
insert into FooStream;

In the above configuration, the system monitors the given directory URI to check whether any file gets created, modified or deleted. If any such activity is detected, an input event is generated in the 'FooStream' stream. The information included in the event are the filepath, filename, and the status of the file.

EXAMPLE 3

@source(type='fileeventlistener',dir.uri='file://abc/xyz', monitoring.interval='200')
define stream FileListenerStream (filepath string, filename string, status string);
@sink(type='log')
define stream FooStream (filepath string, filename string, status string);
from FileListenerStream
select *
insert into FooStream;

In the above configuration, the system monitors the given directory URI every 200 milliseconds to check whether any file gets created, modified or deleted. If any such activity is detected, an input event is generated in the 'FooStream' stream. The information included in the event are the filepath, filename, and the status of the file.

API Docs - v2.0.23¶

File¶

isDirectory (Function)¶

isExist (Function)¶

isFile (Function)¶

lastModifiedTime (Function)¶

size (Function)¶

archive (Stream Function)¶

copy (Stream Function)¶

create (Stream Function)¶

delete (Stream Function)¶

move (Stream Function)¶

rename (Stream Function)¶

search (Stream Function)¶

searchInArchive (Stream Function)¶

unarchive (Stream Function)¶

Sink¶

file (Sink)¶

Source¶

file (Source)¶

fileeventlistener (Source)¶