MAN page from CentOS Other silk-common-3.19.1-3.el8.x86_64.rpm


Section: SiLK Tool Suite (5)
Updated: 2021-01-04


silk.conf - SiLK site configuration file 


The "silk.conf" SiLK site configuration file is used to associatesymbolic names with flow collection information stored in SiLK Flowrecords.

In addition to the information contained in the NetFlow or IPFIX flowrecord (e.g., source and destination addresses and ports, IP protocol,time stamps, data volume), every SiLK Flow record has two additionalpieces of information that is added when rwflowpack(8) converts theNetFlow or IPFIX record to the SiLK format:

The sensor typically denotes the location where the flow data wascollected; e.g., an organization that is instrumenting its borderrouters would create a sensor to represent each router. Each sensorhas a unique name and numeric ID.
The flowtype represents information about how the flow was routed(e.g., as incoming or outgoing) or other information about the flow(e.g., web or non-web). The packing process categorizes each flowinto a flowtype. Each flowtype has a unique name and numeric ID.

Note that the binary form of SiLK flow records represent the sensorand flowtype by their numeric IDs, not by their names.

For historic reasons, one rarely speaks of the flowtype of a SiLK Flowrecord, but instead refers to its class and type. Everyflowtype maps to a unique class/type pair. The classes and types havenames only; they do not have numeric IDs. Note that flowtype andtype are different concepts despite the similarity of their names.

A class is generally used to represent topological features of thenetwork with different collections of sensors, since every activesensor must belong by one or more classes. Every class must have aunique name.

A type is used to distinguish traffic within a single topological areabased on some other dimension. For example, incoming and outgoingtraffic is generally distinguished into different types. Web trafficis also frequently split into a separate type from normal traffic inorder to partition the data better. The type names within a classmust be unique, but multiple classes may have a type with the samename.

As stated above, each class/type pair maps to a unique flowtype.

The "silk.conf" file defines

the mapping between sensor names and sensor IDs
the names of the available classes
the sensors that belong to each class
the names of the types in each class
the mapping from a class/type pair to a flowtype ID
the mapping between a flowtype name and a flowtype ID
the default class to use for rwfilter(1) and rwfglob(1) queries
for each class, the default types to use for rwfilter andrwfglob
the layout of the directory tree for the SiLK archive relative to theroot directory
a default value for the --packing-logic switch to rwflowpack(8)

In normal usage, the "silk.conf" file will be located at the root ofthe SiLK data spool referenced by the SILK_DATA_ROOTDIR environmentvariable, or specified on the command line using the --data-rootdirflag. This ensures that the sensor and class definitions in the siteconfiguration match the data in the flow records you retrieve.

If you cannot place the site configuration file in the data rootdirectory, or the file in that location is incorrect, you can use theSILK_CONFIG_FILE environment variable to specify the location of yourconfiguration file (including the file name). Many SiLK commandsprovide the --site-config-file switch which allows you to specifythe name of the site configuration file on the command line.

By having the site configuration information outside of the SiLKtools, a single SiLK installation can be used to query different datastores (though each invocation of a command can only query one storagelocation).

Any additions or modifications to the "silk.conf" file will be seenby all SiLK applications upon their next invocation. There are someimportant things to keep in mind when modifying the "silk.conf" file:

Once data has been collected for a sensor or a flowtype, the sensor orflowtype should never be removed or renumbered. SiLK Flow files storethe sensor ID and flowtype ID as integers; removing or renumbering asensor or flowtype breaks this mapping. In order to keep the mappingconsistent, old sensor and flowtype definitions should remainindefinitely. Completely unused sensors or flowtypes may be removed,but the IDs of the remaining sensors and flowtypes must not bemodified.
The path to the files in the SiLK data store often involve the sensorname, flowtype name, class name, and/or type name. If any of thosenames are changed, it will be necessary to rename all the previouslypacked data files that have the former name as part of their path.
If the SiLK installation at your site is distributed across multiplehosts (for example, if packing occurs on a machine separate fromanalysis), it is important to synchronize changes to the "silk.conf"files.
The packing logic plug-in file, packlogic-*.so (e.g.,packlogic-twoway(3), packlogic-generic(3)), used byrwflowpack(8) checks for specific class names, type names, andflowtype names at start up, and it will exit with an error if thenames it expects do not exist. In addition, it checks that theflowtype IDs it has match with those in the "silk.conf" file. Whennew flowtypes are added, the packlogic-*.so file will need to beupdated if rwflowpack is to generate SiLK Flow records with the newflowtype.
When rwflowpack reads incoming flow records, those records areassociated with a sensor name as determined by the sensor.conf(5)file. rwflowpack uses the "silk.conf" file to map the sensor nameto the sensor ID, and it stores the sensor ID in the SiLK records itcreates. Changes to the "silk.conf" and "sensor.conf" files mayneed to be coordinated.


In the site configuration file, each line may be blank, or contain anyamount of leading whitespace, which is ignored. At any location in aline, the character "#" indicates the beginning of a comment, whichreaches until the end of the line. (If a literal "#" symbol isrequired in the argument of any command, it may be quoted as describedbelow.) These comments are ignored.

Each non-empty line begins with a command name, followed by one ormore arguments. Command names are a sequence of non-whitespacecharacters, not including the characters "#" or """ (see below forvalid commands). Arguments may either be textual atoms (any sequenceof non-whitespace characters, non-"#"-or-""" characters, includingnumerals and punctuation), or quoted strings. Quoted strings beginwith the character """ and end with the character """, and allow forC-style backslash escapes in between. The character "#" inside aquoted string does not begin a comment, and whitespace is allowedinside a quoted string.

For the commands supported by "silk.conf" and described below, unlessa command explicitly states that it is used by particularapplications, it should be considered used by all of the SiLK analysistools and the packing tools flowcap(8), rwflowpack(8), andrwflowappend(8).

There are three contexts for commands: top-level, class block, andgroup block contexts. The class block and group block contexts areused to describe individual features of classes and groups, whiletop-level commands are used to describe the entire configuration, andto define sensors.

The valid commands for each context are described below. 

Top-Level Commands

class class-name
The "class" command begins a new class block. It takes as anargument the name of the class being defined. Each class must have aunique name. A class block is closed with the "end class"command. See below for a list of commands valid inside class blocks.

The class name must begin with a letter, must not be longer than 32characters, and may not contain whitespace characters or the characterslash ("/").

A site that does not use multiple classes should define a single classwith a name like "all" or "default".

To be valid, a configuration file must contain at least one classdefinition.

Example: class all

default-class class-name
rwfilter(1) and rwfglob(1) will use a default class when theuser does not specify an explicit --class. This command specifiesthat default class; the class must have been created prior to thiscommand. If more than one default class is set, the last definitionencountered is used.

Example: default-class all

group group-name
Sensor groups are a convenient way of defining named groupings ofsensors for inclusion in classes. They cannot currently be used inthe SiLK command-line tools, but only in the configuration file. The"group" command takes as an argument the group to be defined, andbegins a group block. A group block is closed using the "end group"command. See below for details on valid commands within group blocks.

Example: group test-sensors

include file-name
The "include" command is used to include the contents of anotherfile. This may be used to separate large configurations into logicalunits. (Note, however, that all sensors, classes, groups, and typesmust be declared before they may be referenced.)

Example: include ``silk-2.conf''

packing-logic file-name
The "packing-logic" command provides a default value for the--packing-logic switch on rwflowpack(8). The value is the pathto a plug-in that rwflowpack loads; the plug-in provides functionsthat determine into which class and type a flow record will becategorized. The path specified here will be ignored when the--packing-logic switch is explicitly specified to rwflowpack orwhen SiLK has been configured with hard-coded packing logic.

Example: packing-logic ``''

path-format format-string
File and directory locations relative to the SILK_DATA_ROOTDIR maybe defined using the "path-format" command. The "path-format" isused by rwflowpack and rwflowappend(8) when writing data to thedata repository, and it is used by rwfilter and rwfglob whenreading or listing files in the data repository. This command takes aformat string specification that supports the following"%"-conversions:
The textual class name
The textual flowtype name for this class/type pair (see also %f)
The hour (24-hour clock) as a two-digit, zero-padded number
The textual sensor name (see also %n)
The textual type name
The year as a four-digit, zero-padded number
The day of the month as a two-digit, zero-padded number
The flowtype ID, as an unpadded number (see also %F)
The month of the year as a two-digit, zero-padded number
The sensor ID, as an unpadded number (see also %N)
The default file name, which is equivalent to "%F-%N_%Y%m%d.%H"
A literal "%" character

A "%" followed by any other character is an error.

For example, to place all spooled files directly in the data rootdirectory, the path format %x could be used. To use two levels ofhierarchy, the first containing the year and month, and the secondcontaining the day and sensor name, like "2006-01/23-alpha/...", theformat would be "%Y-%m/%d-%N/%x".

If no path format is set by the configuration file, the default pathformat of "%T/%Y/%m/%d/%x" is used.

All path formats are currently required to end in "/%x" so thatinformation may be extracted from the file name. This requirement maybe lifted in the future.

Example: %C/%T/%Y/%m/%d/%x

sensor sensor-id sensor-name
sensor sensor-id sensor-name sensor-description
Individual sensor definitions are created with the "sensor" command.This command creates a new sensor with the given name and numeric ID.Sensor names must begin with a letter, must not be longer than 64characters, and may not contain whitespace characters or thecharacters slash ("/") or underscore ("_").

The sensor line may may also provide an optional description of thesensor, enclosed in double quotes. The description can be usedhowever your installation chooses to use it. The description may beviewed by specifying the "describe-sensor" field to rwsiteinfo(1).(When using sensor descriptions, the file's "version" must be 2.)

It is an error to define two different sensors with the same sensor IDor the same sensor name.

NOTE: It is extremely important not to change the sensor-id orsensor-name for a given sensor once that sensor is in use. Thesensor-id field is stored numerically in SiLK data files, and thesensor-name field is used to construct file names within the dataroot directory.

Example: sensor 0 S001

Example: sensor 0 S001 ``Primary connection to ISP''

version version-number
The "version" command declares that this configuration file conformsto a given version of the configuration file format. If the tools donot support this version of the configuration file, they will reportan error. Currently, versions 1 and 2 of the format is defined,where version 2 indicates that sensor descriptions are present.

It is a recommended practice to include the version number at thebeginning of all configuration files for compatibility with futureversions.

Example: version 1


Class Block Commands

The commands inside a class block define the class's types, itsdefault types, the sensors that belong to it, and the mapping from theclass/type pair to the flowtype name and flowtype ID.
end class
The "end class" command ends the definition of a class. Following an"end class" command, top-level commands are again accepted.

Example: end class

default-types type-name ...
When no types are specified for the "rwfilter" or "rwfglob"commands, the default set of types for the selected class is used.Each of the types listed in this command is included as a default typeof the class.

Example: default-types in inweb

sensors sensor-name-or-group-ref ...
The "sensors" command is used to associate sensors with a class. Inshort, to declare that these sensors have data for this class. Eachitem in the list must be either the name of a sensor or the name of asensor group preceded by an at ("@") character. When you add asensor group, it is the same as individually adding each sensor inthat group to the class.

Example: sensors my-sensor-1 my-sensor-2 @my-group-1

type flowtype-id type-name [ flowtype-name ]
The "type" command defines a type name within the current class, andit specifies the flowtype ID to use for that class/type pair. Inaddition, the "type" command may specify a flowtype name. Theflowtype ID and flowtype name must be unique across the entire"silk.conf" file (and any included files). If a flowtype name is notspecified, a default flowtype name is constructed by concatenating thename of the class and the name of the type. (e.g. the type "in" inthe class "all" would have a flowtype name of "allin".) Within aclass, each type must have a unique name, but multiple classes may usethe same type name. The type name and flowtype name must begin with aletter, must not be longer than 32 characters, and may not containwhitespace characters or the character slash ("/").

As with sensors, it is important to be careful when renumberingflowtype IDs or renaming types or flowtypes because the numeric IDsare stored in data files, and the textual names are used as portionsof file and path names.

Example: type 0 in

Example: type 1 out out


Group Block Commands

A group block is a convenience used to define a list of sensors.
end group
Close a group block by using the "end group" command. Following thiscommand, top-level commands are again accepted.

Example: end group

sensors sensor-name-or-group-ref ...
Sensors are associated with a sensor group by means of the "sensors"command within a group block. Each item in the list must be eitherthe name of a sensor or the name of a sensor group preceded by an at("@") character. When you add a sensor group, it is the same asindividually adding each sensor in that group to the group beingdefined.

Example: sensors my-sensor-1 my-sensor-2 @my-group-1



rwfilter(1), rwfglob(1), rwsiteinfo(1), sensor.conf(5),flowcap(8), rwflowpack(8),packlogic-twoway(3),packlogic-generic(3), rwflowappend(8), silk(7),SiLK Installation Handbook



Top-Level Commands
Class Block Commands
Group Block Commands

This document was created byman2html,using the manual pages.