Paul Bunyan supports very powerful and flexible filter functionality. Filters are currently used in defining queries, configuring connections to message servers, and for selecting messages to be routed to message handlers or forwarded from other servers. In subsequent versions, they will also be used for configuring which messages a server prefilters (tosses) immediately and for prioritizing messages for rolloff deletion. All filters are ultimately used to screen log messages out – there are a bunch of log messages on the left, some subset of them make it through the filter and move on to the right where they are then viewed, handled, forwarded, imported, exported, whatever.
Filters are composed of one or more selectors. Each selector is in turn essentially a stack of field matching criteria for one or more log message fields. Selectors are used to select log messages. For instance, messages can be selected that were generated on 9/9/09 on the computer named “PROD A” or for that matter any of the computers whose names simply contain the sub-string “PROD ”. Or things can be broken up into two selectors to select all messages generated either on 9/9/09 OR on “PROD A”. Criteria can be specified for any of the fields using strings, sub-strings, time/date ranges, etc. and can be specified all in one selector to yield an ‘AND’ relationship or in separate selectors to yield a resultant set corresponding to an ‘OR’ relationship. Note that selectors don’t intrinsically include or exclude anything – they simply match log messages so that the containing filter can decide what to do with them.
Selectors are global to the workspace rather than being local to individual filters. Typically a number of named selectors are created and configured to isolate well-defined sets of log messages such as ‘Client’, ‘Server’, or ‘Connectivity Server’ messages. These selectors can then be used in different combinations in various queries to yield sets of messages pertinent to particular problems. For instance, a communication problem could be diagnosed by viewing the messages identified by the ‘Client’ and ‘Connectivity Server’ selectors. Or another query could be designed specifically for monitoring client / server transactions sans network traffic by including ‘Client’ and ‘Server’ messages while excluding ‘Connectivity Server’ messages. It is for this mix and match usage that selectors are made globally available in the workspace and can be reused when configuring queries, connections, message handlers, and any other objects with filters. Note also that selectors created in one workspace can be imported into another workspace and reused in that manner as well.
A filter is, two arrays of selectors and a flag that governs how those two arrays are used. The two selector arrays are referred to as the primary array and the exception array and the flag, referred to as the filter type, determines whether they are applied in inclusion or exclusion fashion. Semantically, an inclusion filter includes all messages matching any selector in the primary array except those matching any selector in the exception array. An exclusion filter excludes all messages matching any selector in the primary array except those matching any selector in the exception array. An easier way to think of it is to momentarily ignore the exception arrays and simply remember that an inclusion filter allows specification of only those messages to be included whereas an exclusion filter allows specification of only those messages to be excluded. One is simply chosen in favor of the other depending on which takes less work – mathematically speaking, they are functionally interchangeable inasmuch as the directive “Include ALL except XXX” is equivalent to “Exclude XXX”, just more work. Reintroducing the exception arrays on either filter type just provides that last bit of fine control over the resulting subset.