Argent for Compliance all version

Date

Tuesday, 5 May 2015

Summary

Applications often write log debug information to ASCII/Text Files for software developers and system engineers alike to review and help resolve application issues when they occur

Argent for Compliance File Log Rule can help automate looking for key text phrases within these ASCII/Text Files

Technical Background

Argent for Compliance File Log Rule assumes the log file is a line-based ASCII or Unicode text file

In other words, the Engine always assume log records are delimited by a new line (LF) or a new line/carriage return combination (LF/CR)

Log File can either contain or does not contain date/time string for line records as a whole

Users cannot specify the File Log Rule so as to treat some parts of file containing date/time string and other parts are not

If this is required then a Custom VBScript/PowerShell Script Rule can be used

When a log file is deemed not containing date/time string, the ‘*’ should be used in the Date/Time format field

The Engine uses solely the character position in the file to remember what have been read before

Plus when an Alert is fired, the current machine time is used for the Event time

Example

File log exert from Windows log file:

‘ACME.Websites.Website.log4net-ACMEWBSVP05.Errors.log20150501.020150501.0’

To Alert on text phrase “ERROR ACME.Web.Global – Object reference not set to an instance of an object” the following setup is required

Create a new “Windows File Log Rule” from the “Windows File Log Rules” or in the case of UNIX or Linux from “LINUX/Unix File Log Rules”

Completed Rule: each section is discussed below

Click For Full Size

Log File: …ACME.Websites.Website.log4net-ACMEWBSVP05.Errors.log20150501.020150501.*

The wild character “*” acts as a multi file matcher when an application can potentially create multiple files e.g. sample.log.0, sample.log.1 etc

If multi file matching is required then note the “File Path Advanced Settings” option “Scanning Option”:

“The Latest File Only”

“All Matching Files Sequentially”

“All Matching Files In Parallel”

Date/Time: yyyy-MM-dd HH:mm:ss,mms

The sample log file has a date/time format as 2015-05-01 02:16:47,484

The format is configured from the “Format” and “Verify And Explain” buttons

Log Parsing Specification

The “Setup Wizard” button can aid in creating the initial specification

Note the “Separator Field” has a space ” &nbsp” by default, and is seen in the specification as Delimiter = ” ”

When the columns are delimited by a space this is the desired grouping:

To Alert on the required text phrase without distinguishing between [13] ERROR ACME.Web.Global… and [15] ERROR ACME.Web.Global…it is best to split the line data otherwise an Alert would be fired for each, even when “Event Format: Combine Events With Latest Description” is used

Example Test Rule Result:

Note that the sample log “ERROR” text spans multiple rows, therefore to match all text the option NewLineAfterTimeStamp = Yes can be used to distinguish differentiating error messages, but is not essential unless there is actually is a newline after the timestamp

Once the “Setup Wizard” is complete the Log Parsing Specification would be:

Fields structure with the log text:

Use the “Test Sample Data” button to confirm your settings are correct

Click For Full Size

Note “Line 1” and “Line 2” are combined when using NewLineAfterTimeStamp = Yes is used, this is normal and “Line 3” to “Line n” will appear in the Time column, this is normal

With the option NewLineAfterTimeStamp = No the Sample Data is:

Optional File Log Filter

To add a filter click the

button

The drop-down menu “Field: Description” holds the text that the text phrase in text-box “Text” is to match

Note: alternatively the drop-down menu “Field: The Whole Log Message” could be used especially if NewLineAfterTimeStamp = No is used since the Description data would not hold the extra text data

The tick-box option “Match Whole Words Only” avoids matching partial words

Option: Just Monitor

This option as the name suggests just monitors the log file and does not archive any of the log text

If log reporting is required, the option “Both Monitor And Archive File” should be used allowing the filtered text data to be saved to the Argent Database where the default table is “ARGSOFT_COMPLIANCE_LOG_ARCHIVE” or a nominated database of the users choosing

The “Optional File Log Filter” should be entered as seen on the “Monitoring” tab to the “Archiving” tab, unless all the log text is required for archiving

Use unfiltered with caution, as the database can grow unpredictably

Test Rule Results

With NewLineAfterTimeStamp = Yes

Click For Full Size

With NewLineAfterTimeStamp = No

Click For Full Size

Note that “NewLineAfterTimeStamp = Yes” throws multiples Events/Alert while “No” only throws one Event/Alert

References

Configuring File Log Rules

https://help.argent.com/#wp_file_log

KBI 310897 Enhancement: Log File Information Added To File Log Rule

(Argent for Compliance 3.1A-1404 and above)
https://help.argent.com/#KBI_310897

KBI 310789 Issue Addressed: Windows File Log Rule Generates Combined Events Showing Only The Last Two Occurrences

(Argent for Compliance 3.1A-1310-A or below)
https://help.argent.com/#KBI_310789

KBI 310700 New Feature: New Time Format Keywords In File Log Rule

(Argent for Compliance 3.1A-1310-A and later)
https://help.argent.com/#KBI_310700

KBI 310735 New Feature: New Time Format Keywords In File Log Rule Of Argent For Compliance

(Argent Global Manager 3.1A-1310-C and above)
https://help.argent.com/#KBI_310735

KBI 311142 Issue Addressed: File Log Lines Are Not Read When File Server Time Is Out Of Sync And Ignore Minutes Are Too Small

(Argent Advanced Technology all versions)
https://help.argent.com/#KBI_311142

Resolution

N/A