Application Logs for Flat File Configuration

The documentation below describes the new version of the Alert Logic console, which was recently updated. This version will become the default in early 2020. For more information about the new navigation, see Managed Detection and Response Navigation Menu Updates.

The application logs feature allows you to configure applications with functional APIs to automatically collect logs from multiple sources, using tag and topology-based rules in the Alert Logic console. You can also copy an existing log configuration to create a new one, and edit it as necessary.

The log collection configuration is a streamlined workflow with specific application log templates. Log collection allows you to configure flat file logs, which are log messages stored in flat text files. Flat files are common a log message format for web servers and other server software.

To access the log collection setup, click the menu icon () from the Dashboards page. Click Configure, and then click the deployment for which you want to configure collections. On the left navigation panel, click Logs, and then click Application Logs.

This feature is only available for Managed Detection and Response Professional and Enterprise customers who have no legacy flat file configurations.

Application log collections

The Application Logs page lists applications logs that have templates with predefined fields, including ones that are not yet enabled. Click the drop-down menu to view all application logs or only view existing application logs that are enabled. You can use the search bar to find a specific log collection definition. You can preview, edit, duplicate, add or remove rules from existing application logs, and add new application logs.

Add a new application log with rules

The collection method and policy determines which flat file log messages to collect, how to separate log messages within a flat file, and how to read the time of each log message. Rules determine the asset tags, assets, or topology elements from which Alert Logic collects logs.

After you have filled out all required fields and scoped rules, you must turn on Collect for the application to start collecting log data.

To add a new application log with rules:

  1. Click the add icon ().
  2. Under Name Application, type a name for your application in the field.
  3. To automatically enable collection from the application, select Collect.
  4. Under Collection Method and Policy, in the Application File Path field, type the path information.
  5. In the File Name or Pattern field, type the file name or date pattern of the flat file name. Alert Logic can only collect flat file log messages that match the pattern.
    htaccess.* is a file name with a pattern. The * represents the time stamp of the flat file log. Alert Logic accepts a variety of date formats.
  6. In the File Name Rotation Scheme section, select a file name rotation scheme. The format must match the format of your flat file log name. Alert Logic recommends you specify the rotation scheme format of your flat file name.
    The default Auto-Detect identifies many rotation schemes. If you are unsure of the format, or if you do not find the specific format from the drop-down menu, select Auto-Detect.
  7. In the Multiline Handling section, select a multiline handling option:
    • If all of your flat file messages contain log entries with a single or separate line, select Single line log messages.
    • If all of your flat file log messages contain log entries that span multiple lines, select Log messages with multiple lines, and then select and enter a configuration:
      • If the lengths of your log messages are consistent, select Each log message spans a fixed number of lines, and then specify the number of lines.
      • If the lengths of your log messages are not consistent:
        1. Select Each log message follows a known pattern.
        2. Select the appropriate Pattern application.

          Pattern application options:

          • At the beginning of message: A line that matches the specified pattern marks the beginning of a new message; non-matching lines are grouped into the prior message.
          • In the middle of message: A line that does not match the specified pattern marks the middle of a new message; matching lines are grouped into the prior message.
          • At the end of message: A line that matches the specified pattern marks the end of a message; non-matching lines prior to that are grouped into this message.
        3. Type the Pattern for the log message.
        4. If your pattern is a Perl Compatible Regular Expression (PCRE), select Regular expression.
  8. In Timestamp Rule section, select a timestamp rule option:
    • To use the timestamp from the collector, select Set message time as collect time.
    • To use an existing timestamp, select Parse file name using a pre-defined timestamp format, and then choose a format from Select a format.
    • To use a custom timestamp, select Parse file name using a custom timestamp format, and then enter a format for the date string in the expanded configuration area. In the Format of date string field, type a format for the date string, and follow the on-screen instructions.
  9. Click SAVE AND NEXT.
  10. In the Add Rules section, search for assets from which to collect logs.
  11. Select the tags or topology elements from the drop-down list. Tags are separated by AND, and topology elements are separated by OR. Tags and topology elements are separated by OR.
  12. Click SAVE.
  13. Ensure Collect is turned on for the application you just created in the Application Logs list.

Enabled application logs

To view enabled application logs only, click the drop-down menu in the Application Logs list, and then select Enabled Logs. You can also use the search bar to find a specific application log collection. Click Preview to view details on enabled application logs.

Edit an existing application log

To edit an existing application log:

  1. From the preview, click the edit icon () for the application log collection you want to edit.
  2. Make the necessary changes. For more information, see Application Logs for Flat File Configuration.
  3. Click SAVE.

Duplicate an existing application log

To duplicate an existing application log:

  1. From the preview, click the duplicate icon () for the application log collection you want to copy.
  2. Make the necessary changes. For more information, see Application Logs for Flat File Configuration.
  3. Click SAVE.

Add or remove rules

  1. From the preview, click the rules icon () for the application log collection you want to scope rules.
  2. In the Add Rules section, select or clear the tags or topology elements from the drop-down list.
  3. Click SAVE.