Configure Log Sources

A log source is a software or hardware component that produces log data. Multiple types of sources exist, and multiple methods exist to retrieve log data from the sources. The Alert Logic console allows you to create, edit, and update log sources, archive or restore old sources, and perform other tasks.

Alert Logic supports the following log source types:

All deployments:

  • Flat file logs—Log messages stored in flat text files collected by an Alert Logic agent or appliance
  • Syslog—A way for network devices to send event messages to a logging server
You cannot create a Syslog source through the Alert Logic console. You can only do so if you install an agent on a supported *NIX system, or if you configure a source device to send messages in the RFC 5424 Syslog format to an Alert Logic appliance or remote collector.
  • Windows event logs—Log messages captured by the Windows Event Log service, and collected by an Alert Logic agent or appliance

AWS deployments only:

  • AWS CloudTrail logs—Log messages generated by the AWS CloudTrail service that record AWS API activity in your account
  • AWS S3 logs—Log records that provide details about a single access request, such as the requester, bucket name, request time, request action, response status, and any error codes

Azure deployments only:

  • Azure Activity logs—Logs that provide insight into the operations performed on resources in your subscription
  • Azure App Service web server logs—Logs that provide detailed error information for HTTP failure status codes, failed requests, or HTTP transactions using the W3C extended log file format

After you provision and install the Alert Logic agent on your target host, the agent automatically creates an associated log source in the Alert Logic console and configures it with the default collection configuration policy for that log source type. You must create and configure new collection sources with existing collection policies to meet more specific requirements. For more information about Log Management policies, see Log Management policies.

Create and maintain flat file log collection sources

Before you can create a flat file collection source, you must create a flat file collection policy. For more information, see Create a flat file policy.

To create a flat file collection source:

  1. From the Deployments page, click the deployment for which you want to create a flat file collection source.
  2. Click Scope of Protection.
  3. Click CONFIGURE LOG SOURCES.
  4. Click the Add icon ( ).
  5. From Source Log Type, select Flat File Collection.
  6. In Source Name, enter a descriptive name.
  7. Select Enable Collection.
  8. Select Use an Appliance or Use an Agent, as appropriate to your setup.
    • To use a physical appliance to collect flat file logs, select a Collector and the corresponding IP address (of the log source, not the appliance).
    You can use only a physical appliance for remote flat file log collection.
    • To use an agent to collect flat file logs, select Use an Agent, and then Select a Host from the drop-down menu.
  9. Select Use an existing Policy or Create a New Policy.
  10. Under Collection Alerts, select one or more alert options.
  11. In the Tags field, type one or more easily filtered tags.
  12. Click SAVE.

To update a flat file collection source:

  1. From the Deployments page, click the deployment for which you want to update the log source.
  2. Click Scope of Protection.
  3. Click CONFIGURE LOG SOURCES.
  4. Place your cursor over the desired collection source and click the pencil icon ().
  5. Make the necessary updates.
  6. Click SAVE.

Syslog collection

You cannot create a Syslog source through the Alert Logic console. You can create a Syslog collection source only if you install an agent on a supported *NIX system, or if you configure a source device to send messages in the RFC 5424 Syslog format to an Alert Logic appliance or remote collector.

Create and maintain Windows event log collection sources

Before you can create a Windows event log collection source, you must create a Windows event log collection policy. For more information, see Create a Windows event log collection policy.

You can create remote Windows Eventlog sources in the Alert Logic console only through a physical appliance. Agent-based log sources are automatically created when you install and provision the Alert Logic agent on your target host.

To create a Windows event log collection source:

  1. From the Deployments page, click the deployment for which you want to create a Windows event log collection source.
  2. Click Scope of Protection
  3. Click CONFIGURE LOG SOURCES.
  4. Click the Add icon ().
  5. From Source Log Type, select Windows Event Log.
  6. In the Source Name field, type a descriptive name.
  7. Select Enable Collection.
  8. Under Collection Method, select a Collector, and type the IP Address.
  9. Determine whether you want to use an existing policy or create a new policy.
    • To use an existing policy, select Use an existing Policy, and then select a policy.
    • To create a new policy, select Create New Policy, and provide the necessary information. For more information, see Create a Windows event log collection policy.
  10. Under Collection Alerts, select one or more alert options.
  11. Select a Time Zone.
  12. In the Tags field, type an easily filtered tag.
  13. Click SAVE.

To update a Windows event log collection source:

  1. From the Deployments page, click the deployment for which you want to update the log source.
  2. Click Scope of Protection.
  3. Click CONFIGURE LOG SOURCES.
  4. Place your cursor over the desired collection source and click the pencil icon ().
  5. Make the necessary updates.
  6. Click SAVE.

Create and maintain AWS CloudTrail log collection sources

You must create an AWS CloudTrail log source in the Alert Logic console to collect CloudTrail logs. To complete this action, you need the following AWS account information:

  • SQS queue to which CloudTrail publishes events.
  • IAM role credentials

To create an AWS CloudTrail collection source:

  1. From the Deployments page, click the deployment for which you want to create a CloudTrail log collection source.
  2. Click Scope of Protection
  3. In the left navigation area, click CONFIGURE LOG SOURCES.
  4. Click the Add icon ().
  5. From Source Log Type, select AWS CloudTrail.
  6. In Source Name, type a descriptive name.
  7. Select Enable Collection.
  8. In Collection Alerts, click the field and select one or more alert options.
  9. In the SQS Queue Name field, type the name of the SQS queue you created to collect CloudTrail logs.
  10. From AWS Region, specify the region in which you created the SQS queue in the previous steps.
  11. Select or create a IAM Role.
    • To use an existing IAM Role, select Use an existing IAM Role, and then select the IAM Role to use.
    • To create a new IAM Role:
      1.  Select Create a new IAM Role, log into the AWS console and create a new IAM role. For more information, see Full permission deployment.
      2. In the Alert Logic console, complete the following fields:
        • In Credential Name, enter a descriptive name.
        • In Role ID, enter the Role ARN you created.
        • In External ID, enter the external ID.
  12. Click SAVE.

To update an AWS CloudTrail collection source:

  1. From the Deployments page, click the deployment for which you want to update the log source.
  2. Click Scope of Protection.
  3. Click CONFIGURE LOG SOURCES.
  4. Place your cursor over the desired collection source and click the pencil icon ().
  5. Make the necessary updates.
  6. Click SAVE.

Create and maintain AWS S3 log collection sources

Though this feature appears to all users, only those with an AWS account can utilize it.

To create an AWS S3 collection source:

  1. From the Deployments page, click the deployment for which you want to create an S3 collection source.
  2. Click Scope of Protection
  3. Click CONFIGURE LOG SOURCES.
  4. Click the Add icon ().
  5. From Source Log Type, select S3.
  6. In Source Name, type a descriptive name.
  7. Select Enable Collection.
  8. In Bucket, type the bucket name, followed by the directory name. This bucket name must use a DNS-compliant name. For more information, visit the AWS documentation site.
  9. s3bucketname/root_folder

  10. In File Name or Pattern, type the file name or date pattern of the file log.
  11. In Collection Policy:
    • To use an existing policy, select Use an existing Policy, and then select a policy.
    • To create a new policy, select Create New Policy and select the settings you want. For more information, see Create a S3 collection policy.
  12. In Collection Alerts, click the field and select one or more alert options.
  13. From Time Zone, select a time zone.
  14. Select or create a new IAM Role
    • To use an existing IAM Role, select Use an existing IAM Role. Next, in Existing IAM Role, select the IAM Role to use.
    • To create a new IAM Role:
      1.  Select Create a new IAM Role, log into the AWS console and create a new IAM role. For more information, see Full permission deployment.
      2. In the Alert Logic console, complete the following fields:
        • In Credential Name, enter a descriptive name.
        • In Role ID, enter the Role ARN you created.
        • In External ID, enter the external ID.
  15. In Collection Internal, type a value, in minutes, to indicate how often Log Manager retrieves S3 logs.
  16. In the Tags field, type an easily filtered tag.
  17. Click SAVE.

To update an AWS S3 collection source:

  1. From the Deployments page, click the deployment for which you want to update the log source.
  2. Click Scope of Protection.
  3. Click CONFIGURE LOG SOURCES.
  4. Place your cursor over the desired collection source and click the pencil icon ().
  5. Make the necessary updates.
  6. Click SAVE.

Create and maintain Azure Audit log collection sources

To create an Azure Audit log collection source:

  1. From the Deployments page, click the deployment for which you want to create an audit log collection source.
  2. Click Scope of Protection
  3. Click CONFIGURE LOG SOURCES.
  4. Click the Add icon ().
  5. From Source Log Type, select Azure Audit Logs.
  6. In the Source Name field, type a descriptive name.
  7. Select Enable Collection.
  8. Select one of the following:
    • To use an existing audit account, select Existing Audit Account and select the Azure account you want to use.
    • To create a new audit account, select Add new Audit Account and select the settings you want. Azure will ask you to create a new user name and password.

    If you select Add new Audit Account, verify the account has the proper permissions to allow Alert Logic to read the Azure Audit events.

    To properly set up a role with the minimum permissions required, you must create a custom role in Azure. For more information, read Create custom roles for Azure Role-Based Access Control.

    The role below provides a minimum set of permissions required for Audit Log collection:

    { 
    "Name": "<name of your role>",
    "Id": "<auto-assigned>",
    "IsCustom": true,
    "Description": "<description of the role>",
    "Actions": [
    "Microsoft.Authorization/*/read",
    "Microsoft.Insights/eventtypes/*/read"
    ],
    "NotActions": [
    ],
    "AssignableScopes": [
    "/subscriptions/<add your Subscription ID>" ] }
  9. In Collection Alerts, select one or more alert options.
  10. In Subscription ID, type your Azure Subscription ID.
  11. In Resource Group Filter, type a Resource Group name.
  12. In the Tags field, type an easily filtered tag.
  13. Click SAVE.

To update Azure Audit logs collection sources:

  1. From the Deployments page, click the deployment for which you want to update the log source.
  2. Click Scope of Protection.
  3. Click CONFIGURE LOG SOURCES.
  4. Place your cursor over the desired collection source and click the pencil icon ().
  5. Make the necessary updates.
  6. Click SAVE.

Create and maintain Azure App Service web server logs

Though this feature appears to all users, only those with an Azure account can utilize it.

To create an Azure App Service web server logs collection source:

  1. From the Deployments page, click the deployment for which you want to create an Azure App service web server collection source.
  2. Click Scope of Protection
  3. Click CONFIGURE LOG SOURCES.
  4. Click the Add icon ().
  5. From Source Log Type, select App Service Web Server Logging.
  6. In the Source Name field, type a descriptive name.
  7. Select Enable Collection.
  8. Select one of the following:
    • To use an existing storage account, select Existing Storage Account and select the storage account you want to use.
    • To create a new storage account, select Add new Storage Account and select the settings you want. Azure will ask you to create a new user name and password.

    In the Azure Portal, navigate to the storage account in which you store your logs, click Settings, and then click Access keys to view, copy, and regenerate your account access keys.

  9. In Collection Alerts, click the field and select one or more alert options.
  10. In App Service Name, type the name of your App Service Web application.
  11. In Storage Blob Container, type the storage account container name where your web server logging is located.
  12. In the Tags field, type an easily filtered tag.
  13. Click SAVE.

To update an Azure App Service web server logs collection source:

  1. From the Deployments page, click the deployment for which you want to update the log source.
  2. Click Scope of Protection.
  3. Click CONFIGURE LOG SOURCES.
  4. Place your cursor over the desired collection source and click the pencil icon ().
  5. Make the necessary updates.
  6. Click SAVE.

Azure SQL auditing logs

Though this feature appears to all users, only those with an Azure account can utilize it.

To create an Azure SQL database auditing logs collection source:

  1. From the Deployments page, click the deployment for which you want to create an Azure SQL auditing log collection source.
  2. Click Scope of Protection
  3. Click CONFIGURE LOG SOURCES.
  4. Click the Add icon ().
  5. From Source Log Type, select Azure SQL Auditing.
  6. In the Source Name field, type a descriptive name.
  7. Select Enable Collection.
  8. Select one of the following:
    • To use an existing storage account, select Existing Storage Account and select the storage account you want to use.
    • To create a new storage account, select Add new Storage Account and select the settings you want. Azure prompts you to provide your Credential Name, Storage Account Name, and Access Key.

    In the Azure Portal, navigate to your storage account where your logs are stored, click Settings, and then click Access keys to view, copy, and regenerate your account access keys.

  9. In Collection Alerts, select one or more alert options.
  10. In the Azure SQL Table Name, type your SQL Table Name between SQLDBAuditLogs and YYYYMMDD.
  11. In the Tags field, type an easily filtered tag.
  12. Click SAVE.

To update an Azure SQL database auditing logs collection source:

  1. From the Deployments page, click the deployment for which you want to update the log source.
  2. Click Scope of Protection.
  3. Click CONFIGURE LOG SOURCES.
  4. Place your cursor over the desired collection source and click the pencil icon ().
  5. Make the necessary updates.
  6. Click SAVE.

Mass edit collection sources

The Mass Edit feature allows you to edit policies and tags for all sources, filtered sources, or sources you specify. Also, mass edit contains a mass archive feature.

To mass edit log sources:

  1. On the Log Sources page, click the gear icon ().
  2. Select Mass Edit.
  3. In Apply changes to, select:
    • All Sources to mass edit all sources
    • Only Filtered Sources to mass edit only filtered sources
    • Only Selected Sources to mass edit only selected sources
  4. From Collection Policy, select the collection policy to use.
  5. From Replace Collection Alerts, select an alert to apply to the selected sources.
This action overrides the current alerts that correspond to the selected sources. If you leave this option blank, current alerts will not change.
  1. Select Enable Collection.
  2. In Tags, select a tag option from the drop-down menu. Below, type a tag to follow the rule selected in the drop-down menu.
  3. In Archive Sources, select an option from the drop-down menu.
  4. Click SAVE.

Archive and restore log sources

Archive a collection source to remove the log source entry from the Log Sources page, and make it available for use at a later time.

Archive a collection source

To archive a collection source:

  1. From the Deployments page, click the deployment for which you want to archive log sources.
  2. Click Scope of Protection
  3. Click CONFIGURE LOG SOURCES.
  4. Place your cursor over the desired collection source and click the Archive icon ().
  5. Click ARCHIVE.
You cannot archive a log host or collection source that stops log collection.

Restore an archived collection source

To restore an archived log source:

  1. From the Deployments page, click the deployment to which you want to restore an archived log source.
  2. Click Scope of Protection
  3. Click CONFIGURE LOG SOURCES.
  4. Above the log source table, select Show Archive.
  5. Place your cursor over the desired collection source and click the Archive icon ().
  6. Click RESTORE.

Additional tasks

View collection source information

To view information about a collection source:

  1. From the Deployments page, click a deployment.
  2. Click Scope of Protection.
  3. Click CONFIGURE LOG SOURCES.
  4. Place your cursor over the desired collection source and click on it. A tray will appear with three different tabs:
    • Details: This tab displays all information about the collection source, including the account number, the public host name, when it was created or modified, and the host ID.

    The Status field lists any current errors.

    • Metadata History: This tab displays only the metadata history for the collection source.
    • Status History: This tab displays only the status history, including the current status of the collection source.