Log Review Expert-Enhanced Machine Learning Upgrade

Alert Logic is upgrading the Log Review process with new machine learning algorithms, which allows Alert Logic to deliver a higher level of security value. The Log Review Expert-Enhanced Machine Learning algorithms can automatically detect many log-based anomaly types based on unique patterns and trends learned from your organization.

The Log Review Expert-Enhanced Machine Learning output consist of anomalies generated from the machine learning based detection and escalations raised by Alert Logic analysts during the review process. Alert Logic offers the following detailed reports of your log anomalies with the machine learning Log Review algorithms:

UNIX- and AWS-related Log Review escalations are raised as information level incidents in Alert Logic console in the Incidents page, and are also listed in the Log Review incident section of the Log Anomaly Analysis reports. Anomalies raised based on customer-level, host-level, user-level models and Windows Administrator-related activities are summarized in tables in the Log Anomaly Analysis reports.

Alert Logic analysts continue to oversee Log Review incidents, and the Log Review Expert-Enhanced Machine Learning feature continues to meet your log review compliance requirements.

About machine learning based on Log Review process

The machine learning model-based Log Review is fast, efficient, and accurate, which increases the likelihood of detecting most anomalies through log data. The Log Review Expert-Enhanced Machine Learning model can detect more than 100 anomaly scenarios based on time series, location, and unusual names. The machine learning models are computed based on specific logs customers sent to Alert Logic and are based on at least 90 days worth of data. Anomaly detections can also be triggered by rule-based analytics, which allows anomalies to be detected automatically and reliably, based on your patterns and trends.

Log Review Expert-Enhanced Machine Learning examines security-related logs that alert you to potential security issues, and assist with compliance mandates. Log anomalies are automatically raised based on:

  • unusual counts of certain events
  • unique users accessing a host
  • unusual or suspicious user names
  • blacklists
  • user preference

Examples of log data that Alert Logic reviews are:

  • Windows: Failed logins, changes to privileges, changes to accounts, Active Directory global catalog changes, and others
  • Linux: Sudo access, SSH failed logins, switched user common success/fails, and others
  • AWS: MFA, security group changes, IAM, EC2, S3 changes, user account and access changes, network control changes, and more

The Log Review Expert-Enhanced Machine Learning algorithm then observes and learns patterns and trends, and automatically tunes itself for more accurate security content. Anomalies are then computed and reported in the Daily Log Anomaly Analysis and Monthly Log Anomaly Analysis reports available to you.

Log Review Expert-Enhanced Machine Learning models and outcomes

The Log Review Expert-Enhanced Machine Learning is based on three models:

  • Customer model: Tracks and learns from log data related to customer activity
  • Host model: Tracks and learns from log data related to host changes
  • User model: Tracks and learns from log data related to user activity

The Log Review Expert-Enhanced Machine Learning outcomes include anomaly detections based on customer trends, anomalies based on user trends, and anomalies based on host trends. The following are anomalies that Alert Logic uses to generate incidents for all machine learning models:

Anomaly

Description

high-message-count

a spike in the number of log messages of a given type

high-sourceuser-count

a spike in the number unique source users for a given message type and host

high-targetuser-count

a spike in the number unique target users for a given message type and host

high-sourcehost-count

a spike in the number unique source hosts for a given message type and host/user

high-targethost-count

a spike in the number unique target hosts for a given message type and host/user

unusual-name

a user name is different from the normal names encountered

unusual-location

a host location is different from the normal locations encountered

Alert Logic analysts also generate incidents based on pattern matching and rule-based detection based on your preferences. These include the following Windows administrator activities:

  • High log message count for Windows failed login
  • High log message count for Windows account changed
  • Total message count for Windows account changed

Customer model

The Log Review Expert-Enhanced Machine Learning customer model includes anomalies based on customer trends of log message types for high overall count.

Model Anomaly Included details
Customer

high-message-count

customer

target

time range

log source

customer id and name

message type (parser)

implicated hosts

implicated users

expected count

actual count

Host model

Log Review Expert-Enhanced Machine Learning host model includes anomalies based on host trends of log message types for high count, unusual location and unique users.

Model Anomaly Included details
Host

high-message-count

time range

log source

customer id and name

message type

host IP/name

implicated users

implicated event details

expected count

actual count

Host

high-sourceuser-count

high-targetuser-count

time range

log source

customer id and name

message type

host IP/name

implicated users

implicated event details

expected count

actual count

Host

unusual-location

time range

log source

customer id and name

message type

host IP/name

implicated users

implicated event details

location

User model

The Log Review Expert-Enhanced Machine Learning host model includes anomalies based on user trends of log message types for a high overall count, and anomalies regarding unusual names.

Model Anomaly Included details
User

unusual-name

time range

log source

customer id and name

message type

user name

implicated hosts

implicated event details

User

high-message-count

time range

log source

customer id and name

message type

user name

implicated hosts

implicated event details

expected count

actual count

User

high-sourcehost-count

high-targethost-count

time range

log source

customer id and name

message type

user name

implicated hosts

implicated event details

expected count

actual count