Alert Logic Log Collector for Microsoft Azure Event Hubs
Microsoft Azure Event Hubs is a data streaming platform and event ingestion service that can receive and process millions of events per second. Alert Logic allows you to configure Event Hubs to collect your Azure logs and forward them to Alert Logic.
To grant Alert Logic permission to access Event Hubs for log collection, you must have:
- A Microsoft Azure account with administrative privileges
- An Alert Logic user account with administrative privileges
Alert Logic recommends that you create Azure deployments in the Alert Logic console for each Azure subscription you want to use to collect logs from Event Hubs.
Create an Alert Logic access key
Access keys store your customer information to allow integration of data from your Microsoft Azure account to Alert Logic integrations and APIs. You can create an access key through the Alert Logic console. For more information, see Create and Manage Alert Logic Access Keys .
Download and deploy the ARM template
The Azure Resource Manager (ARM) is the Azure API that allows you to use JSON-formatted ARM templates to deploy resources. You can deploy the ARM template configured to create a new Event Hub along with the log collector, or you can deploy the ARM template configured to subscribe the collector to an existing Event Hub.
Deploy the custom Azure Resource Manager template for a new Event Hub
Alert Logic provides a custom ARM template that you must configure to allow communication between Azure and Alert Logic.
To deploy the ARM template:
- Access the ARM template page in Azure.
-or-
Access the Premium plan ARM template page in Azure, which will deploy an Azure Functions Premium plan with virtual network integration and private endpoints for accessing the storage account of the Azure Functions app. Note that deploying a with the Premium plan template will incur additional costs in your Azure account. For more information, refer to the Azure Functions Premium plan overview. - Provide the following template parameters:
- Application Name—Type the name of the log source to appear in the Alert Logic console.
- Alert Logic Access Key ID—Type the access key ID generated when you created the Alert Logic access key.
- Alert Logic Secret Key—Type the secret key generated when you created the Alert Logic access key.
- Alert Logic API endpoint—Do not change the default parameter value (api.global-services.global.alertlogic.com).
- Alert Logic Data Residency—Do not change the default parameter value (default).
- Event Hub Resource Group—Leave parameter value blank.
- Event Hub Connection String—Leave parameter value blank.
- Event Hub Namespace—Leave parameter value blank.
- Event Hub Name—Type the name of the existing Event Hub.
- Event Hub Max Throughput Units—(Optional) The maximum number of throughput units for the Event Hub.
- Event Hub Auto Inflate Enabled—(Optional) Whether auto-inflate is enabled for the Event Hub.
- Event Hub Partition Count—(Optional) The number of partitions for the Event Hub.
- Event Hub Retention Days—(Optional) The number of days to retain data in the Event Hub.
- Event Hub Consumer Group—Do not change the default parameter value ($Default).
- Event Hub Filter JSON—(Optional) If you want to filter messages and use JSON format to define the filter, type a filter in JSON format (for example, {resultType":"Success"}). Only messages that contain the specified property are collected. If both JSON and regex filter values are provided, logs are collected based on both the values. If you do not want to filter messages, leave this blank.
- Event Hub Filter Regex—(Optional) If you want to filter messages and use regex format to define the filter, type a filter in regex format (for example, /*.Policy or "Policy"). Only messages that contain the specified property are collected. If both JSON and regex filter values are provided, logs are collected based on both the values. If you do not want to filter messages, leave this blank.
- Enable Application Insights—(Optional) Enable or disable Application Insights for monitoring invocation logs. Default value is No. For more information on monitoring Azure functions, refer to Microsoft's guide.
- Select I agree to the terms and conditions stated above.
- Click Purchase.
The new Event Hub you created with this ARM template uses the following event hub scaling parameters:
- Maximum throughput units: 20
- Auto-inflate: Enabled
- Partitions: 32
- Data retention period: 7 days
If you want to use other hub scaling parameter values, you can edit the template or contact Alert Logic for assistance.
To use other hub scaling parameter values:
- On the Azure Custom deployment page, click Edit template, and then make the appropriate changes to the following lines:
- "newEhubMaxThroughputUnits": xx,
- "newEhubPartitionCount": xx,
- "newEhubRetentionDays": xx
- Click Save.
Adjust firewall rules in Event Hub
You must configure the firewall configuration for Event Hub to receive messages.
To adjust the firewall:
- In the new Event Hub you created, on the left panel, under Settings, click Networking.
- In the Public Access tab, for Public network access, select Selected networks.
- In the Firewall section, add your required IP address.
- Select Yes to Allow trusted Microsoft services to bypass this firewall.
- Click Save.
Deploy the custom Azure Resource Manager template for an existing Event Hub
If you want to deploy the ARM template for an Event Hub that you already configured, you must provide additional parameter values when you deploy the Alert Logic custom ARM template.
To deploy the ARM template:
- Access the ARM template page in Azure.
-or-
Access the Premium plan ARM template page in Azure, which will deploy an Azure Functions Premium plan with virtual network integration and private endpoints for accessing the storage account of the Azure Functions app. Note that deploying a with the Premium plan template will incur additional costs in your Azure account. For more information, refer to the Azure Functions Premium plan overview. - Provide the following required template parameters:
- Application Name—Type the name of the log source to appear in the Alert Logic console.
- Alert Logic Access Key ID—Type the access key ID generated when you created the Alert Logic access key.
- Alert Logic Secret Key—Type the secret key generated when you created the Alert Logic access key.
- Alert Logic API endpoint—Do not change the default parameter value (api.global-services.global.alertlogic.com).
- Alert Logic Data Residency—Do not change the default parameter value (default).
- Event Hub Resource Group—Type the resource group for the existing Event Hub.
- Event Hub Connection String—Type the connection string for the existing Event Hub.
- Event Hub Namespace—Type the namespace for the existing Event Hub.
- Event Hub Name—Type the name of the existing Event Hub.
- Event Hub Max Throughput Units—(Optional) The maximum number of throughput units for the Event Hub.
- Event Hub Auto Inflate Enabled—(Optional) Whether auto-inflate is enabled for the Event Hub.
- Event Hub Partition Count—(Optional) The number of partitions for the Event Hub.
- Event Hub Retention Days—(Optional) The number of days to retain data in the Event Hub.
- Event Hub Consumer Group—Type the name of the consumer group of the existing Event Hub.
If the Event Hub has other consumers, create a separate consumer group for the Alert Logic collector and use that name here. For more information, see the Microsoft document, Consumer Groups.
- Select I agree to the terms and conditions stated above.
- Click Purchase.
You ensure the firewall is configured for Event Hub to receive messages. To adjust the firewall, see Adjust firewall rules in Event Hub.
Verify the deployment
After you deploy the template, Alert Logic recommends you perform the following steps to verify the template deployed successfully:
- In the Alert Logic console, select the Azure deployment for the Azure subscription where you deployed this log collector.
- Click CONFIGURE LOG SOURCES, and then filter the list by Push (Office 365, EventHub) collection method.
- Verify that the new Event Hubs log source with the name you provided during deployment appears with the source status of "OK."
If the Event Hub log source does not appear, you can:
- Wait approximately 30 minutes, and see if the new Event Hub log source appears.
- Remove the Azure resource group that contains the log source, and deploy the ARM template again.
- Contact Alert Logic Technical Support at:
- US: (877) 484-8383 (Option 1)
- UK: +44 (0) 203 011 5533 (Option 1)
Integrate with Azure Event Hubs
The following links contain instructions to help you integrate different Azure services with Event Hubs.