Amazon Q Business - Monitoring



Monitoring is crucial for ensuring the reliability, availability, and performance of Amazon Q Business and other AWS solutions. AWS offers following monitoring tools to track Amazon Q Business, detect issues, and trigger automatic actions when needed.

  • AWS CloudTrail tracks and records all API calls and related events made within your AWS account, storing the logs in a specified Amazon S3 bucket. This allows you to identify the users and accounts making the calls, their source IP addresses, and the timestamps of the calls.
  • Amazon CloudWatch keeps a real-time eye on your AWS resources and applications. It helps you collect and track metrics, create custom dashboards, and set alarms that alert you or take action when a metric reaches a set threshold. For instance, it can track CPU usage of your Amazon EC2 instances and automatically launch new ones when needed.
  • Amazon CloudWatch Logs helps you monitor, store, and access log files from various sources like Amazon EC2 instances and CloudTrail. It can track information in these logs and alert you when certain thresholds are met, and also allows you to archive your log data in a secure and durable storage.

Logging API calls using AWS CloudTrail

Amazon Q Business Logging Amazon Q Business API calls using AWS CloudTrail captures all API calls for Amazon Q Business as events. The calls captured include calls from the Amazon Q console and code calls to the Amazon Q API. This allows you to track API calls made by or on behalf of your AWS account.

A trail is a configuration that enables delivery of events as log files to an Amazon S3 bucket that you specify. CloudTrail log files contain one or more log entries. An event represents a single request from any source and includes information about the requested action, the date and time of the action, request parameters, and so on.

Here is an example of a CloudTrail log entry that demonstrates the CreateApplication action:

{
    "eventVersion": "1.08",
    "userIdentity": {
        "type": "AssumedRole",
        "principalId": "principal ID",
        "arn": "ARN",
        "accountId": "account ID",
        "accessKeyId": "access key ID",
        "sessionContext": {
            "sessionIssuer": {
                "type": "Role",
                "principalId": "principal ID",
                "arn": "ARN",
                "accountId": "account ID",
                "userName": "user name"
            },
            "webIdFederationData": {},
            "attributes": {
                "creationDate": "yyyy-mm-ddThh:mm:ssZ",
                "mfaAuthenticated": "false"
            }
        }
    },
    "eventTime": "yyyy-mm-ddThh:mm:ssZ",
    "eventSource": "qbusiness.amazonaws.com",
    "eventName": "CreateApplication",
    "awsRegion": "region",
    "sourceIPAddress": "region",
    "userAgent": "user agent",
    "requestParameters": {
        "name": "name",
        "roleArn": "description",
        "clientToken": "client token"
    },
    "responseElements": {
        "applicationId": "application ID"
    },
    "requestID": "request ID",
    "eventID": "event ID",
    "readOnly": false,
    "eventType": "AwsApiCall",
    "managementEvent": true,
    "recipientAccountId": "account ID",
    "eventCategory": "Management",
    "tlsDetails": {
        "tlsVersion": "TLS version",
        "cipherSuite":  "cipher suite",
        "clientProvidedHostHeader": "qbusiness.us-west-2.api.aws"
    }
}

Logging Apps API calls using AWS CloudTrail

Amazon Q Apps is integrated with AWS CloudTrail, a service that records actions taken by users, roles, or AWS services. CloudTrail captures all API calls to Amazon Q Apps as events, including calls from the web experience, console, and code calls to API operations.

Understanding Amazon Q Apps log file entries

A trail is a setup that sends event logs to an Amazon S3 bucket you choose. These log files contain one or more log entries, each representing a single request with details like the action, date, time, and request parameters. Note that these log files don't show the API calls in a specific order, but rather as individual events.

The following example shows a CloudTrail log entry that demonstrates the GetLibraryItem action.

{
    "eventVersion": "1.09",
    "userIdentity": {
        "type": "AssumedRole",
        "principalId": "principal ID",
        "arn": "ARN",
        "accountId": "account ID",
        "accessKeyId": "access key ID",
        "sessionContext": {
            "sessionIssuer": {
                "type": "Role",
                "principalId": "principal ID",
                "arn": "ARN",
                "accountId": "account ID",
                "userName": "user name"
            },
            "attributes": {
                "creationDate": "yyyy-mm-ddThh:mm:ssZ",
                "mfaAuthenticated": "false"
            }
        },
        "onBehalfOf": {
            "userId": "user ID",
            "identityStoreArn": "ARN"
        }
    },
    "eventTime": "yyyy-mm-ddThh:mm:ssZ",
    "eventSource": "qapps.amazonaws.com",
    "eventName": "GetLibraryItem",
    "awsRegion": "region",
    "sourceIPAddress": "source IP address",
    "userAgent": "user agent",
    "requestParameters": {
        "input": "query input",
        "idc-application-arn": "ARN",
        "application-id": "Q application ID"
    },
    "requestID": "request ID",
    "eventID": "event ID",
    "readOnly": true,
    "eventType": "AwsApiCall",
    "managementEvent": true,
    "recipientAccountId": "account ID",
    "eventCategory": "Management"
}

Monitoring with Amazon CloudWatch

You can monitor Amazon Q Business using Amazon CloudWatch, which collects data and turns it into easy-to-read metrics in near real-time. These metrics are stored for 15 months, allowing you to access historical data and understand how your web application or service is performing over time. You can also set up alarms to watch for specific thresholds and receive notifications or take actions when they're met.

Use CloudWatch Metrics

To use metrics, you must specify the following information:

  • A namespace is a container in CloudWatch where Amazon Q publishes its metrics. To view Amazon Q Business metrics using the CloudWatch ListMetrics API or the list-metrics command, use the namespace "AWS/QBusiness".
  • A metric dimension is a name-value pair that helps identify a metric. For example, "ApplicationId" can be a dimension name. You don't have to specify a metric dimension, it's optional.
  • The metric name. For example, DocumentsIndexed.

The following table shows some common uses for the metrics. These are suggestions to get you started, not a comprehensive list.

How do I? Relevant metrics
How do I track how many documents were indexed successfully? Use the DocumentsIndexed metrics.
How do I monitor end user experience? Use the ThumbsUpCount and ThumbsDownCountmetrics.

View Amazon Q Business Metrics

The following steps show how to access Amazon Q Business metrics using the CloudWatch console.

  • Open the CloudWatch console
  • Choose Metrics, choose the All Metrics tab, and then choose AWS/QBusiness.
  • Choose the metric dimension.
  • Choose the metric that you want from the list, and choose a time period for the graph.

Amazon Q Business Chat Metrics

The following table shows the Chat and conversation management metrics that Amazon Q Business sends to CloudWatch in real time.

Metric name Unit Description
ActionErrorCount Count The number of errors because of actions.
Valid dimensions: ApplicationId, PluginId
ActionInvocationCount Count The number of actions invoked.
Valid dimensions: ApplicationId, PluginId
ChatMessages Count The number of chat messages This metric is emitted every time a chat message is processed.
Valid dimensions: ApplicationId
ChatMessagesWithAttachment Count The number of chat messages with file uploads.
Valid dimensions: ApplicationId
DailyActiveUsers Count The number of active users from the previous day.
Valid dimensions: ApplicationId

Amazon Q Business Index Metrics

The following table shows the Index metrics that Amazon Q Business sends to CloudWatch in real time.

Metric name Unit Description
DocumentCount Count The number of documents. This metric is published every 15 minutes.
Valid dimensions: ApplicationId, IndexId
DocumentsIndexed Count The number of documents that were indexed.
Valid dimensions: ApplicationId, IndexId, DataSourceId
DocumentsFailedToIndex Count The number of documents that failed to index.
Valid dimensions: ApplicationId, IndexId, DataSourceId
DocumentsFailedToIndexDueToCDE Count The number of documents that failed to index because of custom document enrichment.
Valid dimensions: ApplicationId, IndexId, DataSourceId
ExtractedTextSize MB Size of the extracted text
Valid dimensions: ApplicationId, IndexId
Advertisements