Amazon Q Business - Monitoring

Quiz

Monitoring Amazon Q Developer and AWS solution is crucial for their reliability, availability, and performance. You have to identify the cost of Amazon Q subscriptions for specific users and Cost Management, that's where monitoring is required.

AWS Monitoring Tools

AWS provides the following monitoring tools:

AWS CloudTrail: captures API calls and related events made by or on behalf of your AWS account and delivers the log files to a specified Amazon S3 bucket.
Amazon CloudWatch: monitors your AWS resources and applications in real time, allowing you to collect and track metrics, create custom dashboards, and set alarms that notify you or take actions when a specified metric reaches a threshold.
Amazon CloudWatch Logs: monitors, stores, and accesses log files from various sources, such as Amazon EC2 instances and CloudTrail. It also notifies you when certain thresholds are met based on the log file information.

CloudTrail Logs

Amazon Q Developer is integrated with AWS CloudTrail, which records all actions taken in Amazon Q. CloudTrail captures all Amazon Q API calls, including those from the console and code calls to API operations. You can enable continuous delivery of these events to an Amazon S3 bucket or view recent events in the CloudTrail console.

Interpreting Q Developer Log Files

A trail is a configuration that delivers events as log files to a specified Amazon S3 bucket. Each log file contains one or more log entries, which represent individual requests with details like the requested action, date, time, and parameters. Note that log files are not ordered by API call sequence.

The following example shows a CloudTrail log entry that demonstrates the SendMessage action.

{
    "eventVersion": "1.08",
    "userIdentity": {
        "type": "AssumedRole",
        "principalId": "AROAXD12ABCDEF3G4HI5J:aws-user",
        "arn": "arn:aws:sts::123456789012:assumed-role/PowerUser/aws-user",
        "accountId": "123456789012",
        "accessKeyId": "ASIAAB12CDEFG34HIJK",
        "sessionContext": {
            "sessionIssuer": {
                "type": "Role",
                "principalId": "AROAXD12ABCDEF3G4HI5J",
                "arn": "arn:aws:iam::123456789012:role/PowerUser",
                "accountId": "123456789012",
                "userName": "PowerUser"
            },
            "webIdFederationData": {},
            "attributes": {
                "creationDate": "2023-11-28T10:00:00Z",
                "mfaAuthenticated": "false"
            }
        }
    },
    "eventTime": "2023-11-28T10:00:00Z",
    "eventSource": "q.amazonaws.com",
    "eventName": "SendMessage",
    "awsRegion": "us-east-1",
    "sourceIPAddress": "123.456.789.012",
    "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0",
    "requestParameters": {
        "Origin": "https://conversational-experience-worker.widget.console.aws.amazon.com",
        "conversationId": "a298ec0d-0a49-4d2e-92bd-7d6e629b4619",
        "source": "CONSOLE",
        "conversationToken": "***",
        "utterance": "***"
    },
    "responseElements": {
        "result": {
            "content": {
                "text": {
                    "body": "***",
                    "references": []
                }
            },
            "format": "PLAINTEXT",
            "intents": {},
            "type": "TEXT"
        },
        "Access-Control-Expose-Headers": "x-amzn-RequestId,x-amzn-ErrorType,x-amzn-ErrorMessage,Date",
        "metadata": {
            "conversationExpirationTime": "2024-02-25T19:31:38Z",
            "conversationId": "a298ec0d-0a49-4d2e-92bd-7d6e629b4619",
            "conversationToken": "***",
            "utteranceId": "3b87b46f-04a9-41ef-b8fe-8abf52d2c053"
        },
        "resultCode": "LLM"
    },
    "requestID": "19b3c30e-906e-4b7f-b5c3-509f67248655",
    "eventID": "a552c487-7d97-403a-8ec4-d49539c7a03d",
    "readOnly": false,
    "eventType": "AwsApiCall",
    "managementEvent": true,
    "recipientAccountId": "123456789012",
    "eventCategory": "Management"
}

The following example shows a CloudTrail log entry that demonstrates Amazon Q calling the s3:ListBuckets action on your behalf.

{
    "eventVersion": "1.09",
    "userIdentity": {
        "type": "AssumedRole",
        "principalId": "AIDA6ON6E4XEGIEXAMPLE",
        "arn": "arn:aws:iam::555555555555:user/Paulo",
        "accountId": "555555555555",
        "accessKeyId": "AKIAIOSFODNN7EXAMPLE",
        "sessionContext": {
            "sessionIssuer": {
                "type": "Role",
                "principalId": "AIDA6ON6E4XEGIEXAMPLE",
                "arn": "arn:aws:iam::555555555555:user/Paulo",
                "accountId": "555555555555",
                "userName": "Paulo"
            },
            "attributes": {
                "creationDate": "2024-04-10T14:06:08Z",
                "mfaAuthenticated": "false"
            }
        },
        "invokedBy": "q.amazonaws.com"
    },
    "eventTime": "2024-04-10T14:07:55Z",
    "eventSource": "s3.amazonaws.com",
    "eventName": "ListBuckets",
    "awsRegion": "us-east-1",
    "sourceIPAddress": "q.amazonaws.com",
    "userAgent": "q.amazonaws.com",
    "requestParameters": {
        "Host": "s3.amazonaws.com"
    },
    "responseElements": null,
    "additionalEventData": {
        "SignatureVersion": "SigV4",
        "CipherSuite": "ECDHE-RSA-AES128-GCM-SHA256",
        "bytesTransferredIn": 0,
        "AuthenticationMethod": "AuthHeader",
        "x-amz-id-2": "ExampleRequestId123456789",
        "bytesTransferredOut": 4054
    },
    "requestID": "ecd94349-b36f-44bf-b6f5-EXAMPLE9c463",
    "eventID": "2939ba50-1d26-4a5a-83bd-EXAMPLE85850",
    "readOnly": true,
    "eventType": "AwsApiCall",
    "managementEvent": true,
    "recipientAccountId": "555555555555",
    "vpcEndpointId": "vpce-EXAMPLE1234",
    "eventCategory": "Management"
}

Monitoring with CloudWatch

Amazon Q Developer can be monitored using CloudWatch, which collects raw data and processes it into near real-time metrics. These metrics are stored for 15 months, allowing you to access historical data and track performance. You can also set alarms to notify you or take actions when certain thresholds are met.

The Amazon Q Developer service reports the following metrics in the AWS/CodeWhisperer namespace.

Dimensions	Metric	Use case or explanation
Count	Invocations	You want to determine how many invocations have been counted over time.
UserCount	DailyActiveUserTrend	You want to determine the number of active users per day.
SubscriptionUserCount	SubscriptionCount	You want to determine the number of users with paying subscriptions.
UniqueUserCount	MonthlyActiveUniqueUsers	You want to determine the number of users who are active in a given month.
ProgrammingLanguage, SuggestionState, CompletionType	GeneratedLineCount	You want to determine the number of lines generated by CodeWhisperer.
ProgrammingLanguage, SuggestionState, CompletionType	SuggestionReferenceCount	You want to determine the number of recommendation triggers with references that have taken place.
ProgrammingLanguage	CodeScanCount	You want to determine the number of code scans that have taken place.
ProgrammingLanguage	TotalCharacterCount	The number of characters in your file, including all suggestions from CodeWhisperer.
ProgrammingLanguage	CodeWhispererCharacterCount	The number of characters generated by CodeWhisperer.

Tracking Q Developer Usage

You can identify the cost of Amazon Q subscriptions for specific users with resource IDs through AWS Billing and Cost Management. To do this, create a data export in the console with the "Include resource IDs" option selected.

Print Page