Send logs to AWS CloudWatch from Aiven client#

This article will show you how you can send logs from your Aiven service to the AWS CloudWatch using the Aiven client.

Prerequisites#

This is what you’ll need to send your logs from the AWS CloudWatch using the Aiven client.

Aiven client installed.
An Aiven account with a service running.
An AWS account, and which region it is in.
An AWS Access Key and Secret Key. Generate the credentials by visiting IAM dashboard then click in Users, open the Security credentials tab, and choose Create access key. Click on Download as you will need this shortly.

Important

Your AWS credentials should have appropriate access rights. According to the official AWS documentation, the access rights required for the credentials are:

“logs:DescribeLogStreams” which lists the log streams for the specified log group endpoint.
“logs:CreateLogGroup” which creates a log group with the specified name endpoint.
“logs:CreateLogStream” which creates a log stream for the specified log group.
“logs:PutLogEvents” which uploads a batch of log events to the specified log stream.

Configure the integration#

Open the Aiven client, and log in:
```
avn user login <you@example.com> --token
```
See also

Learn more about avn user access-token
Collect the following information for the creation of the endpoint between your Aiven account and AWS CloudWatch. These are the placeholders you will need to replace in the code sample:

Variable	Description
`PROJECT`	Aiven project where your endpoint will be saved to.
`LOG_GROUP_NAME`	Used to group your log streams on AWS CloudWatch. It is an optional field. If the value is not provided, it’ll be generated for you.
`AWS_REGION`	The AWS region of your account.
`AWS_ACCESS_KEY_ID`	Your AWS access key ID.
`AWS_SECRET_ACCESS_KEY`	Your AWS secret access key.
`ENDPOINT_NAME`	Reference name for this log integration when linking it to other Aiven services.

Create the endpoint between your Aiven account and AWS CloudWatch.

avn service integration-endpoint-create --project PROJECT \
   -d ENDPOINT_NAME -t external_aws_cloudwatch_logs \
   -c log_group_name=LOG_GROUP_NAME \
   -c access_key=AWS_ACCESS_KEY\
   -c secret_key=AWS_SECRET_ACCESS_KEY \
   -c region=AWS_REGION

Collect the ENDPOINT_ID value. You should be able to see information about your endpoint by running:

avn service integration-endpoint-list --project PROJECT

As an example, an output could be something like:

   ENDPOINT_ID                           ENDPOINT_NAME        ENDPOINT_TYPE
====================================  ===================  ===============================
50020216-61dc-60ca-b72b-000d3cd726cb  ENDPOINT_NAME        external_aws_cloudwatch_logs

The output will provide you with the ENDPOINT_ID to identify your endpoint, your customized endpoint name and the endpoint type.

Send logs from an Aiven service to AWS CloudWatch#

Collect the following information for sending the service logs of an Aiven service to your CloudWatch:

Variable	Description
`PROJECT`	The Aiven project where your endpoint is saved.
`ENDPOINT_ID`	Reference name for this log integration when linking it to other Aiven services.
`AIVEN_SERVICE_NAME`	The Aiven service name that you want to send the logs from.

Send logs from the Aiven service to AWS CloudWatch by running:

avn service integration-create --project PROJECT\
   -t external_aws_cloudwatch_logs -s AIVEN_SERVICE_NAME \
   -D ENDPOINT_ID