CloudWatch - integration with Spring application

This document is a quick summary of the all steps that you need to do to have a logs in CloudWatch in a Spring Application. I won't go into detail about each step here because I want to gather all of that different areas needed to accomplish wider goal and I want this manual to be concise and short. 

📌 #1 IAM role 

IAM policy CloudWatchLogsFullAccess needs to be added to IAM Role attached to your ec2.

📌 #2 Spring and Logback

At the very beginning you need as Maven Dependency the Logback library: 

  
  <dependency>
    <groupId>ch.qos.logback</groupId>
    <artifactId>logback-classic</artifactId>
    <version>1.5.6</version>
</dependency>

Then you have to add Logback configuration file in this directory src/main/resources/logback.xml. Below there is an example file content: 

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <include resource="org/springframework/boot/logging/logback/base.xml"/>

    <property name="LOGS" value="./logs" />

    <appender name="Console"
              class="ch.qos.logback.core.ConsoleAppender">
        <layout class="ch.qos.logback.classic.PatternLayout">
            <Pattern>
                %black(%d{ISO8601}) %highlight(%-5level) [%blue(%t)] %yellow(%C{1}): %msg%n%throwable
            </Pattern>
        </layout>
    </appender>

    <appender name="RollingFile"
              class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${LOGS}/your-application-standard-logger.log</file>
        <encoder
                class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <Pattern>%d %p %C{1} [%t] %m%n</Pattern>
        </encoder>

        <rollingPolicy
                class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>${LOGS}/archived/your-application-standard-logger-%d{yyyy-MM-dd}.%i.log
            </fileNamePattern>
            <timeBasedFileNamingAndTriggeringPolicy
                    class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
                <maxFileSize>10MB</maxFileSize>
            </timeBasedFileNamingAndTriggeringPolicy>
        </rollingPolicy>
    </appender>

    <root level="info">
        <appender-ref ref="RollingFile" />
        <appender-ref ref="Console" />
    </root>

    <logger name="com.your.application" level="trace" additivity="false">
        <appender-ref ref="RollingFile" />
        <appender-ref ref="Console" />
    </logger>

</configuration>

📌 #3 Running Application Container 

Container with your Spring Application needs to be started with special configuration - you need to map logs inside the container to outside scope which is some ec2 directory:

docker run -d -v /path-in-the-ec2/logs:/logs <<IMAGE>>

To see the logs in the Docker Container, go into the running container and open the log file: 

docker exec -it <container_id> bash

cat logs/your-application-logger.log

On the other hand you can see the logs in the ec2 like that:

tail -50 /path-in-the-ec2/logs/your-application-standard-logger.log

📌 #4 CloudWatch Agent 

You need to install CloudWatch Agent in your ec2 (following command for the Amazon Linux): 

sudo yum install amazon-cloudwatch-agent

The set up Agent configuration file using command sudo nano /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json and an example content of that file that I've proposed: 

{
    "agent": {
        "region": "your-region"
    },
    "logs": {
        "logs_collected": {
            "files": {
                "collect_list": [
                    {
                        "file_path": "/path-in-the-ec2/logs/your-application-standard-logger.log",
                        "log_group_name": "your-application-group",
                        "log_stream_name": "your-application-stream",
                        "timestamp_format": "%Y-%m-%d %H:%M:%S.%f"
                    }
                ]
            }
        }
    },
    "force_flush_interval": 15
}

Then you can start CloudWatch Agent with appended configuration file:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a append-config -s -c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json

To check the status of the Agent:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status

And you can stop the Agent manually:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a stop

📌 #5 Summary

After that you should go into AWS -> CloudWatch -> Log Groups and see new Log Group.  All described steps as diagram below: