How to configure Logz.io log shipper for Quarkus-based Lambda using logback extension

2023-01-06

tags: quarkus, aws, lambda, logback, logzio

Goal

You have your AWS Lambda which is based on the Quarkus application. At the same time, you’re using logz.io for your log management tool. For whatever reason, you don’t want to use the logz.io shipper that works on Cloudwatch. What you want to do is to use your existing logging solution, logback, to push the logs directly from your app to logz.io.

Let’s get to it.

Add logback to your Quarkus app

Start with adding logback to the Quarkus. Luckily there is already an existing extension we can use, so just add its definition to the pom.xml:

    <dependency>
      <groupId>io.quarkiverse.logging.logback</groupId>
      <artifactId>quarkus-logging-logback</artifactId>
      <version>0.13.0</version>
    </dependency>

Now you can use logback.xml put into your src/main/resources and it will be picked up by Quarkus. For details on how it works, please reach this site.

Add logback appender from logz.io

You need to have a custom appender created by the logz.io team that you can further configure in your logback.xml. Firstly you need a dependency for it in pom.xml:

    <dependency>
        <groupId>io.logz.logback</groupId>
        <artifactId>logzio-logback-appender</artifactId>
        <version>1.0.28</version>
    </dependency>

Configure logback.xml

Now you have the dependencies in place, so configure logback to push the logs to logz.io using their appender. That’s logback.xml:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <appender name="Console" class="ch.qos.logback.core.ConsoleAppender">
        <layout class="ch.qos.logback.classic.PatternLayout">
            <Pattern>
                %white(%d{ISO8601}) %highlight(%-5level) [%blue(%t)] %yellow(%C{1.}): %replace(%yellow([%mdc])){'\[\]',
                ''} %msg%n%throwable
            </Pattern>
        </layout>
    </appender>

    <appender name="LogzioLogbackAppender" class="io.logz.logback.LogzioLogbackAppender">
        <token>${LOGZIO_TOKEN}</token>
        <logzioUrl>https://listener-eu.logz.io:8071</logzioUrl>

        <additionalFields>env=${quarkus.profile}</additionalFields>
        <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <level>INFO</level>
        </filter>
    </appender>

    <root level="info">
        <appender-ref ref="Console"/>
        <appender-ref ref="LogzioLogbackAppender"/>
    </root>

    <!-- Use shutdownHook so that we can close gracefully and finish the log drain -->
    <shutdownHook class="ch.qos.logback.core.hook.DelayingShutdownHook"/>
</configuration>

Mind that you need to pass LOGZIO_TOKEN env property when running your application. After doing it - you’ll see your application logs on the console and also pushed to logz.io.

There is an additional field pushed with logs - it’s env that value will be set to quarkus.profile which you can use to define what the profile application was built with (e.g. dev, preprod, prod, …)

So now send my logs if I’m running it in dev mode!

What you might be interested in is refraining from sending the logs if you’re developing an application locally and not running it on prod. There is also a solution for that.

You might use a conditional configuration of logback and create something like this:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <appender name="Console" class="ch.qos.logback.core.ConsoleAppender">
        <layout class="ch.qos.logback.classic.PatternLayout">
            <Pattern>
                %white(%d{ISO8601}) %highlight(%-5level) [%blue(%t)] %yellow(%C{1.}): %replace(%yellow([%mdc])){'\[\]',
                ''} %msg%n%throwable
            </Pattern>
        </layout>
    </appender>

    <appender name="LogzioLogbackAppender" class="io.logz.logback.LogzioLogbackAppender">
        <token>${LOGZIO_TOKEN}</token>
        <logzioUrl>https://listener-eu.logz.io:8071</logzioUrl>

        <additionalFields>env=${quarkus.profile}</additionalFields>
        <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <level>INFO</level>
        </filter>
    </appender>

    <root level="info">
        <appender-ref ref="Console"/>
        
        <if condition='isDefined("LOGZIO_TOKEN")'>
            <then>
                <appender-ref ref="LogzioLogbackAppender"/>
            </then>
        </if>
    </root>

    <!-- Use shutdownHook so that we can close gracefully and finish the log drain -->
    <shutdownHook class="ch.qos.logback.core.hook.DelayingShutdownHook"/>
</configuration>

Mind the if element. It will be added only if the property LOGZIO_TOKEN is passed. I assume this will not be done in your local dev env.

Just remember that using this conditional configuration functionality of logback you need to add one more dependency to Janino:

    <dependency>
      <groupId>org.codehaus.janino</groupId>
      <artifactId>janino</artifactId>
      <version>3.1.9</version>
    </dependency>

Summary

Using this approach you will be able to send logs from your Quarkus-based AWS Lambda to the logz.io aggregator directly from logback. The solution is also prepared to work in dev mode, so you will not send garbage logs remotely.