DevMode

Now, you can declare your infrastructure dependencies and run your application in local machine by declaring via Spring Beans which Containers to use.

Something like:

package dev.aleixmorgadas;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.devtools.restart.RestartScope;
import org.springframework.boot.test.context.TestConfiguration;
import org.springframework.boot.testcontainers.service.connection.ServiceConnection;
import org.springframework.context.annotation.Bean;
import org.testcontainers.containers.BindMode;
import org.testcontainers.containers.MongoDBContainer;
import org.testcontainers.utility.DockerImageName;

import java.util.List;


@TestConfiguration(proxyBeanMethods = false)
public class TestMyApplication {

    public static void main(String[] args) {
        SpringApplication.from(MyApplication::main)
            .with(TestMyApplicationon.class).run(args);
    }

    @Bean
    @RestartScope
    @ServiceConnection
    MongoDBContainer mongoContainer() {
        return new MongoDBContainer(DockerImageName.parse("mongo:6.0.8"));
    }
}

With this code, we are telling Spring to use a MongoContainer and replace the configuration with the Container one via `@ServiceConnection` annotation.

Shortly after using it you will find something frustrating.

The missing developer experience - Stateful containers

Compared to tests that you want to start clean on each execution, when developing in local you aim to keep the state between runs.

Here we have two options:

1. Testcontainers reuse feature. It tells Testcontainers to not stop the container, and when we start again, it will reuse the same container instance.

2. Add a volume.

1. Reuse container

Let’s declare the mongoContainer as a reusable with a simple flag.

@Bean
@RestartScope
@ServiceConnection
MongoDBContainer mongoContainer() {
    return new MongoDBContainer(DockerImageName.parse("mongo:6.0.8"))
           .withReuse(true);
}

Pro

• The container will keep its state between executions.

Cons

• If we restart our computer or just stop the container, we will lose the sate.

2. Adding a volume

MongoClient example

@Bean
@RestartScope
@ServiceConnection
MongoDBContainer mongoContainer() {
    return new MongoDBContainer(DockerImageName.parse("mongo:6.0.8")) {
            @Override
            public void configure() {
                withFileSystemBind("./.data", "/data/db");
                waitingFor(Wait.forLogMessage("(?i).*waiting for connections.*", 1));
            }

            @Override
            protected void containerIsStarted(InspectContainerResponse containerInfo, boolean reused) {
                // Disable default configuration due to replica set breaking configuration with stateful approach
            }
        };
}

Postgres example

@Bean
@RestartScope
@ServiceConnection
PostgreSQLContainer<?> postgresContainer() {
    return new PostgreSQLContainer<>(DockerImageName.parse("postgres:15"))
        .withFileSystemBind("./.data", "/var/lib/postgresql/data");
}

Pro

• It keeps the state between executions and keeps the data after stopping the container.

Cons

• You have state 🤷‍♂️.

At this point, we fixed the problem with the state, but if we are developing in local, we might need to access the database to check anything.

Accessing the database painlessly - Fixed Random Ports

Testcontainers was originally intended for tests (and lucky us, we are seeing that it applies to more areas than just tests), it has some design decisions meant for testing best practices.

One of those best practices are random ports.

From the host's perspective Testcontainers actually exposes this on a random free port. This is by design, to avoid port collisions that may arise with locally running software or in between parallel test runs.

This is a great decision from a Testing point of view, and I fully recommend following this approach. BUT, when I’m developing in local, I don’t want to change my DB Tool each time the port changes.

We have two options:

1. Fixed ports

2. Testcontainers Desktop

1. Fixed ports

This is the fastest to apply, into your DevMode Configuration.

@Bean
@RestartScope
@ServiceConnection
MongoDBContainer mongoContainer() {
    var container = new MongoDBContainer(DockerImageName.parse("mongo:6.0.8"));
    container.setPortBindings(List.of("55555:27017"));
    return container;
}

Pros

• You don’t need to change your connection confirguration each time you start your application.

Cons

• You might have port collitions.

2. Testcontainers Desktop

Testcontainers is well aware of this problem and they provided a solution for it via the Testcontainers Desktop.

You can see the full feature description here.

It added a new option called `Development Services` in the application.

Which comes with a lot of preconfigured modules which what they do is map the random port to the prefeered port of the infrastructure you are running.

Here an Redis example of how it is configured for you as default config.

Pros

• You don’t need to change your connection confirguration each time you start your application.

Cons

• You need to install Testcontainers Desktop. None 😄

AWS CDK helps you define your cloud application resources using familiar languages such as python, node, and more.

If you setup from the beginning using this guideline, you will have secured your connection between GitHub Actions and AWS without needing to expose the AWS credentials as GitHub Secrets.

Boostrap your infrastructure

CDK provides a way to boostrap all the necessary resources to start deploying using CDK in your aws account.

The next command assumes you have configured in your local machine the aws credentials and region configuration. If not, please refer to the official guideline on how to specify those credentials.

npx cdk bootstrap

After some minutes, you will see in your IAM > Roles a set of roles created by CDK to manage your infrastructure.

We will use them on the next steps to allow GitHub to deploy CloudFormation Stack changes assuming those roles.

Create a new role for github-actions to assume

I found more convinient to create a dedicated role for GitHub Actions to assume with the necessary trust relationship and permissions.

Go to IAM > Roles and Create Role. We will choose Custom trust policy

Trust relationship

We will copy the next code replacing the values of:

• <account_id> with your account id from AWS.

• <org>/<repo> with the repositories that should be able to change AWS resources.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "RoleForGitHub",
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::<account_id>:oidc-provider/token.actions.githubusercontent.com"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringEquals": {
                    "token.actions.githubusercontent.com:aud": "sts.amazonaws.com",
                    "token.actions.githubusercontent.com:sub": [
                        "repo:<org>/<repo>:ref:refs/heads/main",
                        "repo:<org>/<repo>:ref:refs/heads/main"
                    ]
                }
            }
        }
    ]
}

Policies

Next, it will ask us for the policies to add. We will click to the Create new policy button, and we will use the JSON format.

We will copy pase the next code replacing:

• <account> by your AWS account number

• <region> with the region you plan to deploy your resources. The same that cdk created during the `cdk boostrap` step. Copy paste those arn resources.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "assumerolecdk",
            "Effect": "Allow",
            "Action": [
                "sts:AssumeRole",
                "iam:PassRole"
            ],
            "Resource": [
                "arn:aws:iam::<account>:role/cdk-hnb659fds-lookup-role-<account>-<region>",
                "arn:aws:iam::<account>:role/cdk-hnb659fds-deploy-role-<account>-<region>",
                "arn:aws:iam::<account>:role/cdk-hnb659fds-file-publishing-role-<account>-<region>",
                "arn:aws:iam::<account>:role/cdk-hnb659fds-image-publishing-role-<account>-<region>"
            ]
        }
    ]
}

Verify that your role looks like

GitHub Action

Now, you can tell GitHub Action to deploy the resources assuming the new role replacing the <account> and <region> with your particular case.

The next code assumes you are using CDK with node, if you use CDK with python, you need to adapt the code to setup python.

name: Deployment

on:
  push:
    branches: [ "main" ]
  workflow_dispatch:

jobs:
  build:
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
    steps:
    - uses: actions/checkout@v3
    - name: Set up node 18
      uses: actions/setup-node@v3
      with:
        node-version: 18
        cache: npm
    - name: npm install
      run: npm install
    - name: npm tests
      run: npm tests
    - name: Configure AWS Credentials
      uses: aws-actions/configure-aws-credentials@v2
      with:
        aws-region: <your region>
        role-to-assume: arn:aws:iam::<account>:role/github-role
        role-session-name: deployment-session
    - name: cdk synth
      run: npx --yes cdk synth
    - name: cdk deploy
      run: npx --yes cdk deploy --require-approval never

I’m using PostgreSQL for this example.

Tables and Views

• Transactions Table: It records the Transactions you receive with the Asset (€, $, …).

• Account Balance: It gives the sum of all the Transactions to give the Account Current Balance per Asset.

Materialized View Schema

The Account Balance Materialized View looks like:

CREATE MATERIALIZED VIEW IF NOT EXISTS account_balance
AS
SELECT
    name,
    account_id,
    asset,
    sum(value) as balance
FROM
    transactions
GROUP BY name, account_id, asset
WITH NO DATA;

See source code here

Spring @Entity that consumes the View

In order to consume this view via Spring Boot Data JPA, we need to declare as the ID of the @Entity the fields used in the GROUP BY.

class AccountBalanceId(
    val accountId: UUID = UUID.randomUUID(),
    val name: String = "",
    val asset: String = "",
): Serializable

@Entity
@IdClass(AccountBalanceId::class)
@Immutable
class AccountBalance(
    @Id
    val accountId: UUID,
    @Id
    val name: String,
    @Id
    val asset: String,
    val balance: Double
)

interface AccountBalanceRepository: CrudRepository<AccountBalance, AccountBalanceId> {
    fun findByAccountId(accountId: UUID): List<AccountBalance>
}

See @Entity here and @Repository here.

Full Source Code

Working solution here https://github.com/aleixmorgadas/poc-spring-reactive-timescale

Meme for you

I would like to share with you how I managed to deploy a solution to store Domain Events in DynamoDB in 2 days (and mostly because of AWS permission issues...😅).

I wanted a solution to store how the user interacted with different Domain Models and decouple that storage from the monolithic so they can grow independently. I aimed to create something super simple, or at least not that complex, in 2 to 3 days.

I wanted to process events as:

Monolith sends message to ➡️ SQS ➡️ Triggers Lambda to store it into ➡️ DynamoDB Table

Even though DynamoDB might not be an optimal solution to be used as Event Store, I have given it a try for the next reasons:

• It is fully managed by AWS

• It is cheap enough

• We don't require high write throughput nor read for the moment.

• Able to encrypt the content

Implementation

I decided to go with:

• Terraform as Infrastructure as Code

• Python for AWS Lambda

You can find the source code here 😊

Ready to be deployed into your cloud solution 💪😁