Large Language Models (LLMs) are revolutionizing how we interact with information, but they face challenges with accuracy and access to up-to-date data. Retrieval Augmented Generation (RAG) addresses these limitations by grounding the LLM's responses in a dedicated knowledge base.

This article explores a project based on the implementation of a RAG chatbot using Amazon Bedrock and LangChain that enhances the chatbot's ability to provide more contextually relevant, and current information, making it a key approach for a wide range of applications based on generative AI. The full codebase of the project is available in my GitHub repository.

Design considerations

As can be seen in other articles of my Blog, I usually review the design considerations of any project, workload or cloud architecture showcase in order to provide more wide vision about the main purpose of it. For this case, the following design principles guided the development:

• Serverless LLM Architecture : Leveraging Amazon Bedrock's serverless infrastructure eliminates the need for managing infrastructure, simplifying deployment, scaling, and experimentation. This allows developers to focus on building the core logic of the chatbot without worrying about server management.

• Modularity: The codebase is designed with modularity in mind, facilitating easy extension, component swapping, and simplified maintenance as the project evolves. This approach promotes code reusability and makes it easier to integrate new features or adapt to changing requirements.

• Streamlit for Rapid Prototyping: Streamlit's intuitive framework allows for rapid prototyping and development of user interfaces.

• Dockerized Deployment: The project is containerized using Docker, ensuring consistent execution across different environments and simplifying deployment to various platforms.

• Extensibility: The design allows for easy integration with different data sources and future enhancements like personalized responses or multi-modal search.

• Cost-Effectiveness: While not fully optimized for production, cost-effective practices are considered, such as choosing appropriate model sizes and efficient data handling. This helps keep experimentation costs manageable and lays the groundwork for future optimization in a production environment.

Architecture

Key Components

• Knowledge Base: Designed to be flexible, can ingest data from various sources, including CSV files, potentially expanding to databases and other formats in future iterations.

• Vector Database (ChromaDB): Stores vector embeddings of the knowledge base generated using a suitable embedding model like amazon.titan-embed-text-v2:0 . This allows for efficient similarity searches when retrieving relevant information.

• Streamlit: Provides the user interface for interacting with the chatbot. Streamlit's intuitive API and interactive components make it easy to build a user-friendly interface for querying the knowledge base and visualizing responses.

• Amazon Bedrock: Provides access to the foundation models powering the chatbot. Specifically, amazon.titan-embed-text-v2:0 is used for creating text embeddings, while anthropic.claude-3-sonnet-20240229-v1:0 is employed for text generation and driving the conversational aspect of the application. Bedrock's serverless infrastructure simplifies deployment and experimentation with these powerful models.

• LangChain: Orchestrates the interaction between the vector database, LLM, and user interface. It streamlines the development process and manages the retrieval and generation workflow.

Workflow

1. Data Preprocessing

1. The user enters a new question in the Streamlit Chat App.

2. The Chat history is retrieved from memory object and added prior the new message entered.

3. The question is converted to a vector using Amazon Titan Embeddings, then matched to the closests vector in the database to retireve context.

4. The combination of new Question received, Chat history, and Context from the Knowledge base are sent to the model.

5. The model's response is displayed to the user in the StreamLit App.

Data Preprocessing

A critical preprocessing step prepares the data for efficient retrieval and influences the quality of the chatbot's responses. This involves structuring the data and generating embeddings for each piece of information.

The data is organized in a JSON file containing an array of objects. Each object represents a chunk of information and has the following structure, illustrated by this example of Amazon Bedrock Faqs that has been ingested:

{
  "id": 1,
  "document": "What is Amazon Bedrock?\\nAmazon Bedrock is a fully managed service...",
  "metadata": {
    "topic": "bedrock"
  },
  "embedding": [-0.1018051, 0.01927839, 0.004059858, ...]
}

• "id": A unique integer ID for each text chunk.

• "document": The raw text content of the chunk, which includes questions like "What is Amazon Bedrock?" and their corresponding answers. Notice the \\n indicating newline characters within the text.

• "metadata": Currently, metadata includes a "topic" field set to "bedrock", suggesting this data relates to information about Amazon Bedrock. This could be expanded to include other relevant metadata like source or date.

• "embedding": A 768-dimensional vector representing the semantic meaning of the "document". These embeddings are pre-calculated using the amazon.titan-embed-text-v2:0 model, which supports a flexible embedding dimension size, and are crucial for efficient similarity search within the vector database. Storing them directly in the JSON avoids recalculation during query time, significantly improving performance.

This embedding strategy, coupled with the structured JSON format, optimizes retrieval efficiency and allows the chatbot to generate more relevant and contextually appropriate responses.

Conclusion

This RAG chatbot prototype provides a solid starting point for developers looking to explore and experiment with retrieval augmented generation. By combining Amazon Bedrock, Pinecone, and LangChain, we can build intelligent conversational AI systems that are more grounded and informative. The project demonstrates the potential of RAG for building a new generation of conversational applications.

Next Steps

The mentioned GitHub repository amazon-bedrock-rag-chatbot contains the complete code and instructions for running the project in local environment. Looking forward, next feature development could focus on:

• Expanding data ingestion capabilities: Supporting more diverse data sources and formats.

• Improving retrieval accuracy: Experimenting with different retrieval strategies and ranking algorithms.

• Exploring advanced features: Adding personalization, multi-modal search, and more sophisticated user interfaces.

In this post I’ll continue with my series of Real Time data integration in Cloud. This time I’ll explain How I built a real time telemetry data ingestion pipeline using AWS EKS to process UDP data from the F1 2023 Playstation 4 game. The solution efficiently handles the ingestion, processing and visualization of live racing telemetry, all while ensuring scalability, reliability and cost-effectiveness.

One of the reasons because this use case was chosen because F1 car telemetry data is an excellent representation of high-frequency, combined with the complexity of UDP as a protocol make it an ideal scenario to showcase the solution.

As a Cloud Architect, I cannot proceed with the article without remarking that I review and apply the AWS Well Architected Framework in any project but also in my personal projects like this. The major part of the architectural decisions were made according to best practices and design principles of Well Aerchitected Framework. For example, using Spot instances in order to achieve cost efficiency, EKS/Kubernetes for scalability, and Grafana for maintain operational excellence via monitoring.

Design Considerations

As I’ve commented in the Introduction, the architecture was designed with several WAF key principles in mind:

• Scalability: The system is capable of scaling to ingest and process real time data without performance degradation. AWS EKS and kubernetes provide a level of elasticity that allow to scale in and out in terms of Cluster and also more specifically horizontally the pods can leverage HPA (Horizontal Pod Austoscaling) in K8s (Kubernetes).

• Reliability: A TCP HealthCheck sidecar ensures that the UDP listener service remains available, preventing data loss.

• Cost Optimization: Spot instances and ARM-based instances (C6g) were chosen for their lower cost, while maintaining the necessary performance.

• Performance efficiency: Low latency and high troughput is required. for that reason, the combination of Spot Instances Network load balancer and ARM based architecture is the best suited for high performance network operations.

• Security: A Custom private network, with Load Balancers and security groups fine grained to allow only the specified access/permissions. In addition, AWS IAM policies and EKS Teams ensures to manage access securely.

Why UDP ?

I consider to slightly pause here, prior to go deep in the architecture details, and explain a bit the protocol that we are going to ingest in AWS for this architecture, that is UDP (User Datagram Protocol), to handle the telemetry data generated from the F1 2023 game.

UDP is a connectionless protocol, meaning it sends data without establishing a persistent connection between the sender and receiver, unlike TCP (Transmission Control Protocol), which maintains a connection and ensures reliable delivery through acknowledgments and retransmissions.

For the scope of this article is interesting to briefly explain and justify why UDP is being used, and not only in this case (F1 2023 game) but also as a standard to send telemetry or sensors/iot data. There are several reasons for that:

• Low latency: In a fast-paced environment like racing, timing is crucial. UDP allows data to be sent without waiting for ACK from the receiver, unlike TCP, which requires confirmation that each packet has been received successfully. That makes UDP faster by reducing considerably latency.

• Real-Time Data: Telemetry data becomes outdated very quickly, if a packet is lost, it’s often more important to receive the next one as soon as possible rather than retry sending old data.

• High Throughput: UDP can handle a large volume of data without the overhead of managing the connection, errors, or delivering ordered.

• Lightweight: The F1 telemetry is broadcast to multiple clients (such as applications or external systems) UDP is more efficient for multicast communication because don’t require setting up individual connections (as TCP would)

On the other hand, there are a few drawbacks that should be taken into account:

• No guaranteed delivery: As UDP does not provide ACK of received packets, there’s no guarantee that the data will reach the destination.

• No Error Connections: Corrupted data packets may be received without any mechanism to request a retransmission.

• Out-of-Order delivery

• Lack of congestion and Flow control: Unlike TCP, UDP doesn’t adjust retransmission rate based on network conditions. The quantity of data sent at once either is controlled, meaing that there’s a risk of overhelming the receiver if the data stream is too. fast and too frequent.

Let’s remark that for the purpose of our project, some of this drawbacks are not handled as we consider them ‘out of the scope’, but, I still consider interesting to take them into account and include them if this architecture is used as a blueprint for a new workload. It’s important to remark, that for the specific case of real-time F1 car telemetry, the drawbacks are mitigated by the fact that data is constantly being generated and not persisted. It’s only being monitored in real-time.

Architecture

High-level architecture diagram of the solution:

Key Components

• PlayStation 4 with F1 2023: Sends real-time telemetry data via UDP to a specific IP and port. The game allows for the data to be broadcast to the entire network or to a targeted endpoint.

• EKS Cluster: Deployed in the eu-central-1 region via AWS CDK (EKS Blueprints). The cluster is composed of spot instances, specifically ARM-based C6g instances.

• Network Load Balancer (NLB): Exposes the UDP listener service to the public internet, mapping to a static Elastic IP.

• Application Load Balancer (ALB): Provides secure access to the WebSocket server for external clients and to Grafana for real-time telemetry visualization.

• UDP Listener Service: Receives telemetry data from the PlayStation and processes the UDP packets according to the F1 game’s telemetry specification. A sidecar service (Nginx) performs TCP health checks to ensure availability.

• WebSocket Server: Publishes the telemetry data to connected clients. For this project, only speed data is transmitted.

• Grafana OSS: Configured with a WebSocket plugin to allow real-time visualization of telemetry data, enabling users to monitor key metrics like vehicle speed.

Workflow

The diagram has been labeled with the main workflow process for telemetry ingestion. The different steps have been brifly explained next:

1. The PlayStation 4 with F1 2023 sends UDP telemetry data to the Elastic IP associated with the Network Load Balancer.

2. The NLB forwards this data to the UDP listener service running on AWS EKS.

3. A sidecar TCP health check ensures the UDP listener's availability by monitoring the service.

4. The UDP listener processes the telemetry data and forwards it to the WebSocket server.

5. The WebSocket server broadcasts the telemetry data to connected clients.

6. Grafana, connected via the WebSocket plugin, visualizes the data, providing real-time insights into the car's speed.

Demo / Showcase

After deploying the project in my AWS personal account I’ve recorded myself playing a short Race in the official Formula1 Circuit de Barcelona-Catalonia . Obviously is my favourite circuit, as I was born a few kilomentres away from it.

Next we can see the recorded game and, to monitor the telemetry generated, the Logs of Websocket server generated in real-time and the grafana dashboard with speed gauge configured:

Conclusion

This architecture demonstrates how AWS EKS, in combination with other services, can build a scalable, reliable, and cost effective solution for ingestion UDP data. In my opinion, this open source project serves well as a blueprint or template for integrating UDP-based real-time communication into the cloud. The workload is simple and cover only one use case (Car’s speed), but can be considered as a starting point, and can be adapted for other high frequency data streaming use cases, making it flexible and scalable template.
From an operational perspective, I want to mention that I’ve used Infrastructure as code (IaC) by defining the source code of the recources through CDK and the Kubernetes manifests via kustomize and ArgoCD for easy gitops. This ensures a repeatable ans maintanable process for scalins and evolving the olution over time.
For local testing, I’ve used Minikube, in order to emulate Kubernetes, that allowed me to test the deployments before to rolling them out to AWS, ensuring a smooth integration with the target environment (develop for this project)

Next Steps

• The full code of this solution is available in my GitHub repository:

https://github.com/acriado-dev/aws-eks-udp-telemetry

• Future improvements could include processing all Car’s telemetry or adding data analytics for the data ingested. I’m always openned to add some functionality in the project.

Internet of Things (IoT) has revolutionized how we interact with the world. It involves connecting everyday objects to the internet, enabling them to collect, send, and receive data. This connectivity allows for smarter decision-making, automation, and a new level of interactivity between the physical and digital worlds.

As a Cloud Architect, I’ve had the opportunity to work on several IoT cloud projects where the data from connected devices was processed centrally in AWS. These projects involved analyzing the data to drive critical business logic decisions, leading to improved efficiency, enhanced user experiences, and even the creation of new business models.

In this article, I’ll guide you through a step-by-step process of connecting an Arduino MKR1010 to AWS IoT Core. This tutorial is designed to help you understand how to leverage AWS’s powerful cloud services to manage and analyze IoT data, helping you to get started the way for your own innovative projects.

What is Arduino MKR Wifi 1010?

The Arduino MKR WiFi 1010 is the easiest point of entry to basic IoT and pico-network application design. Whether you are looking at building a sensor network connected to your office or home router, or if you want to create a Bluetooth® Low Energy device sending data to a cellphone, the MKR WiFi 1010 is your one-stop-solution for many of the basic IoT application scenarios.

The board is based on the SAMD21 microcontroller, which is a low-power ARM Cortex-M0+ processor, and includes an integrated u-blox NINA-W102 module for WiFi and Bluetooth connectivity.

Development Environment

Prerequisites

1. AWS IoT Account

• Sign up for an AWS account if you don’t already have one: AWS Sign-Up

• Select a predefined AWS region, ideally one closer to your current location, in our case will be Paris (eu-west-3)

Caution: not all regions support AWS IoT

2. Platform.io

I've selected Platform.io as development environment for Arduino projects because it offers enhanced features compared to the traditional Arduino IDE. It integrates seamlessly with Visual Studio Code, providing advanced code editing tools, debugging capabilities, and project management features

• Install Visual Studio Code

• Install Platform.io for VSCode

• I recommend to follow the Platform.io Quick start guide

3. Arduino MKR1010 and USB cable

• Arduino MKR WiFi 1010 board.

• Micro-USB to USB cable to connect the MKR1010 to your computer.

Configuring the Board

In order to securely connect the Board to AWS there is a mandatory step that we have to follow. We have to create an sketch to generate a CSR for a private key generated in an ECC508/ECC608 crypto chip slot. Once the skecth is uploaded to the board, the user is prompted for the following information that is contained in the generated CSR:

• country

• state or province

• locality

• organization

• organizational unit

• common name

We can consider optional all fields except common name. After providing it, the user can also select a slot number to use for the private key.

Let's define the steps to generate the CSR:

• Open Platform.io and ensure that the Board is available

• Create new Project named ArduinoCSRConfig

• Install ArduinoECCX08 library for this Project

• In /src folder, modify main.cpp to include the sketch required, should be like:

/*
  ArduinoECCX08 - CSR (Certificate Signing Request
  The circuit:

  - Arduino MKR board equipped with ECC508 or ECC608 chip

  This example code is in the public domain.
*/
#include <Arduino.h>
#include <ArduinoECCX08.h>
#include <utility/ECCX08CSR.h>
#include <utility/ECCX08DefaultTLSConfig.h>

String readLine() {
  String line;

  while (1) {
    if (Serial.available()) {
      char c = Serial.read();

      if (c == '\r') {
        // ignore
        continue;
      } else if (c == '\n') {
        break;
      }

      line += c;
    }
  }

  return line;
}

String promptAndReadLine(const char* prompt, const char* defaultValue) {
  Serial.print(prompt);
  Serial.print(" [");
  Serial.print(defaultValue);
  Serial.print("]: ");

  String s = readLine();

  if (s.length() == 0) {
    s = defaultValue;
  }

  Serial.println(s);

  return s;
}

void setup() {
  Serial.begin(9600);
  while (!Serial);

  if (!ECCX08.begin()) {
    Serial.println("No ECCX08 present!");
    while (1);
  }

  String serialNumber = ECCX08.serialNumber();

  Serial.print("ECCX08 Serial Number = ");
  Serial.println(serialNumber);
  Serial.println();

  if (!ECCX08.locked()) {
    String lock = promptAndReadLine("The ECCX08 on your board is not locked, would you like to PERMANENTLY configure and lock it now? (y/N)", "N");
    lock.toLowerCase();

    if (!lock.startsWith("y")) {
      Serial.println("Unfortunately you can't proceed without locking it :(");
      while (1);
    }

    if (!ECCX08.writeConfiguration(ECCX08_DEFAULT_TLS_CONFIG)) {
      Serial.println("Writing ECCX08 configuration failed!");
      while (1);
    }

    if (!ECCX08.lock()) {
      Serial.println("Locking ECCX08 configuration failed!");
      while (1);
    }

    Serial.println("ECCX08 locked successfully");
    Serial.println();
  }

  Serial.println("Hi there, in order to generate a new CSR for your board, we'll need the following information ...");
  Serial.println();

  String country            = promptAndReadLine("Country Name (2 letter code)", "");
  String stateOrProvince    = promptAndReadLine("State or Province Name (full name)", "");
  String locality           = promptAndReadLine("Locality Name (eg, city)", "");
  String organization       = promptAndReadLine("Organization Name (eg, company)", "");
  String organizationalUnit = promptAndReadLine("Organizational Unit Name (eg, section)", "");
  String common             = promptAndReadLine("Common Name (e.g. server FQDN or YOUR name)", serialNumber.c_str());
  String slot               = promptAndReadLine("What slot would you like to use? (0 - 4)", "0");
  String generateNewKey     = promptAndReadLine("Would you like to generate a new private key? (Y/n)", "Y");

  Serial.println();

  generateNewKey.toLowerCase();

  if (!ECCX08CSR.begin(slot.toInt(), generateNewKey.startsWith("y"))) {
    Serial.println("Error starting CSR generation!");
    while (1);
  }

  ECCX08CSR.setCountryName(country);
  ECCX08CSR.setStateProvinceName(stateOrProvince);
  ECCX08CSR.setLocalityName(locality);
  ECCX08CSR.setOrganizationName(organization);
  ECCX08CSR.setOrganizationalUnitName(organizationalUnit);
  ECCX08CSR.setCommonName(common);

  String csr = ECCX08CSR.end();

  if (!csr) {
    Serial.println("Error generating CSR!");
    while (1);
  }

  Serial.println("Here's your CSR, enjoy!");
  Serial.println();
  Serial.println(csr);
}

void loop() {
  // do nothing
}

• Click Upload and monitor and, in order to generate a new CSR, define only the common name with the value ArduinoMKR1010

• Copy the generated CSR text including "-----BEGIN CERTIFICATE REQUEST-----" and "-----END CERTIFICATE REQUEST-----" and save it into a .csr file.

Finally we have a CSR to identify the board and be able to communicate with AWs IoT core securely.

NOTE: This locking process is permanent and cannot be undone, but it is necessary to use the crypto element. The configuration set by the sketch allows the use of 5 private key slots with any Cloud provider (or server). A CSR can be regenerated at any time for each of the other four slots.

Connecting to AWS IoT Core

Once the Board is properly initialized and the CSR configured, we can proceed with the tutorial and connect the device to IoT core using MQTT protocol. AWS requires to use X.509 certificates for authentication. Let's follow the steps:

1. Create Arduino Board as AWS IoT Thing

• Access to your AWS account in the predefined region. For the purpose of our example, it is Paris(eu-west-3)

• To create the Board in AWS IoT as a Thing we will use the same name that we used for the common name in the CSR configuration: ArduinoMKR1010

• By the way, we are going to skip the CSR certificate Upload for the Board

2. Create a project in Platform.io environment

• Create e new project in Platform.io, for our example we have named it Hello World - MKR wifi 101

• Install the required libraries:

  • WiFiNINA

  • ArduinoECCX08

  • ArduinoBearSSL

  • ArduinoMqttClient

• As we have two projects in our VsCode workspace, I recommend to set the new created project as a default:

3. Create the Certificate and attach it to the Thing

• Move the .csr file generated in previous steps to a more appropiate folder inside this project, for example into a /resources folder.

• Create the certificate in AWS IoT by updating the csr generated.

• Download the generated .pem.crt certificate file generated in AWS and save it in the /resources folder. We will need this file in the Board in order to communicate with cloud.

• Ensure that the certificate is activated and Attached to the Thing created:

4. Create a Policy

• For the scope of this example, we are going ot create an open policy, that will allow all iot actions on AWS.

• For real life projects, I strongly recommend to create a strict policy and follow the least principle privilege.

• Once the policy is created, it should b attached to the Certificate created in the previous step.

5. Implement the Connecting sketch to AWS IoT Core

• In the project Hello World - MKR wifi 101, modify the file src/main.cppand add the following code:

• ```cpp /* AWS IoT WiFi

  This sketch securely connects to an AWS IoT using MQTT over WiFi. It uses a private key stored in the ATECC508A and a public certificate for SSL/TLS authetication.

  It publishes a message every second to arduino/outgoing topic and subscribes to messages on the arduino/incoming topic.

  The circuit:

  • Arduino MKR WiFi 1010 or MKR1000

    */ #include #include #include #include #include // change to #include for MKR1000

    #include "arduino_secrets.h"

    /////// Enter your sensitive data in arduino_secrets.h const char ssid[] = SECRET_SSID; const char pass[] = SECRET_PASS; const char broker[] = SECRET_BROKER; const char* certificate = SECRET_CERTIFICATE;

    WiFiClient wifiClient; // Used for the TCP socket connection BearSSLClient sslClient(wifiClient); // Used for SSL/TLS connection, integrates with ECC508 MqttClient mqttClient(sslClient);

    unsigned long lastMillis = 0;

    unsigned long getTime() { // get the current time from the WiFi module
    return WiFi.getTime(); }

void connectWiFi() { Serial.print("Attempting to connect to the SSID: "); Serial.print(ssid); Serial.print(" ");

bool connected = false; int retryCount = 0; const int maxRetries = 10;

while (!connected && retryCount< maxRetries) { if (WiFi.begin(ssid, pass) == WL_CONNECTED) { connected = true; } else { // failed, retry Serial.print("."); delay(1000); retryCount++; } }

if (connected) { Serial.println(); Serial.println("You're connected to the network"); Serial.println(); } else { Serial.println(); Serial.println("Failed to connect to the network after multiple attempts."); }

}

void connectMQTT() { Serial.print("Attempting to MQTT broker: "); Serial.print(broker); Serial.println(" ");

while (!mqttClient.connect(broker, 8883)) { // failed, retry Serial.print("."); delay(3000); } Serial.println();

Serial.println("You're connected to the MQTT broker"); Serial.println();

// subscribe to a topic mqttClient.subscribe("arduino/incoming"); }

void publishMessage() { Serial.println("Publishing message");

// send message, the Print interface can be used to set the message contents mqttClient.beginMessage("arduino/outgoing"); mqttClient.print("{\"deviceModel\": \"MKR 1010\" "); mqttClient.print(",\"temperature\": " ); mqttClient.print(random(15, 30)); mqttClient.print(",\"humidity\": "); mqttClient.print(random(1, 500)); mqttClient.print(",\"vehicleId\": \"MKR1010-1"); mqttClient.print("\"}"); mqttClient.endMessage(); }

void onMessageReceived(int messageSize) { // we received a message, print out the topic and contents Serial.print("Received a message with topic '"); Serial.print(mqttClient.messageTopic()); Serial.print("', length "); Serial.print(messageSize); Serial.println(" bytes:");

// use the Stream interface to print the contents while (mqttClient.available()) { Serial.print((char)mqttClient.read()); } Serial.println();

Serial.println(); }

void setup() { Serial.begin(115200); while (!Serial);

if (!ECCX08.begin()) { Serial.println("No ECCX08 present!"); while (1); }

// Set a callback to get the current time // used to validate the servers certificate ArduinoBearSSL.onGetTime(getTime);

// Set the ECCX08 slot to use for the private key // and the accompanying public certificate for it sslClient.setEccSlot(0, certificate);

// Optional, set the client id used for MQTT, // each device that is connected to the broker // must have a unique client id. The MQTTClient will generate // a client id for you based on the millis() value if not set // // mqttClient.setId("clientId");

// Set the message callback, this function is // called when the MQTTClient receives a message mqttClient.onMessage(onMessageReceived); }

void loop() {

if (WiFi.status() != WL_CONNECTED) { int numNetworks = WiFi.scanNetworks(); Serial.println("Discovered " + String(numNetworks) + " Networks"); connectWiFi(); }

if (!mqttClient.connected()) { // MQTT client is disconnected, connect connectMQTT(); }

// poll for new MQTT messages and send keep alives mqttClient.poll();

// publish a message roughly every 1 seconds. if (millis() - lastMillis > 1000) { lastMillis = millis();

publishMessage(); } }

* Create in `/include` folder a file for the secrets named `arduino_secrets.h` that has already been included in the main sketch file.

* `arduino_secrets.h` will contain the secrets and variables required to connect, first to the Wifi and the to the Cloud provider, for example:


```c
#define SECRET_SSID "MIWIFI_SSID"
#define SECRET_PASS "XXXXXXXXXXXX"

#define SECRET_BROKER "a21nf7ui6d7k9-ats.iot.eu-central-1.amazonaws.com"

const char SECRET_CERTIFICATE[] = R"(
-----BEGIN CERTIFICATE-----
MIIClTCCAX2gAwIBAgIVAMtpt+I79Uj24rlM4MVB4Q2mqffdMA0GCSqGSIb3DQEB
CwUAME0xSzBJBgNVBAsMQkFtYXpvbiBXZWIgU2VydmljZXMgTz1BbWF6b24uY29t
IEluYy4gTD1TZWF0dGxlIFNUPVdhc2hpbmd0b24gQz1VUzAeFw0yNDA5MDIxNDA3
MDJaFw00OTEyMzEyMzU5NTlaMCQxCjAIBgNVBAYTASAxFjAUBgNVBAMTDU15TUtS
V2lGaTEwMTAwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAARo/XaUyfCffmCSIyl1
x1Tc4pbi4d1gu+IzOa98Fl52HaxxroFkDrKCiC6cyaK653BFXkRpWcQ7ivmNouO7
Fz06o2AwXjAfBgNVHSMEGDAWgBTwrTWex0zOILY7T5K6neYroFcHYDAdBgNVHQ4E
FgQUiqwBn2n3ixwwL4E8YI5d4b4eWdMwDAYDVR0TAQH/BAIwADAOBgNVHQ8BAf8E
BAMCB4AwDQYJKoZIhvcNAQELBQADggEBAK8iH5HpmGaxw9XQh/zyPsKxGG/dXXWr
U/mbZH6HGnf/Ux3ULnyZElBOB99KgqCutu8+k8lgpR8gLtsfC3XpL+TwIG4UHVak
fqP//CFl+n0DRWZvqw2wyvKBVyP1d9WUcTAW7x1k5ZN3AeXDT1Iy/KIkxhQXHIQi
wA1nw+YPDafHNFAviLKrUYd+Sx9hpKLDLI17609HbeY/9dsZh8MvxUmgjod7jvIy
0EGOgf8EhqyDvTVMb18ZUT0LKCwDWpMug0fZ+XK0kHA98Th0ndkE3997MqJSuY2u
tPND16KqhQPWim/akbJb1yDmP0M5xzF9Vtoh9Tv8JLxynmgc0KDq5/g=
-----END CERTIFICATE-----
)";

• Replace the secrets for the correct ones, specially the SECRET_CERTIFICATE, that should be the one that is contained in the certificate file that was downloaded from AWS in a previous step in resources folder.

• To replace the SECRET_BROKER, the endpoint should be obtained from AWS:

6. Connecting to AWS IoT

• Once the project code is properly added to Hello World - MKR wifi 101 , Ensure that the Board is connected through USB and that the device is available in PIO HOME Devices:

• There are several Project Tasks availables. We need to Build the sketch code and upload it to the Board connected.

• I recommend to use the following tasks:

  • Full Clean

  • Build

  • Upload and Monitor

• Finally, if all steps have worked successfully, a Serial monitor terminal is openned and the Board is going to Connect to the Wifi and send messages to AWS IoT

Testing and Monitoring Data

The board is continously sending a MQTT message to IoT core with random generated data every second with this composition, see example:

 {
  "deviceModel": "MKR 1010",
  "temperature": 22,
  "humidity": 317,
  "vehicleId": "MKR1010-1"
}

1. Monitor Outgoing Data from the Board

2. We have to use the MQTT test client to Subscribe to the topic arduino/outgoing

2. Send a Message to the Board

• We should subscribe to the topic arduino/incoming and publis a message from the MQTT Test client in AWS IoT

• The message sent should appear in the Serial console that we have openned in Platform.io

Conclusion

In this guide, we’ve walked through the process of connecting the Arduino MKR WiFi 1010 to AWS IoT Core using Platform.io in Visual Studio Code. You’ve learned how to set up your development environment, configure AWS IoT Core, and establish communication between your Arduino board and the cloud using the MQTT protocol.

Although this is a specific project, it can be considered the first step toward building scalable IoT solutions. In my opinion, Arduino is one of the best platforms for growing in the IoT space and for enjoying the process of building personal projects. It’s also a viable option for production systems due to its flexibility and ecosystem. Additionally, using Platform.io enhances local development capabilities and simplifies the integration of new tasks, making it an excellent tool for both hobbyists and professionals.

As a next step, I plan to extend this project by creating an AWS IoT Rule that processes the data coming from the Arduino MKR1010 and integrates it with a real-time platform. This will involve uploading the data from the incoming MQTT topic directly to DynamoDB.

I’ll be covering this in my upcoming articles, where I’ll explain the integration in more detail. You can also check out my previous article on building a real-time platform using AWS services here, which will serve as a foundation for the next phase of this project.

Reference

1. Github Project:

https://github.com/acriado-dev/arduino-mkr-1010-aws-iot-core

2. Arduino MKR 1010 Official Docs:

https://store.arduino.cc/en-es/products/arduino-mkr-wifi-1010?gad_source=1&gclid=CjwKCAjwreW2BhBhEiwAavLwfFKUmllSUvjuY6mNsFugNXUCh-aoR3kiqFvGyX0dToSSzvty08dyBhoCE28QAvD_BwE

3. Platform.io

https://docs.platformio.org/en/latest/what-is-platformio.html

4. AWS IoT core Developer guide

https://docs.aws.amazon.com/iot/latest/developerguide/what-is-aws-iot.html

During my work on several projects as a Cloud Architect, I noticed a recurring challenge:

Integrating diverse data sources and delivering real-time updates to multiple clients

Whether it's monitoring environmental conditions in smart agriculture, tracking assets in logistics, or managing connected devices in smart cities, the ability to process and act on data instantly can make a significant difference.

In this post, I'll share my journey of building a serverless Near Real-Time Data Integration Platform using the AWS Serverless Application Model (SAM).

All the infrastructure code is available in a GitHub repository:

https://github.com/acriado-dev/sam-real-time-websockets

Overview

Understanding Real-Time vs Near Real Time

Before diving into the details and architectural components, it's important to clarify a core concept for the platform that sometimes causes confusion. In the context of data processing, what is the difference between "real-time" and "near real-time"?

• Real-Time: Data is processed and delivered as soon as it is generated, with virtually no delay. This is critical in scenarios where immediate action is required. Examples include industrial automation, connected vehicles, and healthcare monitoring.

• Near Real-Time: This refers to a slight delay in data processing, typically ranging from milliseconds to seconds. Examples include data analytics, CDNs, and social media monitoring.

About this Project

To answer the question,

is this platform Real-time or Near Real-Time?

We have to take into account that TRUE Real-Time implies that data is processed and delivered with minimal latency, often microsecons or milliseconds, with strict timing constraints. This level of immediacy is not accomplished with the Cloud resources used for thi project implementation, let's analyze the most significant ones:

• AWS Lambda: Introduces some latency due to cold starts and the execution time of the function. While this latency is generally small (typically within milliseconds to a couple of seconds), it does mean that it cannot be considered "true" Real-Time but rather operates in near Real-Time.

• WebSocket API: This component allow low-latency communication, which is closer to Real-Time, but the overall latency is influenced by the procesisng times of the connected Services.

• DynamoDB and Streams: Provides very fast data retrieval and storage capabilities, but Lambda triggered by streams still introduces additional processing time.

Given these factors, the most accurate description for the platform should be 'near Real-Time'. However, there are some benefits of this approach, from cost efficiency to scalability and simplicity, but remember that depending on the use case you're facing of for your application, Near Real-time could be the best decision.

Architecture

I will explore the project's architecture, detailing various components and explaining how the serverless approach offered an appropiate and efficient solution. In addition, a workflow example will be described in order to understand better How the project works:

Components:

• Client Integration: Frontend application (Mobile & WebApp) that is able to connect to the WebSocket API and receive the real time data updates.

• WebSocket API Gateway: Entry point of the application. It's responsible for handling the WebSocket connections and the messages that are sent to the clients.

• OnConnect Lambda Function: Is triggered when a client connects to the WebSocket API Gateway. Stores the connection information in the DynamoDB table.

• OnDisconnect Lambda Function: Is triggered when a client disconnects from the WebSocket API Gateway. Removes the connection information from the DynamoDB table.

• OnReceiveRealTimeItem Lambda Function: Is triggered when a new item is added to the DynamoDB table. Sends the real time data updates to the clients that are interested in the item.

• RealTimeData DynamoDB Table: Used to store the real time data items. Each item has a unique key and a value that represents the data that is sent to the clients.

• WebSocketConnectionManager DynamoDB Table: Stores the connection information of the clients that are connected to the WebSocket API Gateway. Each item has a unique connectionId and a realTimeItemKey that represents the item that the client is interested in.

• Integration sources: Data sources responsible for sending the real time data updates to the platform. For this project, the following sources has been considered:

  • SDK: AWS SDK and CLI for any language compatible.

  • Data transfer: Any data transfer mechanism that is able to send the data to the platform. For example, data replication from other database.

  • Device Location & Sensors: IoT devices mainly sending sensor data and telemetry. For example, GPS location of a vehicle.

  • REST API: For example, a weather API that sends the weather data to the platform.

  • Async (SNS, SQS): Asynchronous mechanism that is able to send the data to the platform. For example, an SNS topic that sends the data to the platform.

  • S3 Events: Event triggered by an S3 bucket. For example, when a new file is uploaded to the bucket.

Workflow:

Next, we are going to describe the different numbered steps marked in the diagram:

1. The client integration connects to the WebSocket API Gateway.

2. The WebSocket API Gateway triggers the OnConnect Lambda function.

3. The OnConnect and OnDisconnect Lambda functions store and remove the connection information from the WebSocketConnectionManager DynamoDB table.

4. RealTimeData items are added to the RealTimeData DynamoDB table from the integration sources.

5. The OnReceiveRealTimeItem lambda function is triggered through DynamoDB Streams.

6. The OnReceiveRealTimeItem lambda function sends the real time data updates to the clients that are interested in the item.

7. The client integration receives the real time data updates from the WebSocket API Gateway.

Setup and Test the Platform

In order to test the Platform accordingly we are going to use wscat utility to simulate an API client and monitor the websocket API. This way we can allow each tes client to receive the data instantly.

npm install wscat

We can retrieve the WebSocket API endpoint in AWS Console although is useful to have it as an output for the SAM project:

The Endpoint should have the following composition:

wscat -c wss://[api-id].execute-api.[aws-region-id].amazonaws.com/[environment]?realTimeItemKey=[item-key]

Notice that the realTimeItemKey query parameter is required to connect to the WebSocket API. This parameter is used to filter the messages that are sent to the client. The value of this parameter must be the same as the realTimeItemKey attribute of the item that you want to receive updates for.

According to the project specification, the realTimeItemKey is a dynamic value and can be adapted depending on the client/integration needs.

In our example, we are going to use vehicleId as real time item to receive updates from multiple clients, then, the realTimeItemKey should be vehicleId, and the command to connect should be:

wscat -c  wss://svcz00plil.execute-api.eu-central-1.amazonaws.com/develop?vehicleId=3

Let's see some Examples of the three main events of the Platform:

• OnConnect:

• OnDisconnect

• OnReceiveRealTimeItem

Conclusion

This project is a fantastic example of a simple near real-time integration in a Serverless architecture. It can be used as a starting point for implementing a more complex integration or added to another project to handle real-time processing.

In my opinion, it's important to optimize these kinds of data integrations and make them as efficient as possible. Many projects overuse polling, which can be considered an anti-pattern for proper real-time implementation.

Additionally, note that a special part of the project focuses on IaC, as it's crucial to implement workloads that can be reused and reproduced in any Cloud environment (in this case, AWS).

AWS Lambda is a serverless compute service that runs code in response to events and automatically manages the underlying resources for you. You can use AWS Lambda to extend other AWS services with custom logic, or create your own backend services that operate at AWS scale, security and the main feature considered for this lab, performance.

One of the biggest challenges that is common among AWS Lambda and serverless computing is the Performance. Unlike traditional server-based environments where infrastructure resources can be provisioned and fine-tuned to meet specific performance requirements, serverless platforms like Lambda abstract away much of the underlying infrastructure management, leaving less control over performance optimization in the hands of developers. Additionally, as serverless architectures rely heavily on event-driven processing and pay-per-execution pricing models, optimizing performance becomes crucial for minimizing costs, ensuring responsive user experiences, and meeting stringent latency requirements.

What is Lambda LLRT?

Warning

LLRT is an experimental package. It is subject to change and intended only for evaluation purposes.

AWS has open-sourced its JavaScript runtime, called LLRT (Low Latency Runtime), an experimental, lightweight JavaScript runtime designed to address the growing demand for fast and efficient Serverless applications.

LLRT is designed to address the growing demand for fast and efficient Serverless applications. LLRT offers up to over 10x faster startup and up to 2x overall lower cost compared to other JavaScript runtimes running on AWS Lambda. It's built in Rust, utilizing QuickJS as JavaScript engine, ensuring efficient memory usage and swift startup.

Real-Time processing or data transformation are the main use cases for adopting LLRT for future projects. LLRT Enables a focus on critical tasks like event-driven processing or streaming data, while seamless integration with other AWS services ensures rapid response times.

Key features of LLRT

1. Faster Starup Times: Over 10x faster startup compared to other JavaScript runtimes running on AWS Lambda.

2. Cost optimization: Up to 2x overall lower cost compared to other runtimes. By optimizing memory usage and reducing startup time, Lambda LLRT helps minimize the cost of running serverless workloads.

3. Built on Rust: Improves performance, reduced cold start times, memory efficiency, enhanced concurrency and safety.

4. QuickJS Engine: Lightweight and efficient JavaScript engine written in C. Ideal for fast execution, efficient memory usage, and seamless integration for embedding JavaScript in AWS Lambda.

5. No JIT compiler: unlike NodeJs, Bun & Deno, LLRT not incorporates JIT compiler. That contributes to reduce system complexity and runtime size. Without the JIT overhead, CPU and memory resources can be more efficiently allocated.

Overview

The goal of this lab is to conduct a comparative analysis between AWS Lambda's Low Latency Runtime (LLRT) and traditional Node.js runtime, focusing on their performance and efficiency in real-world serverless applications. By deploying identical functions with LLRT and Node.js, with the aim to evaluate their respective startup times, execution speeds, and resource consumption.

The previous results of performance profiling by AWS Labs, available on GitHub, offer valuable benchmarks and insights:

• LLRT - DynamoDB Put, ARM, 128MB:

• Node.js 20 - DynamoDB Put, ARM, 128MB:

Lab

The lab is composed by 2 Lambda functions "llrt-put-item" and "nodejs-put-item", both designed to put any received event into a DynamoDB table named "llrt-sam-table".

Implemented using AWS SAM, the project leverages AWS Lambda Power Tuning for performance profiling and benchmarking. While resembling the default benchmark by AWS Labs, all code developed is accessible on my personal GitHub repository, facilitating easy replication and further exploration of the lab's results:

https://github.com/acriado-otc/example-aws-lambda-llrt

Benchmarking: LLRT vs Node.js

To benchmark the lab, AWS Lambda Power Tuning has been chosen as a open-source tool developed by AWS Labs. AWS Lambda Power Tuning is a state machine powered by AWS Step Functions that will helps us to get results for both lab's Lambda functions and get results for cost and/or performance in a data-driven way.

While you can manually run tests on functions by selecting different memory allocations and measuring the time taken to complete, the AWS Lambda Power Tuning tool allows us to automate the process.

The state machine is designed to be easy to deploy and fast to execute. Also, it's language agnostic.

Deployment

To deploy AWS Lambda Power Tuning, clone the official repository and choose the preferred deployment method. In my case I used SAM cli for simplicity, but I highly recommend to use AWS CDK for IaC deployments:

https://github.com/alexcasalboni/aws-lambda-power-tuning

Executing the State Machine

Independently of how you've deployed the state machine, you can execute it in a few different ways. Programmatically, using the AWS CLI, or AWS SDK. In the case of our lab, we are going to execute it manually, using the AWS Step Functions web console.

• Once properly deployed, You will find the new state machine in the Step Functions Console in the AWS account and region defined for the lab:

• Find it and click "Start execution".

• A descriptive name and input should be provided.

• We are going to execute llrt-put-item first, that's the input used**:**

{
  "lambdaARN": "arn:aws:lambda:eu-west-1:XXXXXXXXXXX:function:llrt-sam-LlrtPutItemFunction-42aAa2of5wqn",
  "powerValues": [
    128,
    256,
    512,
    1024
  ],
  "num": 500,
  "payload": {"message": "hello llrt lambda"},
  "parallelInvocation": true,
  "strategy": "speed"
}

• All other fields should remain by default. Finally, find it and click "Start execution".

• After some seconds/minutes the Execution should have the status "Succeded"

• Repeat the same process for the nodejs-put-item lambda function. The unique field to change from the input should be the function's ARN and optionally the payload.

• While running each execution, we can see the status/progress of the step function execution with the graph view:

Results and Analysis

In order to get accurate results several executions have been made for each Lambda function, with 'speed' strategy. Next we are presenting the results for each Lambda separately, and a final comparison:

• LLRT Lambda function:

{
  "power": 1024,
  "cost": 9.519999999999999e-8,
  "duration": 6.25,
  "stateMachine": {
    "executionCost": 0.00025,
    "lambdaCost": 0.0001901263,
    "visualization": "https://lambda-power-tuning.show/#gAAAAQACAAQ=;REREQWbm5kIREdFAAADIQA==;ata9Muy90zTBcEwzwXDMMw=="
  }
}

• Nodejs

{
  "power": 1024,
  "cost": 1.512e-7,
  "duration": 8.666666666666666,
  "stateMachine": {
    "executionCost": 0.00025,
    "lambdaCost": 0.00025384590000000003,
    "visualization": "https://lambda-power-tuning.show/#gAAAAQACAAQ=;d7fdQ7y7iEKamatBq6oKQQ==;Ckp6Nc+VmzRwbUY0ilkiNA=="
  }
}

• Comparison

After several executions of the AWS Power Tuning step function. It's clear that both lambda functions has a similar performance for 1024MB and 512MB of memory allocation. The huge difference appears in the interval from 128MB to 256MB, where LLRT is more suitable option and should be considered for small functions.

In terms of cost, for LLRT 128MB is the cheaper average execution, and 256MB the wrost, meanwhile for NodeJs function the wrost cost 128MB, just the opposite of LLRT. However, it's important to remark that for this laboratoy, the criteria applied for the benchmark has been 'speed'. To get more accurate cost results should be executed with 'cost' strategy accordingly.

Conclusion

LLRT clearly can add advantages for some projects, such as faster startup times and potential cost savings, but it currently lacks the stability, support, and real-world testing required to be considered for production use.

For developers working with smaller serverless functions that prioritize rapid response times and efficient resource utilization, LLRT is an alternative to traditional Lambda runtimes. However, it's essential to evaluate carefully LLRT and consider the specific requirements of your application before.

As LLRT continues to evolve and mature, it's adoption will increase for sure, and progressively a refinement of this runtime for latency-sensitive use cases will be provided by AWS. In the meantime, let's monitor LLRT's development and wait for news, specially for production environments.

References and resources

• https://github.com/awslabs/llrt

• https://aws.amazon.com/lambda/features/

• https://bellard.org/quickjs/

• https://docs.aws.amazon.com/lambda/latest/operatorguide/profile-functions.html

• https://serverlessrepo.aws.amazon.com/applications/arn:aws:serverlessrepo:us-east-1:451282441545:applications~aws-lambda-power-tuning

AWS re:Invent stands as a pivotal learning conference hosted by AWS, serving the global cloud-computing community. This immersive in-person event goes beyond traditional conferences, offering keynote announcements, extensive training and certification opportunities, over 2,000 technical sessions, an expansive Expo, and engaging after-hours events. The sheer scale and breadth of AWS re:Invent make it a must-attend for cloud professionals worldwide.
In the context of this article, we'll narrow our focus to the specific realm of Cloud Architecture and DevOps. As a Cloud Architect, I find immense value in exploring and dissecting the announcements that directly impact my professional domain. Join me in uncovering the latest innovations and advancements unveiled during AWS re:Invent 2023, with a keen emphasis on services and updates relevant to Cloud Architecture and DevOps.

Keynote Highlights

During almost one week, the main AWS executives appeared and we have several interesting Keynotes trying to summarize the next year predictions and the impact of them. Despite generative AI (GenAI) dominated the conference as a central theme, there was more than 140 announces from different topics.
From the point of view of Cloud Architecture, I highly recommend the Keynote of Dr. Werner Vogels, but here ara a link to the rest of them:

• CEO Keynote with Adam Selipsky: Amazon Web Services CEO shares his perspective on cloud transformation and highlights innovations in data, infrastructure, and artificial intelligence and machine learning.

• Monday Night Live Keynote with Peter DeSantis: Senior Vice President of AWS Utility Computing, dives deep into the engineering that powers AWS services.

• Keynote with Dr. Swami Sivasubramanian: Vice President of Data and AI at AWS explores the powerful relationship between humans, data, and AI, unfolding right before us.

• Keynote with Dr. Werner Vogels: Amazon.com’s VP and CTO, covers best practices for designing resilient and cost-aware architectures, and discusses why artificial intelligence is something every builder must consider when developing systems and the impact this will have in our world.

Cutting-Edge Services and Features:

Developer tools

Generative AI / Machine Learning

Application Integration

Cost Optimization

Database

Storage

Security

Conclusion

As a Cloud Architect, I consistently prioritize alignment with the Cloud Pillars and Well-Architected frameworks, not only within AWS projects but also across various cloud providers. These frameworks serve as standard best practices for cloud adoption. AWS re:Invent stands out as one of the premier IT innovation events globally, adopting a comprehensive approach to presenting information. All pillars and best practices are meticulously covered, showcasing AWS's commitment to innovation and solidifying its leadership in the cloud services domain.
Choosing a concise list of announcements for Cloud Architecture and DevOps from the presented at AWS re:Invent is challenging. The summarized tables aim to reflect my perspective on the most impactful services, particularly from an architect's standpoint.
While AWS re:Invent 2022 had a focus on low code/no-code innovations, this year has unmistakably brought generative AI to the forefront, influencing various resources concurrently. Amazon Q, in particular, appears to be a revolutionary way to interact with AWS documentation, addressing a common daily practice for numerous profiles. Its potential to enhance the already robust documentation with a layer of intelligent generative AI is promising.
For those who missed keynotes and sessions, they are available for viewing on the AWS re:Invent site: AWS re:Invent 2023

AutoScaling in Kubernetes is a critical feature that allows applications to automatically adapt to workload variations, ensuring optimal performance and efficient resource management within the cluster. The Horizontal Pod Autoscaler (HPA) is a tool that automates the adjustment of the number of pod replicas based on predefined metrics.

In this guide, you'll learn how auto-scaling works in Kubernetes through a real example HPA configuration.

What is Horizontal Pod Autoscaler (HPA)?

• Scales the number of pod replicas horizontally based on metrics like CPU and memory utilization.

• Ideal for applications with varying workload demands and where adding or removing identical replicas is effective.

• Offers flexibility in adapting to changing traffic and resource needs.

• Requires that applications are designed to be stateless or capable of handling pod replication.

Key Considerations

In the journey to optimize your Kubernetes deployment and ensure efficient autoscaling, it's essential to weigh the factors that influence your choice of an autoscaling solution. There are several considerations that should be aligned with your application's requirements and broader objectives:

1. Application Suitability: Evaluate if your application can benefit from horizontal scaling.

2. Resource Metrics: Ensure that your performance bottlenecks relate to metrics like CPU and memory.

3. Cluster Size: Consider the cluster's size and capacity for effective autoscaling.

4. Monitoring and Alerting: Implement monitoring and alerting for resource utilization and performance.

5. Pod Design: Ensure pods are stateless and replicable for effective HPA.

6. Testing: Thoroughly test HPA in a non-production environment.

7. Scaling Policies: Define clear scaling policies aligned with your objectives.

8. Resource Quotas: Be aware of resource quotas to prevent overscaling.

9. Cost Implications: Understand how autoscaling affects cloud infrastructure costs.

10. Maintenance and Updates: Keep configurations and policies up to date.

11. Alternative Solutions: Consider alternative autoscaling solutions like Vertical Pod Autoscaler or custom controllers if they better fit your use case.

HPA Configuration Example

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: app-autoscaler
  namespace: app-namespace
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: app-deployment
  minReplicas: 5
  maxReplicas: 15
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 75
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
      - type: Percent
        value: 25
        periodSeconds: 10
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
      - type: Percent
        value: 100
        periodSeconds: 15
      - type: Pods
        value: 2
        periodSeconds: 15
      selectPolicy: Max

Explanation:

• metadata: In this section, you set the name and namespace of the HPA.

• spec: Here, you define the HPA's characteristics:

  • scaleTargetRef: It selects the resource to be automatically scaled, in this case, the Deployment named "app-deployment."

  • minReplicas: It ensures that at least 5 pod replicas are always running.

  • maxReplicas: It allows a maximum of 15 pod replicas.

• metrics: These metrics specify the criteria for scaling decisions. In this example, CPU usage is used with an average utilization target of 75%.

• behavior: It configures the scaling behavior:

  • scaleDown: Specifies how scaling down occurs. If CPU usage decreases by 25% within a 10-second window, it will reduce the number of replicas.

  • scaleUp: Defines how scaling up is performed. If CPU usage exceeds 100% within a 15-second window or if the number of pods exceeds the 15-second window by 2, it will add more replicas.

Monitoring HPA in Kubernetes

Monitoring Horizontal Pod Autoscaler (HPA) is essential to ensure effective autoscaling. Several monitoring mechanisms should be considered and used accordingly depending on several factors, such as the project, application and team expertise. For the scope of this article, we're going to evaluate prometheus/grafana as the main monitoring approach:

1. Install Prometheus and Grafana:

   • Deploy Prometheus and Grafana in your Kubernetes cluster using Helm charts or manual installation.

   • Set up Prometheus to scrape HPA-related metrics by adding a scrape configuration to your Prometheus config.

   • Create or import Grafana dashboards tailored for HPA monitoring. Customize them to fit your needs.

2. Visualize Key Metrics:

   • In Grafana, visualize metrics like CPU and memory utilization, replica counts, and scaling events.

     • Example:

• Set up alerts for scaling events and resource utilization thresholds.

1. Evaluate Scaling Performance:

   • Monitor HPA behavior, ensuring it scales pods in response to changing workloads.

   • Analyze historical data to optimize scaling configurations.

2. Continuous Monitoring:

   • Regularly review and adjust your monitoring setup to proactively address scaling issues and anomalies.

3. Enhance with Logging and Tracing:

   • Consider adding logging and tracing solutions for deeper insights during scaling events.

Conclusion

The use of the Horizontal Pod Autoscaler (HPA) in Kubernetes is essential to ensure that applications can automatically adapt to changes in workload demand. This configuration maintains a minimum number of active replicas (5 in this case) and scales up to a maximum of 15 replicas based on CPU utilization.

As resource needs change, the HPA will automatically adjust the number of replicas to ensure optimal performance and efficient resource management within the cluster. AutoScaling is fundamental for ensuring availability and performance in dynamically changing Kubernetes environments.

This guide is your roadmap to seamlessly connect your developer workstation to an Amazon Elastic Kubernetes Service (EKS) cluster and harness the power of Lens for effective monitoring and management. In this walkthrough, we'll walk you through the essential steps required to establish this connection, ensuring you have the tools and configurations in place to streamline your Kubernetes operations.

Whether you're an experienced Kubernetes practitioner or just beginning your journey with EKS, this guide will equip you with the knowledge and resources to make the process straightforward. Let's embark on this journey to unlock the potential of EKS and Lens for your Kubernetes development and administration needs.

Steps to Follow

Step 1: Configure the AWS Environment

You need to obtain the "Access Key" and "Secret Access Key" to interact with AWS services programmatically or through the AWS CLI.

https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html

Step 2: Install kubectl

Kubectl is a command-line tool that allows you to interact with Kubernetes clusters. You can download and install it by following the instructions in the official Kubernetes documentation:

https://kubernetes.io/docs/tasks/tools/install-kubectl/

Step 3: Install Lens

Lens is a Kubernetes cluster management tool that allows you to visualize and manage Kubernetes clusters centrally. You can download and install Lens from the official website:

https://k8slens.dev/

Step 4: Configure kubectl for EKS

With the AWS CLI properly installed and configured, to install kubectl, you need to execute the following command:

aws eks --region eu-central-1 update-kubeconfig --name cluster-stack

• Verify the Connection:

To ensure that kubectl is configured correctly and can connect to the EKS cluster, you should run the following command:

kubectl config get-contexts

You should see a list of Kubernetes contexts, including the context for your EKS cluster that was created earlier.

Step 6: Open Lens and Add the Cluster

Open the Lens application on your local machine.

• Click "Add Cluster" on the main screen.

• Select "Kubeconfig" as the authentication method.

• Click "Browse" and select the kubectl configuration file generated in Step 4 (usually located at ~/.kube/config).

• Click "Next" and provide a name for the cluster.

• Click "Connect" to add the cluster to Lens.

You should now be able to view and manage your EKS cluster in Lens remotely from your local machine.

Conclusion

To put it simply, Lens is a fantastic choice for connecting to your Amazon Elastic Kubernetes Service (EKS) clusters. Why? Well, it's like having a super user-friendly control center for all things Kubernetes. It makes monitoring and managing your clusters easier.

With Lens, you get a clear and easy-to-use dashboard cluster's health and performance. So, if you want to make your Kubernetes life easier and more efficient, Lens is the way to go!

Amazon S3 (Simple Storage Service) is a scalable and durable cloud-based storage service by Amazon Web Services (AWS) for storing and retrieving data, files, and objects. In this tutorial, we will explore the capability of enabling versioning in Amazon S3. Whether you're a seasoned cloud architect or a newcomer to the cloud, understanding how versioning works and how to implement it can be a game-changer in maintaining data consistency, recovering from accidental deletions, and complying with regulatory requirements.

What is versioning in S3?

S3 versioning is a feature on Amazon S3 that allows you to preserve, retrieve, and manage multiple versions of objects stored in a bucket.

Key concepts

• It is used to preserve, retrieve, and restore every version of every object stored in an S3 bucket.

• Done at the S3 Bucket level.

• Can be enabled from the AWS Console / SDKs / API.

• Once enabled, cannot be completely disabled, the alternative is placing the bucket in a "versioning-suspended" state.

• A drawback of having multiple versions of an object is you are billed multiple times (since the objects are getting stored in S3 each time).

• In order to avoid having multiple versions of the same object, S3 has a feature called Lifecycle Management. This allows us to decide on what to do when multiple versions of an object are piling up.

• One advantage of versioning is, we can provide permissions on versioned objects, i.e., we can define which version of an object is public and which one is private.

Preparation

• Create an AWS Account: If you don't already have an AWS account, you'll need to create one. Go to the AWS website and follow the sign-up process.

• Access Key and Secret Access Key: To interact with AWS services programmatically or through the AWS CLI, you'll need to generate access keys. These keys consist of an Access Key ID and a Secret Access Key. They are essential for authenticating your AWS requests.

• Configure AWS CLI: If you plan to use the AWS Command Line Interface (CLI), you'll need to install it on your local machine. Once installed, configure the CLI with your access key and region. This step is necessary for performing actions on your AWS resources, including S3 buckets.

• Select the Target S3 Bucket: Decide which S3 bucket you want to enable versioning for. Ensure you have the necessary permissions to modify the bucket settings.

Practical example

In the following steps, we will guide you through the process of enabling versioning in Amazon S3:

• Create new bucket

• Enable versioning

• In order to test it, upload a new object. (For the purpose of this tutorial we have used a fancy orc avatar):

• Make the bucket public with a Bucket policy, use the Policy generator and grant public access to getObject in S3:

• Finally, we upload the same object but with different color balance to differentiate clearly the versions:

Conclusion

In conclusion, enabling versioning in Amazon S3 is a fundamental step towards enhancing the resilience, security, and control of your data in the cloud. By following the steps outlined in this guide, you've empowered yourself with a powerful tool for data preservation, recovery, and compliance. Versioning ensures that no data is lost to accidental deletions, provides a historical record of changes, and safeguards your critical information.

Serverless frameworks are gaining popularity for their cost-effectiveness, scalability, and rapid development capabilities and increasingly, many companies consider that the future of computing is serverless.

A serverless architecture represents an approach to creating and operating applications and services without the need for direct infrastructure management. Although serverless applications are ultimately executed on servers, the responsibility for overseeing these servers is shifted to a particular service provider, such as Amazon Web Services (AWS) utilizing 'lambda functions,' or Google Cloud Platform through 'Cloud functions.'

Effective migration is crucial to unlock the advantages of serverless architecture. Migrating existing applications or developing new ones in a serverless environment offers benefits like scalability, cost-efficiency, and simplified management. However, without a well-planned and executed migration strategy, organizations may struggle to realize these advantages. Effective migration ensures a smooth transition, minimizes disruption, and maximizes the potential of serverless computing for agility, cost savings, and improved performance.

The main goal of this article is to present the most relevant migration strategies to serverless applications. It's crucial to understand all the migration strategies, and there are several considerations to take into account and questions to answer before deciding which strategy best fits the migration.

Notice that the migration patterns presented in this article are common patterns and should be considered a good starting point for any migration because they cover the basis for many serverless applications.

Serverless Fundamentals

In order to provide more context for the article, let's dig more into the Key advantages of serverless applications, and some common use cases:

What does 'Serverless' exactly means?

• No Server Management

• Flexible Scaling

• Automated high availability

• No idle capacity

Top Five use cases for Serverless Computing

• Media Processing

• Event-Driven Applications

• Building APIs

• Chatbots

• Webhooks

Migration Challenges

While serverless computing has many benefits, it also has some challenges that must be acknowledged or addressed before you can be successful. The purpose of this article is to present them briefly.It's important to understand that those considerations are not only related to Architecture problems, security and compliance are also important topics.

• Cold Start Latency: Serverless functions experience a delay when they're invoked for the first time, known as "cold start latency."

• Vendor Lock-In: Each Serverless providers offer their unique services and tools Transitioning away from a specific provider can be difficult and costly.

• Legacy Systems Integration: Migrate from traditional systems to serverless may be required to integrate with legacy systems. That adds a customization layer with increasing complexity and difficult maintainability.

• Resource Limitations: The different Serverless platforms, impose resource limits, such as execution time and memory. In some cases, the limitations are applied to different Regions. High-Scale projects could be affected in terms of performance.

• Security Concerns: Ensure the security of data in a serverless environment is a big challenge. Developers must carefully configure security settings and access controls to prevent unauthorized access and data breaches.

• Cost Management: Misconfigured functions or excessive usage can lead to unexpected expenses.

• Compliance and Data Residency: In regulated industries, can be complex when data is distributed across serverless functions and services.

• Monitoring and Debugging: Identifying issues and performance bottlenecks may require specialized tools.

The Migration process

Initial approach

3 preliminary questions should be done prior to accomplishing the migration and decide the strategy:

• How is implemented the computing infrastructure?

• How is approached application development?

• How is approached application deployment?

Answer completely or partially these 3 paradigms is required to decide the strategy. At least all 3 points should be considered and evaluated accordingly.

Infrastructure abstraction

Understand the main infrastructure abstraction and architecture modernization allow us to group the different strategies as well, and can answer part of the questions introduced in the initial approach. For the scope of this project there are 3 abstractions considered :

Server based

Definition:

Also known as a server-centric infrastructure, is an IT architecture where a central server or multiple servers play a crucial role in providing services, managing resources, and storing data for clients or end-users.

Considerations:

• monolithic

• Single Artifact releases

• Usually have some manual deployments

• Single Technology stack

• Minimal impact moving application to the cloud

Containerized

Definition:

Containers are lightweight, portable, and isolated environments that encapsulate an application and all the libraries, runtime components, and configuration settings it needs to run. This approach offers several advantages in terms of efficiency, scalability, and ease of management

Considerations:

• Platform independence

• Environment parity

• Straightforward deployments

• Portability

• Security policies of container images and runtime

APIs and microservices

Definition:

APIs and microservices are two interconnected concepts that play a crucial role in modern software development and architecture. They are often used together to build scalable, flexible, and maintainable applications. APIs are sets of rules and protocols that allow one software application or component to interact with another, and the microservices structures the application into those components, being a collection of small, independent services.

Considerations:

• Event-driven microservices

• CI/CD with polyglot technology stacks

• Frequent releases.

• Applications need to be rewritten.

Migration patterns

Leapfrog

As the name suggests, with the leapfrog pattern, you bypass interim steps and go straight from an on-premises legacy architecture to a serverless cloud architecture.

Example Scenario: Image Processing

Traditional Setup:

• You have a web app that processes images on a dedicated server.

• Server maintenance, scaling, and cost management are challenges.

Serverless Migration:

• Rewrite the image processing code as a serverless function (e.g., AWS Lambda).

• Configure AWS API Gateway to expose an HTTP endpoint that triggers the "UploadImage" Lambda function to S3.

• Trigger this function whenever an image is uploaded to an S3 bucket.

Lift and Shift

In this model, existing applications are kept intact initially.

Developers experiment with Serverless tools, like for instance, Cloud functions, in low-risk internal scenarios such as log processing or scheduled tasks. Progressively, other serverless components are adopted for tasks such as data transformations and parallelization of processes.

At a certain stage in the adoption process, it's recommended to take a strategic look at how more serverless and microservices infrastructure might address different business goals.

Then, create a production workload as a pilot, and with initial success and lessons learned in the small processes adopted to serverless previously, more applications could be migrated incrementally to serverless.

Example Scenario: Content Management System (CMS) Migration

Traditional Setup:

• You have a traditional CMS running on a virtual server or on-premises infrastructure.

• The CMS serves content, manages user accounts, and handles user-generated content.

Lift and Shift Migration to Serverless:

• Identify a serverless platform, such as a fully managed cloud service like AWS Amplify or Firebase.

• Create a serverless application using this platform.

• Replicate the functionality of your existing CMS within this serverless environment.

• Migrate your content, user data, and user-generated content to the new serverless application.

• Adjust DNS settings or routing to direct traffic to the new serverless application.

• Perform testing to ensure the new serverless CMS functions correctly and serves content as expected.

Strangler

With the strangler pattern, an organization incrementally and systematically decomposes monolithic applications by creating APIs and building event-driven components that gradually replace components of the legacy application.

Distinct API endpoints can point to old compared to new components and safe deployment options (such as canary deployments) let you point back to the legacy version with very little risk.

New feature branches can be serverless first, and legacy components can be decommissioned as they are replaced.

Example Scenario: Legacy eCommerce

Traditional Setup:

• You have a legacy eCommerce application running on a monolithic server.

• It's challenging to maintain and update the old codebase.

Strangler Migration to Serverless:

• Start by identifying a specific function, like product image resizing, within the monolithic app.

• Rewrite this function as a serverless microservice (e.g., AWS Lambda).

• Expose this microservice via an API Gateway endpoint.

• Gradually, route new product image resizing requests to the serverless microservice while the rest of the eCommerce app remains on the monolithic server.

• Over time, refactor and migrate other functions in a similar manner.

• Eventually, the entire application is decomposed into serverless microservices.

• Benefits: Incremental modernization, reduced risk, and improved agility without a complete system rewrite.

Top migration questions you need to answer:

Despite the most common migration pattern for moving complex applications is the strangler pattern, where you refactor and rewrite parts of your application with serverless, in many cases, move to serverless coincide with decompose monolith in order to implement Event-Driven microservices.

For the scope of this post, is interesting to give some examples of questions that require to be answered:

• What does this application do and how are its components organized?

• How does the application scale and what components drive the capacity you need?

• Do you have schedule-based tasks?

• Do you have workers listening to a queue?

• Where can you refactor or enhance functionality without impacting the current implementation?

• What is the infrastructure cost/budget to run your workload?

• What will be the cost/investment of your team’s time to maintain the application once it is in production.