• Discover best practices: Get a comprehensive understanding of how FinOps works to control cloud costs and improve budgeting accuracy.
• Learn from our expert: who will share actionable strategies and real-world examples to help you apply FinOps in your organization.
• Boost accountability and efficiency: Understand how to drive collaboration and ownership in managing cloud resources effectively.

To access the video, please fill in the form

Speakers

Dawid Janasik
Senior DevOps engineer / Technical Leader

MAREK DOBKOWSKI
Head of Microsoft Practice / Senior Azure Architect  

To access the video, please fill in the form

Speakers

Marek Czachorowski
Head of Data & AI Solutions

Krzysztof Bratnicki
Business Development Manager  

What is DevSecOps? 

In the realm of modern craft, there arose a practice known among the learned as DevSecOps, a union of three great disciplines: development, security, and operations. It was not unlike the forging of a mighty alliance, wherein each realm must contribute its strength, lest their endeavors fall prey to the shadows of vulnerability. 

In days past, many development teams would wait until the final hour, when the code was near release, before calling upon the wardens of security. Yet this path was fraught with peril, for to uncover weaknesses so late would often cost dear in time, gold, and effort. But the wisest of realms soon saw another way. 

Benefits of DevSecOps 

By weaving security into every phase of the software development lifecycle – whether in the laying of the first line of code or the final shaping of the product – teams could safeguard their work from the outset and prevent a number of security issues. Through deep collaboration and the magic of automation, they crafted a system where all shared the burden of protection. Thus, no longer was security the task of the few, but of all who toiled together, forging a shield that would hold fast even in the face of the darkest threats. 

In this way, DevSecOps was not merely a method, but a way of ensuring that no creation would leave the forges unguarded against the unseen dangers.  

DevSecOps vs DevOps 

In terms of traditional software development, the crafting of great projects was long governed by the old ways, where the work was divided into phases – like the seasons of the year – each flowing one after the other.  

There would come a time for planning, then design, and only after would the labor of development begin, followed by the testing and binding of all parts into a whole. Yet, this process, while orderly, was as slow as the march of the seasons themselves. In an age where customers’ desires are ever-shifting, such a pace no longer sufficed. 

Worse still, security specialists were oft called upon at the very end, when the work was near complete, to cast their protections over the product. Alas, this lateness was fraught with danger, for vulnerabilities uncovered at such a time could unravel much of the work that had come before. 

Thus, many turned to a new way: the DevOps model. Here, rather than waiting for the long seasons to pass, the DevOps teams delivered smaller, yet finely crafted parcels of work – each one polished and ready – rather than undertaking vast projects that stretched on for years.  

In this way, the teams of development and operations joined forces, testing and refining their work as they went. By using the tools of automation and forging standardized processes, they moved with great speed, yet kept the quality of their software product intact. 

DevSecOps for enhanced application security 

But in time, the guilds realized there was a further step to be taken. They sought not just swiftness, but security woven into every step of their journey. And so was born DevSecOps, where security was not an afterthought but a companion to every stage of the craft. From the very first whispers of planning, security was present, and in the fires of development, it was tested and shaped alongside the code. The burden of security threat protection no longer lay on the shoulders of a few, but on the entire fellowship of creators. This new way became known to some as ‘shift left security’, for it brought the guardians of protection to the forefront of the process, ensuring that no ill would befall their work from the very start of the journey to its end. 

Read also:

• Azure Cost Management 101
• Cloud-native applications: what do you need to know?

Why is DevSecOps important? 

Many foes are seeking to gain entry and plunder company’s most valuable treasures – its data and assets. One of the most common practices they can employ, is the exploitation of weaknesses, hidden deep within the folds of the organization’s own software. These vulnerabilities, if left unchecked, are like cracks in the foundation of a mighty fortress, and through them, adversaries can slip in unnoticed, wreaking havoc from within. 

Such breaches can be devastating. They consume both time and money, and in their wake, they leave scars upon a company’s name, causing trust to wither among clients and partners alike. 

DevSecOps culture 

But there is hope within DevSecOps. By following this path, the guilds of development, security, and operations stand together, ever watchful. The framework is like a vigilant sentry, reducing the risk of sending forth software riddled with flaws, misconfigurations, or vulnerabilities. Through constant vigilance and the weaving of security into every phase of creation, they close the gates through which bad actors might pass, fortifying their code so that it may stand strong against the onslaught of those who would seek to exploit it. Thus, the company stands firm, its reputation unshaken, its defenses prepared for the battles yet to come. 

Continuous Integration 

In the world of software development, there once was a time when the labor of many artisans was brought together only at the final hour, when all the pieces were near completion. But it was often in those moments, at the very end, that flaws were revealed, and the seams of their work would unravel, leaving them with a tangle of issues too great to swiftly resolve. 

But a new method arose, known as Continuous Integration, a practice where the builders of code did not wait for the end to unite their works. Instead, they would commit their efforts to a central repository many times throughout the day, like artisans returning to the hearth to meld their creations together piece by piece. Each time they did so, their work was automatically tested and integrated, ensuring that all parts fit together in harmony. 

By catching integration issues and bugs early, long before the final forge was set aflame, this approach saved the guilds from the chaos of last-minute discoveries.  

Continuous Delivery 

Building upon the foundation of Continuous Integration, a further practice emerged, known as Continuous Delivery. Where the former ensured that code was swiftly integrated and tested, Continuous Delivery took this one step further, automating the journey from the builder’s hands to a staging environment, where code would be tested. 

Once the code reached this staging ground, it did not rest. The system immediately set to work, not only with unit testing, but with a series of trials to ensure that all aspects of the creation were sound. The user interface was inspected to ensure it responded as intended, the seams of integration were examined for any weaknesses, and the APIs were tested to confirm they communicated well between systems. The code was also tested under the weight of simulated traffic, to see if it could bear the burden of the many users it would one day serve. 

The aim of Continuous Delivery is simple: to consistently deliver code that was not only complete, but of true value.  

Continuous Security 

DevSecOps practice, one of the most vital elements is the weaving of security into every step of the software’s journey, from the first step of design to the final unveiling. No longer could security be treated as an afterthought, called upon only when the work was nearly complete. Instead, it became a core part of the process, guiding the work like an unseen but ever-present hand. 

From the earliest stages, when the blueprints of the software were still taking shape, the guilds would engage in threat modeling, a process of foresight where they sought to uncover any potential dangers. They did not wait for the enemy to strike, but anticipated its moves, fortifying their code against unseen attacks before they could take root. 

As the work progressed, automated security testing was woven into every stage of the DevOps workflow. Automation played a key role, testing the code continuously, from the developers’ own environments to the farthest reaches of the deployment pipeline. No phase was left unguarded; no section of code went untested. 

Through this constant vigilance – testing early and testing often – the teams were able to find and mend weaknesses swiftly. And so, they delivered their software products with confidence, knowing that each line of code was protected. With DevSecOps, the road to production became one of fewer pitfalls and greater security, allowing organizations to deliver secured software swiftly and with minimal issues.   

Communication and collaboration 

In the practice of DevSecOps, the strength of the software does not lie solely in the application security tools or the processes but in the fellowship of those who undertake the journey together. It is a path that demands more than mere skill; it calls for deep collaboration and unity of purpose among individuals and teams. 

When the developers commit their work to the central repository in the practice of Continuous Integration, conflicts in code inevitably arise. But it is through collaboration – where minds come together, and voices are heard – that these challenges are resolved. Developers, security experts, and operations alike must work side by side, swiftly addressing these conflicts so the flow of progress is not hindered. 

But beyond the technical work, there is a greater need – communication. Teams must speak openly and often, sharing their visions and aligning around the same goals. Without this shared understanding, the work would drift in many directions, and the efforts of one might undo the labors of another. In DevSecOps approach, every hand contributes to the same creation, and every voice is heard in the great chorus.  

DevSecOps best practices: shift security left 

Planning and development 

Introducing security policies and addressing security risks early into the rhythm of development sprints is akin to fortifying the foundations of a great structure before the first stones are laid. By addressing vulnerabilities in the early stages, teams not only reduce the risk of future threats but also save valuable time, for it is far easier to mend potential flaws before the code has been built and added to the greater whole.  

At the start of every sprint, during planning and development, threat modeling becomes a crucial tool, a map guiding the teams to uncover and mitigate potential dangers long before they can occur. By identifying these threats early, security is no longer something that is added at the end, but something that is woven into the very fabric of the application from the outset. 

To ensure security, before the code is committed to the shared repository, automated checks are employed, acting as vigilant sentries. Integrated development environment (IDE) security plug-ins provide developers with immediate feedback, warning them if their code harbors a potential risk. These automated checks catch flaws early, empowering developers to address them before they can take root. 

As the code continues its journey, passing from one set of hands to the next, the software is further refined during the code review. Here, someone with the knowledge of security steps forward, offering their insight and making recommendations to bolster the work. This expertise ensures that, by the time the code is ready to move on, it is fortified against threats, functional, and valuable.   

Code commit 

A cornerstone of the DevSecOps process lies in the practice of continuous integration, a discipline that ensures code is not simply created in isolation but is constantly woven into the central repository, allowing teams to catch issues before they can fester. Developers, like diligent artisans, commit their code several times a day, ensuring that each piece of work seamlessly fits into the greater whole. This frequent integration allows potential conflicts or errors to be discovered early, long before they can threaten the stability of the project. 

However, to truly safeguard the craft, it is vital to introduce security into this phase. Automated security checks must stand guard alongside the integration process. These include scanning third-party libraries and dependencies – those external pieces of code that, while useful, may harbor unseen vulnerabilities. Unit testing ensures the smallest parts of the code function as they should, while static application security testing (SAST) reviews the code for weaknesses, searching for hidden threats that might otherwise go unnoticed. 

But safeguarding the code itself is not enough. The continuous integration (CI) and continuous delivery (CD) infrastructure, which carries this code from creation to deployment, must also be protected. Role-based access controls (RBAC) play a crucial role in this defense, limiting access to the system based on the specific roles of individuals. By ensuring that only those with the right permissions can interact with the CI/CD infrastructure, teams protect it from attackers who might seek to run malicious code or steal credentials. 

In this way, the continuous integration process becomes not only a means to unite code swiftly and efficiently but also a stronghold against external threats. Security is built into every layer, from the automated checks that scan the code to the protections guarding the very systems that bring the work to life. 

Building and testing 

In DevSecOps, where vigilance is paramount, the test environment serves as a proving ground for code before it ventures into production. Here, automated security scripts are seeking out potential threats that may have slipped past earlier defenses. By running these tests in a controlled environment, teams can uncover hidden vulnerabilities and ensure their work remains strong and secure.  
 
DAST 

Among the many tests that can be employed during this phase is Dynamic Application Security Testing (DAST), which simulates real-world attacks against the running application. Unlike static tests, DAST operates while the application is live, identifying vulnerabilities such as: cross-site scripting, SQL injection, and other dangerous flaws. 

Infrastructure scanning  

Infrastructure scanning follows, casting its gaze across the entire architecture, from servers to networks, searching for weaknesses in the foundational layers that might allow an attacker entry. For those employing containers as part of their deployment strategy, container scanning ensures that these lightweight units of software do not harbor vulnerabilities in their dependencies or configurations, fortifying them before they are deployed. 

Cloud configuration validation 

In the age of the cloud, where infrastructure is often abstracted and spread across vast digital environments, cloud configuration validation becomes crucial. By checking the configurations of cloud resources, teams can ensure that no misconfigurations – such as excessive permissions or insecure access points – expose their environments to unnecessary risk. 

Lastly, security acceptance testing ensures that all necessary security requirements are met. This step serves as the final safeguard, confirming that the code and infrastructure are not only functional but fortified against known threats and risks minimized. 

Production 

Once the application has been deployed to production and stands in the real world, some organizations take a proactive step to uncover any remaining weaknesses by engaging in penetration testing. This practice is more than just another test – it is a deliberate attempt to breach the application as an attacker might, with real-world tactics and determination. 

In penetration testing, skilled individuals, often referred to as ethical hackers, adopt the mindset of a potential adversary. They probe the application for weaknesses, using the same strategies and tools a malicious actor might employ. These tests can range from exploiting known vulnerabilities in third-party components to more sophisticated attacks aimed at bypassing the application’s defenses. 

The goal is simple: to expose any weaknesses that might have slipped through the earlier layers of security testing, those that could potentially be exploited. By simulating real-world attack scenarios, penetration testing reveals how the application holds up under direct assault, whether it’s vulnerable to unauthorized access, data breaches, or other forms of compromise. 

This phase is crucial for understanding not just theoretical vulnerabilities but how the system behaves in a live environment, where the s 1takes are highest. Penetration testing provides organizations with invaluable insights into the robustness of their defenses, allowing them to patch any remaining weaknesses before an actual attacker can exploit them. Thus, it becomes the final line of preparation, ensuring that the application is truly ready to stand firm against threats in the production environment. 

Operation 

Even with the most robust DevSecOps process, no system is entirely resistant to evolving threats. This is why continuous monitoring of applications becomes essential once they are deployed. By maintaining constant vigilance, organizations can quickly detect, respond to, and mitigate any new vulnerabilities, unforeseen threats and risks before they cause significant harm. 

Monitoring tools look for signs of irregularities, scanning for vulnerabilities, unauthorized access attempts, or other suspicious activities that might signal a breach or weakness. These tools provide real-time insights, alerting teams to potential issues the moment they arise.  

To further strengthen this defense, analytics data plays a key role. By analyzing patterns and trends in security events, teams can evaluate the effectiveness of their security posture. This data offers valuable insights into how well current defenses are performing, allowing organizations to track whether they are improving over time or if new vulnerabilities are emerging. It also highlights areas that may require optimization, guiding future efforts in reinforcing the system.  

Bear in mind, however, that in the world of security, the battle is never truly over. 

Read also:

• Cloud-Agnostic Applications: Pros and Cons
• Horizontal vs Vertical Scaling: A 101 Guide
• How to gain a cloud advantage? Here are 7 cloud computing trends for 2023 – 2025 

Implementing DevSecOps and automating security best practices 

I bid you to consider these tools as you embark upon the journey of DevSecOps automation within your organization. Some are like fruit hanging low upon the bough, easily gathered and swiftly put to use, while others may lie deeper within the forest, requiring more effort to attain. Yet, though the path to them may be more difficult, the rewards they yield are well worth the quest. 

Want to gain cost and competitive advantage in the cloud?

Consult with Marek Dobkowski, Head of Microsoft Practice, for expert guidance.

Let's talk!

Recommended DevSecOps tools 

Trivvy 

Trivy has risen to prominence as a trusted solution among open-source security scanners, valued for its reliability, swiftness, and simplicity. It offers a far-reaching array of security checks, making it a vital companion for those seeking to fortify their DevSecOps practices. For teams looking to secure their realms of code and infrastructure, Trivy stands as a steadfast tool, ever vigilant and ready to ensure the safety of their creations. 

Targets (what Trivy can scan): 

• Container Image 
• Filesystem 
• Git Repository (remote) 
• Virtual Machine Image 
• Kubernetes 
• AWS 

Scanners (what Trivy can find there): 

• OS packages and software dependencies in use (SBOM) 
• Known vulnerabilities (CVEs) 
• IaC issues and misconfigurations 
• Sensitive information and secrets 
• Software licenses 

Trufflehog 

TruffleHog tool is a masterful seeker of concealed passwords and keys. Like a skilled ranger, TruffleHog ventures where few dare to tread, unearthing the hidden secrets that, if left unchecked, could spell doom for the unwary. 

How TruffleHog wields its power: 

• Detect: TruffleHog scours the history of all platforms, much like a wise lorekeeper sifting through ancient scrolls, seeking out long-forgotten secrets. Yet it looks not only in the obvious places but also in the whispers of comments, the hidden folds of Docker images, and other obscure corners. 
• Analyze: TruffleHog reveals the true nature of the secrets it uncovers, discerning what resources and permissions are tied to API keys and other tokens. Remarkably, it achieves this without ever needing to peer into the provider’s vault. 
• Prevent: To stop the ill-fated inclusion of secrets from the very beginning, TruffleHog sets traps at key points, using pre-commit and pre-receive hooks. These safeguards ensure that no sensitive data is unintentionally leaked before it ever leaves the developer’s hand. 
• Remediate: TruffleHog continues to track the fate of discovered keys and secrets. It verifies that remediation is complete, sending reminders on preferred platforms and providing knowledge to users on how to properly manage and secure the keys that were once at risk. 

Why TruffleHog is a worthy ally: 

• Comprehensive multi-branch analysis: TruffleHog does not simply guard the main road but patrols every path. It scans all branches, not just the primary one, ensuring the same level of vigilance across the entire project. This is especially valuable in larger domains where many branches are being worked on in tandem. 
• Credential verification: TruffleHog employs programmatic verification, testing each credential using its own protocol or API. This removes the false trails, ensuring that only real threats are brought to light. 
• Open-source fellowship: As with any great alliance, TruffleHog thrives through the support of an open-source community. Many dedicated hands join together to audit and improve the tool, ensuring that no single voice carries undue weight. The community checks and balances each other’s work, so that trust is shared among all. 
   

Snyk 

This platform guards the entirety of an application’s journey – from the very first lines of code to its deployment in the cloud. Through its guidance, developers may discover and mend vulnerabilities before they are ever loosed upon the world. 

The powers of Snyk: 

• Snyk open source: Snyk scours open-source libraries and dependencies, seeking out vulnerabilities. And when such flaws are found, it does not merely warn the developer but offers a swift means to mend them, restoring the strength of the code. 
• Snyk code: As code is written, Snyk watches in real-time, finding and fixing vulnerabilities within the very heart of the application. It is like a companion at the developer’s side, ever watchful and ready to lend its aid.  
• Snyk container: In the context of containers and Kubernetes, where applications are housed, Snyk’s gaze does not falter. It delves into container images, finding and repairing potentially harmful vulnerabilities.  
• Snyk Infrastructure as Code: With great foresight, Snyk peers into the blueprints of infrastructure itself, examining the configurations of Terraform and Kubernetes code. Should it find any insecurity in the very foundation, it offers swift guidance on how to rectify these flaws, ensuring that the structure remains strong and secure. 

Pre-commit 

 Pre-commit is a powerful system for managing and maintaining pre-commit hooks across many programming languages. Pre-commit ensures that no errant detail is left unchecked before the code is sent for review. 

In the world of Git, hook scripts act as a safeguard, catching simple errors before they reach the eyes of a reviewer. Whenever a developer commits their work, these hooks spring into action, pointing out issues such as missing semicolons, trailing whitespace, or forgotten debug statements. By addressing these small matters early, Pre-commit allows the reviewer to focus on the grand architecture of the changes, rather than wasting time on trivial style errors. 

Wazuh 

Free and open to all, Wazuh is skilled in the arts of threat prevention, detection, and response. It is a protector capable of defending the realms of on-premises fortresses, virtualized strongholds, containerized ships, and vast cloud kingdoms alike. 

The strength of Wazuh lies in two parts: its endpoint security agents, which are deployed like watchful sentinels to the systems they protect, and its management server, a wise and ever-alert overseer. 

The agents gather knowledge and data from the systems they monitor, and the management server collects, analyzes, and interprets this information, ever vigilant for signs of danger. 

Wazuh, whem integrated with Elastic Stack, offers seamless navigation through security alerts, enhancing visibility and threat detection. By combining it with SIEM and XDR, you can gain protection for IT assets, responding to potential security dangers.  
 
Use-cases: 

• Configuration assessment 
• Malware detection 
• File integrity monitoring 
• Threat hunting 
• Log data analysis 
• Vulnerability detection 
• Incident response 
• Regulatory compliance 
• IT hygiene 
• Containers security 
• Posture management 
• Workload protection 
   

Successful DevSecOps and DevOps Integration 

The integration should be as natural as the turning of the seasons, an organic and seamless process that unfolds with time. It is not a single task to be completed and forgotten, but a continuous journey.   Though at times it may call for a shift in the very culture of the organization, such change is not forced, but arises naturally 

And as in all great works of creation, what we forge must be shaped by the needs of people. The processes we follow and the tools we wield must be chosen wisely, fitting the unique contours of our organization Only then can the integration thrive, as both DevOps and DevSecOps become not separate disciplines, but part of the same living tapestry, woven together in purpose and vision. 

Read also:

• DevOps monitoring systems
• Azure Durable Functions – the extension to Azure Functions

To access the video, please fill in the form

Meet our speaker

Amadeusz Kryze
Principal DevOps Leader at Inetum

The Dev[Sec]Ops Technology Division Manager, with over ten years of experience in the DevOps and DevSecOps industry, focused on gathering and implementing best practices tailored to individual project needs.  

To access the video, please fill in the form

Speaker

Dawid JANASIK

Software / Systems Engineer & DevOps Team Leader in Inetum with over 10 years spent working on many challenging projects for different clients & many industries. He is a certified engineer working on extending his know-how regarding DevOps & cloud-native solutions.

Despite the rapid advancement of technology and the constant release of new models and SDKs, it can be difficult for developers to know where to begin. While there are many polished end-to-end sample applications available for .NET developers to use as a reference, some may prefer to build their applications incrementally, starting with the basics and gradually adding more advanced features.

Building a console-based .NET chat app with solutions like Semantic Kernel

This post aims to guide developers in building a simple console-based .NET chat application from scratch, with minimal dependencies and fuss. The ultimate goal is to create an application that can answer questions based on both the data used to train the model and additional data provided dynamically. Each code sample provided in this post is a complete application, allowing developers to easily copy, paste, and run the code, experiment with it, and then incorporate it into their own applications for further refinement and customization.

How to get started with Semantic Kernel SDK. Learn how to use it

To begin, make sure you have .NET 8 installed, and create a simple console app.dotnet

new console -o chat-sample-app-01 --use-program-main 

cd chat-sample-app-01

This creates a new directory chat-sample-app-01 and populates it with two files: chat-sample-app-01.csproj and Program.cs. We then need to bring in one NuGet package: Microsoft.SemanticKernel.

dotnet add package Microsoft.SemanticKernel

Find out more at Microsoft Learn

Planning, orchestration, and multiple plugins

Instead of referencing specific AI-related packages such as Azure.AI.OpenAI, I have opted for an open source Semantic Kernel kit to streamline various interactions and easily switch between different implementations for faster experimentation. Semantic Kernel offers a collection of libraries that simplify working with Large Language Models (LLMs) by providing abstractions for various AI concepts, allowing for the easy substitution of different implementations. It also includes many concrete implementations of these abstractions, wrapping numerous other SDKs, and offers support for planning, orchestration, and multiple plugins. This post will explore various aspects of Semantic Kernel, but my primary focus is on its abstractions.

While I have tried to keep dependencies to a minimum for the purpose of this article, there is one more I cannot avoid: you need access to an LLM. The easiest way to get access is via either OpenAI or Azure OpenAI. For this post, I am using Azure OpenAI. You will need three pieces of information for the remainder of the post:

• Your API key and endpoint provided to you in the Azure portal
• A chat model, or to be more precise, the deployment name of your model. I use GPT-4-32k (0613), which as of this writing has a context window of 32K tokens. I’ll explain more about what it is later.
• An embedding model. I use text-embedding-3-large.

Let’s make it as easy as possible

With that out of the way, we can dive in. Believe it or not, we can create a simple chat app in just a few lines of code. Copy and paste this into your Program.cs:

using Microsoft.SemanticKernel;

namespace chatSampleApp01
{
    class Program
    {
        static string deploymentName = Environment.GetEnvironmentVariable("AI:OpenAI:DeploymentName")!;
        static string endpoint = Environment.GetEnvironmentVariable("AI:OpenAI:Endpoint")!;
        static string apiKey = Environment.GetEnvironmentVariable("AI:OpenAI:APIKey")!;

        static async Task Main(string[] args)
        {
            //Initialize Semantic Kernel
            var builder = Kernel.CreateBuilder();
            builder.AddAzureOpenAIChatCompletion(deploymentName, endpoint, apiKey);
            
            var kernel = builder.Build();

            //Question and Answer loop
            string question;
            while (true)
            {
                Console.Write("Me: ");
                question = Console.ReadLine()!;
                Console.Write("Mine Copilot: ");
                Console.WriteLine(await kernel.InvokePromptAsync(question));
                Console.WriteLine();
            }
        }
    }
}

To prevent accidentally revealing my API key, which should be safeguarded like a password, I have stored it in an environment variable and accessed it using GetEnvironmentVariable. Then I created a new kernel using the Semantic Kernel APIs and added an OpenAI chat completion service to it. The Microsoft.SemanticKernel package we imported earlier includes references to client support for both OpenAI and Azure OpenAI, eliminating the need for additional components to communicate with these services. With this configuration, we can now run our chat app using dotnet run, enter questions, and receive responses from the service. 

The expression await kernel.InvokePromptAsync(question) is the core of the interaction with the LLM, where it captures the user’s input and sends it to the LLM, receiving a string response in return. Semantic Kernel is equipped to handle various function types, including prompt functions for text-based AI interactions and standard .NET methods capable of executing any C# code. These functions can be triggered directly by the user, as shown in this example, or as part of a “plan” where a set of functions is provided to the LLM to formulate a strategy to achieve a specified objective. Semantic Kernel can execute these functions as per the plan (I will show it later). Additionally, some models support a “function calling” feature, which is also simplified by Semantic Kernel.

Leveraging Semantic functions: reusable prompts, dynamic input handling, and plugins

In this instance, “function” refers to the user’s input, such as the question “What is Inetum Polska?” which is then processed by the LLM through the InvokePromptAsync method. To clarify the concept of “function,” we can extract it into a separate entity using the `CreateFunctionFromPrompt“ method, allowing us to reuse the same function for multiple inputs. This approach eliminates the need to create a new function for each input, but requires a way to incorporate the user’s input into the existing function. Semantic Kernel supports this through prompt templates, which include placeholders that are filled with the appropriate variables and functions. For example, if the sample is run again with a request for the current time, the LLM will not be able to provide an answer:

To anticipate such inquiries, we can equip the LLM with the necessary information within the prompt itself. I have registered a function with the kernel that provides the current date and time. Subsequently, I created a prompt function that utilizes a prompt template to invoke this time function during the prompt’s rendering. This template also incorporates the value of the $input variable. It is possible to pass any number of arguments with arbitrary names using a KernelArguments dictionary; in this case, I have chosen to name one “input”. Functions are organized into collections known as “plugins”.

using Microsoft.SemanticKernel;

namespace chatSampleApp02
{
    class Program
    {
        static string deploymentName = Environment.GetEnvironmentVariable("AI:OpenAI:DeploymentName")!;
        static string endpoint = Environment.GetEnvironmentVariable("AI:OpenAI:Endpoint")!;
        static string apiKey = Environment.GetEnvironmentVariable("AI:OpenAI:APIKey")!;

        static async Task Main(string[] args)
        {
            //Initialize Semantic Kernel
            var builder = Kernel.CreateBuilder();
            builder.AddAzureOpenAIChatCompletion(deploymentName, endpoint, apiKey);

            // Create the prompt function as part of a plugin and add it to the kernel.
            builder.Plugins.AddFromFunctions(
                pluginName: "DateTimeHelpers",
                functions: [
                    KernelFunctionFactory.CreateFromMethod(
                        ()=> $"{DateTime.UtcNow:r}", "Now", "Gets the current date and time"
                    )
                ]);

            var kernel = builder.Build();

            var kernelFunction = KernelFunctionFactory.CreateFromPrompt(
                promptTemplate: @"
                    The current date and time is {{ datetimehelpers.now }}.
                    {{ $input }}"
                );

            //Question and Answer loop
            string question;
            while (true)
            {
                Console.Write("Me: ");
                question = Console.ReadLine()!;
                Console.Write("Mine Copilot: ");
                Console.WriteLine(await kernelFunction.InvokeAsync(kernel, new() { ["input"] = question }));
                Console.WriteLine();
            }
        }
    }
}

When the function is activated, it renders the prompt by calling the previously registered ‘Now’ function and integrating its output into the prompt. Now, posing the same question yields a more comprehensive answer. 

Does LLM have memory? How to use Semantic Kernel to overcome statelessness in chat agents 

We have made significant strides: with just a few lines of code, we have crafted a basic chat agent that can field repeated questions and provide responses. Moreover, we have managed to furnish it with extra prompt information to aid in answering questions it would otherwise be unable to tackle. Yet, in doing so, we have also fashioned a chat agent devoid of memory, lacking any awareness of prior conversations: 

To remedy the statelessness of LLMs and their lack of memory, we must maintain a record of our chat history and incorporate it into each prompt request. This can be done manually by integrating the chat history into the prompt, or we can rely on Semantic Kernel to handle it for us, which in turn can depend on the clients for Azure OpenAI, OpenAI, or any other chat service. The latter approach involves using the registered IChatCompletionService to create a new chat, which is essentially a compilation of all messages. This method not only processes requests and outputs responses but also archives them into the chat history. 

using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;

namespace chatSampleApp03
{
    class Program
    {
        static string deploymentName = Environment.GetEnvironmentVariable("AI:OpenAI:DeploymentName")!;
        static string endpoint = Environment.GetEnvironmentVariable("AI:OpenAI:Endpoint")!;
        static string apiKey = Environment.GetEnvironmentVariable("AI:OpenAI:APIKey")!;

        static async Task Main(string[] args)
        {
            //Initialize Semantic Kernel
            var builder = Kernel.CreateBuilder();
            builder.AddAzureOpenAIChatCompletion(deploymentName, endpoint, apiKey);
            var kernel = builder.Build();

            //Create new chat
            var chatService = kernel.GetRequiredService<IChatCompletionService>();
            var chat = new ChatHistory(
                    systemMessage: "You are an AI assistant that helps people find information."
                );

            //Question and Answer loop
            string question;
            while (true)
            {
                Console.Write("Me: ");
                question = Console.ReadLine()!;
                chat.AddUserMessage(question);

                Console.Write("Mine Copilot: ");
                var answer = await chatService.GetChatMessageContentAsync(chat);
                chat.AddAssistantMessage(answer.Content!);
                Console.WriteLine(answer);
                
                Console.WriteLine();
            }
        }
    }
}

With that chat history rendered into an appropriate prompt, we then get back much more satisfying results: 

In a practical application, it is crucial to consider various additional factors, such as the data processing limitations of language models, referred to as the “context window.” The `GPT-4-32k (0613) model that I am using here can handle ~32000 tokens, where a token can be a full word, part of a word, or a single character. Additionally, each token incurs a cost for every interaction. Therefore, when transitioning from a trial phase to full production, it becomes essential to monitor the chat history’s data volume closely and manage it by removing unnecessary parts, etc.

We can enhance the user experience by adding a small segment of code that accelerates the interaction. These large language models (LLMs) generate responses by predicting the next token, so while we have been displaying the complete response once it is fully generated, we can actually present it in real time as it is being formulated. This functionality is available in Semantic Kernel through IAsyncEnumerable, which allows for convenient integration using await foreach loops to stream the response incrementally.

Does LLM know everything? Connectors and RAG with Semantic plugins and native functions

We have now reached a point where we can pose questions and receive answers, maintain a record of these exchanges to refine future responses, and even broadcast our findings. But is our work complete? Not quite.

As it stands, the only information available to LLM for providing answers is the data it was initially trained on, plus any additional information we explicitly include in the prompt (like

the current time, as previously mentioned). Consequently, if we inquire about topics outside the LLM’s training or areas where its knowledge is lacking, the responses we receive may be unhelpful, misleading, or entirely incorrect, which are often referred to as ‘hallucinations’.

The question is about the latest C# 12 changes, which were released after this version of the GPT-4-32k (0613) model was released (November 2023 vs October 2021). The model has no information about the newest capabilities, so it does not give a reasonable answer to the first question. We need to find a way to teach it about the things the user is asking about.

We know the way to teach LLM: include the necessary information in the prompt. For instance, the Microsoft Learn articles:

• Relationships between language features and library types 
• Version and update considerations for C# developers 
• The history of C# 
• What’s new in C# 11 
• Breaking changes in Roslyn after .NET 6 all the way to .NET 7 
• What’s new in C# 12 
• Breaking changes in Roslyn after .NET 7 all the way to .NET 8 
• What’s new in C# 13 
• Breaking changes in Roslyn after .NET 8 all the way to .NET 9 
• What’s new in .NET 8 
• What’s new in the .NET 8 runtime 
• What’s new in the SDK and tooling for .NET 8 
• What’s new in containers for .NET 8 

which were published after the training of this GPT-4-32k (0613) model have a detailed section regarding the new capabilities of C# 12. By incorporating this content into the prompt, we can supply the LLM with the required knowledge. In the following example, I have expanded the previous code to download the web page content and then insert it into a user message.

This approach ensures that the LLM is provided with the latest information to assist with the query.

using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using System.Text;

namespace chatSampleApp05
{
    internal class Program
    {
        static string deploymentName = Environment.GetEnvironmentVariable("AI:OpenAI:DeploymentName")!;
        static string endpoint = Environment.GetEnvironmentVariable("AI:OpenAI:Endpoint")!;
        static string apiKey = Environment.GetEnvironmentVariable("AI:OpenAI:APIKey")!;

        static async Task Main(string[] args)
        {
            //Initialize Semantic Kernel
            var builder = Kernel.CreateBuilder();
            builder.AddAzureOpenAIChatCompletion(deploymentName, endpoint, apiKey);
            var kernel = builder.Build();

            //Create new chat
            var chatService = kernel.GetRequiredService<IChatCompletionService>();
            var chat = new ChatHistory(
                    systemMessage: "You are an AI assistant that helps people find information."
                );

            // Download a documents and add all of its contents to our chat
            var articleList = new List<string>
            {
                "https://raw.githubusercontent.com/dotnet/docs/main/docs/csharp/whats-new/relationships-between-language-and-library.md",
                "https://raw.githubusercontent.com/dotnet/docs/main/docs/csharp/whats-new/version-update-considerations.md",
                "https://raw.githubusercontent.com/dotnet/docs/main/docs/csharp/whats-new/csharp-version-history.md",
                "https://raw.githubusercontent.com/dotnet/docs/main/docs/csharp/whats-new/csharp-11.md",
                "https://raw.githubusercontent.com/dotnet/roslyn/main/docs/compilers/CSharp/Compiler%20Breaking%20Changes%20-%20DotNet%207.md",
                "https://raw.githubusercontent.com/dotnet/docs/main/docs/csharp/whats-new/csharp-12.md",
                "https://raw.githubusercontent.com/dotnet/roslyn/main/docs/compilers/CSharp/Compiler%20Breaking%20Changes%20-%20DotNet%208.md",
                "https://raw.githubusercontent.com/dotnet/docs/main/docs/csharp/whats-new/csharp-13.md",
                "https://raw.githubusercontent.com/dotnet/roslyn/main/docs/compilers/CSharp/Compiler%20Breaking%20Changes%20-%20DotNet%209.md",
                "https://raw.githubusercontent.com/dotnet/docs/main/docs/core/whats-new/dotnet-8/overview.md",
                "https://raw.githubusercontent.com/dotnet/docs/main/docs/core/whats-new/dotnet-8/runtime.md",
                "https://raw.githubusercontent.com/dotnet/docs/main/docs/core/whats-new/dotnet-8/sdk.md",
                "https://raw.githubusercontent.com/dotnet/docs/main/docs/core/whats-new/dotnet-8/containers.md"

            };
            
            var articleStringBuilder = new StringBuilder();

            using (var httpClient = new HttpClient())
            {
                foreach (var article in articleList) {
                    articleStringBuilder.Append(await httpClient.GetStringAsync(article));
                }

                chat.AddUserMessage($"Here's some additional information: {articleStringBuilder.ToString()}");
            }

            string question;
            StringBuilder stringBuilder = new StringBuilder();

            //Question and Answer loop
            while (true)
            {
                Console.Write("Me: ");
                question = Console.ReadLine()!;
                chat.AddUserMessage(question);

                stringBuilder.Clear();
                Console.Write("Mine Copilot: ");

                await foreach (var message in chatService.GetStreamingChatMessageContentsAsync(chat))
                {
                    Console.Write(message);
                    stringBuilder.Append(message.Content);
                }
                Console.WriteLine();
                chat.AddAssistantMessage(stringBuilder.ToString());
                Console.WriteLine();
            }
        }
    }
}

and the result?

We have gone over the context window almost twice without adding any history to the conversation. We obviously need to include less information, but still need to ensure it is relevant information. RAG will help us… 

Interested in building AI-powered apps? Connect with Marek Dobkowski to explore Microsoft solutions that simplify your workflow and enhance productivity. Schedule a meeting

Using RAG

Retrieval Augmented Generation (RAG) essentially means looking up relevant information and incorporating it into the prompt. Instead of including all possible information in the prompt, we index the additional information we care about. When a question is asked, we use that question to find the most relevant indexed content and add just that specific content to the prompt. To facilitate this process, we need embeddings. 

An embedding can be thought of as a vector (array) of floating-point values that represents the content and its semantic meaning. We can use a model specifically designed for embeddings to generate such a vector for a given input, and then store both the vector and the original text in a database. Later, when a question is posed, we can process that question through the same model to produce a vector, which we then use to find the most relevant embeddings in our database. We are not necessarily looking for exact matches, but rather for sufficiently similar ones. The term ‘close’ here is quite literal, as the lookups are typically performed using distance measures like cosine similarity. For instance, consider this program: 

using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Embeddings;
using System.Numerics.Tensors;

namespace chatSampleApp06
{
    internal class Program
    {
        static string embeddingDeploymentName = Environment.GetEnvironmentVariable("AI:OpenAI:EmbeddingDeploymentName")!;
        static string endpoint = Environment.GetEnvironmentVariable("AI:OpenAI:Endpoint")!;
        static string apiKey = Environment.GetEnvironmentVariable("AI:OpenAI:APIKey")!;

        static async Task Main(string[] args)
        {
            //Experimental
            #pragma warning disable SKEXP0010,SKEXP0001

            //Initialize Semantic Kernel
            var builder = Kernel.CreateBuilder();

            builder.AddAzureOpenAITextEmbeddingGeneration(embeddingDeploymentName, endpoint, apiKey);

            var kernel = builder.Build();

            var input = "What is a reptile?";
            var examples = new string[]
            {
                "What is a reptile?",
                "¿Qué es un reptil?",
                "Was ist ein Reptil?",
                "A turtle is a reptile.",
                "Eidechse ist ein Beispiel für Reptilien.",
                "Crocodiles, lizards, snakes, and turtles are all examples.",
                "A frog is green.",
                "A grass is green.",
                "A cat is a mammal.",
                "A dog is a man's best friend.",
                "My best friend is Mike.",
                "I'm working at Inetum Polska since 2013."
            };

            // Generate embeddings for each piece of text
            var embeddingGenerator = kernel.GetRequiredService<ITextEmbeddingGenerationService>();
            var inputEmbedding = (await embeddingGenerator.GenerateEmbeddingsAsync([input])).First();

            var exampleEmbeddings = (await embeddingGenerator.GenerateEmbeddingsAsync(examples)).ToArray();
            var similarities = new List<Tuple<float, string>>();

            // Print the cosine similarity between the input and each example
            for (int i = 0; i < exampleEmbeddings.Length; i++)
            {
                similarities.Add(
                    new Tuple<float, string>(
                        TensorPrimitives.CosineSimilarity(exampleEmbeddings[i].Span, inputEmbedding.Span), 
                        examples[i]));
            }

            similarities.Sort((x,y) => y.Item1.CompareTo(x.Item1));

            Console.WriteLine("Similarity\tExample");
            foreach (var similarity in similarities) {
                Console.WriteLine($"{similarity.Item1:F6}\t{similarity.Item2}");
            }
            Console.ReadLine();
        }
    }
}

This process utilizes the AzureOpenAI embedding generation service to obtain an embedding vector (using the text-embedding-3-large model mentioned earlier in the post) for both an input and several other pieces of text. It then compares the resulting embedding for the input with the embeddings of those other texts, sorts the results based on similarity, and prints them out. 

Let’s incorporate this concept into the chat app. In this round, I have augmented the previous chat example with a few things: 

• In order for Semantic Kernel to handle the embedding generation through its abstractions, we need to include its Memory package. Please note the –prerelease flag, as this is an evolving area. While some Semantic Kernel components are stable, others are still in development and therefore marked as prerelease. 

dotnet add package Microsoft.SemanticKernel.Plugins.Memory --prerelease 

• Next, I need to create an ISemanticTextMemory for querying. I achieved this by using MemoryBuilder to combine an embeddings generator with a database. I specified the Azure OpenAI service as my embeddings generator using the WithAzureTextEmbeddingGenerationService method. For the store, I registered a VolatileMemoryStore instance using the WithMemoryStore method. Although we will change this later, it will suffice for now. VolatileMemoryStore is essentially an implementation of Semantic Kernel’s IMemoryStore abstraction that wraps an in-memory dictionary. 
• I downloaded the text and used Semantic Kernel’s TextChunker to break it into pieces. Then, I saved each piece to the memory store using `SaveInformationAsync“. This process generates an embedding for the text and stores the resulting vector along with the input text in the dictionary. 
• When it is time to ask a question, instead of just adding the question to the chat history and submitting it, we first use the question to perform a SearchAsync on the memory store. This generates an embedding vector for the question and searches the store for the closest vectors. I have it return the three closest matches, append the associated text together, add the results to the chat history, and submit it. After submitting the request, I remove this additional context from the chat history to avoid sending it again in subsequent requests, as it can consume much of the allowed context window. 

And full source code… 

using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using Microsoft.SemanticKernel.Connectors.OpenAI;
using Microsoft.SemanticKernel.Memory;
using Microsoft.SemanticKernel.Text;
using System.Net;
using System.Text;
using System.Text.RegularExpressions;

namespace chatSampleApp07
{
    internal class Program
    {
        static string deploymentName = Environment.GetEnvironmentVariable("AI:OpenAI:DeploymentName")!;
        static string embeddingDeploymentName = Environment.GetEnvironmentVariable("AI:OpenAI:EmbeddingDeploymentName")!;
        static string endpoint = Environment.GetEnvironmentVariable("AI:OpenAI:Endpoint")!;
        static string apiKey = Environment.GetEnvironmentVariable("AI:OpenAI:APIKey")!;

        static async Task Main(string[] args)
        {
            //Experimental
            #pragma warning disable SKEXP0001, SKEXP0010, SKEXP0050

            //Initialize Semantic Kernel
            var builder = Kernel.CreateBuilder();
            builder.AddAzureOpenAIChatCompletion(deploymentName, endpoint, apiKey);
            var kernel = builder.Build();

            //Initialize Memory Builder
            var memoryBuilder = new MemoryBuilder()
                .WithMemoryStore(new VolatileMemoryStore())
                .WithAzureOpenAITextEmbeddingGeneration(
                    deploymentName: embeddingDeploymentName,
                    endpoint: endpoint,
                    apiKey: apiKey
                );

            var memory = memoryBuilder.Build();

            // Download a documents and add all of its contents to our chat
            var articleList = new List<string>
            {
                "https://raw.githubusercontent.com/dotnet/docs/main/docs/csharp/whats-new/relationships-between-language-and-library.md",
                "https://raw.githubusercontent.com/dotnet/docs/main/docs/csharp/whats-new/version-update-considerations.md",
                "https://raw.githubusercontent.com/dotnet/docs/main/docs/csharp/whats-new/csharp-version-history.md",
                "https://raw.githubusercontent.com/dotnet/docs/main/docs/csharp/whats-new/csharp-11.md",
                "https://raw.githubusercontent.com/dotnet/roslyn/main/docs/compilers/CSharp/Compiler%20Breaking%20Changes%20-%20DotNet%207.md",
                "https://raw.githubusercontent.com/dotnet/docs/main/docs/csharp/whats-new/csharp-12.md",
                "https://raw.githubusercontent.com/dotnet/roslyn/main/docs/compilers/CSharp/Compiler%20Breaking%20Changes%20-%20DotNet%208.md",
                "https://raw.githubusercontent.com/dotnet/docs/main/docs/csharp/whats-new/csharp-13.md",
                "https://raw.githubusercontent.com/dotnet/roslyn/main/docs/compilers/CSharp/Compiler%20Breaking%20Changes%20-%20DotNet%209.md",
                "https://raw.githubusercontent.com/dotnet/docs/main/docs/core/whats-new/dotnet-8/overview.md",
                "https://raw.githubusercontent.com/dotnet/docs/main/docs/core/whats-new/dotnet-8/runtime.md",
                "https://raw.githubusercontent.com/dotnet/docs/main/docs/core/whats-new/dotnet-8/sdk.md",
                "https://raw.githubusercontent.com/dotnet/docs/main/docs/core/whats-new/dotnet-8/containers.md"
            };

            var collectionName = "microsoft-news";

            using (var httpClient = new HttpClient())
            {
                var allParagraphs = new List<string>();

                foreach (var article in articleList)
                {
                    var content = await httpClient.GetStringAsync(article);
                    var lines = TextChunker.SplitPlainTextLines(content, 64);
                    var paragraphs = TextChunker.SplitPlainTextParagraphs(lines, 512);

                    allParagraphs.AddRange(paragraphs);
                }

                for (var i = 0; i < allParagraphs.Count; i++)
                {
                    await memory.SaveInformationAsync(collectionName, allParagraphs[i], $"paragraph[{i}]");
                }
            }

            //Create new chat
            var chatService = kernel.GetRequiredService<IChatCompletionService>();
            var chat = new ChatHistory(
                    systemMessage: "You are an AI assistant that helps people find information."
                );

            string question;
            var responseBuilder = new StringBuilder();
            var contextBuilder = new StringBuilder();

            //Question and Answer loop
            while (true)
            {
                Console.Write("Me: ");
                question = Console.ReadLine()!;

                await foreach (var result in memory.SearchAsync(collectionName, question, limit: 3))
                {
                    contextBuilder.AppendLine(result.Metadata.Text);
                }

                var contextToRemove = -1;
                if (contextBuilder.Length > 0)
                {
                    contextBuilder.Insert(0, "Here's some additional information: ");
                    contextToRemove = chat.Count;
                    chat.AddUserMessage(contextBuilder.ToString());
                }

                chat.AddUserMessage(question);

                responseBuilder.Clear();
                Console.Write("Mine Copilot: ");

                await foreach (var message in chatService.GetStreamingChatMessageContentsAsync(chat,null, kernel))
                {
                    Console.Write(message);
                    responseBuilder.Append(message.Content);
                }

                Console.WriteLine();
                chat.AddAssistantMessage(responseBuilder.ToString());

                if (contextToRemove >= 0)
                {
                    chat.RemoveAt(contextToRemove);
                }

                Console.WriteLine();
            }
        }
    }
}

The text chunking code divided the documents into 104 “paragraphs” resulting in 104 embeddings being created and stored in the database. The exciting part is that with all these embeddings, when we pose our question, the database retrieves the most relevant material and adds the additional text to the prompt. Now, when we ask the same questions as before, we receive a much more helpful and accurate response:

Storing memories

Naturally, we do not want to reindex all documents every time the application starts. Imagine this was a public website facilitating chats with thousands of users and hundreds of documents reindexing all content each time – the application restart process would not only be time-consuming but also unnecessarily expensive. For instance, the Azure OpenAI embedding model I use costs €0.000121 per 1,000 tokens (Azure OpenAI Service pricing), meaning indexing just those documents costs a couple of cents (but remember: “scale makes a difference”).

Therefore, we should switch to using persistent storage. Semantic Kernel provides various IMemoryStore implementations, and we can easily switch to one that persists in the results. For example, let’s switch to one based on Sqlite. To do this, we need another NuGet package:

dotnet add package Microsoft.SemanticKernel.Connectors.Sqlite --prerelease

and with that, we can change just one line of code to switch from the VolatileMemoryStore:

.WithMemoryStore(new VolatileMemoryStore())

to the SqliteMemoryStore:

.WithMemoryStore(await SqliteMemoryStore.ConnectAsync("data\\rag-data.db"))

Sqlite is an embedded SQL database engine that operates within the same process and stores its data in standard disk files. In this case, it will connect to a rag-data.db file, creating it if it does not already exist. However, if we were to run this, we would still end up generating the embeddings again, as our previous example did not include a check to see if the data already existed. Therefore, our final step is to add a guard to prevent this redundant work.

var collectionName = "microsoft-news";
var collections = await memory.GetCollectionsAsync();

if (!collections.Contains(collectionName))
{                       
    ... // same code as before to download and process the documents
}
else
{
    Console.WriteLine($"Found '{collectionName}' in RAG database");
}

You get the idea. Here is the complete version using Sqlite:

using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using Microsoft.SemanticKernel.Connectors.OpenAI;
using Microsoft.SemanticKernel.Connectors.Sqlite;
using Microsoft.SemanticKernel.Memory;
using Microsoft.SemanticKernel.Text;
using System.Text;

namespace chatSampleApp08
{
    internal class Program
    {
        static string deploymentName = Environment.GetEnvironmentVariable("AI:OpenAI:DeploymentName")!;
        static string embeddingDeploymentName = Environment.GetEnvironmentVariable("AI:OpenAI:EmbeddingDeploymentName")!;
        static string endpoint = Environment.GetEnvironmentVariable("AI:OpenAI:Endpoint")!;
        static string apiKey = Environment.GetEnvironmentVariable("AI:OpenAI:APIKey")!;

        static async Task Main(string[] args)
        {
            //Experimental
            #pragma warning disable SKEXP0001, SKEXP0010, SKEXP0020, SKEXP0050

            //Initialize Semantic Kernel
            var builder = Kernel.CreateBuilder();
            builder.AddAzureOpenAIChatCompletion(deploymentName, endpoint, apiKey);
            var kernel = builder.Build();

            //Initialize Memory Builder
            var memoryBuilder = new MemoryBuilder()
                .WithMemoryStore(await SqliteMemoryStore.ConnectAsync("data\\rag-data.db"))
                .WithAzureOpenAITextEmbeddingGeneration(
                    deploymentName: embeddingDeploymentName,
                    endpoint: endpoint,
                    apiKey: apiKey
                );

            var memory = memoryBuilder.Build();

            var collectionName = "microsoft-news";
            var collections = await memory.GetCollectionsAsync();
            
            if (!collections.Contains(collectionName))
            {                       
                // Download a documents and add all of its contents to our chat
                var articleList = new List<string>
                {
                    "https://raw.githubusercontent.com/dotnet/docs/main/docs/csharp/whats-new/relationships-between-language-and-library.md",
                    "https://raw.githubusercontent.com/dotnet/docs/main/docs/csharp/whats-new/version-update-considerations.md",
                    "https://raw.githubusercontent.com/dotnet/docs/main/docs/csharp/whats-new/csharp-version-history.md",
                    "https://raw.githubusercontent.com/dotnet/docs/main/docs/csharp/whats-new/csharp-11.md",
                    "https://raw.githubusercontent.com/dotnet/roslyn/main/docs/compilers/CSharp/Compiler%20Breaking%20Changes%20-%20DotNet%207.md",
                    "https://raw.githubusercontent.com/dotnet/docs/main/docs/csharp/whats-new/csharp-12.md",
                    "https://raw.githubusercontent.com/dotnet/roslyn/main/docs/compilers/CSharp/Compiler%20Breaking%20Changes%20-%20DotNet%208.md",
                    "https://raw.githubusercontent.com/dotnet/docs/main/docs/csharp/whats-new/csharp-13.md",
                    "https://raw.githubusercontent.com/dotnet/roslyn/main/docs/compilers/CSharp/Compiler%20Breaking%20Changes%20-%20DotNet%209.md",
                    "https://raw.githubusercontent.com/dotnet/docs/main/docs/core/whats-new/dotnet-8/overview.md",
                    "https://raw.githubusercontent.com/dotnet/docs/main/docs/core/whats-new/dotnet-8/runtime.md",
                    "https://raw.githubusercontent.com/dotnet/docs/main/docs/core/whats-new/dotnet-8/sdk.md",
                    "https://raw.githubusercontent.com/dotnet/docs/main/docs/core/whats-new/dotnet-8/containers.md"
                };

                using (var httpClient = new HttpClient())
                {
                    var allParagraphs = new List<string>();

                    foreach (var article in articleList)
                    {
                        var content = await httpClient.GetStringAsync(article);
                        var lines = TextChunker.SplitPlainTextLines(content, 64);
                        var paragraphs = TextChunker.SplitPlainTextParagraphs(lines, 512);

                        allParagraphs.AddRange(paragraphs);
                    }

                    for (var i = 0; i < allParagraphs.Count; i++)
                    {
                        await memory.SaveInformationAsync(collectionName, allParagraphs[i], $"paragraph[{i}]");
                    }
                }
            }
            else
            {
                Console.WriteLine($"Found '{collectionName}' in RAG database");
            }

            //Create new chat
            var chatService = kernel.GetRequiredService<IChatCompletionService>();
            var chat = new ChatHistory(
                    systemMessage: "You are an AI assistant that helps people find information."
                );

            string question;
            var responseBuilder = new StringBuilder();
            var contextBuilder = new StringBuilder();

            //Question and Answer loop
            while (true)
            {
                Console.Write("Me: ");
                question = Console.ReadLine()!;

                await foreach (var result in memory.SearchAsync(collectionName, question, limit: 3))
                {
                    contextBuilder.AppendLine(result.Metadata.Text);
                }

                var contextToRemove = -1;
                if (contextBuilder.Length > 0)
                {
                    contextBuilder.Insert(0, "Here's some additional information: ");
                    contextToRemove = chat.Count;
                    chat.AddUserMessage(contextBuilder.ToString());
                }

                chat.AddUserMessage(question);

                responseBuilder.Clear();
                Console.Write("Mine Copilot: ");

                await foreach (var message in chatService.GetStreamingChatMessageContentsAsync(chat, null, kernel))
                {
                    Console.Write(message);
                    responseBuilder.Append(message.Content);
                }

                Console.WriteLine();
                chat.AddAssistantMessage(responseBuilder.ToString());

                if (contextToRemove >= 0)
                {
                    chat.RemoveAt(contextToRemove);
                }

                Console.WriteLine();
            }
        }
    }
}

Now, when we run it, the first invocation will still index everything, but after that, the data will already be indexed:

and subsequent invocations are able to simply use it.

While Sqlite is a fantastic tool, it is not specifically optimized for performing these types of searches. In fact, the code for this SqliteMemoryStore in SK merely enumerates the entire database and performs a `CosineSimilarity“ check on each entry.

// from: https://github.com/microsoft/semantic-kernel/blob/9264b3e0b42e184b7e9e8b2a073d8a721c4af92a/dotnet/src/Connectors/Connectors.Memory.Sqlite/SqliteMemoryStore.cs#L135

await foreach (var record in this.GetAllAsync(collectionName, cancellationToken).ConfigureAwait(false))
{
    if (record is not null)
    {
        double similarity = TensorPrimitives.CosineSimilarity(embedding.Span, record.Embedding.Span);
        ...
    }
}

For real scale and the ability to share data across multiple frontends, we need a dedicated ‘vector database’ designed for storing and searching embeddings. There are many such vector databases available now, including Azure AI Search, Chroma, Milvus, Pinecone,

Qdrant, Weaviate, and many more… We can easily set one of these up, change our `WithMemoryStore“ call to use the appropriate connector, and we are ready to go. Let’s proceed with that. For this example, I have chosen Azure AI Search.

I add the relevant Semantic Kernel “connector” to my project:

dotnet add package Microsoft.SemanticKernel.Connectors.AzureAISerach --prerelease 

and then add a couple of lines: 

static string azureAISearchEndpoint = Environment.GetEnvironmentVariable("AI:AzureAISearch:Endpoint")!;
static string azureAISearchApiKey = Environment.GetEnvironmentVariable("AI:AzureAISearch:APIKey")!;

change from:

.WithMemoryStore(await SqliteMemoryStore.ConnectAsync("data\\rag-data.db"))

to:

.WithMemoryStore(new AzureAISearchMemoryStore(azureAISearchEndpoint, azureAISearchApiKey)) 

And that’s it! The application works as before but much faster.  

Enhancing your Semantic integration: use the Semantic best practices and considerations

Wow! Clearly, I have left out many crucial details that any real application would need to address. For instance, how should the data being indexed be cleaned, normalized, and chunked? How should errors be managed? How can we limit the amount of data sent with each request, such as restricting chat history or the size of the found embeddings? How to make the application more secure (API Key vs Managed Identity)? Which service is the best for storing all the information?

And there are many other considerations, including making the UI much more attractive than my basic Console.WriteLine calls. Despite these missing details, I hope it is evident that you can start integrating this kind of functionality into your applications right away.

Also read: Introduction to Python Pandas Libraries

How can Azure DB be used to its highest potential?  How can you easily scale up the Azure SQL Database and reduce costs? By using the portal, you can meet all the performance requirements and save money at the same time. Read on to learn how! 

The Serverless Azure SQL Database: key features

Serverless Azure SQL Database offers a multitude of features and benefits, including automatic pause and resume capabilities, built-in high availability, and robust security measures. It is designed to simplify performance management and reduce costs, as you only pay for the computing resources you use. 

However, there are also potential drawbacks to consider. The auto-pause feature, while cost-effective, can lead to a delay in database availability, which might not be suitable for applications requiring instant access. Additionally, while the serverless model can handle fluctuating workloads, it might not be the best fit for workloads with consistent high-performance needs, as the scaling process can introduce a slight latency. 

Choosing the right purchasing model for the Azure SQL Database 

The Azure SQL Database offers a fully managed platform as a service (PaaS) database engine that aligns with your specific performance and budgetary requirements. When selecting a purchasing model for the Azure SQL Database, based on your chosen deployment model, you have the following options: 

• Virtual core (vCore)-based purchasing model: This model allows you to choose between provisioned or serverless compute tiers: 
  • In the provisioned compute tier, you have the ability to allocate a fixed amount of compute resources dedicated to your workload. 
  • The serverless compute tier offers flexibility in terms of autoscaling compute resources within a set range and includes cost-saving measures by pausing databases during inactivity (billing only for storage) and resuming them when activity picks up. The cost per vCore unit is more economical in the provisioned compute tier compared to the serverless tier. 
• Database transaction unit (DTU)-based purchasing model: This model offers combined compute and storage packages, which are optimized for typical workloads. 

Find out more about them at the official Microsoft Learn website 

Serverless vs provisioned – key differences

When you compare serverless and provisioned compute tiers in the Azure SQL Database, you can find a couple of key differences: 

Serverless Compute Tier:

• Autoscaling: Compute resources scale automatically based on workload demands.
• Automatic pausing and resuming: Databases automatically pause during inactivity, reducing costs as you are billed only for storage, and resume when activity itself resumes.
• Cost-effective: You pay only for the compute resources you use, which can lead to cost savings during periods of low or no activity, so it is ideal for workloads with intermittent and unpredictable usage patterns, such as development and testing environments.

Provisioned compute tier: 

• Fixed resources: Compute resources are pre-allocated and dedicated to your workload. 
• Consistent performance: Ideal for workloads with predictable usage patterns that require consistent performance. 

Reducing the cost of the Azure SQL Database – a real-world example

Let’s take a look together at the following real case from one of my Azure environments. 

You can observe that during business hours when users are actively using an application, the system requires much more computing power. Another peak can be observed after midnight when the system calculates aggregates (derived data) based on transactional data provided on the last day. 

First, I tried a serverless compute layer, which (at first glance) fits my application usage perfectly and costs me $568 monthly. 

Then, I started to think about how to reduce the cost of the Azure SQL Database. 

My first idea was to use the provisioned compute tier (2 vCores => $385 monthly), but it does not handle peak business hours… 

The provisioned compute tier does not support auto-scaling, but if it did, what would be the cost impact? 

Based on this model, it will cost me $517 monthly, which comes with $51 monthly savings (almost a 9% cost reduction). 

I think that the introduction of a pausing mechanism allows me to save $50-60 monthly extra. 

Another observation is that at a lower price you have more available computing power – the average 1.4 vCPU (serverless) vs 2.7 vCPU (provisioned). In other words, you can do more while paying less. To summarize, the provisioned compute tier or a DTU-based purchasing model may be cost-effective, but the service doesn’t support auto-scaling as some of us would expect… But using Microsoft Azure, we can set up a workflow that auto-scales an Azure SQL Database instance to the next immediate tier when a specific condition is met.  

For example: scale up the database as soon as it goes over 85% CPU usage for a sustained period of 5 minutes or scale down when CPU usage is  below 40%. 

By using this tutorial you will achieve that as well. 

Optimize your Azure SQL Database and take your project to the next level Schedule a consultation with Marek today to streamline your database management and maximize efficiency Book a call

Step-by-step tutorial. Automating Azure SQL Database scaling based on CPU usage

Step #1: Deploy an Azure Automation Account 

The scaling operation will be carried out through a PowerShell runbook within an Azure Automation account. Navigate to the Azure Portal, use the search bar to find Automation, and proceed to set up a new Automation Account. 

Step #2: Deploy an Azure Automation Account 

With our Automation Account now in place, we are ready to proceed with the scripting process. Create a new runbook and paste the following code into it: 

The script provided utilizes the Webhook data transmitted from the alert. This data is packed with valuable insights regarding the resource that triggers the alert, enabling the script to autonomously scale any database without requiring parameters. It simply needs to be activated by an alert that employs the Common Alert Schema on an Azure SQL database. 

param 

( 

    [Parameter (Mandatory = $false)] 

    [object] $WebhookData 

) 

 

#If webhook data is received from an Azure Alert, proceed to execute the workflow. 

if ($WebhookData) { 

    # Get the data object from WebhookData 

    $WebhookBody = (ConvertFrom-Json -InputObject $WebhookData.RequestBody) 

     

    # Get the info needed to identify the SQL database (depends on the payload schema) 

    $schemaId = $WebhookBody.schemaId 

 

    Write-Verbose "schemaId: $schemaId" -Verbose 

    if ($schemaId -eq "azureMonitorCommonAlertSchema") { 

        # Common Metric Alert schema 

        $essentials = [object] ($WebhookBody.data).essentials 

        Write-Output $essentials 

        # Get the first target only as this script doesn't support multiple targets 

        $alertTargetIdArray = (($essentials.alertTargetIds)[0]).Split("/") 

        $subscriptionId = ($alertTargetIdArray)[2] 

        $resourceGroupName = ($alertTargetIdArray)[4] 

        $resourceType = ($alertTargetIdArray)[6] + "/" + ($alertTargetIdArray)[7] 

        $serverName = ($alertTargetIdArray)[8] 

        $databaseName = ($alertTargetIdArray)[-1] 

        $status = $essentials.monitorCondition 

     

 

        # If alert that triggered the runbook is Activated or Fired, it means we want to autoscale the database. 

        # When the alert gets resolved, the runbook will be triggered again but because the status will be Resolved, no autoscaling will happen. 

        if (($status -eq "Activated") -or ($status -eq "Fired")) { 

            try { 

                "Logging in to Azure using managed identity for automation account..." 

                Connect-AzAccount -Identity 

            } 

            catch { 

                Write-Error -Message $_.Exception 

                throw $_.Exception 

            } 

 

            # Gets the current database details, from where we will capture the Edition and the current service objective. 

            # With this information, the below if/else will determine the next tier that the database should be scaled to. 

            # Example: if DTU database is S6, this script will scale it to S7. This ensures the script continues to scale up the DB in the case CPU keeps pegging at 100%. 

 

            $currentDatabaseDetails = Get-AzSqlDatabase -ResourceGroupName $resourceGroupName -DatabaseName $databaseName -ServerName $serverName 

            $edition = $currentDatabaseDetails.Edition 

            if ($edition -in @("Basic", "Standard", "Premium")) { 

                Write-Output "Database is DTU model." 

 

                $dtuTiers = (Get-AzSqlServerServiceObjective -Location $currentDatabaseDetails.Location) | Where-Object { $_.Enabled -eq $true -and $_.Edition -eq $edition -and $_.CapacityUnit -eq "DTU" } | Select-Object -ExpandProperty ServiceObjectiveName 

 

                $maxDtuTier = ($dtuTiers | Select-Object -Last 1) 

 

                if ($currentDatabaseDetails.CurrentServiceObjectiveName -eq $maxDtuTier) { 

                    Write-Output "DTU database is already at highest tier ($maxDtuTier). Suggestion is to move to Business Critical vCore model with 32+ vCores." 

                } 

                else { 

                    for ($i = 0; $i -lt $dtuTiers.length; $i++) { 

                        if ($dtuTiers[$i] -eq $currentDatabaseDetails.CurrentServiceObjectiveName) { 

                            $targetServiceObjectiveName = $dtuTiers[$i + 1] 

 

                            Write-Output "Scaling up database $databaseName to $targetServiceObjectiveName" 

                            Set-AzSqlDatabase -ResourceGroupName $resourceGroup -DatabaseName $databaseName -ServerName $serverName -RequestedServiceObjectiveName $targetServiceObjectiveName 

                            break 

                        } 

                    } 

                } 

            } 

            elseif ($edition -in @("GeneralPurpose", "BusinessCritical", "Hyperscale")) { 

                Write-Output "Database is vCore model." 

                 

                $vCoreTiers = (Get-AzSqlServerServiceObjective -Location westeurope) | Where-Object { $_.Enabled -eq $true -and $_.Edition -eq $edition -and $_.CapacityUnit -eq "VCores" -and $_.Family -eq $currentDatabaseDetails.Family -and $_.SkuName -eq $currentDatabaseDetails.SkuName } | Select-Object -ExpandProperty ServiceObjectiveName 

                $maxvCoreTier = ($vCoreTiers | Select-Object -Last 1) 

 

                if ($currentDatabaseDetails.CurrentServiceObjectiveName -eq $maxvCoreTier) { 

                    Write-Output "vCore database is already at highest tier ($maxvCoreTier)." 

                } 

                else { 

                    for ($i = 0; $i -lt $dtuTiers.length; $i++) { 

                        if ($vCoreTiers[$i] -eq $currentDatabaseDetails.CurrentServiceObjectiveName) { 

                            $targetServiceObjectiveName = $vCoreTiers[$i + 1] 

 

                            Write-Output "Scaling up database $databaseName to $targetServiceObjectiveName" 

                            Set-AzSqlDatabase -ResourceGroupName $resourceGroup -DatabaseName $databaseName -ServerName $serverName -RequestedServiceObjectiveName $targetServiceObjectiveName 

                            break 

                        } 

                    } 

                } 

            } 

            else { 

                Write-Error "The database edition '$edition' is not supported." 

            } 

 

             # All done, closing alert automatically 

             $alert = [object] ($WebhookBody.data).essentials.alertId 

             $pos = $alert.lastIndexOf("/") 

             $alertId = $alert.Substring($pos + 1, 36) 

             Write-Output "Closing alert $alertId" 

             Update-AzAlertState -AlertId $alertId -State "Closed" -Comment "Required action was executed automatically by autoscaleupsqldb-rb runbook. No further action requied, hence closing this alert." 

 

        } else { 

            Write-Error "The alert status - $status - is not in expected state." 

        } 

    } 

    else {         

        Write-Error "The alert data schema '$schemaId' is not supported." 

    } 

} 

else { 

    Write-Error -Message "Webhook data - $WebhookData - is in expected format." 

} 

 

And one more for scaling down… 

param 

( 

    [Parameter (Mandatory = $false)] 

    [object] $WebhookData 

) 

 

#If webhook data is received from an Azure Alert, proceed to execute the workflow. 

if ($WebhookData) { 

    # Get the data object from WebhookData 

    $WebhookBody = (ConvertFrom-Json -InputObject $WebhookData.RequestBody) 

     

    # Get the info needed to identify the SQL database (depends on the payload schema) 

    $schemaId = $WebhookBody.schemaId 

 

    Write-Verbose "schemaId: $schemaId" -Verbose 

    if ($schemaId -eq "azureMonitorCommonAlertSchema") { 

        # Common Metric Alert schema 

        $essentials = [object] ($WebhookBody.data).essentials 

        Write-Output $essentials 

        # Get the first target only as this script doesn't support multiple targets 

        $alertTargetIdArray = (($essentials.alertTargetIds)[0]).Split("/") 

        $resourceGroupName = ($alertTargetIdArray)[4] 

        $serverName = ($alertTargetIdArray)[8] 

        $databaseName = ($alertTargetIdArray)[-1] 

        $status = $essentials.monitorCondition 

     

 

        # If the alert that triggered the runbook is Activated or Fired, it means we want to autoscale the database. 

        # When the alert gets resolved, the runbook will be triggered again but because the status will be Resolved, no autoscaling will happen. 

        if (($status -eq "Activated") -or ($status -eq "Fired")) { 

            try { 

                "Logging in to Azure using managed identity for automation account..." 

                Connect-AzAccount -Identity 

            } 

            catch { 

                Write-Error -Message $_.Exception 

                throw $_.Exception 

            } 

 

            # Gets the current database details, from where we will capture the Edition and the current service objective. 

            # With this information, the below if/else will determine the next tier that the database should be scaled to. 

            # Example: if DTU database is S6, this script will scale it to S7. This ensures the script continues to scale up the DB in case CPU keeps pegging at 100%. 

 

            $currentDatabaseDetails = Get-AzSqlDatabase -ResourceGroupName $resourceGroupName -DatabaseName $databaseName -ServerName $serverName 

            $edition = $currentDatabaseDetails.Edition 

            if ($edition -in @("Basic", "Standard", "Premium")) { 

                Write-Output "Database is DTU model." 

 

                $dtuTiers = (Get-AzSqlServerServiceObjective -Location $currentDatabaseDetails.Location) | Where-Object { $_.Enabled -eq $true -and $_.Edition -eq $edition -and $_.CapacityUnit -eq "DTU" } | Select-Object -ExpandProperty ServiceObjectiveName 

 

                $minDtuTier = ($dtuTiers | Select-Object -First 1) 

 

                if ($currentDatabaseDetails.CurrentServiceObjectiveName -eq $minDtuTier) { 

                    Write-Output "DTU database is already at lower tier ($minDtuTier)." 

                } 

                else { 

                    for ($i = ($dtuTiers.length - 1); $i -gt 0; $i--) { 

                        if ($dtuTiers[$i] -eq $currentDatabaseDetails.CurrentServiceObjectiveName) { 

                            $targetServiceObjectiveName = $dtuTiers[$i -1] 

 

                            Write-Output "Scaling down database $databaseName to $targetServiceObjectiveName" 

                            Set-AzSqlDatabase -ResourceGroupName $resourceGroup -DatabaseName $databaseName -ServerName $serverName -RequestedServiceObjectiveName $targetServiceObjectiveName 

                            break 

                        } 

                    } 

                } 

            } 

            elseif ($edition -in @("GeneralPurpose", "BusinessCritical", "Hyperscale")) { 

                Write-Output "Database is vCore model." 

                 

                $vCoreTiers = (Get-AzSqlServerServiceObjective -Location westeurope) | Where-Object { $_.Enabled -eq $true -and $_.Edition -eq $edition -and $_.CapacityUnit -eq "VCores" -and $_.Family -eq $currentDatabaseDetails.Family -and $_.SkuName -eq $currentDatabaseDetails.SkuName } | Select-Object -ExpandProperty ServiceObjectiveName 

                $minvCoreTier = ($vCoreTiers | Select-Object -First 1) 

 

                if ($currentDatabaseDetails.CurrentServiceObjectiveName -eq $minvCoreTier) { 

                    Write-Output "vCore database is already at lower tier ($minvCoreTier)." 

                } 

                else { 

                    for ($i = ($dtuTiers.length - 1); $i -gt 0; $i--) { 

                        if ($vCoreTiers[$i] -eq $currentDatabaseDetails.CurrentServiceObjectiveName) { 

                            $targetServiceObjectiveName = $vCoreTiers[$i - 1] 

 

                            Write-Output "Scaling down database $databaseName to $targetServiceObjectiveName" 

                            Set-AzSqlDatabase -ResourceGroupName $resourceGroup -DatabaseName $databaseName -ServerName $serverName -RequestedServiceObjectiveName $targetServiceObjectiveName 

                            break 

                        } 

                    } 

                } 

            } 

            else { 

                Write-Error "The database edition '$edition' is not supported." 

            } 

 

             # All done, closing alert automatically 

             $alert = [object] ($WebhookBody.data).essentials.alertId 

             $pos = $alert.lastIndexOf("/") 

             $alertId = $alert.Substring($pos + 1, 36) 

             Write-Output "Closing alert $alertId" 

             Update-AzAlertState -AlertId $alertId -State "Closed" -Comment "Required action was executed automatically by autoscaleupsqldb-rb runbook. No further action requied, hence closing this alert." 

 

        } else { 

            Write-Error "The alert status - $status - is not in expected state." 

        } 

    } 

    else {         

        Write-Error "The alert data schema '$schemaId' is not supported." 

    } 

} 

else { 

    Write-Error -Message "Webhook data - $WebhookData - is in expected format." 

} 

 

Step #3: Trigger Automation runbook(s) via the Azure Monitor Alert 

Create a new alert rule on your Azure SQL Database: 

The next step will require several different setups: 

1. Scope of the alert: this will be auto-populated if the New Alert Rule is clicked from within the database itself. 
2. Condition: when should the alert get triggered by selecting a signal and defining its logic?
3. Actions: when the alert gets triggered, what will happen? 
4. Details: location, name, and other configuration values.

Condition 

For this example, the alert will monitor the CPU consumption every 1 minute from the last 5 minutes. When the average goes over 85%, the alert will be triggered: 

Actions 

After the signal logic is created, we need to indicate what the alert is to do when it gets fired. We will do this with an action group. When creating a new action group, two tabs will help you configure triggering the runbook. 

Basics: 

Actions:

After saving the action group, add the remaining details to the alert.

Create another alert and action for scaling down with the condition: CPU percentage is less than 40%. 

That’s it! The alerts are now enabled and will auto-scale the database when fired. The runbook will be executed twice per alert: once when fired and another when resolved, but it will only perform a scale operation when fired.

Summary 

The serverless Azure SQL Database offers a potentially cost-effective solution for database management, balancing performance and scalability with features like automatic pause and resume, high availability, and robust security. Key benefits include cost savings of $50-$60 per month from automatic pausing and flexibility in resource usage. However, it may not be ideal for applications needing instant access or consistent high performance due to potential delays and slight latencies during scaling.

Interesting statistics behind “Why migrate from VMware to Azure”  

• Nearly 50 percent of IT professionals consider cost the most significant factor in looking for an alternative solution, according to a survey by Gartner.  
• In November 2023, VMmare was acquired by Broadcom. The change in ownership came with new licensing models. Under some circumstances, a subscription model that replaces lifetime licenses will be beneficial, but may also involve an increase in costs in some cases.  

VMware on-premises – when is it better to stay?

The new subscription model may prove to be a better choice in some situations for certain organizations. For example, companies that don’t have technical support will get it bundled with the subscription. This model also represents a more flexible way to adjust the license cost to the real-world usage of resources.  

So why do companies migrate VMS to Azure?

Despite the potential benefits, many organizations decide to migrate VMware to the Azure cloud “as is” to gain greater cost efficiency. This way, they do not need to incur the cost of on-premises infrastructure and gain more flexibility.  

Azure VMware Solution (AVS) allows them to combine the potential of VMware capabilities with the potential of the cloud: business continuity and perfect scaling. Furthermore, no refactoring is needed, licenses are included, and consistency with on-premises workloads is assured. 

Migrate VMware – how to prepare for the migration process in 4 simple steps

1. Plan – before migrating from on-premises VMware to Azure, you need to carefully think over and plan the transition. You can handle it in-house or, in case any issues arise, take advantage of the opportunity for an advisory call with cloud experts. 

2. Collaborate – you have a transition process ahead of you. From now on, cooperation and good communication with all stakeholders involved will work in your favor. The use of project management tools for this purpose is standard today, especially in Agile projects. 

3. Assess – evaluate the existing environment to identify dependencies, applications, and resources that require migration. A proper automation tool like Azure Migrate will come in handy here. There are many such tools available on the market. In our experience, Microsoft solutions provide everything you need out of the box. 

4. Set up – connect to Azure and ensure all necessary permissions and access rights are in place. You need to set up accounts and credentials in the Azure portal to walk through migration smoothly.  

Assessing your current VMware environment

When starting your migration process to Azure, it is essential to evaluate your current on-premises workloads to determine their cloud readiness, identify any risks that may arise, and estimate the associated costs and level of complexity. Below is how you do it. 

Essential steps before you start: 

• Set up a free Azure account if you don’t have one yet.  
• Ensure you have discovered servers to assess.  

1) Choose the assessment method 

Assess based either on server configuration data/metadata or on the collection of dynamic performance data. 

2) Assess and Migrate using the Azure Migration Tool 

This can be done in the Discovery and Assessment tab, by selecting Assess and then going to Azure VM. You can use the Azure tutorial for a detailed overview of all the configurations needed at this step.  

3) Review your assessment  

As a result, you will obtain recommendations regarding Azure readiness. These include: 

• Ready for Azure,  
• Ready with conditions, 
• Not ready for Azure, 
• Readiness unknown (in case of data availability problems). 

You will also get the estimated monthly costs and associated storage expenses. Remember, you can make use of Azure Cost Management tools when migrating workloads to Azure. 

Evaluating Azure subscription options

Select the Azure subscription option that best fits your organization’s needs by evaluating cost, scalability, and the features you need. 

Before deploying the Azure subscription, consider familiarizing yourself with the saving options Azure offers. The great thing is that the Azure assessment you performed, in addition to providing relevant information on your cloud readiness, storage requirements, and recommended cloud region, will also suggest the best cost optimization options (that is, Azure Reservations, Azure Saving Plans, or Azure Hybrid Benefits for saving on computing costs).  

Setting up your Azure environment

Setting up your Azure environment involves selecting the right tools and resources. This will allow you to streamline the process of moving your data and applications to the Azure environment, saving time and reducing potential errors. Once done, the next step is to set up an Azure environment for your organization. We recommend using the Azure Migrate native tool.  
 
The steps include: 

1) Deployment of the Azure Migration appliance 

You will need to create an appliance server and follow the configuration steps detailed in the Azure tutorial. 

2) Start continuous discovery 

To access the servers’ configuration and performance data, your appliance must connect with the vCenter Server. Continuous discovery allows you to catalog software, analyze dependencies, and locate SQL Server instances and databases.  

Creating Azure VM instances

Creating Azure VM instances can be done by replicating Azure VMs to a secondary location or transferring virtual machines from VMware to Azure. The process involves specifying the Virtual Machine name and selecting the appropriate settings for the migration.  

Azure Migrate project – executing the migration

Performing an Azure Migrate server assessment is necessary to understand the current environment and define the migration settings. Utilizing Azure Migration and Modernization tools can simplify and streamline the process. Choose the right migration method (agentless or agent-based as described in the Microsoft resources), considering, e.g.:  

• Permissions needed to run the migration, 
• Number of VM migrations to be performed at once, 
• Disk limits – selection of storage account/number of disks in Azure, 
• Passthrough disks to be used as a storage source, 
• Deployment steps that will depend on the method. 

Run a test migration to ensure data integrity

Data integrity refers to the accuracy, completeness, and consistency of data. According to a Gartner study, due to data quality problems, 80% of data migrations won’t meet their business objectives, and the cost of bad data annually goes into the millions of dollars. That’s why migration tests are conducted when migrating databases. VMWare’s move to the cloud is no different.  

Run a test migration to ensure data integrity before conducting a full migration process. This involves moving a subset of data to a test environment to check for any errors or missing information. Then, a clean-up test migration should be performed to remove any redundant data. Otherwise, you may incur extra expenses in Azure. The data integrity check is carried out in two steps: 

1) Validating replication 

Checking if every sector changed in the source disk was replicated to the target one.  

2) Ensuring data consistency  

Ensuring that the data sent to the Azure disks matches the data copied from the source disks.  

Also read: Azure cost optimization 

VMware migration to Azure – post-migration optimization

Your journey doesn’t end with the cloud migration itself. Post-migration optimization is crucial. Ensuring proper Backup and Recovery solutions and using Managed Services will be the steps you take to ensure business continuity.  

Implementing Azure Backup and Disaster Recovery solutions 

Choose a backup solution for the VMware Virtual Machines, such as Microsoft Azure Backup Server (MABS).  For optimal results, consider using Azure Managed Services. Such services are great whenever you need to enhance efficiency.  

Summary – your migration to the Azure VMWare solution

In a recent webinar, an Inetum expert emphasized why is it worth considering moving on-premises VMware VSphere to Azure. Built-in tools offered by Microsoft allow for accurate estimation of the costs involved in the migration process. Making use of Azure cost calculations provided by the platform is also important. Additional features such as Azure security solutions and the general scalability of Azure Cloud give many companies great options to keep their existing virtualization features in a modern environment. 

Migrate on-premises VMware VMS to Azure with us!

Do you find transferring your VMware to Azure using an Azure migration tool a challenge? Contact us to make your transition seamless!

Contact us!

Understanding Azure Cost Management

Cost optimization on the Azure platform involves using tools and strategies that allow you to minimize the costs associated with using the cloud. 

The importance of cost optimization in Azure is a key element in the effective and efficient use of cloud computing. Cost optimization in Azure is not just about minimizing expenses; it has a significant impact on overall operational efficiency. 

One of the most important aspects of cost optimization in Azure is monitoring resource utilization. This allows you to identify unnecessary or unused resources and turn them off, saving costs. In addition, cost optimization on the Azure platform also includes the use of process automation tools, which allows you to reduce the costs associated with infrastructure management. 

Read also:  Azure Cost Management 101. How to optimize cloud costs?

Zero-Cost Azure Cost Optimization Best Practice 

Cost optimization is becoming an integral part of the cloud strategy, enabling organizations to achieve the highest performance with minimal financial outlays. To achieve this, it is crucial to remember about practice that do not generate additional costs. 

Use of native Azure tools 

Azure provides a comprehensive suite of tools specifically crafted to streamline the monitoring and optimization of costs within the cloud environment. One notable tool is Azure Cost Management and Billing, a centralized platform that empowers users to meticulously track their expenditures. Through this tool, organizations can gain insights into resource consumption patterns, identify cost drivers, and make informed decisions on where cost optimizations can be implemented. 

Furthermore, Azure Advisor plays a crucial role by offering personalized recommendations for resource management. This intelligent advisory service leverages Machine Learning algorithms to analyse usage patterns and suggests optimizations tailored to the specific needs of the organization. These recommendations span diverse aspects, ranging from right-sizing virtual machines to identifying underutilized resources, providing actionable insights for efficient cost management. 

Virtual machine management 

The dynamic scaling of resources based on real-time demand is a cornerstone of efficient cost optimization. Leveraging the auto-scaling feature within Azure allows organizations to seamlessly adjust the number of virtual machines in response to fluctuating workloads. During periods of increased demand, the system can automatically provision additional resources, ensuring optimal performance without incurring unnecessary costs during periods of reduced activity. 

The auto-scaling mechanism can be finely tuned to respond to predefined metrics such as CPU utilization or network traffic, providing a flexible and responsive infrastructure. This not only enhances the overall efficiency of resource utilization but also aligns with the principles of pay-as-you-go, where costs are incurred only when resources are actively needed. 

Data warehouse optimization 

Strategic management of data warehousing significantly influences overall costs in the cloud. Adopting practices such as data partitioning and compression becomes instrumental in curbing expenses related to storage. Data partitioning involves dividing large datasets into smaller, more manageable segments, allowing for more efficient storage and retrieval. Compression, on the other hand, reduces the physical storage space required for data, leading to direct cost savings in terms of cloud storage. 

Azure provides tools and features that enable organizations to implement these optimization strategies seamlessly. By incorporating these practices, businesses not only economize on storage costs but also enhance the performance of data-intensive workloads, contributing to an overall more cost-effective cloud deployment. 

Network traffic management 

Effective control of data transfer costs is paramount in the pursuit of cloud cost optimization. Conscious management of network traffic emerges as a crucial strategy in achieving this objective. Organizations can strategically minimize data transfer between Azure regions by strategically placing resources or utilizing region-specific services. Additionally, the adoption of Content Delivery Network (CDN) services aids in optimizing the delivery of content by strategically caching data at edge locations, reducing latency, and mitigating unnecessary data transfer costs. 

By implementing robust network traffic management practices, organizations can mitigate the impact of data transfer costs on their overall cloud expenditure. This involves a strategic balance between optimizing the performance of applications and minimizing unnecessary data movement, ultimately contributing to a more efficient and cost-effective cloud infrastructure.

Take Advantage of Azure Cloud Optimization trends 

Trends in Azure cost optimization are increasingly focused on both strategic resource management and the utilization of specific Azure services. Organizations can significantly reduce their Azure spend without compromising performance by: 

• Strategic Resource Management: This trend involves the careful allocation and management of resources to ensure they are used efficiently. By strategically managing resources, organizations can avoid unnecessary Azure costs. 
• Utilization of Specific Azure Services: Services like Azure Hybrid Benefit and Azure Reservations are being increasingly used for cost optimization. Azure Hybrid Benefit allows organizations to use their on-premises Windows Server and SQL Server licenses in Azure, leading to significant cost savings. Azure Reservations, on the other hand, provide discounted prices for one- or three-year commitments on various Azure services. 
• Regular Cost Analysis: Regularly analysing Azure costs can help identify areas where spending can be reduced without compromising performance. Tools like Azure Cost Management are being used to monitor, allocate, and optimize cloud costs. These tools provide insights into where organizations are accruing costs and offer suggestions for reducing unnecessary spending. 
• Use of Azure Spot VMs and Reserved Instances: Azure Spot VMs offer unused Azure capacity at a significant discount, while reserved instances allow organizations to commit to Azure resources over a one- or three-year period in exchange for a discount. Both of these strategies help optimize the Azure budget. 
• Leveraging Azure Cost Optimization Strategies: Using server licenses with Azure and Azure Kubernetes Service can help organizations find the most cost-effective approach for their specific workloads. 
• Trend of Containerization: The use of container technologies like Docker and Kubernetes allows for efficient management and deployment of applications in the cloud. This not only improves performance but also contributes to cost reduction. 
• Security as a Key Element of Cost Optimization: Preventing security breaches is more cost-effective than addressing them after they occur. Azure offers a range of tools and services to protect data and applications in the cloud, which can significantly aid in cost optimization efforts. 

Future Perspective: The Development of Azure policy 

The future of cost optimization in the cloud is promising and is expected to develop significantly in the coming. This is because as many as 94% of organizations are already in the cloud, but 69% of them don’t have a defined cloud strategy. It includes the development of advanced tools and strategies to help companies manage and reduce cloud expenses. One such tool is Azure Policy, which helps organizations enforce and maintain compliance with corporate standards and service level agreements.  

These policies can play a key role in controlling and minimizing cloud costs by setting guidelines and automatically implementing cost-saving measures. Additionally, the development of more sophisticated automation tools will further improve the process of resource allocation and cost control. Technologies such as Machine Learning and Artificial Intelligence solutions can provide companies with insights and recommendations for optimizing cloud spend. 

CLOUD ENGINEERING Make the best use of the cloud! Enter the digital world taking advantage of our cloud competences! Get started now!

Tips and Best Practices to Optimize your Azure – Summary  

Utilizing cost optimization techniques, such as resource management, use of monitoring and analysis tools, as well as understanding and optimizing pricing models, can significantly enhance cloud performance while simultaneously reducing costs. 

Inetum, as an experienced specialist in the implementation and cost optimization of cloud solutions, can assist in fully exploiting the potential of Microsoft Azure. Thanks to our experience and knowledge, we are able to provide tailored solutions that will help achieve optimal cloud performance at minimal costs. 

Remember, cost optimization is a continuous process that requires regular evaluation and adjustment. Therefore, using the services of experts such as Inetum can bring significant benefits to your organization. 

“The success of implementing GenAI solutions is influenced by several factors, including the competencies of the implementation team, the tools and licensing forms used, appropriate business processes, and cost-effectiveness. The costs of such an investment can be optimized, for instance, by using properly selected algorithms that require less computing power and, consequently, generate fewer costs.

Conducting an AI Maturity Assessment is beneficial to achieve optimal conditions in these areas. As part of this assessment, an experienced team will evaluate the organization’s readiness to implement GenAI solutions and provide recommendations for future work.” – adds Marek Czachorowski, Practice Leader of Cloud Engineering at Inetum.

So how to save money on the cloud and scale it all right? One way is to take advantage of solutions from public cloud providers. In this guide, we examine solutions of the leader in public cloud services, that is Azure Saving Plans offer. Find out what they are, how they differ from Azure Reserved Instances, and how to purchase them and use them most effectively.

How to Buy and Manage Azure Savings Plans?

According to respondents to Flexera’s “State of the Cloud 2023” report, cost management is now even more important than cyber security. Over 80% of respondents surveyed say that managing the cost of the cloud is their number 1 challenge. In times of economic concerns, companies are seeking support from FinOps practices or building the Cloud Centers of Excellence. 

It’s worth knowing that Azure savings plans for computing capacities are also available. Let’s look at what public cloud leader Microsoft offers in their Azure Portal. Below we explain how to gain access to the savings plans.

What Azure Savings plan for computing is?

The idea behind the savings plan for computing is simple: you commit to spending a fixed hourly amount on compute services for one or three years and in doing so, you will be able to save up to 65% on pay-as-you-go prices.

How do you buy a savings plan?

1. First, ensure you meet the conditions for purchasing a savings plan – check your billing plan.
2. Note that savings plan discounts are only applicable to resources connected to subscriptions acquired through an Enterprise Agreement (EA), Microsoft Customer Agreement (MCA), or Microsoft Partner Agreement (MPA).
3. Once done, proceed to Azure Portal, and navigate to All services, and then choose Savings plans. From there, click Add to buy a saving plan.

Azure Savings Plans Benefits Explained

Understanding Savings Plan Benefits for Compute Usage

• Working on your terms – you decide whether to commit for one year or three years, and determine the hourly plan based on recommendations for past compute usage.  
• Optimizing consumption of selected services  – as part of savings plans, lower prices are available for selected services, including Azure Virtual Machines, Azure App Service, container instances, Azure Functions, and more.
• Saving even more – the longer the commitment time, the greater the benefits and savings. You can combine savings offers, such as with Azure Hybrid Benefit explained below. 
•  Gaining flexibility – even though you cannot modify or cancel the commitment, if your usage exceeds estimated figures, you may add another savings plan to cover the extra usage for computing resources.
  Key differences between Azure Reservation vs. Savings Plan A savings plan is not the only way to save expenditures. Azure gives you various options – another s is the Azure Reservation service. How do they differ?
• Azure Reservation – it means a commitment to a specific virtual machine type in a given cloud region. Choose reserved instances for stable workloads or if you do not expect any changes eg. to the cloud region. Azure reserved instances provide cost savings compared to pay-as-you-go pricing and ensure the availability of resources for your specific workload. It’s important to carefully consider your workload requirements and usage patterns before reserving instances.
• Azure Savings Plan – it is a commitment to spending a fixed hourly amount in total on compute services. This option is recommended for dynamic workloads. Azure savings plan offers a pricing model that provides significant cost savings on consistent compute usage in Azure regions. By purchasing an Azure Savings plan, you can achieve savings across multiple Azure regions and select compute services covered by the plan.

Getting the most out of Azure Saving Plans

It’s a good idea to carry out thorough research on the topic before committing to a specific plan. Below are two popular options.

Utilizing Azure Advisor to Leverage Savings Plan Discounts

Azure Advisor is available for free in the Azure Portal. Using Azure Advisor can be compared to having your cloud assistant, which with Machine Learning capabilities at its service, helps you recognize underutilized resources and ensure optimal usage of virtual machines and their scale sets. Then, the Advisor recommends shutting down or resizing given resources.

But Azure Advisor is also helpful in estimating recommendations on appropriate plans for your company.  Based on hourly and total on-demand usage costs, calculations are created to help you choose the right option.

Get inspired to save money with minimal effort and read about this powerful tool in our guide: Azure Advisor – the Microsoft Cost Management Tool.

Utilizing Azure Hybrid Benefit with Savings Plans

Azure Hybrid Benefit is a special licensing offer for Azure cloud migrations.
In simple terms, Azure Hybrid Benefit is a program that allows new Azure customers to use their existing on-premises licenses for certain Microsoft software products. This way, they can reduce the cost of running them in the Azure cloud.

For instance, if someone already has licenses for certain Microsoft software (Windows Server or SQL Server) and wants to move their workloads to the cloud, Azure Hybrid Benefit lets them apply existing licenses and run the same product within Azure.

By utilizing Azure Hybrid Benefit with new Savings Plans, you can potentially save more money on Azure compute services.

Purchasing Azure Savings plan – summary 

A savings plan allows you to lower cloud costs and optimize selected Azure resources at the same time. These plans cannot be canceled or changed during the contract, so the choice must be well-thought-out depending on the workloads, services, and cloud regions used. Also, you need to consider any planned changes in this context in the future. While there are a lot of knowledge sources on the Microsoft Learn portal, it’s good to know the opinions of cloud migrations and cost optimization experts. You don’t have to be alone in making this decision!

Read also: Azure Cost Management 101. How to optimize cloud costs?

Contact our subject matter experts if you want to quickly choose the right plan and start saving money right away. 

CLOUD ENGINEERING

Make the best use of the cloud!

Enter the digital world taking advantage of our cloud competences! Get started now!