Why is CSDE important in the banking industry? Use case, benefits, and challenges 

In an increasingly interconnected world, the landscape of banking and financial services is evolving at a rapid pace. In response, established design patterns such as Command Sourcing and Deterministic Execution have been subject to significant attention. In the article, I will discuss some of the most important aspects related to CSDE and the ways it can improve customer experience through the implementation of digital banking products.

Challenges of digital banking   

Traditional brick-and-mortar banking has undergone a digital transformation, paving the way for digital banks that operate online. This digital evolution has also brought forth the need for more innovative approaches to enhance system performance, streamline operations, and facilitate automation.  

Command Sourcing and Deterministic Execution are not silver bullets that can solve all the challenges faced by digital banks and financial institutions. The 3 major challenges are as follows:   

• Security concerns – Banking systems are designed with maximum due diligence but there are no systems in the world that are 100 percent resistant to cyberattacks. 
• Technical resilience – Thousands of engineers around the world work every day to make banking systems failsafe. The downtime of a banking application is not only an impediment for users but may also bring about potential security and reputation problems for the banking institution. The cost of application downtime can run into the millions. Hidden costs may lie in the loss of trust of discouraged users, who may decide to switch service providers.  
• Users are changing their banking habits – Online banking, cashless transactions, and mobile banking solutions are trending. 80% of people prefer online banking instead of visiting banks, according to research. Banks need to adjust their products to constantly changing customer expectations. 

Nonetheless, Command Sourcing and Deterministic Execution are pivotal in the development of high-performance systems where non-deterministic behavior is highly undesirable. These techniques, when combined with the power of new technologies, can lead to the generation of valuable data and enable real-time processing. These are essential aspects of modern banking services addressing the most sophisticated user experience requirements.  

Delving deeper into Command Sourcing in digital banking 

At its core, Command Sourcing is related to Event Sourcing. It embodies an alternate method of persisting and replicating the state of a digital banking application by storing its history as a sequence of commands or requests. This approach means stepping away from traditional CRUD-based (Create, Read, Update, Delete) methods that many software systems utilize. 

Within a digital banking context, Command Sourcing allows for the establishment of a more manageable and debuggable framework for data alterations. It necessitates data storage only once it enters the system. Subsequently, the system state exists in the memory, bolstering the efficiency and speed of operations. This shift to a deterministic sequence of operations is beneficial not only to day-to-day operations. It also simplifies audits – a crucial aspect of compliance in financial services. The inherent traceability of actions performed within the system serves as a comprehensive audit log, thereby offering increased transparency and reliability. 

Read also: We develop the flagship FinTech software of our British client

Exploring the implications of Deterministic Execution 

In the realm of software programming, deterministic execution is of critical importance. It ensures that when a specific set of inputs is fed into a program, the results generated will be the same, regardless of when or how many times the program is executed. This level of predictability forms the crux of reliable and reproducible computations in the world of digital banking. 

Unfortunately, certain practices and external factors can introduce non-deterministic behavior into software programs. This means that despite having identical inputs, the outputs from these programs may vary. Aspects like those listed below can contribute to unpredictability:  

• System date/time,  
• random number generation, 
• I/O operations,  
• threading,  
• hashed key-based, hash table/hash map iteration, 
• mutable shared state,  
• and code changes.

These elements make testing and debugging significantly more complex and cumbersome. Why is it so?  

1. System date/time: Any reliance on the system date/time can introduce non-determinism into your software. This is because the system date/time is continually changing, which can cause the same function to return different results depending on when it is executed. This makes it difficult to test your software in a consistent and repeatable manner, and it can also make bugs harder to reproduce and debug. 
2. Random number generation: Functions or operations that use random numbers are inherently non-deterministic. That’s because each call to a random number generator, by definition, produces a different output. This can lead to different execution paths in your software and can make it hard to reproduce bugs. In testing, a common solution is to use a fixed seed for the random number generator so that it produces the same sequence of “random” numbers each time. 
3. I/O operations: Disk read/write operations, network calls, and other input/output (I/O) operations can cause non-deterministic behavior because the state of the disk or network can change between executions. For example, a network request might succeed at one point and fail immediately after because of a network hiccup. This variability can make testing and debugging tricky because failures can be intermittent and hard to reproduce. 
4. Threading: Multithreaded programs can be especially hard to debug because the order in which threads are scheduled and executed can vary between runs. This can lead to race conditions where the output of the program depends on the precise timing of threads, which is hard to control or predict. Moreover, many common debugging tools are not well-equipped to handle multithreaded programs, further complicating the debugging process. 
5. Hashed key-based, hash table/hash map iteration: The order in which entries are retrieved from hash-based collections like hash tables or hash maps may be non-deterministic, depending on the hashing algorithm and the current state of the collection. This can make your software’s output vary between runs, even with the same input, which complicates testing and debugging. 
6. Mutable shared state: If different parts of your program are sharing and modifying the same data, this can lead to unpredictable behavior. For example, one thread might be reading data while another thread is modifying it, leading to a race condition. Debugging such issues can be challenging because the problem often disappears when you try to observe it. 
7. Code changes: Any modifications to the code that alter the functionality of a function could lead to different outputs for the same input, thus introducing non-determinism. While changes are a natural part of the software development process, they can introduce bugs and make existing ones harder to track down. 

However, strategies can be adopted to promote deterministic execution.  

1. Run a single-threaded process: This eliminates the complexities of thread scheduling and concurrency. For this to be possible, async/await tasks or similar constructs should not be used.  
2. Avoid usage of DateTime fields and randomly generated numbers: When the flow of control in your business logic is being determined, DateTime fields or any randomly generated numbers should not be used as they can lead to different outcomes on different executions. 
3. Avoid disk read/write operations: Any form of data I/O can introduce non-determinism due to factors beyond your control like disk availability or network issues.  
    Avoid external service calls: Calling external services to get data may lead to non-determinism due to potential changes in the external service or data over time. Never ask – instead rely on the fact that you are being told of the data change. For example, when receiving an order, don’t ask for the price of the underlying financial instrument, but instead, just apply the price you have and make sure that you are told when the underlying’s price has changed. 
4. Use deterministic collections: Data storage should be carried out using collections that provide deterministic behavior, i.e., given the same operations in the same order, they should always produce the same outcome.  
5. Avoid sharing mutable data: Sharing mutable data can introduce unpredictability, so it should be avoided. Any necessary data sharing should be performed using immutable data structures.  
6. Version commands: Command versioning ensures that any changes to your business logic are traceable and consistent. Each version of a command should always produce the same output for the same inputs, enabling deterministic behavior.   
7. Reflect business logic changes based on command versions: This way, even when business logic changes, the functions will always produce the same output for the same version of the command, ensuring consistency and predictability. 

When to use Command Sourcing and Deterministic Execution? 

The synergy between command sourcing and deterministic execution can be leveraged to great effect in a variety of scenarios within the financial services sector. In the context of event sourcing within a digital bank, for instance, the application’s state reconstruction process can benefit immensely from deterministic execution. This would entail applying events in a predictable order and ensuring that concurrent events do not cause inconsistencies or anomalies. 

Command Sourcing and Deterministic Execution in practice: An illustrative use case 

Consider a digital bank that processes hundreds of thousands of transactions daily. Using Command Sourcing, each transaction (command) is logged as it enters the system. This provides the capability to reconstruct the system’s state at any given point merely by replaying the stored commands. 

To complement this, deterministic execution is implemented, ensuring that regardless of when a transaction is processed, the system will always generate the same result for a given set of inputs. This reliable predictability greatly enhances the performance, reliability, and overall efficiency of the banking system. 

Complex systems within digital banks can find it challenging to achieve complete determinism due to factors like external dependencies or time-based events. In such cases, the concept of eventual consistency can be adopted, with techniques like versioning, conflict resolution, or consensus algorithms employed to reconcile differences in distributed systems. 

The benefits of Command Sourcing and Deterministic Execution in the financial services industry 

Implementing Command Sourcing and Deterministic Execution within digital banking systems provides several key benefits: 

• Predictability: The principle of deterministic execution guarantees that given the same inputs, the system will always produce identical outputs. This predictability simplifies system behavior analysis and testing. 
• Reproducibility: The practice of Command Sourcing allows you to reproduce any past system state by replaying commands, a valuable tool for debugging and issue resolution. 
• Auditability: With all commands logged, a comprehensive audit trail is available for review, offering a level of transparency which is highly valued in banking and financial services. 
• Resiliency: In the event of a system crash or failure, the state of the system can be restored by replaying the stored commands, ensuring quick recovery and minimal downtime. 

Considerations for implementation 

Despite their considerable benefits, implementing these techniques is not without its challenges: 

• Overhead: Storing every command can add significant overhead to the system, potentially impacting performance and leading to increased storage costs. 
• Privacy and security: Stored command data may contain sensitive information, raising concerns about data protection and privacy. 
• Complexity: The shift from traditional CRUD patterns to CSDE can add considerable complexity to the system and may necessitate additional training for the team. 
• Data Volume: In high-throughput systems, the volume of commands can grow exponentially, potentially leading to issues with storage space and command replay speed. 
• Versioning: Handling changes to commands over time can pose a significant challenge, often requiring command versioning. 

Expert knowledge Hire dedicated team of developers! Entrust your company FinTech project to dedicated development team. Get started now!

New technologies in financial data processing and user experience – conclusion 

In an era where financial services are undergoing significant digital transformation, Command Sourcing and Deterministic Execution represent potent tools that can considerably enhance the robustness, maintainability, and reliability of banking systems.

By capitalizing on new technologies, digital banks can leverage these techniques to manage valuable data in real time and optimize their architectural framework. However, as with all technological tools, meticulous planning and careful implementation are essential to overcoming potential challenges and fully harnessing their benefits to further enhance the digital banking solutions user experience.