New Capabilities
3forge's AMI script now includes additional support for concurrency and pausability without the complexity of traditional thread-based development.
Why It Matters
The performance advantages of multi-threading are critical for applications connecting to remote data sources and processing large data sets. New concurrency and pausability support in AMI Script aim to optimize code that does not return immediately, including remote system calls as well as code that processes large data sets. Concurrency and Pausability allow 3forge to implement behaviors that perform like multi-threading, but lack the complexity and support burden.
Threading Explained
In Computer Science, a thread represents the processing of a single instruction stream by a processor (CPU). A single-threaded application is a contained set of instructions executed in a single stream. While some applications process and quickly terminate, other applications run indefinitely like a user interface (UI).
The term "Multi-Threading" comes into play when the computer is responsible for executing multiple instruction streams concurrently within an application. Threading is analogous to having one person writing a document versus multiple people. With one person, there is no need to coordinate updates or merge changes. With multiple people, the document may come together quicker, but the composition process requires coordination to prevent changes from being overwritten.
Example: Single Threaded Trade Reporting Application
The Trade Reporter is a single-threaded process that builds a "new trades" report every 30 seconds, by:
- Querying for trade data (blocking call)
- Querying for related counterparty reference data (blocking call)
- Rendering a report file (e.g. CSV) based on both data sets
- Publishing the report via email or SFTP (blocking call)
The reporting process happens sequentially as part of a single instruction stream. The Trade Reporter makes a blocking call to a Reference Data Manager that is hosted on a separate server. The Reference Data Manager's threading model is not relevant. However, it is important to understand that it listens for reference data requests and then builds and transmits a response, which takes precious time that slows the Trade Reporter down.
Performance Limitations
The single-threaded version of the Trade Reporter runs well for a while, until trade volumes and reference data volumes increase significantly. The process has to make two remote system calls to (1) get trade data, and (2) get client reference data. If the firm is now trading with 3x the number of counterparties and producing 5x the volume of trades, these calls will take longer and put the Trade Reporter's ability to build the report in under 30 seconds at risk.
Performance Tuning: Cache and Query Reference Data with a Separate Thread
In part because reference data does not change as often as trade data, a decision is made to employ a separate thread to cache the reference data, making the results available for fast query from a local Map-based data cache. This approach minimizes blocking during the report generation process.
Since the blocking call to the reference data server has been removed from the main execution stream, the Trade Reporter speeds up significantly as the calls to the reference data cache return immediately. The report can now be produced easily within the 30 second interval.
So far, so good, but as with the previously mentioned example of multi person document creation, the reference data cache updates must be coordinated or "thread-safe" in tech speak. If there is no coordination, the reporter may end up querying against an empty or partially filled cache that is in the process of being rebuilt. Coding mechanisms such as semaphores and synchronized blocks can help with this, but the resulting code is significantly more complex versus the single-threaded variant.
Asynchronous 3forge Development: You're Already Using It
Asynchronous execution via pausable, coroutine statements has been a seamless part of AMI script for years now. Many developers may not have realized it, but certainly appreciate not having their user interface freeze up when sending an email or executing one or more large database queries.
Coroutines offer the ability to suspend and resume execution without the complexity of threading. 3forge's implementation adheres to the async/await pattern, which aims to allow code to resemble synchronous design, but ultimately allows for asynchronous execution without adding threads. Therefore, a statement in 3forge is declared like any other synchronous code block, but instead of immediate execution, it is instead prepared for execution and queued by the active thread.
The Scope of a Pausable Statement
When a method contains a pausable statement, it is considered a "pausable formula". All statements before the deferred statement are executed inline. The deferred statements, including subsequent code, are queued for later execution. Without a "dispatch" block, the deferred statement boundary is determined to be the end of the method. Thus, a method that has two deferred statements will still execute sequentially unless concurrent blocks are used.
Concurrency
Concurrency allows multiple blocks of pausable code to be executed simultaneously. For example, if a method is required to query data from three data sources, a concurrent approach would allow for all sources to be queried at the same time and avoid "one after the other" sequential execution.
Increased Performance by Mixing Concurrency and Pausability
In 3forge, multiple pausable statements can be executed in parallel in a concurrent code block. New AMI Scripting elements for Concurrency and Pausability include the concurrent{} block, which wraps a set of pausable statements that are executed in parallel, and the dispatch{} block, which defines a pausable statement block.
Concurrent Execution Is Durable to Failure
A dispatched block of code can fail and be isolated with appropriate error-handling, allowing the remaining streams of execution to complete and return.
When to Use Pausable Statements
While powerful in many circumstances, there are scenarios where pausable statements do not make sense, including scenarios where a block of code is repeatedly executed in a tight loop (e.g. a column filter). To prevent system performance bottlenecks, 3forge does not allow pausable statements to be executed in lambda scenarios, including SQL statements and Web UI formulas.
