I recently published an article about how to bind non-transactional resources like web services / microservices into global distributed transactions so that recovery is handled automatically. Over the years I have often had to integrate "non-transactional" systems into Java EE application servers and data consistency was often a topic of discussion or even a non-functional requirement. I've put "non-transactional" into quotes because often the systems contain ways of ensuring data consistency, for example using calls to compensate, but the systems aren't what you might traditionally call transactional. There is certainly no way of configuring a Java EE application server to automatically handle recovery for such resources. The following is a list of patterns that we compiled, showing different ways to maintain consistency when faced with the task of integrating a non-transactional system. Write job to database - The common scenario whereby you want to send say an email confirmation after a sale is made. You cannot send the email and then attempt to commit the sales transaction to your database, because if the commit fails, the customer receives an email stating that they have bought something and you have no record of it. You cannot send the email after the sales transaction is committed to your database, because if the sending of the email fails (e.g. the mail server is temporarily down), the customer won't get their confirmation, perhaps with a link to the tickets that they bought. One solution is to write the fact that an email needs to…
Author: Ant
Mysql versions prior to 5.7 do not fully support two phase commit
While doing some tests for the recently released generic JCA adapter which is capable of binding remote calls to microservices (as well as other things) into JTA transactions, I discovered a bug in Mysql 5.6 which has been around for nearly ten years. The test scenario was a crash after the "prepare" phase of the XA transaction, after both the database and the generic connector vote to commit the transaction. After crashing the database or the application server, the transaction manager will try to recover and commit the transaction in the database. But while testing the crashes with Mysql 5.6 rather than Mysql 5.7, I was having the problem that the database never actually committed the transaction, meaning that the system as a whole was in an inconsistent state. There were absolutely no problems with the generic connector, just the database. The application server continuously logged that there was an incomplete transaction but was unable to complete the transaction in the database. During the simulated database crash, the application returned a HeuristicMixedException to indicate to the user that something is not and will not ever be consistent. The error logged by JBoss was: ...WARN (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid: <...eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK ...> I spent time debugging in the Mysql driver code but eventually came across MySQL Bug #12161 which has only just been closed after being open for nearly 10 years! It is…
Global Data Consistency in Distributed (Microservice) Architectures
UPDATE: Now supports Spring and Spring Boot outside of a full Java EE sever. See new article for more details! UPDATE 2: See this new article for more details about using asynchronous calls to remote applications to guarantee global data consistency I've published a generic JCA resource adapter on Github available from Maven (ch.maxant:genericconnector-rar) with an Apache 2.0 licence. This let's you bind things like REST and SOAP web services into JTA transactions which are under the control of Java EE application servers. That makes it possible to build systems which guarantee data consistency, with as little boiler plate code as possible, very easily. Be sure to read the FAQ. Imagine the following scenario... Functional Requirements ... many pages of sleep inducing requirements... FR-4053: When the user clicks the "buy" button, the system should book the tickets, make a confirmed payment transaction with the acquirer and send a letter to the customer including their printed tickets and a receipt. ... many more requirements... Selected Non-Functional Requirements NFR-08: The tickets must be booked using NBS (the corporations "Nouvelle Booking System"), an HTTP SOAP Web Service deployed within the intranet. NFR-19: Output Management (printing and sending letters) must be done using COMS, a JSON/REST Service also deployed within the intranet. NFR-22: Payment must be done using our Partner's MMF (Make Money Fast) system, deployed in the internet and connected to using a VPN. NFR-34: The system must write sales order records to its own Oracle database. NFR-45: The data related to a…
Is Asynchronous EJB Just a Gimmick?
In previous articles (here and here) I showed that creating non-blocking asynchronous applications could increase performance when the server is under a heavy load. EJB 3.1 introduced the @Asynchronous annotation for specifying that a method will return its result at some time in the future. The Javadocs state that either void or a Future must be returned. An example of a service using this annotation is shown in the following listing: The annotation is on line 4. The method returns a Future of type String and does so on line 10 by wrapping the output in an AsyncResult. At the point that client code calls the EJB method, the container intercepts the call and creates a task which it will run on a different thread, so that it can return a Future immediately. When the container then runs the task using a different thread, it calls the EJB's method and uses the AsyncResult to complete the Future which the caller was given. There are several problems with this code, even though it looks exactly like the code in all the examples found on the internet. For example, the Future class only contains blocking methods for getting at the result of the Future, rather than any methods for registering callbacks for when it is completed. That results in code like the following, which is bad when the container is under load: This kind of code is bad, because it causes threads to block meaning that they cannot do anything useful during that…
Javascript everywhere
I can think of at least two scenarios when you might need to run the same algorithms in both a Javascript and a Java environment. A Javascript client is running in offline mode and needs to run some business logic or complex validation which the server will need to run again later when say the model is persisted and the server needs to verify the consistent valid state of the model, You have several clients which need access to the same algorithms, for example a Javascript based single page application running in the browser and web service which your business partners use which is deployed in a Java EE application server. You could build the algorithm twice: once in Java and once in Javascript but that isn't very friendly in terms of maintenance. You could build it once in just Java and make the client call the server to run the algorithm, but that doesn't work in an offline application like you can build using HTML 5 and AngularJS, and it doesn't make for a very responsive client. So why not build the algorithm just once, using Javascript, and then use the javax.script Java package which first shipped with Java SE 7 and was improved with Java SE 8, in order to execute the algorithm when you need to use it from Java? That is precisely what I asked myself, and so I set about building an example of how to do it. The first thing I considered was deployment…
A reactive and performant Spray + Akka solution to “Playing with concurrency and performance in Java and Node.js”
In my previous post I examined a fictitious trading engine and compared a Java based blocking solution to a Node.js based non-blocking solution. At the end of the post I wrote that: I suspect that following the recent success of Node.js, more and more asynchronous Java libraries will start to appear. Well such libraries already exist, for example: Akka, Spray, and this Mysql async driver. I set myself the challenge of creating a non-blocking Java based solution using exactly those libraries, so that I could compare its performance to that of the Node.js solution created for the last article. The first thing you might have noticed is that these are all Scala based libraries, but I wrote this solution in Java even though it is a little less syntactically elegant. In the last article I introduced a solution based upon Akka whereby the trading engine was wrapped in an actor. Here, I have dropped Tomcat as the HTTP server and replaced it with Spray, which neatly integrates the HTTP server straight into Akka. In theory this should make no difference to performance, because Spray is NIO just as Tomcat 8 is, out of the box. But what attracted me to this solution was that overall, the number of threads is greatly reduced, as Spray, Akka and the async Mysql library all use the same execution context. Running on my Windows development machine, Tomcat has over 30 threads compared to just a few over 10 for the solution built here, or…
Playing with concurrency and performance in Java and Node.js
Imagine a simple market where buyers and sellers interested in the same product come together to trade. For each product in the market, buyers interested in the product could form an orderly queue, sorted on a "first come, first serve" basis. Each buyer could then approach the cheapest seller and trade, purchasing as much of the product from the seller as they wish for the price dictated by the seller. Should no seller be offering the product at a price low enough, the buyer could step to the side, giving the next buyer the opportunity to trade. Once all buyers have had the chance to make a trade, and after all products in the market have been through the cycle, the whole process can start again, after satisfied buyers and sellers leave and new ones take their place. In the internet age, there is no reason why buyers and sellers could not trade on a virtual platform, using this type of algorithm, from the comfort of their armchair. Indeed, trading platforms like this have existed for many years. While basic, this type of problem becomes interesting when used to build a computer based trading engine. Simple questions pose challenges: How could the market scale up across multiple cores? How could the market scale out across multiple machines? Inherently, the answers boil down to requiring some form of concurrency so that such a trading engine can scale. Typically I would jump into writing a Java based solution using perhaps an execution…
Rule Engine for Node.js
Based on my original blog posting, this posting introduces a ported Javascript version of my rule engine, for Node.js. I have the requirement to use a rule engine. I want something light weight and fairly simple, yet powerful. While there are products out there which are super good, I don't want something with the big learning overhead. And, I fancied writing my own! Here are my few basic requirements: Use some kind of expression language to write the rules, It should be possible to store rules in a database, Rules need a priority, so that only the best can be fired, It should also be possible to fire all matching rules, Rules should evaluate against one input which can be an object like a tree, containing all the information which rules need to evaluate against Predefined actions which are programmed in the system should be executed when certains rules fire. So to help clarify those requirements, imagine the following examples: 1) In some forum system, the administrator needs to be able to configure when emails are sent. Here, I would write some rules like "when the config flag called sendUserEmail is set to true, send an email to the user" and "when the config flag called sendAdministratorEmail is true and the user has posted less than 5 postings, send an email to the administrator". 2) A tarif system needs to be configurable, so that the best tarif can be offered to customers. For that, I could write rules like these:…
Simple rule engine updated
I have taken the time to upgrade my simple Java rule engine so that it supports Java 8 lambdas and streams and it is now published in Maven Central. The code is now also available on GitHub. First off, the Maven dependencies. The following dependency is for the basic rule engine which is compatible with Java 6. <dependency> <groupId>ch.maxant</groupId> <artifactId>rules</artifactId> <version>2.1.0</version> </dependency> If you want to use lambdas from Java 8, then you need to also add a dependency as follows: <dependency> <groupId>ch.maxant</groupId> <artifactId>rules-java8</artifactId> <version>2.1.0</version> </dependency> This allows you to write code like that found on lines 15 and 20. Rule rule1 = new Rule("R1", "input.p1.name == \"ant\" && input.p2.name == \"clare\"", "outcome1", 0, "ch.maxant.produits", "Règle spéciale pour famille Kutschera"); Rule rule2 = new Rule("R2", "true", "outcome2", 1, "ch.maxant.produits", "Régle par défault"); List<Rule> rules = Arrays.asList(rule1, rule2); //to use a lambda, construct a SamAction and pass it a lambda. IAction<MyInput, BigDecimal> action1 = new SamAction<MyInput, BigDecimal>( "outcome1", i -> new BigDecimal("100.0") ); IAction<MyInput, BigDecimal> action2 = new SamAction<MyInput, BigDecimal>( "outcome2", i -> new BigDecimal("101.0") ); List<IAction<MyInput, BigDecimal>> actions = Arrays.asList(action1, action2); Engine e = new Engine(rules, true); MyInput input = new MyInput(); Person p1 = new Person("ant"); Person p2 = new Person("clare"); input.setP1(p1); input.setP2(p2); BigDecimal price = e.executeBestAction(input, actions); assertEquals(new BigDecimal("101.0"), price); If you want to pass a Stream to the Engine rather than a Collection, then use the sub-class found in the second library, for example: Stream<Rule> streamOfRules = getStreamOfRules(); //to pass in a stream, we need to use…
The effects of programming with Java 8 Streams on algorithm performance
Multi-paradigm programming with Java has been possible for many years, with its support for a mix of service oriented, object oriented and aspect oriented programming. Java 8 with its lambdas and java.util.stream.Stream class, is good news because it lets us add the functional programming paradigm into the mix. Indeed there has been a lot of hype around lambdas. But is changing our habits and the way we write our code a wise thing, without first getting to know the dangers that might lurk? Java 8's Stream class is neat because it lets you take a collection of data and chain multiple functional calls on that data together, making for tidy code. Map/reduce algorithms are a good example, where you take a collection of data and aggregate it by first selecting or modifying data out of a complex domain and simplifying it (the "map" part), and then reducing it to a single useful value. Take for example the following data classes (written in Groovy so that I get code generation of constructors, accessors, hash/equals and toString methods for free!): //Groovy @Immutable class City { String name List<Temperature> temperatures } @Immutable class Temperature { Date date BigDecimal reading } I can use those classes to construct some random weather data in a list of City objects, e.g.: private static final long ONE_DAY_MS = 1000*60*60*24; private static final Random RANDOM = new Random(); public static List<City> prepareData( int numCities, int numTemps) { List<City> cities = new ArrayList<>(); IntStream.range(0, numCities).forEach( i -> cities.add( new…
