Our Purely Functional Microservice in Production

Recently I was interviewing a candidate for a job on my team at BackstopSolutions and he said something like: “I read your blog post about Microservices and was wondering if the lack of follow up meant the project was a failure.” He actually expressed this sentiment in a much nicer way as this was an interview but still it got me thinking: "I should do an update on the status of our microservice and explain how it could be purely functional."

The service is implemented in Clojure, uses Datomic for persistence, and integrates via REST.

The service has been in production for several months now and has gone through a number of versions of its API. Implementing HATEAOS worked out well as it allows us decouple having to keep the service and front end in lock step. Since this is a microservice, we could keep both applications in step with each other as we own both ends of the interface. However this co-ordination is not without cost.  Release dates slip b/c of bugs in either system.  There are less bottlenecks in production releases as the front and back end can be deployed to production independently of each other. Not having to co-ordinate releases is a wondrous decoupling. It didn’t take long to implement HATEAOS and maintenance costs for the discoverability aspect have not materialized.

Datomic is a special snowflake of a database and that it why I believe it is the future. That is a bold statement deserving of a whole other blog post so I will merely list a few of Datomic’s features that we exploit:
  1. It never forgets. Think of it like source control but with transactions replacing commits.
  2. Time is a first class citizen. This is really an outgrowth of the above feature but is worth pointing out on its own.
  3. It uses the incredibly compose-able Datalog query language instead of SQL and can include clojure.core expressions in its queries.
  4. It can easily be used as a graph database.

Those are some crazy advantages. Since Datomic never forgets, ANY query to our service that specifies a time can never have a different return value. That is what I mean when I say that our service is “purely functional.”  This enables the Rails app that works with our micro service to cache every query it makes FOREVER. The service without caching is still crazy fast considering the incredibly complex queries is has to run. Like any graph database, speed is determined by what percentage of the graph a node can access. Even if a customer stretches the engine to its maximum, our ability to cache forever means they will take that hit but once. Of course you can query our microservice with no time specified where it will default to now and not cache. This is used only for debugging and is never used by our Rails application.  

The graph database aspect of Datomic combined with its understanding of clojure.core enable us to spend days instead of months (or even years) implementing custom business logic. Backstop Solutions is a Business to Business (B2B) Software As A Service (SAAS) firm.  In B2B there is but one constant:  All business’ processes are unique. Period: End of story. 

There may be some similarities and generalities between companies in the same competitive space but these similarities are revealed to be mostly illusionary when you try to implement specific customers. Drill down into any business’ rules and you will find unique requests from every client.  Think of it this way: In an efficient market what use is there for complete duplication? The clients who's processes are a competitive advantage hate any SAAS offering that can not implement aspects of their system in the SAAS product. With this microservice we can. The actual challenge is to get clients to simplify their rules enough so that they themselves can understand the rules. This may seem confusing until you consider that Backstop Solutions has clients with many offices in many different lands so we are often the first people to implement a logically consistent system that fits all of their needs, people, and processes.

Datalog is a subset of Prolog. Prolog is a logic based language as is SQL. Where it crushes SQL like a bug is its compose-ability. You can create a bunch of “rules” and mix and match these rules just like you are used to composing functions. This is amazing. I can not stress this enough. I continually implement stories in a fraction of the estimated time because I discover that most of the query is already written and I just have to tack a little on the end, or beginning, or middle. 

What is even more wonderful is that even if I ever get into a tight corner with Datalog I can fall back on all the functions in clojure.core.  How many times has it infuriated you that the version of SQL you are using can’t support a completely logical function and so you have to do something horrible? Now imagine being able to use all of clojure.core in SQL. Exactly.


I should stop and point out that I had the idea for this service 4 years ago during a conversation with David Chelimsky. As he described a problem he was solving with a graph database I realized that we had a completely different but functionally similar problem where I work. Thanks for always being inspiring David.

Comments

Post a Comment

Popular posts from this blog

What's a Good Flog Score?

I've been using Flog to measure the complexity of my Ruby code and I was wondering how others interpret Flog numbers. I'll go first with my opinions: Score of Means 0-10 Awesome 11-20 Good enough 21-40 Might need refactoring 41-60 Possible to justify 61-100 Danger 100-200 Whoop, whoop, whoop 200 + Someone please think of the children (note: these are scores for an individual method) The 20 - 60 range is interesting to me. Sometimes the complexity of what you're doing will justify the score and other times the method is in desperate need of refactoring. The highest Flog score I've seen in the wild was in the 300's. What's you're personal high and what do you think of my rankings? In other news, I've redesigned the look of my blog so if you only see my post through a reader you might want to stop on by, take a look, and tell me what you think.
Read more

SICP Wasn’t Written for You

The number of software luminaries who sing the praises of “ Structure and Interpretation of Computer Programs ” (referred to as SICP) is such a long list that you might think only a crazy person would take issue with it. However, to ignore SICP’s problems and continue to blindly recommend it seems just as crazy. SICP was the textbook for MIT’s introductory programming class and was a bit of a departure from other into to computer science textbooks at the time.  Wikipedia sums it up nicely:  “Before SICP, the introductory courses were almost always filled with learning the details of some programming language, while SICP focuses on finding general patterns from specific problems and building software tools that embody each pattern.”  Which sounds awesome, but does essentially say that abstract principles will be introduced before the nuts and bolts of a language.  If you think about that for a minute, you may see where the problems will be. When I was training to be a...
Read more

Point Inside a Polygon in Ruby

Recently my team needed to find out if a point was inside a given polygon and, as usual, we found some code on the internet that did what we wanted. And, as usual, there wasn't much explanation as to what it was doing. Here's the code (after we converted it to Ruby): 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 def contains_point? (point) c = false i = -1 j = self .size - 1 while (i += 1 ) < self .size if (( self [i].y <= point.y && point.y < self [j].y) || ( self [j].y <= point.y && point.y < self [i].y)) if (point.x < ( self [j].x - self [i].x) * (point.y - self [i].y) / ( self [j].y - self [i].y) + self [i].x) c = !c end end j = i end return c end end So, you know, it works and I'm grateful to somebody for posting it but we had no idea how it works. So we wrapped 2 tests around it (one where the point is not in the polygon and another where it is...
Read more