Radar 2016

The 2016 JUXT Clojure Technology Radar

Loading radar..

Welcome to our Clojure tech radar. This is an opinionated view collated from our collective experience of building projects with Clojure. There will be many libraries that should be on here but aren’t, and partly this is down to the awesome Clojure ecosystem, and how we take some of it for granted.

Whilst we have direct experience of the vast majority of items on the radar, some will be on here that we haven’t used. We will explicity mention this where appropriate in the write up.

No radar can ever be perfect, but we hope that this radar will be useful for those deploying Clojure in real-world projects, and we welcome your feedback and suggestions.

Our Radar is derived from the well known ThoughtWorks technology radar.

Rings

We use the same rings as ThoughtWorks do: hold, assess, trial and adopt.

Hold means that we are holding off using this technology for now for some given reason. It may be a library that we consider simply too raw and new, or it could be a tool that we had issues with in the past and that there are valid alternatives.

Assess means that we think this libary is worth checking out, but we don’t consider to be an automatic choice. It may be a new library that we haven’t used but wish to, or one that you should definitely weigh up the pros and cons for.

Trial means go try this library. It may still be new or there may be cases where it’s not entirely suitable, but if it in anyway fits your problem case, then you should give it strong consideration.

Adopt this is a library that we have used in anger and is a solid choice.

Quadrants

We’ve divided the radar up into four quadrants. ClojureScript and Tools should be self explanatory. Infrastructure refers to a group of libraries that have a high architectural footprint on your application. The smaller libraries, we’ve simply put into libraries.

Infrastucture

We’ve used Stuart Sierra’s Component in anger on many projects to manage the lifecycles of state in our programs (such as database connections). This fits in with the Reloaded workflow pattern, so we don’t have to keep bouncing the REPL during development. Although this was a high profile large step for Clojure architectures, there is concern from some that the introduction of protocols can proliferate, and that the layers of abstraction and indirection are increased. Mount is a new alternative that uses namespaces and vars as oppose to protocols and records. We haven’t used it yet and so it’s at trial, here’s an interesting discussion on it.

Ring is the dominant HTTP web application framework, providing an out of the box HTTP server, middleware and more. Ring has been battle tested on many large projects. Http-Kit is an alternative that follows the Ring spec and is thus compatible as a drop in replacement, but is built from the ground up to support "highly concurrent a/synchronous" web applications. We’ve put this at trial owing to our lack of usage of the server component (the client lib is great). Aleph is another Ring compatible contender, built on top of the event streaming library Manifold. We’ve put Aleph as adopt.

Compojure-API is a power-up for the plain old Compojure over Ring stack. It integrates tightly with Swagger and Schema, giving you a Swagger UI for nearly free, and Schema coercions (i.e. converting to and from dates and UUIDs inside of JSON payloads). Swagger is a beautiful present you can give to your third-party server consumers, describing your APIs.

We’re fans of Pedestal, a web framework bringing the concept of interceptors that can be chained together and executed asynchronously. Frankie says why interceptors are preferable over the conventional approach of using Ring middleware and handler functions; "data beats function composition". We’re using Pedestal on many of our projects, and so we view it as adopt.

We feel that Liberator is also a valid option when you need good HTTP compliance, but for common use-cases of building simple websites and simple REST services, it could be over-doing it. Liberator isn’t trivial with its state machine, and using Pedestal or Compojure/Ring returning explicit HTTP status codes may be easier.

New emerging web service libraries include yada and Kekkonen. yada, like Liberator nails the HTTP spec in a resource-oriented fashion, using convention over explicit configuration. We’re rating it as assess, because it’s still in beta. Kekkonen is also at alpha-stage and aims to hold similar ground in the API definition space, with integration with Swagger and Schema etc. We haven’t used it, but we are keeping an eye on it.

Buddy is a security framework we’ve had success with. It’s at trial because it’s relatively new for a security library, and all the features you need may not already exist. We’d recommend giving it a go though. It has many components including a crypto API, access control for ring middleware, and authentication of JWT (json web tokens). Frankie gives an example usage of the latter.

Storm is a 'distributed realime computation system' we’ve used, but it involves a lot of devops, and it’s non-trivial to get running in the REPL. Onyx is a more modern, Clojure-centric alternative that we would use instead on our next project, when we have the complex requirements of spreading a pipeline of computational work over a cluster of nodes. core.async would be our first choice when you don’t need to spread work over parallel machines or processes, but you still want to the separate work up, and to potentially utilise a number of cores.

Riemann is a highly concurrent realtime event processing platform we’ve used on previous projects, to integrate with 3rd party services such as Logstash, Pagerduty, and to send emails upon incident. We’ve placed it at assess. It’s a great and powerful platform, but a lot of what we’ve used Riemann for you can do with the third parties now directly, and libraries such as metrics-clojure help us to do event aggregation directly in the application.

Avout is an ambitious project to provide a 'distributed implementation of Clojures multi-version-concurrency-control STM'. Basically creating atoms with the data persisted into a central store such as ZooKeeper. We place this on hold owing to some outstanding issues recorded in the GitHub that affected our previous experience. If you like the genernal idea of Avout, then check out zookeeper-atom which is simpler and aims to do one thing well.

Datomic is a great Clojure immutable database and a compelling choice. We’ve used it in anger on various projects and would classify it as adopt.

Libraries

On the database side we’ve got HoneySQL, Yesql and Korma. We’ve used HoneySQL extensively on various projects, which allows you to build up SQL via Clojure data structures. There’s the occasional time you need to break out of the DSL and either drop down to using raw SQL (supported), or you can add your own SQL terms via extension points. Yesql takes a different approach to HoneySQL where you treat SQL as raw text. This can be a cleaner, but less amendable for when you need to dynamically build up complex SQL statements. Korma is comprised out of macros, which makes composability more difficult and for this reason we’ve used it less. For more information on the various SQL libaries, see this good Clojure SQL libary showdown.

Compojure is a the dominant HTTP routing library. We feel there are modern alternatives as so Compojure is at trial. Bidi is one such alternative (for ClojureScript also) that we’ve used extensively that allows you to define bi-directional routing and to keep the routes as data. Tripod is a another brand new routing library, that ports the routing aspects of Pedestal for use with Ring middleware and Clojurescript.

We’ve already mentioned Prismatic’s Schema library, which has proven invaluable on a number of projects. As well as its use for data validation, the coercion facilities can be extremely useful. Spectre is a new tool that promises to make working with immutable data easier. We haven’t used it yet, but it looks great and could be essential. We’ve put it at trial.

Nomad is a good adopt choice for managing application configuration, marrying the EDN format and reader conditionals so that you can have powerful and flexible config files. Aero is similar, but newer and more limited, therefore at trial. Environ is a small lib that allows you to grab environment settings from various sources. We’ve put this at trial; it’s great, but it adds variability as to where settings come from.

Prone is great Ring middleware for helping to display and diagnose exceptions.

Cheshire is an extremely well battle tested and performant option you can adopt for deserialising and serialising JSON.

If you need to deal with binary data packets, transforming them back and forth from Clojure data-structures, then Gloss is a lifesaver.

Metrics-Clojure is an invaluable tool for batching up metrics for periodically sending to a third-party service such as Riemann or Logstash. If you’re holding back collecting metrics because it’s expensive to log and send each one individually, consider this approach.

If you want to keep your logging config purely in Clojure, then Timbre is getting a lot of love for this and is placed as adopt.

clj-time is an indepensible library for wrapping the joda-datetime Java library, giving it more of an immutable Clojurey feel.

Instaparse is indispensable for building parsers to process large formally-defined languages.

For template languages, we’ve not yet established a clear winner. Hiccup is great for building up HTML like data-structures and then converting them to actual HTML. Given the sheer usefulness of Hiccup (even when it’s not the main templating language) we’ve put it at adopt. Some feel that Hiccup code can be hard to parse for others not used to Clojure (i.e. CSS designers), and so the Enlive approach is often taken. With Enlive you work with XHTML templates, and have some behind-the-scenes code to re-render the templates based on real data. Some feel that this decoupling makes the whole flow harder to follow. Template rendering aside, Enlive is a great tool for parsing HTML, and should always considered for this use case. Taken as a whole, we’ve put at trial.

Stencil follows the Mustache spec and is a well battle tested option (performance is excellent). It is limited though as a templating language, and can frustrate with it "logic-less" philosophy, therefore trial. Selmer is based on the Django templating approach, and looks like a compelling alternative (we haven’t used). It should allow you to do much more in templates than with Mustache.

ClojureScript

Lein-cljsbuild interacts with the ever-improving ClojureScript compiler and is a must (unless you’re using Boot, which is also a viable option). Figwheel (and boot) have revolutionised the experience of developing ClojureScript, automatically pushing changes to the browser. We say lein-cljsbuild and FigWheel are adopt because they simply just 'work', and they vastly enrich the developer experience.

Om is the standard choice for a React ClojureScript wrapper. We’ve used it successfully alongside Om-tools to reduce the boilerplate. Om has a large on-going development effort behind it, and the future iteration of Om Next is very interesting. Some developers can find Om a touch complex, in particular with the availabilty of component-local state and cursors for interacting with the global application state.

Reagent takes a different approach and bypasses much of this complexity, boiling React down to just functions and atoms, and for this reason it’s become very popular (the radar on this page is built using it, rendering an SVG using Clojure functions accessing data in a Reagent atom). re-frame offers a layer on top of Reagent helping to structure your application and event flow. Both Reagent and re-frame are ripe for adoption as an alternative to Om (although Om is also a great choice).

Quiescent is a lightweight wrapper around React.js. We haven’t used it directly, but it is definitely worth evaluating. Therefore trial.

For more resources on Clojurescript React wrappers, check out this post covering a comparison of the main players.

Sablono allows you to build up regular HTML React components using the familiar Hiccup syntax. You can use Sablono with Reagent and Om, and is most definitely adopt.

Datascript allows you to have a database in ClojureScript that supports Datalog. It supports most of the Datomic queries with documented exceptions. We’ve put it at trial. It could be overkill for simple websites that would benefit from straightforward React (with the data housed in atoms), but for more complex websites it could be suitable.

Tooling

Paredit (or an alternative tool to help you with parens), is a must.

For building Clojure projects, Lein is the dominant choice. Boot is relatively new and offers a programmatic alternative in difference to Lein’s declarative style. We like Boot and use it for some projects (such as building this website), but we’ve had some teething problems with it for larger projects (such as building up complex uberjars). Some of these niggles are being ironed out, and it’s worth checking out. Boot vs Lein is really a philosophical battle over programmatic vs declarative, and it will be interesting to see if boot gets mass adopted.

Gradle could be the best multi-language build tool for the JVM as a whole, but its support for Clojure is not near the maturity of other languages such as Java and Groovy. Therefore it’s a hold.

For editors, CIDER is the main player for Emacs and has lots of awesome features (debugger, eval’ing code inline). The main issue people complain about is when you start upgrading and you get stuck in between versions and their dependencies. This has become an annoyance many of us have come to live with. clj-refactor is an Emacs addon that complements CIDER, providing lots of refactoring goodness omitted from the CIDER mothership. clj-refactor is fit for use, but is a bit more experimental than CIDER in its choice of features and will carry on evolving, and so we’ve put it at trial.

If you’re using IntelliJ and want to stick with it, then adopt Cursive. The debugging and refactoring support has been excellent from day one. Cursive is so good, it’s swaying some established Emacs/CIDER users.

Garden is a good tool for building up CSS from Clojure data structures. We’re putting it at assess because whilst it’s a good fit for some projects, it’s not an hugely active project and is still missing some features such as conversion from other preprocessors such as Sass and Less, which is useful if you want to use mixins from third parties such as Bootstrap and Google Material Designs.

For testing, we rely heavily on clojure.test. We’ve also used Midje, a tool to 'make tests more readable', but we’ve had a couple of tussles with it (such as reporting, reloading code, working with the macros). We’ve got it at hold, but other development teams have used it extensively, and have taken to the DSL.

We have commonly used Joplin for migrating a variety of different databases, including PSQL and Elasticsearch.

Honourable Mentions

We have used Logback for logging with frequent success. It’s written in Java, and is a splendid example of Java/Clojure interop. clj-http is an old friend that wraps the Apache HTTP client library.

Authors

Jon Pither, Malcolm Sparks

Contributors/Reviewers

Our thanks to the efforts of many in the JUXT network who have given opinions and suggestions for this radar, including Ernestas Lisauskas, Kris Jenkins, Oliver Hine, Frazer Irvine, David Humphreys, Frankie Sardo, Martin Trojer, David Smith, Dalibor Novak, Sam Newman, Stathis Sideris.

Further Resources

Checkout the Clojure Toolbox for a more expansive listing of what’s in the Clojuresphere.