In his article Why I chose OCaml as my primary language, my friend Xavier Van de Woestyne presents, in the section Dependency injection and inversion, two approaches to implementing dependency injection: one using user-defined effects and one using modules as first-class values. Even though I’m quite convinced that both approaches are legit, I find them sometimes a bit overkill and showing fairly obvious pitfalls when applied to real software. The goal of this article is therefore to briefly highlight the ergonomic weaknesses of both approaches, and then propose a new encoding of inversion and dependency injection that I find more comfortable (in many cases). In addition, this gives an example of using objects in OCaml, which are often overlooked, even though in my view OCaml’s object model is very interesting and offers a lot of practical convenience.
This approach is by no means novel and is largely the result of several experiments shared with Xavier Van de Woestyne during multiple pair-programming sessions. However, a precursor of this encoding can be found in the first version of YOCaml (using a Left Kan Extension/Freer Monad rather than a Reader). It's also interesting we can find a similar approach in the recent work around Kyo, an effect system for the Scala language, which uses a similar trick based on subtyping relationships to type an environment.
Why use dependency injection?
There are plenty of documents that describe (sometimes a bit aggressively) all the benefits of dependency injection, which are sometimes extended into fairly formalized software architectures (such as hexagonal architecture). For my part, I find that dependency injection makes unit testing a program trivial, which I think is reason enough to care about it (For example, the time-tracker I use at work, Kohai, uses Emacs as its interface and the file system as its database. Thanks to inversion and dependency injection, we were able to achieve high test coverage fairly easily).
Effect system and dependency injection
There are many different ways to describe an effect handler, so in my view it is difficult to give a precise definition of what an Effect system is. However, in our modern interpretation, the goal is more to suspend a computation so it can be interpreted by the runtime (the famous IO monad), and an Effect system is often described as a systematic way to separate the denotational description of a program, where propagated effects are operational “holes” that are given meaning via a handler, usually providing the ability to control the program’s execution flow (its continuation), unlocking the possibility to describe, for example, concurrent programs. In this article, I focus on dependency injection rather than on building an effect system because that would be very pretentious (my article does not address performance or runtime concerns). Similarly to how exception handling can be seen as a special case of effect propagation and interpretation (where the captured continuation is never resumed), I also see dependency injection as a special case, this time, where the continuation is always resumed. It’s quite amusing to see that dependency injection and exception capturing can be considered two special cases of effect abstraction, differing only in how the continuation is handled.
The Drawbacks of Modules and User-Defined Effects
As I mentioned in the introduction, I believe that both approaches proposed by Xavier are perfectly legitimate. However, after using both approaches in real-world software, I noticed several small annoyances that I will try to share with you.
Using modules
Using modules (through functors or by passing them as values) seems to be the ideal approach for this kind of task. However, the module language in OCaml has a different type system, in which type inference is severely limited. From my point of view, these limitations can lead to a lot of verbosity whenever I want to imagine that my dependencies come from multiple different sources. For example, consider these two signatures:
The first one provides basic manipulation of the file system (very simple, with no handling of permissions or errors):
module type FS = sig val read_file : path:string -> string option val write_file : path:string -> content:string -> unit end
The second one simply allows logging to standard output (also very basic, without support for log levels):
module type CONSOLE = sig val log : string -> unit end
The first way to depend on both signatures is to introduce a new signature that describes their combination:
module type FS_CONSOLE = sig include FS include CONSOLE end let show_content ( module H : FS_CONSOLE ) = match H . read_file ~ path : " /foo/bar " with | None -> H . log " Nothing " | Some x -> H . log x
Or simply take the dependencies as two separate parameters:
let show_content ( module F : FS ) ( module C : CONSOLE ) = match F . read_file ~ path : " /foo/bar " with | None -> C . log " Nothing " | Some x -> C . log x
Indeed, constructing the module on the fly, directly in the function definition, is not possible. Although very verbose, this expression is rejected by the compiler:
# let show_content ( module H : sig include FS include CONSOLE end ) = match H . read_file ~ path : " /foo/bar " with | None -> H . log " Nothing " | Some x -> H . log x ; ; Lines 1 - 4 , characters 30 - 10 : Error : Syntax error : invalid package type : only module type identifier and with type constraints are supported
Moreover, beyond the syntactic heaviness, I find the loss of type inference quite restrictive. While in situations where you don’t need to separate and diversify dependency types this isn’t a big deal, I find that this approach sometimes forces unnecessary groupings. I often ended up in situations where, for all functions that had dependencies, I had to provide a single module containing them all, which occasionally forced me, in testing scenarios, to create dummy functions. A very frustrating experience!
Using User-Defined-Effects
I proudly claimed that dependency injection is a special case of using an Effect System, so one might wonder: could using OCaml’s effect system be a good idea? From my understanding, the integration of effects was primarily intended to describe interactions with OCaml’s new multi-core runtime. In practice, the lack of a type system tracking effects makes, in my view, their use for dependency injection rather cumbersome. Indeed, without a type system, it becomes, once again in my view, difficult to mentally keep track of which effects have been properly handled. In YOCaml, we recorded effects in a module called Eff that encodes programs capable of propagating effects in a monad (a kind of IO monad). This allows us to handle programs a posteriori (and thus inject dependencies, of course) but restricts us in terms of handler modularity. Indeed, it assumes that in all cases, all effects will be interpreted. And, in the specific case of YOCaml, we usually only want to either continue the program or discard the continuation (which can be done trivially using an exception). Control-flow management is therefore a very powerful tool, for which we have very little use.
In practice, there are scenarios where using OCaml’s effects seems perfectly legitimate (and I think I have fairly clear ideas about why introducing a type system for effects is far from trivial, particularly in terms of user experience):
When you want to perform backtracking
When you want to express concurrency libraries, schedulers, etc., which makes a lot of sense in libraries like Eio, Miou, or Picos
So, in the specific case of dependency injection, I have the intuition that using OCaml’s effects gives too much power, while putting significant pressure on tracking effects without compiler assistance, making them not entirely suitable.
Using objects
It’s not very original to use objects to encode a pattern typically associated with object-oriented programming. However, in functional programming, it’s quite common to encounter encodings of dependency injection sometimes referred to as Functional Core, Imperative Shell. Although relatively little used and sometimes unfairly criticized, OCaml’s object model is actually very pleasant to work with (its theoretical foundation is even extensively praised in Xavier’s article, in the section Closely related to research). To my knowledge, OCaml is one of the rare mainstream languages that draws a very clear separation between objects, classes, and types. Objects are values, classes are definitions for constructing objects, and objects have object types, which are regular types, whereas classes also have types that are not regular, since classes are not regular expressions but rather expressions of a small class language.
In order to integrate coherently into OCaml and with type inference, the object model relies on four ingredients: structural object types (object types are structural, their structure is transparent), row variables, equi-recursive types, and type abbreviations.
In practice, an object type is made up of the row of visible members (associated with their types) and may end with a row variable (to characterize closed/open object types). This row variable can serve as the subject of unification as objects are used together with their types.
A quick way to become aware of the presence of the row variable is to simply write a function that takes an object as an argument and sends it a message (in OCaml, sending a message is written with the syntax obj # message ):
# let f obj = ( obj # foo ) + 10 ; ; val f : < foo : int ; . . > -> int = < fun >
In the return type, the row variable indicating that the object type is open is represented by <..> . It is thanks to this variable that we can perform dependency injection, finely tracked by the type system and guided by inference.
Simple example
To keep things simple, let’s just revisit the classic teletype example, which we could write in direct style like this:
# let teletype_example () = print_endline " Hello, World! " ; print_endline " What is your name? " ; let name = read_line () in print_endline ( " Hello " ^ name ) val teletype_example : unit -> unit = < fun >
Let’s rewrite our example by taking an object as an argument, which will serve as the handler:
# let teletype_example handler = handler # print_endline " Hello, World " ; handler # print_endline " What is your name? " ; let name = handler # read_line () in handler # print_endline ( " Hello " ^ name ) val teletype_example : < print_endline : string -> 'a ; read_line : unit -> string ; . . > -> 'a = < fun >
To convince ourselves of the compositionality of our dependency injection, let’s imagine the following function, whose role is to simply log traces of the execution:
# let log handler ~ level message = handler # do_log level message val log : < do_log : 'a -> 'b -> 'c ; . . > -> level : 'a -> 'b -> 'c = < fun >
By using the teletype_example and log functions, we can directly observe the precision of the elision, showing that our handler object was open each time:
# let using_both handler = let () = log handler ~ level : " debug " " Start teletype example " in teletype_example handler val using_both : < do_log : string -> string -> unit ; print_endline : string -> 'a ; read_line : unit -> string ; . . > -> 'a = < fun >
All the requirements of our using_both function are (logically) correctly tracked. We can now implement a dummy handler very simply, using immediate object syntax:
# using_both object method do_log level message = print_endline ( " [ " ^ level ^ " ] " ^ message ) method print_endline value = print_endline value method read_line () = " Pierre " end ; ; [ debug ] Start teletype example Hello , World What is your name ? Hello Pierre - : unit = ()
This approach is extremely similar to using polymorphic variants, notably popularised by Mirage 3, for composable error handling, which share many features with objects (structural subtyping, rows).
Typing, sealing and reusing
In our examples, we were guided by type inference. However, in OCaml, it is common to restrict the generality of inference using explicit signatures. As we have seen, some of our inferred signatures are too general and arguably not very pleasant to write:
< do_log : string -> string -> unit; print_endline : string -> 'a; read_line : unit -> string; .. >
Fortunately, type abbreviations allow us to simplify this notation. By using class types and certain abbreviations, we can simplify the type expressions of our functions that require injection:
First, we will describe our types using dedicated class types, let's start with console :
class type console = object method print_endline : string -> unit method read_line : unit -> string end
Now let's write our loggable interface:
class type loggable = object method do_log : level : string -> string -> unit end
Now, let's rewrite our three functions inside a submodule (to make their signatures explicit):
module F : sig val teletype_example : #console -> unit val log : #loggable -> level:string -> string -> unit end = struct let teletype_example handler = handler # print_endline " Hello, World " ; handler # print_endline " What is your name? " ; let name = handler # read_line () in handler # print_endline ( " Hello " ^ name ) let log handler ~ level message = handler # do_log ~ level message end
And now, we can describe our using_both function as taking an object that is the conjunction of loggable and console , like this:
module G : sig val using_both : -> unit end = struct let using_both handler = let () = F . log handler ~ level : " debug " " Start teletype example " in F . teletype_example handler end
At the implementation level, the separation between inheritance (as a syntactic action) and subtyping (as a semantic action) allows us to benefit from this kind of mutualization directly at the call site. For example, let's implement a handler for console and loggable :
class a_console = object method print_endline value = print_endline value method read_line () = " Pierre " end class a_logger = object method do_log ~ level message = print_endline ( " [ " ^ level ^ " ] " ^ message ) end
Even though it would be possible to constrain our classes by the interfaces they implement (using class x = object (_ : #interface_) ), it is not necessary because structural subtyping will handle the rest (and it also allows us to potentially introduce intermediate states easily). We can now use inheritance to share functionality between our two handlers:
# G . using_both object inherit a_console inherit a_logger end ; ; [ debug ] Start teletype example Hello , World What is your name ? Hello Pierre - : unit = ()
One could argue that it’s still a bit verbose, but in my view, this approach offers much more convenience. We know statically the capabilities that need to be implemented, we retain type inference, and we have very simple composition tools. At this point, I have, subjectively, identified scenarios for using the different approaches discussed in this article:
When you need to control the program’s continuation (essentially for backtracking or concurrency), it’s preferable to use effects (hoping that one day we’ll be able to type them ergonomically).
When you want to introduce types (without parametric polymorphism) into dependencies, first-class modules work very well.
When you want to introduce types that can have type parameters (for example, to express the normal forms of our dependencies via a runtime value, e.g., through a monad), functors are suitable.
When you want to do simple dependency injection, guided by type inference and requiring the ability to fragment or share handlers, objects are perfectly suitable.
However, the approach based on first-class modules or objects can heavily pollute a codebase, whereas effect interpretation and functor instantiation can remain at the edges of the program. Let’s look at the final part of this article, which explains how to reduce the aggressive propagation of handlers.
Injected dependencies as part of the environment
Currently, our approach forces us to pass our handler explicitly from call to call, which can drastically bloat business logic code. What we would like is the ability to only worry about the presence of dependencies when it is actually necessary. To achieve this, we’ll use a module whose role will be to pass our set of dependencies in an ad-hoc manner:
module Env : sig type ('normal_form, 'rows) t val run : 'rows -> ('normal_form, 'rows) t -> 'normal_form val perform : ('rows -> 'normal_form) -> ('normal_form, 'rows) t val return : 'normal_form -> ('normal_form, 'rows) t val ( let* ) : ('a, 'rows) t -> ('a -> ('b, 'rows) t) -> ('b, 'rows) t end = struct type ( 'normal_form , 'rows ) t = 'rows -> 'normal_form let run env comp = comp env let perform f = f let return x _ = x let ( let* ) r f x = ( fun y -> ( f y ) x ) ( r x ) end
Our module allows us to separate the description of our program from the passing of the environment. The only subtlety lies in the (let*) operator, a binding operator that lets us simplify the writing of programs. Let's define some helpers based on our previous interfaces:
module U : sig val print_endline : string -> (unit, #console) Env.t val read_line : unit -> (string, #console) Env.t val log : level:string -> string -> (unit, #loggable) Env.t end = struct let print_endline str = Env . perform ( fun h -> h # print_endline str ) let read_line () = Env . perform ( fun h -> h # read_line () ) let log ~ level message = Env . perform ( fun h -> h # do_log ~ level message ) end
And now, we can describe our previous programs using our let* syntax shortcut:
let comp = let open Env in let* () = U . log ~ level : " Debug " " Start teletype example " in let* () = U . print_endline " Hello, World! " in let* () = U . print_endline " What is your name? " in let * name = U . read_line () in U . print_endline ( " Hello " ^ name )
And we can handle it using Env.run :
# Env . run object inherit a_console inherit a_logger end comp ; ; [ Debug ] Start teletype example Hello , World ! What is your name ? Hello Pierre - : unit = ()
Some prefer to pass handlers explicitly to avoid relying on binding operators. Personally, I really like that the operators make it explicit whether I’m dealing with a pure expression or one that requires dependencies, and I find that binding operators make the code readable and easy to reason about.
Under the hood
Looking at the signature of the Env module, many will notice that it is a Reader Monad (a specialization of ReaderT transformed with the Identity monad), which is sometimes also called a Kleisli . In practice, this allows us to be parametric over the normal form of our expressions. For simplicity in the examples, I’ve minimized indirection, but it is entirely possible to project our results into a more refined monad, defining a richer runtime (and potentially supporting continuation control, effect interleaving, etc.).
To conclude
Some might be surprised—indeed, I’ve fairly closely applied the Boring Haskell philosophy. I literally used objects for dependency injection (the extra-classical approach to DI) and rely on a ReaderT to access my environment on demand, which is a very common approach in the Haskell community.
The only idea here, which isn’t particularly novel, is to use subtyping relationships to statically track the required dependencies, and rely on inheritance to arbitrarily compose handlers. From my point of view, this drastically reduces the need to stack transformers while still keeping modularity and extensibility. By relying on a different normal form than the identity monad, it’s possible to achieve results that are surprisingly pleasant to use and safe, as demonstrated by Kyo in the Scala world. In OCaml, the richness of the object model (and its structural capabilities) is a real advantage for this kind of encoding, allowing a drastic reduction of the boilerplate needed to manage multiple sets of dependencies.
In practice, I’ve found this approach effective for both personal and professional projects (and, importantly, very easy to explain). One limitation is the inability to eliminate dependencies when they are partially interpreted in ways other than by partial function application. Still, the encoding remains neat for at least three reasons:
It allows us to statically track dependencies in the context of dependency injection.
It makes it easy to write testable programs by providing handlers adapted for unit tests.
It provides another example showing that OCaml’s objects are really powerful and can offer fun solutions to well-worn problems.
In the not-so-distant future, we might even imagine providing handlers more lightly using Modular Implicits.
Thank you for reading (if you made it this far), and a special thanks to Xavier Van de Woestyne for his article that inspired me to write this one, and to Jonathan Winandy for showing me Kyo and helping me rephrase some sentences.
Bibliography