GoKawiilSomewhere in 2019 I started a project that aimed to bring some of Clojure features to Lua runtime - fennel-cljlib. It was a library for Fennel that implemented a basic subset of clojure.core namespace functions and macros. My goal was simple - I enjoy working with Clojure, but I don’t use it for hobby projects, so I wanted Fennel to feel more Clojure-like, besides what it already provides for that.
This library grew over the years, I implemented lazy sequences, added immutability, made a testing library, inspired by clojure.test and kaocha, and even made a port of clojure.core.async . It was a passion project, I almost never used it to write actual software. One notable exception is fenneldoc - a tool for documentation generation for Fennel libraries. And I haven’t seen anyone else use it for a serious project.
The reason for that is simple - it was an experiment. Corners were cut, and Fennel, being a Clojure-inspired lisp is not associated with functional programming the same way Clojure is. As a matter of fact, I wouldn’t recommend using this library for anything serious… yet.
Recently, however, I started a new project: ClojureFnl. This is a Clojure-to-Fennel compiler that uses fennel-cljlib as a foundation. It’s still in early days of development, but I’ve been working on it for a few months in private until I found a suitable way to make things work in March. As of this moment, it is capable of compiling most of .cljc files I threw at it, but running the compiled code is a different matter. I mean, it works to some degree, but the support for standard library is far from done.
;; Welcome to ClojureFnl REPL ;; ClojureFnl v0.0.1 ;; Fennel 1.6.1 on PUC Lua 5.5 user=> ( defn prime? [ n ] ( not ( some zero? ( map #( rem n % ) ( range 2 n ))))) # <function : 0 x89ba7c550> user=> ( for [ x ( range 3 33 2) :when ( prime? x )] x ) (3 5 7 11 13 17 19 23 29 31) user=>
However, there was a problem.
My initial implementation of immutable data structures in the itable library had a serious flaw. The whole library was a simple hack based on the copy-on-write approach and a bunch of Lua metatables to enforce immutability. As a result, all operations were extremely slow. It was fine as an experiment, but if I wanted to go further with ClojureFnl, I had to replace it. The same problem plagued immutableredblacktree.lua, an implementation of a copy-on-write red-black tree I made for sorted maps. It did a full copy of the tree each time it was modified.
For associative tables it wasn’t that big of a deal - usually maps contain a small amount of keys, and itable only copied levels that needed to be changed. So, if you had a map with, say, ten keys, and each of those keys contained another map with ten keys, adding, removing or updating a key in the outer map meant only copying these ten keys - not the whole nested map. I could do that reliably, because inner maps were immutable too.
But for arrays the story is usually quite different. Arrays often store a lot of indices, and rarely are nested (or at least not as often as maps). And copying arrays on every change quickly becomes expensive. I’ve mitigated some of the performance problems by implementing my version of transients, however the beauty of Clojure’s data structures is that they’re quite fast even without this optimization.
Proper persistent data structures
... continue reading