GoKawiilA long, long time ago, mighty matklad used to write great posts about how Rust tooling works. Those were great times, but alas, the last rust-analyzer blog post dates 2023
I am no matklad, but I'm building Rust Glancer, an experimental Rust LSP, for quite a while now. It's probably the most interesting and ambitious project I've worked on, and I want to share some things I've learned while working on it.
This will be a (hopefully coherent) story about how Rust LSPs work, from the perspective of both rust-analyzer and Rust Glancer: how things that seem easy turn out to be hard, things that seem hard turn out to be even harder, and things I didn't expect to exist at all somehow do.
Obviously, a single blog post can't cover everything, this will be a very technical but still architectural overview rather than a deep dive into any particular topic brought up along the way. Those will come as separate posts, granted I won't be lazy.
Otherwise, be ready for many anecdotal chapters that have one thing in common: building an LSP means having to produce useful answers from partial information.
Disclaimer: I am no expert in building LSPs, and the purpose of this post is to make readers interested in internals and caveats of LSPs rather than give an unambiguous and formal design overview. I intentionally try not to use compiler jargon, and use approximate phrasing in many places to focus on the overall meaning rather than precision. There are plenty of links in the post to more detailed/precise sources, and I recommend checking them out! Also, I have read a lot of rust-analyzer code before and during preparation of this post, but I'm no rust-analyzer maintainer; if I got some things wrong -- sorry.
Where does an LSP start?
LSP server has two opposite ends: the server that implements the Language Server Protocol (as in, "there is this thing I can send requests to and receive well-formed responses") and the actual state that we want to serve (as in, "the sent queries actually do what they need to do and operate over some kind of indexed state"). The first seems to be a solved problem, right? Especially given that tower-lsp-server exists. Welp, not really. Let's start there, and then we will gradually get to indexing once we actually need it.
The LSP begins with a client sending an initialize request which requires you to initialize the LSP (huh). Before you respond, client won't do anything. Once you do, it sends an initialized notification, and all is good, and LSP communication starts.
The problem is: when do you respond to this request? Once the server starts, you have nothing. You don't know anything about the project, and only when you receive this request you will know what's the codebase we're talking about. And in order to actually answer any queries, we need to "index™" it. We don't know what indexing means yet, but it's certainly a lot of work.
... continue reading