Sadiq's Knowledge Vaults

Language Server Protocol

5 min read

It defines a protocol used between an editor and a language server that provides language features like auto-complete, go to definition, etc…
It’s created by Microsoft, along side collaboration with RedHat and Codenvy
When we try to understand it in the context of a code editor, the LSP is process wither a binary like rust analyzer or go PLS or perhaps something executed by the language runtime like TS Server that is executed by node, these communicate with the client, via a process (plugin in most cases) spawn by the client and this process will be own by the editor for lifetime until the session persist(until the editor is closed)
Q Why this LSP exists ?
A If we have M languages and N code editors then every language should support the N code editors, then in total we need to write M x N server implementations
Now LSP is a standard way to write the language servers that will be supported by all languages and all code editors, now we just need to write M + N server implementations

Here are some examples of the communication between Client and Server

  1. Initialization Phase (Client → Server → Client)
  • Client → Server: initialize
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "initialize",
  "params": {
    "processId": 12345,
    "rootUri": "file:///home/user/project",
    "capabilities": {
      "textDocument": {
        "completion": { "dynamicRegistration": true }
      }
    }
  }
}
  • Server → Client: initialize result
{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "capabilities": {
      "textDocumentSync": 2,
      "definitionProvider": true,
      "completionProvider": { "triggerCharacters": ["."] }
    }
  }
}
  • Client → Server: initialized
{
  "jsonrpc": "2.0",
  "method": "initialized",
  "params": {}
}
  1. Completion Request (Auto-Complete)
  • Client → Server: text document/completion
{
  "jsonrpc": "2.0",
  "id": 2,
  "method": "textDocument/completion",
  "params": {
    "textDocument": { "uri": "file:///main.rs" },
    "position": { "line": 5, "character": 10 }
  }
}
  • Server → Client: completion list
{
  "jsonrpc": "2.0",
  "id": 2,
  "result": {
    "isIncomplete": false,
    "items": [
      {
        "label": "println!",
        "kind": 3,
        "detail": "macro_rules! println",
        "insertText": "println!($0)"
      },
      {
        "label": "print!",
        "kind": 3,
        "detail": "macro_rules! print",
        "insertText": "print!($0)"
      }
    ]
  }
}
  1. Go To Definition
  • Client → Server: text document/definition
{
  "jsonrpc": "2.0",
  "id": 3,
  "method": "textDocument/definition",
  "params": {
    "textDocument": { "uri": "file:///app/main.ts" },
    "position": { "line": 10, "character": 16 }
  }
}
  • Server → Client: definition result
{
  "jsonrpc": "2.0",
  "id": 3,
  "result": {
    "uri": "file:///app/utils.ts",
    "range": {
      "start": { "line": 2, "character": 6 },
      "end":   { "line": 2, "character": 15 }
    }
  }
}
  1. Hover Information
  • Client → Server: text document/hover
{
  "jsonrpc": "2.0",
  "id": 4,
  "method": "textDocument/hover",
  "params": {
    "textDocument": { "uri": "file:///index.js" },
    "position": { "line": 8, "character": 12 }
  }
}
  • Server → Client: hover result
{
  "jsonrpc": "2.0",
  "id": 4,
  "result": {
    "contents": [
      {
        "language": "javascript",
        "value": "function greet(name: string): string"
      },
      "Returns a greeting message."
    ]
  }
}
  1. Publish Diagnostics (After Opening File)
  • Client → Server: text document/did open
{
  "jsonrpc": "2.0",
  "method": "textDocument/didOpen",
  "params": {
    "textDocument": {
      "uri": "file:///main.go",
      "languageId": "go",
      "version": 1,
      "text": "package main\nfunc main() { fmt.Println(\"hello\") }"
    }
  }
}
  • Server → Client: text document/publish diagnostics
{
  "jsonrpc": "2.0",
  "method": "textDocument/publishDiagnostics",
  "params": {
    "uri": "file:///main.go",
    "diagnostics": [
      {
        "range": {
          "start": { "line": 1, "character": 22 },
          "end":   { "line": 1, "character": 25 }
        },
        "message": "undefined: fmt",
        "severity": 1
      }
    ]
  }
}
  1. Formatting Request
  • Client → Server: text document/formatting
{
  "jsonrpc": "2.0",
  "id": 5,
  "method": "textDocument/formatting",
  "params": {
    "textDocument": { "uri": "file:///bad.py" },
    "options": { "tabSize": 4, "insertSpaces": true }
  }
}
  • Server → Client: formatting result
{
  "jsonrpc": "2.0",
  "id": 5,
  "result": [
    {
      "range": {
        "start": { "line": 0, "character": 0 },
        "end":   { "line": 2, "character": 0 }
      },
      "newText": "def main():\n    print(\"Hello\")\n"
    }
  ]
}
  1. Shutdown Sequence
  • Client → Server: shutdown
{
  "jsonrpc": "2.0",
  "id": 6,
  "method": "shutdown",
  "params": null
}
  • Server → Client: shutdown result
{
  "jsonrpc": "2.0",
  "id": 6,
  "result": null
}
  • Client → Server: exit
{
  "jsonrpc": "2.0",
  "method": "exit"
}

Tree Sitter

LSP is for understanding whole project structure, packages installed and provide definitions, references, completions,… But Tree Sitter focus on one file at a time and get quick and incremental feedback from that file, and it’s not a compiler nor a interpreter, it’s a library which only understands text to be more specific it’s a parser generator tool which takes the grammar(input) and transforms it into a parser and will be embedded into the application which will be running the tree sitter
It also responsible to generate the Concrete Syntax Tree(CST) also called as “full syntax tree” which is a highly simplified AST, which is used to ask questions regarding the code(see how Cursor uses it). Here are some aspects why we use Tree Sitter

  1. Incremental - Just compute the changes nodes in the tree, rather computing whole tree again
  2. Error Recovery - It means, it knows when to show the error, for example when we’re typing the code, if it acts fast then the whole line will be a error, rather it incrementally shows the error bracket by bracket and at last after our typing completes, it shows the errors on the whole
  3. Queries - Store in the form of AST that let the editor to perform certain operations

Here is the flow via Tree Sitter Load grammar → parse source text into syntax tree → on edits update tree incrementally → run queries over tree to find / manipulate nodes → feed results into editor/tool UI

Graph View