Mixing JavaScript Runtimes

Published on: Fri Dec 16 2022

Introduction

The year 2022 has definitely been the year of TypeScript for me. I’m at a point where I use the T3 Stack pretty much exclusively for any hobby and professional project. The only modifications I make are usually wrapping it in a Turborepo and switching out NextJS & React for Astro & SolidJS whenever the use case allows it. But Tailwind, Prisma, tRPC and TypeScript have been part of every project from the last 12 months, big and small.

I’d heard of Deno when it was first announced back in 2018 by Ryan Dahl, the creator of Node, in his infamous talk 10 Things I Regret About Node.js. I immediately took a look and was in love with the vision. Up to that point I was already using Node, but I was still stuck in the MongoDB MERN Stack hype and thought that “SQL was dead” and other nonsense…

In the last 6 months, when building a really big software in a Full-Stack TypeScript Turborepo, I’ve definitely noticed that I hit a kind of Node / Web Development fatique. And to be fair this wasn’t only because of Node, but also because of React (damn you useEffect). Nevertheless I was getting incredibly annoyed spending my time settings up tsconfigs, package.json, managing dependencies between packages and trying to integrate digitalocean serverless functions into the repository (spoiler: don’t).

Fed up and always with the thought of “This is such a waste of precious time” in the back of my mind, I decided to work on a hobby project (a battleship game). I had already made some progress, but I was struggling with a lot of the web socket logic. I decided to switch out React for Solid, in the hopes that it would make the develoment and debugging easier. I then saw, that the Deno YouTube channel released a video where they showed how to use Vite with Deno, so I decided to also toss out Node for Deno, just to see how it would work out. And what can I say. After the initial struggle of making a web socket http server in Deno everything went smoothly and the DX (Developer Experience) was a 10x improvement over Node & React. Honestly, it had been a long time, since I felt that excited about writing code.

The Problem

When I then read that Deno had come out with official support for 80-90% of npm packages, I was super hyped. Only to quickly realize, that the ”npm:” prefix did not apply to local npm packages, as is the case in a monorepository. That means that although I’ve got a folder with a package.json and node code in my packages, the module resolution between Node and Deno is very different, so it wouldn’t allow me to import the package directly in Deno. In node you install a package with npm install ... (or any other package manager of your choice), which caches the code of that package in the node_modules folder. So when you import a package in node with import x from "npm-package" it will fetch the code from the folder inside of node_modules. What turbo does (among other things of course), is symlink local packages into that node_modules directory, which allows us to use and import them like regular npm packages across the whole code base.

So if I want to use the package node-utils in another package and I add it to the dependencies in the package.json like so:

{
  ...
  "dependencies": {
    "node-utils": "*"
  }
}

If I run npm install, it will result in the following node_modules directory:

A screenshot of the node_modules folder

You can see that the “node-utils” package is symlinked to the local package. Now I can just write import x from "node-utils" in the package where I added the dependency, and it works.

Problems arise when you want to make Deno work with my Node packages. You would think that it wouldn’t be that much more work to make it available to us, but I won’t judge a feature that I don’t understand on a deeper technical level. So as long as we can’t import local node packages just like we can with packages in the npm registry, I’ll give you my way around it that works today.

Using Node in Deno

The Setup

You can find all of the code in this repo: daniellionel01/mixing-js-runtimes

To go beyond console logging hello world from Node in Deno, we will work with a more ellaborate codebase.

We’ll start with a Turborepo consisting of two packages: node-utils and node-code.

node-utils can export a bunch of small utilities. To keep things simple, we’ll only have a math utility with a single function.

Here’s some code:

// packages/node-utils/math.ts
export function add (x: number, y: number): number {
  return x + y
}

// packages/node-utils/index.ts
export * from "./math.ts"

node-code is a package that we’ll use to demonstrate that we can call our code from both runtimes easily. It uses code from our node-utils package and the npm package chalk, which is very useful for styling terminal output.

// packages/node-code/package.json
{
  "name": "@pkg/node-code",
  "version": "1.0.0",
  "main": "index.ts",
  "types": "index.ts",
  "scripts": {
    "dev": "tsx main.ts"
  },
  "dependencies": {
    "@pkg/node-utils": "*",
    "chalk": "^5.2.0"
  }
}

// packages/node-code/index.ts
import { add } from "@pkg/node-utils"
import chalk from "chalk"

export function runMyCode() {
  console.log("Running node code...")
  console.log("3 + 5 = ", add(3, 5))
}

// packages/node-code/main.ts
import { runMyCode } from "./index.ts"

runMyCode()

So far so good. We can now call our dev script in node-utils and we’ll get the following output

$ yarn workspace node-code dev
Running node code...
3 + 5 = 8

It’ll be a little more colorful when you run it on your local machine. But ok, we’ve got very basic Node code working. So let’s call this code from Deno.

Getting Deno involved

To demonstrate our case, we’ll create a new package deno-code.

// packages/deno-code/deps.ts
import chalk from "npm:chalk@5"
import * as utils from "@pkg/node-utils"
import * as node from "@pkg/node-code"

export { chalk, utils, node }

// packages/deno-code/main.ts
import { chalk, utils, node } from "./deps.ts"

console.log(chalk.yellow.bold("Running deno code..."))
console.log("8 + 2 =", utils.add(8, 2))

node.runMyCode()

// deno.json
{
  "tasks": {
    "cache": "deno cache packages/deno-code/deps.ts",
    "dev": "deno run -A packages/deno-code/main.ts"
  },
  "importMap": "./import_map.json"
}

// import_map.json
{
  "imports": {
    "@pkg/node-utils": "./packages/node-utils/index.ts",
    "@pkg/node-code": "./packages/node-code/index.ts",
    "chalk": "npm:chalk@5"
  }
}

Et voila! Yeah that’s basically it. We can now do the following:

$ deno task dev
Running deno code...
8 + 2 = 10
Running node code...
3 + 5 = 8

Explanation

So to explain what we did up there a little more, let’s go through the thought process.

The trick is to keep the Node code Deno compatible, which is not very difficult, since both runtimes use standard TypeScript (we’ll talk about caveats in the next section).

The last step is to make any imports to local or npm packages in Node work in Deno. That’s where import_map.json comes in. In this file we can create straight forward mappings between import names and the file or package they refer to - kind of like a macro. And since we don’t need fancy node_module symlink magic in Deno, we can just use the direct file path to the entry file of the package. In node we don’t have the “npm:*” prefix, so we just add it in the mapping.

Using Deno in Node

Now this is great. We can use our existing Node code in our new Deno code! But… what if I need to use parts of my Deno code in my existing Node code?

I’ve got you covered.

dnt

denoland/dnt is a package to transpile Deno code automatically to Node compatible code.

We can transpile it by making a build script that does all the work for us.

// scripts/build_npm.ts
import { build, emptyDir } from "https://deno.land/x/dnt@0.32.0/mod.ts"
import { join } from "https://deno.land/std@0.168.0/path/mod.ts"

await emptyDir("./npm/deno-code")

const dirname = new URL(".", import.meta.url).pathname

const entry = join(dirname, "../packages/deno-code/main.ts")
const map = join(dirname, "../import_map.json")

await build({
  importMap: map,
  typeCheck: false,
  test: false,
  entryPoints: [entry],
  outDir: "./npm/deno-code",
  shims: {
    deno: true
  },
  packageManager: "yarn",
  package: {
    name: "@npm/deno-code",
    version: Deno.args[0],
    description: "Deno code"
  }
})

{
  "tasks": {
    ...
    "npm": "deno run -A scripts/build_npm.ts 1.0.0"
  },
  ...
}

Running deno task npm will create a directory npm/deno-code that we can import and use in our Node code.

Using it

Let’s create a new package deno-from-node.

// packages/deno-from-code/package.json
{
  "name": "@pkg/deno-from-node",
  "private": true,
  "version": "1.0.0",
  "main": "index.ts",
  "types": "index.ts",
  "scripts": {
    "dev": "tsx index.ts"
  },
  "dependencies": {
    "@npm/deno-code": "*"
  }
}

// packages/deno-from-node.index.ts
import "@npm/deno-code"

Now we can do the following:

$ yarn workspace @pkg/deno-from-node dev
Running deno code...
8 + 2 = 10
Running node code...
3 + 5 = 8

Excellent! We have achieved ultimate compatibility between Node and Deno code.

You could automate this even further by iterating all Deno packages programmatically (for example all directories in the packages/* directory that don’t contain a package.json or have a Deno specific file, maybe deps.ts) and build the corresponding package.

You should probably also not commit the npm package and add it to a .gitignore.

Inception

Alright, but… what if I want to use a Node module in my Deno code, but that Node module in turn uses other Deno code?

The point is, that it all just boils down to managing imports. So if I wanted to use that deno-from-node package in another Deno package, I would just have to add the @npm/deno-from-node to the import_map.json.

Caveats

There a couple of ways to easily mess up compatibility between NodeJS and Deno code.

No file extension in import path

import x from "./some-file"

This works fine in Node, but in Deno the full URL of any file or module is required to work reliably. So you’ll import it like this, making it still work in Node and also in Deno:

import x from "./some-file.ts"

Using Node specific APIs

When you’re writing a module in Node, you might use node specific APIs, such as __dirname or process.

To still be able to use them in Deno, you will need to provide polyfills for those variables. To make global objects in Deno, you’ll use the window object.

declare global {
  interface Process {
    pid: number
  }
  interface Window {
    __dirname: string;
    process: Process;
  }
}

window.process = { pid: Deno.pid }
window.__dirname = new URL('.', import.meta.url).pathname;

Just know that this isn’t always an elegent solution and you are bound to run into some ugly code. After all, all of this runtime mingeling is inherently hacky.

Where am I?

This is a useful function to detect which runtime you’re code is operating in. You might be able to do some funky dynamic imports here if you’re struggling with very specific node or deno imports or APIs.

export function isNode(): boolean {
  return "process" in globalThis && "global" in globalThis;
}

Closing

In closing I think it’s very exciting that we can extend certain parts of a big software in Deno. I’ll definitely get a lot of use out of this technique. Obviously in the real world things are going to get messier than our example with such limited logic and cross dependencies. But as long as you understand the mechanisms behind module resolution, import maps and such, you should be able to figure it out.