Deno shims for Node.js APIs - Deep Dive

Overview - Deno shims for Node.js APIs

What is it?

Deno shims for Node.js APIs are pieces of code that let programs written for Node.js run in Deno. They act like translators, making Node.js functions and modules work in Deno's environment. This helps developers use familiar Node.js features while benefiting from Deno's modern design. Shims fill the gaps where Deno and Node.js differ.

Why it matters

Without these shims, developers would struggle to move their Node.js code to Deno because many Node.js APIs don't exist in Deno. This would mean rewriting large parts of code, wasting time and effort. Shims make it easier to adopt Deno by reusing existing code, speeding up development and reducing bugs. They help bridge two worlds, making the transition smooth.

Where it fits

Before learning about Deno shims, you should understand basic JavaScript, Node.js APIs, and Deno's runtime environment. After this, you can explore how to migrate Node.js projects to Deno or how to write cross-platform code that works in both runtimes.

Mental Model

Core Idea

Deno shims for Node.js APIs are like adapters that let Node.js code plug into Deno's environment by mimicking missing Node.js features.

Think of it like...

Imagine you have a phone charger made for one country’s plug shape, but you want to use it in another country with different sockets. A shim is like a plug adapter that fits your charger into the new socket without changing the charger itself.

┌───────────────┐       ┌───────────────┐       ┌───────────────┐
│ Node.js Code  │──────▶│ Node.js APIs  │──────▶│ Node.js Runtime│
└───────────────┘       └───────────────┘       └───────────────┘
         │                      ▲                       ▲
         │                      │                       │
         │                      │                       │
         ▼                      │                       │
┌─────────────────────┐        │                       │
│ Deno Shims for Node │────────┘                       │
│ APIs (Adapters)     │                                │
└─────────────────────┘                                │
         │                                            │
         ▼                                            │
┌───────────────┐       ┌───────────────┐             │
│ Deno Code     │──────▶│ Deno APIs     │─────────────┘
└───────────────┘       └───────────────┘

Build-Up - 7 Steps

1

Foundation

Understanding Deno and Node.js Basics

Concept: Learn what Deno and Node.js are and how they run JavaScript outside the browser.

Node.js is a popular runtime that lets you run JavaScript on servers. It has many built-in APIs like file system access and networking. Deno is a newer runtime that also runs JavaScript and TypeScript but with a different design focusing on security and modern features. They both run JavaScript but have different built-in tools and ways to do things.

Result

You know that Node.js and Deno are similar but have different built-in APIs and security models.

Understanding the basic differences between Deno and Node.js is key to seeing why shims are needed to connect their worlds.

2

Foundation

What Are APIs and Why They Differ

Concept: APIs are sets of tools and functions a runtime provides to interact with the system. Node.js and Deno have different APIs.

Node.js offers APIs like 'fs' for file system, 'path' for file paths, and 'process' for environment info. Deno has its own APIs with different names and behaviors, often using web standards. For example, Deno uses 'Deno.readFile' instead of Node's 'fs.readFile'. This means code written for Node.js APIs won't run directly in Deno.

Result

You understand that Node.js and Deno have different APIs, so code using Node.js APIs needs adjustment to run in Deno.

Knowing that APIs differ explains why direct code reuse is hard and why shims are necessary.

3

Intermediate

Introducing Shims: The Compatibility Layer

🤔Before reading on: do you think shims rewrite Node.js code or just provide missing APIs? Commit to your answer.

Concept: Shims provide missing Node.js APIs in Deno without changing the original Node.js code.

A shim is a piece of code that implements Node.js APIs using Deno's features. It acts as a bridge so Node.js code can call familiar functions, but under the hood, Deno does the work. For example, a shim might implement 'fs.readFile' by calling Deno's file reading functions. This way, Node.js code runs in Deno with minimal changes.

Result

Node.js code using common APIs can run in Deno because shims fill the missing parts.

Understanding that shims mimic APIs rather than rewrite code helps grasp their role as adapters, not translators.

4

Intermediate

Common Node.js APIs Shimmable in Deno

🤔Before reading on: which Node.js APIs do you think are easiest or hardest to shim in Deno? Commit to your answer.

Concept: Some Node.js APIs are easier to shim because Deno has similar features; others are harder due to design differences.

APIs like 'fs' (file system), 'path' (file paths), and 'buffer' (binary data) are commonly shimmed because Deno supports similar operations. However, APIs tied to Node.js internals like 'process' or native modules are harder or impossible to shim fully. Shims often cover the most used APIs to maximize compatibility.

Result

You know which Node.js APIs can be shimmed well and which are challenging, guiding realistic expectations.

Knowing the limits of shims prevents frustration and helps plan code migration or compatibility layers effectively.

5

Intermediate

How to Use Deno Shims in Your Project

Concept: Learn how to add and configure shims so your Node.js code runs in Deno.

Deno provides official and community shims as modules you can import. You add them at the start of your code to patch global objects or provide missing modules. For example, importing 'std/node/fs.ts' adds Node.js 'fs' API support. You may need to adjust import paths and permissions. This setup lets your Node.js code run with minimal changes.

Result

Your Node.js code runs in Deno using shims, reducing rewrite effort.

Knowing how to integrate shims is crucial to practical migration and leveraging Deno's features safely.

6

Advanced

Limitations and Performance of Shims

🤔Before reading on: do you think shims run as fast as native Node.js APIs or slower? Commit to your answer.

Concept: Shims add a compatibility layer that can affect performance and have limitations compared to native APIs.

Because shims translate Node.js API calls to Deno's APIs, there is some overhead. Also, some Node.js features rely on native C++ bindings not available in Deno, so shims can't fully replicate them. This means some code may run slower or behave differently. Developers must test and optimize critical parts or rewrite them for Deno.

Result

You understand that shims are a tradeoff between compatibility and performance.

Recognizing shim limitations helps avoid surprises in production and guides when to rewrite code.

7

Expert

Internals: How Shims Patch Global Environment

🤔Before reading on: do you think shims replace Node.js globals or add missing ones in Deno? Commit to your answer.

Concept: Shims work by patching or adding global objects and modules so Node.js code finds expected APIs in Deno.

Under the hood, shims modify Deno's global scope by defining objects like 'process', 'Buffer', or 'require' that Node.js code expects. They also provide module implementations that export Node.js API functions. This patching happens at runtime before user code runs. It tricks Node.js code into thinking it runs in Node.js, while Deno handles the actual operations.

Result

You see how shims create a fake Node.js environment inside Deno.

Understanding this patching mechanism explains how shims enable compatibility without changing user code.

Under the Hood

Deno shims intercept calls to Node.js APIs by defining global variables and modules that mimic Node.js behavior. They use Deno's native APIs internally to perform operations like file access or networking. This interception happens by loading shim modules early, which patch or add missing objects and functions. The runtime then routes Node.js API calls through these shim implementations, allowing Node.js code to run unmodified.

Why designed this way?

This design allows maximum code reuse without rewriting Node.js code. Instead of changing user code, shims adapt the environment, making migration easier. Alternatives like transpiling code or rewriting APIs would be more complex and error-prone. The shim approach balances compatibility, maintainability, and performance.

┌─────────────────────────────┐
│ User Node.js Code           │
│ (calls Node.js APIs)        │
└─────────────┬───────────────┘
              │
              ▼
┌─────────────────────────────┐
│ Deno Shim Modules           │
│ (patch globals, provide APIs)│
└─────────────┬───────────────┘
              │
              ▼
┌─────────────────────────────┐
│ Deno Native APIs            │
│ (file system, network, etc.)│
└─────────────────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Do shims rewrite your Node.js code to run in Deno? Commit yes or no.

Common Belief: Shims automatically convert all Node.js code to Deno code behind the scenes.

Reality: Shims do not rewrite code; they provide missing APIs so Node.js code can run as-is in Deno.

Why it matters: Believing shims rewrite code leads to expecting perfect compatibility and no manual changes, causing confusion when some code still breaks.

Quick: Are all Node.js APIs fully supported by Deno shims? Commit yes or no.

Common Belief: Deno shims cover every Node.js API perfectly.

Reality: Shims cover many common APIs but not all, especially those relying on native Node.js internals or C++ addons.

Why it matters: Assuming full coverage causes unexpected runtime errors or missing features in production.

Quick: Do shims run Node.js code at native Node.js speed? Commit yes or no.

Common Belief: Shims run Node.js code just as fast as Node.js itself.

Reality: Shims add overhead and can be slower than native Node.js APIs due to translation layers.

Why it matters: Ignoring performance impact can cause slow applications and poor user experience.

Quick: Can shims fix security differences between Node.js and Deno? Commit yes or no.

Common Belief: Shims make Deno behave exactly like Node.js, including security model.

Reality: Shims do not change Deno's security model; Deno remains more secure by default.

Why it matters: Misunderstanding this can lead to security assumptions that are incorrect, risking vulnerabilities.

Expert Zone

1

Some shims selectively patch only parts of Node.js APIs to avoid conflicts with Deno's native features, requiring deep knowledge to customize.

2

Shims must carefully handle asynchronous behavior differences between Node.js and Deno to avoid subtle bugs in event loops and timers.

3

Advanced shims can dynamically detect environment capabilities to provide optimized implementations, balancing compatibility and performance.

When NOT to use

Avoid shims when your project relies heavily on native Node.js addons or low-level internals not replicable in Deno. Instead, rewrite critical parts using Deno's native APIs or use cross-platform libraries designed for both runtimes.

Production Patterns

In production, teams use shims to gradually migrate large Node.js codebases to Deno, running tests to identify unsupported APIs. They combine shims with conditional code to handle runtime differences and optimize performance-critical paths by rewriting them natively.

Connections

Polyfills in Web Development

Shims are similar to polyfills that add missing web browser features to older browsers.

Understanding polyfills helps grasp how shims provide missing APIs to enable compatibility without rewriting code.

Adapter Design Pattern in Software Engineering

Shims implement the adapter pattern by converting one interface (Node.js APIs) to another (Deno APIs).

Recognizing shims as adapters clarifies their role in bridging incompatible interfaces elegantly.

Language Interpreters and Virtual Machines

Shims act like a lightweight virtual machine layer that interprets Node.js API calls in Deno's runtime.

Knowing interpreter design helps understand the runtime patching and call routing shims perform.

Common Pitfalls

#1 Assuming all Node.js code runs without changes in Deno with shims.

Wrong approach:

import 'https://deno.land/std/node/fs.ts';
// Run complex native addon code without changes

Correct approach:

Use shims for common APIs but rewrite or replace native addons with Deno-compatible code.

Root cause:

Misunderstanding shim coverage and limitations leads to runtime failures.

#2 Not granting required permissions in Deno when using shims.

Wrong approach:

deno run app.ts
// No permissions flags

Correct approach:

deno run --allow-read --allow-net app.ts
// Permissions granted for file and network access

Root cause:

Ignoring Deno's security model causes permission errors even if shims are correct.

#3 Importing shims after user code runs.

Wrong approach:

import './app.ts';
import 'https://deno.land/std/node/fs.ts';
// Shims loaded too late

Correct approach:

import 'https://deno.land/std/node/fs.ts';
import './app.ts';
// Shims loaded first to patch environment

Root cause:

Loading order misunderstanding prevents shims from patching globals before use.

Key Takeaways

Deno shims for Node.js APIs let you run Node.js code in Deno by providing missing APIs as adapters.

They patch Deno's environment at runtime to mimic Node.js globals and modules without rewriting your code.

Shims cover many common Node.js APIs but have limits, especially with native addons and internals.

Using shims requires understanding Deno's security model, permissions, and loading order to avoid errors.

Shims are a practical bridge for migration but may impact performance and sometimes require rewriting critical code.

Practice

1. What is the main purpose of Deno shims for Node.js APIs?

easy

A. To convert Deno code into Node.js code automatically

B. To allow using Node.js modules and APIs in Deno environment

C. To replace Deno's runtime with Node.js runtime

D. To compile TypeScript code into JavaScript

Solution

1. Step 1: Understand what shims do

   Deno shims provide compatibility layers that mimic Node.js APIs inside Deno.

2. Step 2: Identify the purpose of these shims

   The main goal is to let developers use Node.js modules and APIs in Deno without rewriting code.

3. Final Answer:

   To allow using Node.js modules and APIs in Deno environment -> Option B

4. Quick Check:

   Deno shims = Node.js API compatibility [OK]

Hint: Shims let Deno run Node.js code easily [OK]

Common Mistakes:

• Thinking shims convert code automatically
• Confusing runtime replacement with API compatibility
• Assuming shims compile TypeScript

2. Which is the correct way to import the Node.js 'path' module shim in Deno?

easy

A. import path from 'https://deno.land/std/node/path.ts';

B. import { path } from 'https://deno.land/std/node/path.ts';

C. import * as path from 'https://deno.land/std/node/path.ts';

D. import path from 'node:path';

Solution

1. Step 1: Check Deno shim import style

   Deno shims for Node.js modules are imported as namespaces using * as syntax.

2. Step 2: Match the correct import statement

   Only import * as path from 'https://deno.land/std/node/path.ts'; correctly imports the shim as a namespace.

3. Final Answer:

   import * as path from 'https://deno.land/std/node/path.ts'; -> Option C

4. Quick Check:

   Namespace import = import * as [OK]

Hint: Use 'import * as' for Node.js shims in Deno [OK]

Common Mistakes:

• Using default import syntax which is invalid here
• Trying named import when module exports namespace
• Using 'node:path' which is Node.js native, not Deno shim

3. What will be the output of this Deno code using Node.js 'path' shim?

import * as path from 'https://deno.land/std/node/path.ts';
console.log(path.basename('/foo/bar/baz.txt'));

medium

A. 'baz.txt'

B. '/foo/bar/baz.txt'

C. 'baz'

D. SyntaxError

Solution

1. Step 1: Understand path.basename behavior

   The basename function returns the last part of a path, including extension.

2. Step 2: Apply to given path string

   For '/foo/bar/baz.txt', the basename is 'baz.txt'.

3. Final Answer:

   'baz.txt' -> Option A

4. Quick Check:

   path.basename('/foo/bar/baz.txt') = 'baz.txt' [OK]

Hint: basename returns last filename with extension [OK]

Common Mistakes:

• Expecting the full path as output
• Confusing basename with dirname
• Syntax errors from wrong import (not here)

4. Identify the error in this Deno code using Node.js 'fs' shim:

import { readFileSync } from 'https://deno.land/std/node/fs.ts';
const data = readFileSync('./file.txt', 'utf8');
console.log(data);

medium

A. readFileSync does not accept encoding as second argument in Deno shim

B. The import path is incorrect, should be 'https://deno.land/std/node/fs.ts'

C. readFileSync is asynchronous and cannot be used like this

D. No error, code works as expected

Solution

1. Step 1: Verify the import

   Named import { readFileSync } from Deno's std/node/fs.ts is valid as functions are named exports.

2. Step 2: Check readFileSync arguments

   The function accepts encoding as a string ('utf8') as second argument, matching Node.js API.

3. Final Answer:

   No error, code works as expected -> Option D

4. Quick Check:

   readFileSync(path, 'utf8') valid [OK]

Hint: Named imports and string encoding work in Deno fs shim [OK]

Common Mistakes:

• Believing encoding must always be an options object
• Confusing with Deno's native synchronous APIs
• Overlooking that fs shim closely mimics Node.js

5. You want to use the Node.js 'events' module shim in Deno to create an event emitter. Which code snippet correctly creates an emitter and listens for a 'ping' event?

hard

A. import * as events from 'https://deno.land/std/node/events.ts';
const emitter = new events.EventEmitter();
emitter.addListener('ping', () => console.log('pong'));
emitter.emit('ping');

B. import events from 'https://deno.land/std/node/events.ts';
const emitter = new events.EventEmitter();
emitter.on('ping', () => console.log('pong'));
emitter.emit('ping');

C. import EventEmitter from 'https://deno.land/std/node/events.ts';
const emitter = new EventEmitter();
emitter.on('ping', () => console.log('pong'));
emitter.emit('ping');

D. import { EventEmitter } from 'node:events';
const emitter = new EventEmitter();
emitter.on('ping', () => console.log('pong'));
emitter.emit('ping');

Solution

1. Step 1: Check correct import style for 'events' shim

   The 'events' module shim exports as a namespace, so import * as events is correct.

2. Step 2: Verify usage of EventEmitter and event methods

   Creating emitter with new events.EventEmitter() and using addListener or on works; here addListener is used correctly.

3. Step 3: Confirm emit triggers listener

   Calling emitter.emit('ping') triggers the listener printing 'pong'.

4. Final Answer:

   import * as events from 'https://deno.land/std/node/events.ts'; const emitter = new events.EventEmitter(); emitter.addListener('ping', () => console.log('pong')); emitter.emit('ping'); -> Option A

5. Quick Check:

   Namespace import + EventEmitter + addListener = correct [OK]

Hint: Use namespace import and new events.EventEmitter() [OK]

Common Mistakes:

• Using default import which is invalid
• Importing from 'node:events' which is Node.js native
• Confusing on() and addListener() usage with wrong import