HMR api specs

hmr.md

HMR Runtime API Specification (Node.js readFileVm Variant)

1. Introduction

This document provides a comprehensive specification for the Hot Module Replacement (HMR) runtime, specifically tailored for a Node.js environment where update chunks are loaded from the filesystem and executed within the current V8 context using the vm module. This runtime enables developers to update modules in a running Node.js application without a full server restart, significantly speeding up development workflows.

1.1. Purpose of this HMR Runtime

The primary purpose of this HMR runtime is to:

• Detect changes to modules in a running application
• Fetch these changes (update chunks) from the filesystem
• Evaluate the new module code
• Determine which modules need to be updated and how (e.g., re-evaluation, disposal, propagation to parent modules)
• Apply these updates with minimal disruption to the application state
• Provide hooks and mechanisms for modules to manage their state during updates (e.g., dispose handlers, accept handlers)

1.2. Target Environment

This runtime is specifically designed for:

• Node.js: It uses Node.js specific modules like require('fs'), require('path'), and require('vm')
• Webpack: The structure and naming conventions (__webpack_require__, chunkId, moduleId) indicate it's part of a webpack-generated bundle. It integrates with webpack's internal module system and HMR API.

1.3. Key Concepts

• Module: A unit of code, typically a JavaScript file, managed by webpack. Each module has a unique moduleId.
• Chunk: A larger file generated by webpack that can contain multiple modules. Updates are often delivered as chunks.
• Update Manifest: A JSON file (typically ending in .hot-update.json) generated by webpack during a recompilation. It lists which chunks have been updated, removed and modules to remove if any.
• Update Chunk: A JavaScript file (typically ending in .hot-update.js) containing the new code for updated modules and potentially new runtime code.
• webpack_require: Webpack's internal runtime function used for loading modules, accessing the module cache, and other internal operations. This HMR runtime heavily extends and interacts with it.

2. Global State and Configuration

2.1. installedChunks

var installedChunks = __webpack_require__.hmrS_readFileVm = __webpack_require__.hmrS_readFileVm || {
  "index": 0
};

Purpose: Tracks the loading status of chunks in the application.

• 0: Chunk is already loaded
• Promise: Chunk is currently loading
• undefined: Chunk is not loaded

2.2. HMR State Variables

var currentModuleData = {};           // Data passed between module dispose/accept cycles
var installedModules = __webpack_require__.c;  // Reference to webpack's module cache
var currentChildModule;               // Currently loading child module
var currentParents = [];              // Parent modules of currently loading module
var registeredStatusHandlers = [];    // Callbacks for status changes
var currentStatus = "idle";           // Current HMR status
var blockingPromises = 0;             // Count of blocking operations
var blockingPromisesWaiting = [];     // Queue of callbacks waiting for unblock
var currentUpdateApplyHandlers;       // Handlers for applying current update
var queuedInvalidatedModules;         // Modules queued for invalidation

3. Core HMR Runtime Functions

3.1. Module Hot Object Creation

createModuleHotObject(moduleId, me)

Purpose: Creates the module.hot object that provides the HMR API to individual modules.

Parameters:

• moduleId (string): The unique identifier for the module
• me (object): The module object from webpack's module cache

Returns: Hot object with the following API:

{
  // Module API
  active: true,
  accept: function(dep, callback, errorHandler),
  decline: function(dep),
  dispose: function(callback),
  addDisposeHandler: function(callback),
  removeDisposeHandler: function(callback),
  invalidate: function(),
  
  // Management API
  check: hotCheck,
  apply: hotApply,
  status: function(listener),
  addStatusHandler: function(listener),
  removeStatusHandler: function(listener),
  
  // Data from previous dispose call
  data: currentModuleData[moduleId]
}

3.2. Module Hot API Methods

accept(dependencies, callback, errorHandler)

Purpose: Accept code updates for specified dependencies or self.

Signatures:

// Accept self updates
module.hot.accept();
module.hot.accept(errorHandler);

// Accept dependency updates
module.hot.accept(dependency, callback, errorHandler);
module.hot.accept([dependencies], callback, errorHandler);

Parameters:

• dependencies (string|string[]): Module dependencies to accept updates for
• callback (function): Called when dependencies are updated
• errorHandler (function): Called when update fails

Example:

if (module.hot) {
  module.hot.accept('./handler.js', function() {
    // Handle update of ./handler.js
    console.log('Handler updated!');
  });
}

decline(dependencies)

Purpose: Decline updates for specified dependencies or self.

Signatures:

// Decline self updates
module.hot.decline();

// Decline dependency updates
module.hot.decline(dependency);
module.hot.decline([dependencies]);

dispose(callback) / addDisposeHandler(callback)

Purpose: Add a handler that executes when the module is disposed (before update).

Parameters:

• callback (function): Function called with data object for state transfer

Example:

if (module.hot) {
  module.hot.dispose(function(data) {
    // Save state before module disposal
    data.savedState = currentState;
  });
}

removeDisposeHandler(callback)

Purpose: Remove a previously added dispose handler.

invalidate()

Purpose: Mark the current module as invalidated, triggering an update check.

3.3. Management API Methods

check(autoApply)

Purpose: Check for available updates and optionally apply them.

Parameters:

• autoApply (boolean): Whether to automatically apply updates

Returns: Promise that resolves with updated module IDs or null if no updates

Example:

module.hot.check(true).then(function(updatedModules) {
  if (updatedModules) {
    console.log('Updated modules:', updatedModules);
  } else {
    console.log('No updates available');
  }
}).catch(function(err) {
  console.error('Update failed:', err);
});

apply(options)

Purpose: Apply previously checked updates.

Parameters:

• options (object): Configuration options for the apply process
  • ignoreUnaccepted (boolean): Continue despite unaccepted modules
  • ignoreDeclined (boolean): Continue despite declined modules
  • ignoreErrored (boolean): Continue despite errors
  • onUnaccepted (function): Callback for unaccepted modules
  • onDeclined (function): Callback for declined modules
  • onDisposed (function): Callback for disposed modules
  • onErrored (function): Callback for errors

Returns: Promise that resolves with updated module IDs

status(listener)

Purpose: Get current HMR status or register a status change listener.

Parameters:

• listener (function, optional): Callback for status changes

Returns: Current status string if no listener provided

Status Values:

• "idle": No update process running
• "check": Checking for updates
• "prepare": Preparing update
• "ready": Update ready to apply
• "dispose": Disposing old modules
• "apply": Applying new modules
• "abort": Update aborted
• "fail": Update failed

4. Webpack Runtime Integration

4.1. Update Manifest Loading

webpack_require.hmrM()

Purpose: Fetch the update manifest file to determine available updates.

Implementation:

__webpack_require__.hmrM = function() {
  return new Promise(function(resolve, reject) {
    var filename = require('path').join(__dirname, "" + __webpack_require__.hmrF());
    require('fs').readFile(filename, 'utf-8', function(err, content) {
      if(err) {
        if(err.code === "ENOENT") return resolve(); // No updates available
        return reject(err);
      }
      try { resolve(JSON.parse(content)); }
      catch(e) { reject(e); }
    });
  });
}

Returns: Promise resolving to update manifest object or undefined if no updates

Manifest Format:

{
  "c": ["chunk1", "chunk2"], // Updated chunk IDs
  "r": ["oldChunk"],         // Removed chunk IDs
  "m": ["moduleId1"]         // Removed module IDs
}

4.2. Chunk Update Loading

webpack_require.hmrC.readFileVm(chunkIds, removedChunks, removedModules, promises, applyHandlers, updatedModulesList)

Purpose: Load update chunks for the specified chunk IDs.

Parameters:

• chunkIds (string[]): Array of chunk IDs that have updates
• removedChunks (string[]): Array of chunk IDs that were removed
• removedModules (string[]): Array of module IDs that were removed
• promises (Promise[]): Array to push loading promises into
• applyHandlers (function[]): Array to push apply handler functions into
• updatedModulesList (string[], optional): Array to collect updated module IDs

Implementation Details:

1. Registers the main applyHandler for the update cycle
2. Initializes update state variables
3. Loads update chunks using loadUpdateChunk
4. Sets up dynamic import HMR hook if needed

loadUpdateChunk(chunkId, updatedModulesList)

Purpose: Load and execute a single update chunk file.

Parameters:

• chunkId (string): The chunk ID to load
• updatedModulesList (string[], optional): Array to collect updated module IDs

Returns: Promise that resolves when chunk is loaded and processed

Implementation:

function loadUpdateChunk(chunkId, updatedModulesList) {
  return new Promise(function(resolve, reject) {
    var filename = require('path').join(__dirname, "" + __webpack_require__.hu(chunkId));
    require('fs').readFile(filename, 'utf-8', function(err, content) {
      if(err) return reject(err);
      var update = {};
      require('vm').runInThisContext(
        '(function(exports, require, __dirname, __filename) {' + content + '\n})', 
        filename
      )(update, require, require('path').dirname(filename), filename);
      
      var updatedModules = update.modules;
      var runtime = update.runtime;
      
      for(var moduleId in updatedModules) {
        if(__webpack_require__.o(updatedModules, moduleId)) {
          currentUpdate[moduleId] = updatedModules[moduleId];
          if(updatedModulesList) updatedModulesList.push(moduleId);
        }
      }
      if(runtime) currentUpdateRuntime.push(runtime);
      resolve();
    });
  });
}

4.3. Compilation Hash Access

webpack_require.h()

Purpose: Get the current compilation hash for the webpack build.

Returns: String representing the current compilation hash

Implementation: This function is typically defined in hot-update chunks and can be either:

• A hardcoded hash value: __webpack_require__.h = () => "d07870e94108fb010d12"
• A dynamically generated value: __webpack_require__.h = () => Date.now().toString()

Usage: Used internally by webpack for:

• Generating hot-update file names via __webpack_require__.hu(chunkId)
• Generating manifest file names via __webpack_require__.hmrF()
• Tracking compilation versions for cache invalidation

Example:

// Get current compilation hash
const currentHash = __webpack_require__.h();

// Used in filename generation
const updateFileName = chunkId + "." + __webpack_require__.h() + ".hot-update.js";
const manifestFileName = "index." + __webpack_require__.h() + ".hot-update.json";

webpack_require.hu(chunkId)

Purpose: Generate the filename for a hot-update chunk file.

Parameters:

• chunkId (string): The chunk ID to generate filename for

Returns: String representing the hot-update chunk filename

Implementation:

__webpack_require__.hu = (chunkId) => {
  return "" + chunkId + "." + __webpack_require__.h() + ".hot-update.js";
}

webpack_require.hmrF()

Purpose: Generate the filename for the hot-update manifest file.

Returns: String representing the hot-update manifest filename

Implementation:

__webpack_require__.hmrF = () => ("index." + __webpack_require__.h() + ".hot-update.json");

4.4. Module Invalidation

webpack_require.hmrI.readFileVm(moduleId, applyHandlers)

Purpose: Register a module for invalidation and update.

Parameters:

• moduleId (string): The module ID to invalidate
• applyHandlers (function[]): Array to push apply handlers into

Usage: Called when a module calls module.hot.invalidate() or needs individual update handling.

5. Update Application Process

5.1. Apply Handler

The applyHandler function orchestrates the entire update application process:

1. Analysis Phase: Determine which modules are affected by updates
2. Validation Phase: Check if updates can be accepted
3. Disposal Phase: Clean up old modules and call dispose handlers
4. Application Phase: Install new module code and call accept handlers

5.2. Module Effect Analysis

getAffectedModuleEffects(updateModuleId)

Purpose: Analyze the impact of updating a specific module.

Returns: Object describing the update effect:

{
  type: "accepted" | "declined" | "self-declined" | "unaccepted",
  moduleId: string,
  chain: string[],           // Dependency chain
  outdatedModules: string[], // Modules that need updating
  outdatedDependencies: {    // Dependencies that need updating
    [parentId]: [dependencyId, ...]
  }
}

5.3. Update Application Phases

Dispose Phase

1. Remove outdated chunks from installedChunks
2. Call dispose handlers for each outdated module
3. Deactivate modules (module.hot.active = false)
4. Remove modules from webpack cache
5. Update parent-child relationships

Apply Phase

1. Install new module factories in __webpack_require__.m
2. Execute new runtime code
3. Call accept handlers for updated dependencies
4. Re-require self-accepted modules
5. Handle errors and report status

6. Error Handling

6.1. Error Types

• accept-errored: Error in accept handler
• accept-error-handler-errored: Error in accept error handler
• self-accept-errored: Error in self-accept handler
• self-accept-error-handler-errored: Error in self-accept error handler

6.2. Error Callbacks

The apply options object supports various error callbacks:

{
  onErrored: function(info) {
    console.error('HMR Error:', info.type, info.error);
  },
  ignoreErrored: false // Set to true to continue despite errors
}

7. Status Management

7.1. Status Transitions

idle → check → prepare → ready → dispose → apply → idle
                    ↓         ↓
                  abort ← fail

7.2. Status Handlers

Register callbacks to monitor HMR status changes:

module.hot.addStatusHandler(function(status) {
  console.log('HMR Status:', status);
});

8. Advanced Usage Examples

8.1. HTTP Server Hot Replacement

var requestHandler = require("./handler.js");
var server = require("http").createServer();
server.on("request", requestHandler);
server.listen(8080);

if(module.hot) {
  module.hot.accept("./handler.js", function() {
    server.removeListener("request", requestHandler);
    requestHandler = require("./handler.js");
    server.on("request", requestHandler);
  });
}

8.2. CSS Hot Replacement

var addStyleTag = require("./addStyleTag");
var element = addStyleTag(".rule { attr: name }");

if(module.hot) {
  module.hot.accept();
  
  var removeStyleTag = require("./removeStyleTag");
  module.hot.dispose(function() {
    removeStyleTag(element);
  });
}

8.3. Context-based Hot Replacement

var context = require.context("./modules", false, /\.js$/);
var modules = {};

context.keys().forEach(function (key) {
  var module = context(key);
  modules[key] = module;
  initializeModule(key, module, false);
});

if (module.hot) {
  module.hot.accept(context.id, function () {
    var reloadedContext = require.context("./modules", false, /\.js$/);
    var changedModules = reloadedContext.keys()
      .map(function (key) {
        return [key, reloadedContext(key)];
      })
      .filter(function (reloadedModule) {
        return modules[reloadedModule[0]] !== reloadedModule[1];
      });
      
    changedModules.forEach(function (module) {
      modules[module[0]] = module[1];
      initializeModule(module[0], module[1], true);
    });
  });
}

9. Best Practices

9.1. Module Design for HMR

1. Stateless Modules: Design modules to be easily replaceable
2. State Preservation: Use dispose handlers to save/restore state
3. Resource Cleanup: Always clean up resources in dispose handlers
4. Error Handling: Implement proper error handling in accept callbacks

9.2. Performance Considerations

1. Selective Acceptance: Only accept updates for modules that can handle them
2. Minimize Side Effects: Reduce global state modifications
3. Efficient Disposal: Clean up resources promptly in dispose handlers

9.3. Debugging HMR

1. Status Monitoring: Use status handlers to track HMR state
2. Error Logging: Implement comprehensive error logging
3. Update Tracking: Log which modules are being updated

if (module.hot) {
  module.hot.addStatusHandler(function(status) {
    console.log('[HMR] Status:', status);
  });
  
  module.hot.accept(function(err) {
    console.error('[HMR] Accept error:', err);
  });
}

This specification provides a complete reference for the HMR runtime implementation in the Node.js readFileVm variant, covering all APIs, functions, and usage patterns for effective hot module replacement in Node.js applications.