JavaScript AST Analysis

Overview
Background
Key Concepts
Implementation
References
Notes

Overview

Abstract Syntax Trees (ASTs) provide a structured representation of JavaScript source code, enabling static analysis, code metrics collection, and automated transformations. This research explores using AST parsing libraries like Esprima and Acorn to extract statistical insights from JavaScript codebases, including complexity metrics, dependency analysis, and code quality indicators.

Background

JavaScript's dynamic nature makes static analysis challenging, yet AST-based tools have become essential for:

Code linting and style enforcement (JSLint, JSHint, ESLint)
Minification and bundling (UglifyJS, Closure Compiler)
Code coverage and complexity analysis
Automated refactoring and code generation

The ECMAScript specification defines the grammar that AST parsers implement, with the Mozilla Parser API establishing a de facto standard for AST node representation.

Key Concepts

Parser Libraries

Esprima

Full ECMAScript 5.1 compliance (ES6 support emerging)
Produces detailed AST with location information
Supports tolerant parsing mode for error recovery
~2000 lines of code, well-documented

var esprima = require('esprima');
var ast = esprima.parse('var x = 42;', { loc: true, range: true });

Acorn

Smaller and faster than Esprima (~1000 lines)
Plugin architecture for syntax extensions
Used by projects like Tern and Webpack

var acorn = require('acorn');
var ast = acorn.parse('var x = 42;', { locations: true });

AST Node Types

Common node types for statistical analysis:

Node Type	Description	Complexity Weight
FunctionDeclaration	Named function definitions	High
FunctionExpression	Anonymous/assigned functions	High
IfStatement	Conditional branches	Medium
ForStatement	Loop constructs	Medium
CallExpression	Function invocations	Low
MemberExpression	Property access (a.b, a[b])	Low

Code Metrics

Cyclomatic Complexity

Measures the number of independent paths through code:

function calculateCyclomaticComplexity(ast) {
  var complexity = 1;
  estraverse.traverse(ast, {
    enter: function(node) {
      switch (node.type) {
        case 'IfStatement':
        case 'ConditionalExpression':
        case 'ForStatement':
        case 'WhileStatement':
        case 'DoWhileStatement':
        case 'CatchClause':
        case 'LogicalExpression':
          complexity++;
          break;
        case 'SwitchCase':
          if (node.test) complexity++;
          break;
      }
    }
  });
  return complexity;
}

Halstead Metrics

Vocabulary: n = n1 + n2 (unique operators + operands)
Length: N = N1 + N2 (total operators + operands)
Volume: V = N * log2(n)
Difficulty: D = (n1/2) * (N2/n2)

Lines of Code (LOC) Variants

Physical LOC: Raw line count
Logical LOC: Statement count from AST
Comment density: Comment nodes / total nodes

Implementation

AST Walker Pattern

var estraverse = require('estraverse');
var stats = {
  functions: 0,
  variables: 0,
  conditionals: 0,
  loops: 0,
  calls: 0
};

estraverse.traverse(ast, {
  enter: function(node, parent) {
    switch (node.type) {
      case 'FunctionDeclaration':
      case 'FunctionExpression':
        stats.functions++;
        break;
      case 'VariableDeclarator':
        stats.variables++;
        break;
      case 'IfStatement':
      case 'ConditionalExpression':
        stats.conditionals++;
        break;
      case 'ForStatement':
      case 'WhileStatement':
      case 'DoWhileStatement':
        stats.loops++;
        break;
      case 'CallExpression':
        stats.calls++;
        break;
    }
  }
});

Dependency Extraction

function extractDependencies(ast) {
  var deps = [];
  estraverse.traverse(ast, {
    enter: function(node) {
      if (node.type === 'CallExpression' &&
          node.callee.name === 'require' &&
          node.arguments[0] &&
          node.arguments[0].type === 'Literal') {
        deps.push(node.arguments[0].value);
      }
    }
  });
  return deps;
}

Reporting Output

File: app.js
=====================================
Physical LOC:     245
Logical LOC:      189
Functions:        12
Cyclomatic:       24 (avg: 2.0 per function)
Maintainability:  68.4
Dependencies:     ['express', 'lodash', 'async']

References

Esprima - ECMAScript Parsing Infrastructure
Acorn - A small, fast JavaScript parser
Estraverse - ECMAScript AST Traversal
Mozilla Parser API
McCabe, T.J. (1976) "A Complexity Measure" IEEE Transactions on Software Engineering
Halstead, M.H. (1977) "Elements of Software Science"

Notes

Consider using escomplex for comprehensive complexity metrics
JSHint/ESLint build upon these AST techniques for linting
Source maps enable mapping minified code back to original AST locations
The ESTree specification standardizes AST format across tools