A history of assertions

README.md

What are assertions?

Assertions are a means to verify that program invariants have not been violated. Put simply: you assert that impossible things are not happening, and if that assertion ever fails — if the impossible is happening — then your program should halt execution immediately, generally for two reasons:

• If your program has done something that should (if your program is properly written) be logically impossible, then you will likely only detect the problem after the fact. You don't know what went wrong, you don't know how it went wrong, and you don't know the scope of the damage, and this means both that your program can't reliably correct the error and that your program can no longer trust itself at all. The least dangerous thing to do is stop executing immediately — crash, on purpose, before your program can do anything else that shouldn't be possible.

• Depending on your choice of programming language and environment, halting execution may give you an opportunity to inspect the program's state and investigate why something impossible happened — investigate what mistake you made. The failing program may produce a crash dump that you can analyze, or you may run the program in a debugger so that if the assertion fails in real time, the debugger will let you inspect the program's state before letting the program fully die.

A history of assertions

The oldest example of "assertions" in computer science that Wikipedia is aware of comes from a 1947 paper (Planning and Coding of problems for an Electronic Computing Instrument by Goldstine and von Neumann (yes, that one); Part II, Vol. 1, page 12). The authors designed some visual conventions for diagramming a program, and these included an "assertion box" meant to display known invariants — things that, logically, must be true at some point in a program's execution if the program has been implemented correctly. Alan Turing endorsed this same idea in 1949, stating that a program should include assertions in order to aid a human reviewer in checking a programmer's work. Wikipedia loses the trail after that, though.

In program code

In September 1973, Revised Report on the Algorithmic Language Algol 68 was published, as part of the overall process of standardizing ALGOL 68. Section 9.2 describes "pragmatics"¹ or "pragmats" for short, a precursor to #pragma directives in C, and as an example of possible pragmats that an implementation might define, they describe a hypothetical ASSERT pragmat which would indicate "that the compilation may check for the truth, or attempt to prove the correctness, of some assertion." Section 3.5 uses this hypothetical ASSERT pragmat in a code sample, as an informational note to the reader. This is the earliest example I can find of assertions that have any sort of language-level support and are meant to be checked automatically (whether by the compiler or by the program itself).

In 1984, the second version of AT&T's UNIX System V was released, alongside a reference manual for C programmers. UNIX System V included its own standard library, and this version introduced a ton of new functionality, including the assert macro. This macro was specified such that in programs compiled with the NDEBUG macro (i.e. Release builds), assertions are a no-op; in all other programs, an assertion failure displays an error and forcibly aborts the program, creating an opportunity² for investigation. Circa 1985, AT&T attempted to codify UNIX System V's existing behaviors into the SVID standard. The later POSIX standard would borrow from SVID, and among the borrowed content was the spec for the assert macro. In turn, ISO's C99 standard borrowed from POSIX; and so the behavior of C assertions dates all the way back to 1984.

In program code, but wrong forever

Unfortunately, some languages haven't implemented assertions very well. I contend that properly implemented assertions should have behavior resembling one of the following:

• An assertion failure occurring at run-time should be guaranteed to halt execution of the program immediately. Ideally, an assertion failure should also afford some opportunity to investigate the problem.

• An assertion failure occurring at compile-time should cause compilation of the program to fail.

Python

Python 1.5 (released in April 1999) introduced an assert keyword. Similarly to C, this keyword was designed to be stripped if Python's bytecode compiler was told to produce an optimized build.

Unfortunately, Python's keyword halted program execution by throwing an AssertionError — a perfectly ordinary exception class, which could be caught by user code at any time. This apparently led to many newbie developers seeing assert used in other people's code to catch invalid conditions, yet failing to understand what kind of conditions those should catch — failing to understand the difference between a program invariant and invalid input. These newbies began cargo-culting assert, but misusing it for input validation and catching its thrown exceptions; and by doing this, they created programs which would break if run with optimizations enabled. In turn, some widely-used Python tooling has been designed to indiscriminately warn on any use of the assert keyword, with no means to suppress warnings by line (only by file).

This didn't need to be an issue. If Python wanted assertion failures to use the same mechanism for halting a program as thrown exceptions, for expediency's sake, they could've given the AssertionError class an identifier that is impossible to reference within user-authored code, and special-cased bare except clauses to ignore that class. Other languages and systems have made things unreachable by user code in this fashion; offhand, the examples I know of include:

• PHP 4.0.1 introduced the create_function API, which spawns functions whose names contain null bytes.
• When compiling C++ code and mangling the fully-qualified names of identifiers scoped to anonymous namespaces (e.g. for RTTI generation), MSVC will generate a randomized identifier for each anonymous namespace in the form of a hex literal (i.e. 0x12345678, such that the mangled name is for something like 0x12345678::my_class). Legal identifiers cannot start with a digit.

Python's original assert implementation predates the more public of these examples, but the later jump from Python 2 to Python 3 would've been a perfect opportunity to break compatibility and make assert do the right thing. Sadly, that didn't happen.

JavaScript

In 2006, Joe Hewitt created the excellent "Firebug" add-on for Firefox, essentially inventing JavaScript debugging tools as we know them today. One Firebug feature was the Console API, which would inject a console singleton into pages and allow those pages to log messages to Firebug's console. This included a console.assert function which was intended to log assertion failures and throw exceptions. Unfortunately, however, it only displayed assertion failures, without halting execution or breaking into a debugger, making these "assertions" in name only.³

It's understandable for the author and maintainers of an unofficial browser add-on to accidentally write a bad assertion implementation. What baffles me is that the maintainers of JavaScript as a language copied Firebug's console behavior into the language standard pretty much verbatim and apparently without any review whatsoever. By the time console.assert shipped as a standardized API (per Can I Use, the earliest implementation was Internet Explorer 8, which shipped in 2009), the web's backwards-compatibility burden was already well understood. It's easy to find sources on things like Quirks Mode dating back to 2007. On top of that, web standards are decided on by a collaboration which includes browser vendors — organizations which create browsers in languages that'd had sensible assertion behavior for a quarter of a century by then.

And of course, when someone finally noticed that console.assert did not assert things, backwards-compatibility was exactly the rationale cited for intentionally keeping it broken forever. In fact, non-browser JavaScript run-times have even gone as far as to deliberately ruin their own assertion functionality for the sake of parity. Today, sixteen years after support for this broken spec first shipped, only the following remedies exist:

• You can write JavaScript to manually replace console.assert with something that actually asserts a truth and allows that assertion to meaningfully fail. The general pattern is to wrap console.assert and then throw an error (and pray that nothing catches and swallows those indiscriminately) after calling the original function.

• You can enable a specific setting on Google Chrome's developer tools to break into the debugger when console.assert produces a log. (Firefox does not have comparable options; how the mighty have fallen from the Firebug days.)

Footnotes

1. Section 1.3 ("Pragmatics") of Draft Report on the Algorithmic Language ALGOL 68 (MR93) (1968) introduces the concept of "pragmatic" remarks, which "do not form part of the definition of the language but are intended to help the reader to understand the implications of the definitions and to find corresponding sections." It seems that by 1973, those involved in ALGOL's standardization process had realized the utility of having "pragmatic" remarks as a feature of the language itself. ↩

2. From what I can find, UNIX System V had a tool called sdb ("symbolic debugger"). Per the 1986 programmer's manual (Vol. 5: Languages and Support Tools, page 111), a program compiled with the -g compiler switch would contain enough debug information for sdb to be able to analyze program aborts, displaying a stack trace as well as the values of all relevant variables. This means that as early as 1986, assertions were explicitly an analytical tool: they were only functional in debug builds, and assertion failures triggered an abort that gave programmers a chance to inspect the program state at the time of the assertion failure. You can assert that the program state is valid, and should it ever not be valid, you can inspect it to figure out what broke and how. ↩

3. As far as I can tell, Firebug's console.assert was broken from the start, although it had always worked properly in Firebug Lite, a library that web developers could add to their sites during testing to emulate Firebug in non-Firefox browsers. (Firebug Lite worked by throwing an error. That isn't ideal — those errors can be caught, so this would've reproduced the Python problem had it caught on — but it's the best a pure-JavaScript implementation can do.) Looking through both Firebug's modern repo and its pre-Firebug 1.11 repo, I was able to piece together the following:

   • console.assert as implemented in Firebug 1.1 didn't throw an error and so couldn't have halted execution of the calling code.

   • console.assert as implemented in Firebug 1.2 threw an error by accident, and didn't properly log and display it. This bug (whose reports are preserved here and here) stemmed from a mistake made when attempting to retrieve a stack trace. The logAssert function (called by console.assert to display an assertion failure) relied on a helper function to retrieve the stack trace, storing the return value in local variable trace and then checking if (trace && trace.frames[0]). However, the helper function didn't return falsy on failure; it returned a string (intended to be displayed to the user).

   • console.assert as implemented in Firebug 1.3 no longer threw any error. That helper function for returning stack traces still returned a string on failure, so presumably, the Firebug 1.2 bug was resolved by just improving the function's reliability and not by actually fixing the underlying problem. Given their comments on the relevant issues, Firebug's developers don't appear to have identified why the bug was fixed. This is understandable, as Firebug 1.2 also featured relatively large architectural changes to how the console API was injected into pages. Finding the source of the problem wouldn't have been quite as simple as just looking over code changes.

   • In Firebug 1.4, console.error was modified so that if it was called with just one argument, it would display its error-style message using the same underlying function as console.assert (i.e. logAssert). This confirms that by Firebug 1.4, the extension's maintainers considered console.assert's incorrect behavior to be by design.

   So yeah, Firebug's assertions were busted from the start. ↩