Logic for Programmers: a tools comparison
Nine verification tools on one axis, installed and measured
Table of Contents
1. Introduction
Hillel Wayne's Logic for Programmers reached 1.0 in 2026 after five years and fifteen public alphas. It is a survey: one chapter of logic basics, then eleven chapters applying them, four of which hand you a tool you probably do not have.
This note is not a review. It is the tools comparison the book implies but does not tabulate, with two things the book cannot give you — what each tool costs to get running on a real machine, and a failure mode they share that only shows up once you try to automate them.
Companion to Alloy Specification Language, which goes deeper on Alloy specifically and compares it against TLA+ and Lean 4. This one is wider and shallower: nine tools, one axis.
Working repository: jwalsh/logic-for-programmers.
2. The organizing axis
Wayne names it in §2.6 and returns to it in §11.4 and appendix C: the ability-guarantee tradeoff. The more a formalism can express, the less it can promise about answering. Every tool in the book sits somewhere on it, and the book's structure is a walk from one end to the other.
The left column always answers. The middle answers within a scope — Alloy's
check is bounded, TLC explores a finite state space, a property test samples.
The right column is the honest end: SMT can return unknown, Dafny needs
invariants you supply, Prolog can diverge.
Nothing on the right is "better". Wayne's field observation is the useful one: SMT solvers get used for satisfaction problems in software engineering, while the constraint solvers get used for optimization problems in business. Which end of the axis you live on is a fact about your industry.
3. What each tool actually costs
Measured, not estimated. Every row was installed on one macOS arm64 machine and run against both a minimal example and one of the book's own samples.
| Ch | Tool | Install | Runtime | First useful output |
|---|---|---|---|---|
| 4 | Hypothesis | pip |
Python | minutes |
| 5 | icontract | pip |
Python | minutes |
| 6 | Dafny | brew / dotnet | .NET | hours — you must write invariants |
| 7 | SQLite | preinstalled | --- | minutes |
| 9 | Alloy 6 | jar, manual | JVM | ~an hour |
| 10 | TLA+ / TLC | jar, manual | JVM | a day |
| 11 | MiniZinc | brew (+ Gecode) | native | minutes |
| 11 | Z3 | brew or pip | native | minutes |
| 12 | SWI-Prolog | brew | native | minutes |
| 12 | clingo | brew | native | minutes |
| 12 | Picat | manual tarball | native | — (no brew formula) |
The real barrier in this book is not conceptual difficulty. It is that chapters
9 and 10 need JVM jars with no package-manager story, and chapter 6 needs a .NET
toolchain. Everything else is one brew install away. A reader who bounces off
the second half usually bounced off an install, not off temporal logic.
Two version traps worth stating:
- Alloy must be 6 or later. Temporal modeling landed in 2021; most Alloy material online — including the canonical book — predates it and emulates time with a signature-based encoding. §9.3 uses the modern operators.
- MiniZinc bundles Gecode and a CBC-class ILP backend, so §11.2's comparison between solver classes runs without further installs. Running the same model against both backends turns §11.4's table into an experiment.
4. The finding: verification tools report success by failing
This is the part that does not appear in the book, because it only bites when you try to put these tools behind a gate.
| Tool | The result you want | Exit status |
|---|---|---|
| Dafny | this proof does not verify | nonzero |
| TLC | this invariant is violated | nonzero |
| Alloy | a counterexample exists | zero, but SAT |
| SQLite | this constraint did fire | nonzero per statement |
| Z3 | unsat on the negated theorem |
zero |
Three of five signal their most valuable answer through a failure channel. A harness written on the ordinary assumption — nonzero means broken — inverts the meaning of the thing it is checking.
Concretely, in a dependency gate over the book's own samples:
set -o pipefailwithtool | grepreturns the tool's exit status rather than the match. Dafny correctly rejecting a bad proof and TLC correctly finding an overdraft were both scored as failures. The gate was red because the book was right.- The book's
basic_invariants.sqlends with three INSERTs whose own comments read "Will fail saying …". The constraint violations are the demonstration; a clean run would mean the invariants were not being enforced.
Alloy deserves separate mention because the inversion is inside the tool:
| Command | SAT means |
UNSAT means |
|---|---|---|
run |
an instance exists — good | over-constrained — bad |
check |
a counterexample exists — bad | none in scope — good |
Same token, opposite verdicts, decided by a keyword elsewhere in the file. Any
automation that greps for SAT without knowing which command produced it is
guessing.
4.1. The general rule
Assert the specific expected failure, never the absence of failure. And assert
in matched pairs: that qr-loop-valid.dfy verifies and qr-loop-invalid.dfy
is rejected; that wire.tla is clean and wire03.tla violates its invariant;
that the buggy Alloy model yields a counterexample and the fixed one does not.
Asserting only the positive half passes against a checker that never checks.
This connects directly to CPRR: a refutation gate is only meaningful if it has been observed refuting something.
5. The idea worth stealing from the book
Independent of tooling. Wayne's §5.4 generalizes Liskov and Wing's behavioral subtyping into replacement, and then reuses it twice — §7.5 for database schemas, §10.5 for specification refinement. He says so outright on p. 65: there is no common term across all the cases, so subtyping, backwards compatibility, the Liskov substitution principle and refinement get one umbrella word.
The operational form is a shortcut worth having (p. 63):
newtype is compatible with oldtype if it passes all property tests written for oldtype.
That is executable as stated. Running Rectangle's property suite against
candidate types decides the classic Square case, and the pattern of failures
recovers the formal two-clause rule without anyone writing it down: Square
passes the invariant property (every Square is a valid Rectangle — clause 2
holds) and fails both behavioral properties (clause 1 does not). A candidate
with a strengthened precondition fails in the opposite direction.
It also works on things that are not types at all. An API whose listing endpoint silently stops sorting fails the same harness, and Wayne notes that this case — data schema replacement — is the one that never got a name.
6. Where this connects
- Alloy Specification Language — deeper on Alloy's relational logic and its Kodkod/SAT flow; this note places it against eight neighbours instead.
- CPRR Methodology — conjecture and refutation as tracked artifacts. The matched-pair discipline above is the same instinct: a gate is evidence only once it has been seen to fail.
- Four-boundary spec mapping — decision tables (§8) are the cheapest formal specification in the book and the only one needing no software; completeness and soundness are exactly the boundary questions.
7. Reading recommendation
The book is explicitly non-linear: read ch2, then jump. Four honest paths:
| Track | Chapters | Pages | Installs |
|---|---|---|---|
| Weekend | 2, 3, 8 | ~50 | none |
| Testing and correctness | 2, 4, 5, 6 | ~65 | pip, then Dafny |
| Modeling | 2, 9, 10 | ~65 | two JVM jars |
| Solvers | 2, 11, 12 | ~50 | brew |
Chapters 2 and 10 carry 45% of the book's 56 exercises between them, which is a fair signal about where practice matters.
The one thing not to skip on a first pass through chapter 10: §10.5 is prefaced with the author saying he "won't be too sad" if you skip it, and the liveness and fairness material genuinely is optional — but Refinement is buried in there at p. 147, and it closes a thread that opened on p. 60.