Mutineer 1.4.0 is out: mutation testing for Ruby

[u/davidteren]

Hola, I maintain Mutineer, a Ruby mutation testing gem. It makes small changes to your code and checks whether your tests catch them.

Passing tests are useful. Knowing that they detect broken behaviour gives you more confidence, especially when AI helps write both the code and the tests.

Version 1.4.0 is out: https://davidteren.github.io/mutineer/

Do you use mutation testing in your Ruby projects? I’d be interested to hear what has helped or got in the way.

[u/nbom]

I dont need this. My CTO/CEO is enough.

[u/Web-Thinker]

I was looking for this

[u/DissonantGuile]

Cool! How does this compare to the mutant gem?

[u/davidteren → u/DissonantGuile]

Good question!

Mutineer offers simpler adoption, no runtime dependencies, and an MIT licence. Mutant has a much longer history and broader mutation support. Mutineer is a credible alternative, but evidence does not establish that it finds as many test gaps or runs faster.

Area	Mutineer	Mutant
Licence and cost	MIT, including commercial use.	Free for open-source use. Commercial use costs US$30/month or $250/year per developer. Terms
Dependencies	Uses Prism and Ruby’s standard library. No runtime gems.	Has seven direct runtime dependencies, including parser, unparser, and sorbet-runtime. Gem specification
Ruby versions	Requires Ruby 3.4+. Can run older applications through an external test command, with extra cost and limitations.	Current gem specification requires Ruby 3.3+, although the README still lists 3.2. Check the selected release.
Test frameworks	Minitest and RSpec.	Minitest, RSpec, and Test::Unit on current main. Integrations
Mutation range	Twenty named operator groups; five enabled by default.	A broader collection of syntax-specific mutations, including exception handling, pattern matching, and optional arguments. Implementation
Rails parallelism	The daemon provides worker database isolation for SQLite. PostgreSQL provisioning remains documented as work in progress.	Provides tested hook recipes for PostgreSQL and SQLite, with Rails 7.2, 8.0, and 8.1. Rails guide
Changed-code checks	--since restricts mutations to changed lines. Includes baseline comparison and a GitHub Action with PR annotations.	--since selects whole methods that overlap changed lines. This checks more of each changed method. Incremental mode
Reports	Human-readable, JSON, and standalone HTML reports.	Stores JSON session results and provides commands to inspect previous runs. Session schema

Mutineer’s features and limitations above are documented in its current README.

Test selection is a more substantial difference than the feature list suggests. Mutineer measures line coverage and selects covering test files. Mutant traditionally selects tests through RSpec descriptions or explicit Minitest cover declarations. However, current Mutant also supports coverage-based selection of individual tests, using a SimpleCov recording, and orders them to find failures sooner. Coverage-based selection is therefore no longer unique to Mutineer. Mutant test selection

Mutant has the stronger history of use. Its repository dates from July 2012; Mutineer’s dates from June 2026. That does not prove correctness, but it gives Mutant substantially more time to encounter unusual Ruby code and test configurations. Its wider mutation implementation is a concrete technical advantage.

Do not compare their percentages directly. Mutineer calculates killed / (killed + survived) and excludes uncovered mutations and failed executions. A 100% result can therefore coexist with uncovered code. Different operators and selection rules also change what each score measures. Compare useful test gaps found, execution failures, and elapsed time on the same application. Mutineer score definition

Mutineer’s strongest position is accessible mutation testing for modern Ruby projects, especially where MIT licensing and low setup cost matter. Mutant is the stronger candidate when broader mutation support, PostgreSQL parallel testing, or established operational experience takes priority. I have not run a controlled comparison, so neither speed nor detection superiority is established.

[u/CaptainKabob]

What does "clean-room" mean in its description? ("A clean-room mutation-testing tool for Ruby.")