What?

mutastic is a mutation tester for Java. It makes small changes to your compiled code, called mutants, and runs your tests against each one. A mutant that makes a test fail is caught. A mutant no test notices is missed.

I built this on whim after trying out some other tools for improving the quality of and trust of tests generated by coding agents and wanting to explore what a modern version should look like.

Why?

• Coverage says a test ran a line. Mutation testing says whether it would notice the line being wrong.
• A test with no assertions gets full coverage and catches nothing. Generated tests are often like that.
• A miss comes with its evidence and its fix: the inputs your tests used, a test that would catch it, and the command that proves the fix.

How?

Setup

build.gradle.kts

plugins {
    java
    id("io.mutastic") version "0.5.0"
}
 
repositories {
    mavenCentral()
}
 
java {
    toolchain { languageVersion = JavaLanguageVersion.of(25) }
}
 
dependencies {
    testImplementation(platform("org.junit:junit-bom:6.1.3"))
    testImplementation("org.junit.jupiter:junit-jupiter")
    testRuntimeOnly("org.junit.platform:junit-platform-launcher")
}
 
tasks.test { useJUnitPlatform() }

• Nothing to configure for JUnit 6 or TestNG: mutastic reads the settings the test task uses.
• mavenCentral() is required. The plugin fetches the mutastic command from it.
• mutastic runs on JDK 25; your tests don’t have to. They run on whatever JDK the test task uses, 17 or later.

The code

src/main/java/com/example/add/Add.java

public interface Add {
  static long add(long left, long right) {
    return left + right;
  }
}

A test that checks nothing

src/test/java/com/example/add/AddTest.java

@Test
void itWorks() {
  assertDoesNotThrow(() -> Add.add(2, 2));
}

./gradlew mutasticRun

 ✗ src/main/java/com/example/add/Add.java:5 tests pass even when + becomes - in add() mffc6fafe
 ✗ src/main/java/com/example/add/Add.java:5 no test checks what add() returns here m873f138a

   Mutants  ✓ 0 caught · ✗ 2 missed (2)
     Score  0.0%  threshold 80

 [FAIL] Score below 80: 2 missed, 0 not covered.

• The test passes and covers the line, and it would still pass if add subtracted or returned 0.
• A score below the threshold fails the build (80 by default, mutastic { threshold = … }).

A test that checks the answer

src/test/java/com/example/add/AddTest.java

@Test
void itWorks() {
  var result = Add.add(2, 2);
  assertEquals(4, result);
}

   Mutants  ✓ 2 caught (2)
     Score  100.0%  ▲ 100.0 vs last run (code changed)  threshold 80

 [PASS] Every mutant was caught.

• Same coverage, and every mutant is caught.
• The score is compared with the last run, and it says whether the code changed in between.

What 100% doesn’t mean

• 2 × 2 is also 4. The defaults only swap + for -. The stronger set adds + → *:

./gradlew mutasticRun -Pmutastic.operators=stronger

 ✗ src/main/java/com/example/add/Add.java:5 tests pass even when + becomes * in add() m745873f4

   Mutants  ✓ 2 caught · ✗ 1 missed (3)
     Score  66.7%  ▼ 33.3 vs last run (same code)  threshold 80

• The score only covers the mutants generated. It isn’t proof of correctness.

Reading a miss

./gradlew mutasticExplain -Pmutastic.mutant=m745873f4 -Pmutastic.operators=stronger

 [✗ MISSED] tests pass even when + becomes * in add()
   ┌─ src/main/java/com/example/add/Add.java:5
   │
 5 │     return left + right;
   │                 ^ changed to *
   │
   = note: inputs seen: left = 2; right = 2
           covered by AddTest#itWorks (its 1 covering test still passes)
   = help: use inputs where left + right and left * right differ, and assert the result

           // in AddTest
           @Test
           void addWithLeft2AndRight0() {
             assertEquals(2L, Add.add(2L, 0L)); // today 2L · with this mutant 0L
           }

   done when  ./gradlew :mutasticVerify -Pmutastic.mutant=m745873f4 passes
   or ignore  ./gradlew :mutasticIgnore -Pmutastic.mutant=m745873f4 -Pmutastic.reason="<why>"

• ”Inputs seen” is the diagnosis: with 2 and 2, the sum and the product agree.
• The suggested test pins today’s value. Check that today’s value is the one you mean.
• The id is stable. It’s a hash of the class, method, site and change, with no line number, so it survives edits above it.
• Ignoring a mutant records a reason in .mutastic/ignores.yaml, instead of silently lowering the score.

Proving the fix

Change the test to Add.add(2, 3) and assertEquals(5, result), then:

./gradlew mutasticVerify -Pmutastic.mutant=m745873f4 -Pmutastic.operators=stronger

mutastic: m745873f4 is caught by com.example.add.AddTest#itWorks

• verify runs only that mutant, so the check after a fix takes seconds.
• Reports are in build/mutastic/ as mutation.html and mutation.json.

For the same example under PIT, see Bytes, Java, Mutation Testing with PIT.

Sources

• mutastic: Quickstart: run, read a miss, write the test, verify.
• mutastic: Install: the command line, the Gradle plugin and the Maven plugin.
• mutastic: Operators: what each operator changes, and the defaults and stronger sets.
• mutastic: Build plugins: the mutastic { … } block, -Pmutastic. properties and the tasks.
• mutastic on GitHub: source, Apache-2.0, and release notes.
• Output above is from mutastic 0.5.0 on Gradle 9.2.1 and JDK 25, trimmed to the lines discussed.