What?
The gates a build needs once a model writes most of the code, beyond the ones that keep it readable. Second of three: for Readers covers the code, for Operators the running system.
| Category | Tool | Owns |
|---|---|---|
| Type checker | NullAway + JSpecify | Whether a value can be null, answered by the build |
| Mutation testing | mutastic | Whether the tests would notice a wrong line |
| Integration testing | Testcontainers | The seams, tested against the real thing |
| Security | OWASP dependency-check | Known vulnerabilities in what you pulled in |
| Supply chain | Gradle dependency locking / verification metadata | Dependencies pinned as tightly as the tools |
| Supply chain | CycloneDX | An SBOM: what went into the artifact |
| Gate | CI | The checks running on every change, not when you remember |
Why?
- Agents pull transitive dependencies by reflex. Bound the blast radius at the manifest.
- A rule in the build is only enforced if something runs the build.
- Dependencies already make promises. Spring Framework 7, Spring Boot 4 and Spring AI 2.0 are JSpecify-annotated. Ignoring the annotations means ignoring the contract.
- Security and supply-chain gates are never waived.
How?
Type checker: Make null a compile error
dependencies {
errorprone("com.uber.nullaway:nullaway:0.14.2")
implementation("org.jspecify:jspecify:1.0.1")
}
options.errorprone {
check("NullAway", CheckSeverity.ERROR)
option("NullAway:OnlyNullMarked", "true") // check only packages marked @NullMarked
option("NullAway:JSpecifyMode", "true")
}@NullMarked
package com.example.bakery.kitchen;
import org.jspecify.annotations.NullMarked;- JSpecify is the standard. Spring deprecated its own
org.springframework.langannotations for it. Mark packages@NullMarked, and only what can be null@Nullable. - Adopt one package at a time.
OnlyNullMarkedchecks only packages with a@NullMarkedpackage-info.java. Get a package to zero, then mark the next. - Limits. Field-initialisation checks are incomplete, and generics support is still maturing.
The Checker Framework is complete but much slower. On Java 21, add
-XDaddTypeAnnotationsToSymbol=true; Java 25 doesn’t need it.
Mutation testing: Measure whether the tests would notice
Don’t just measure whether you have tests; measure whether those tests detect the failures you care about.
plugins { id("io.mutastic") version "0.5.0" }
repositories { mavenCentral() } // the plugin fetches the mutastic command from it
mutastic {
threshold = 80 // a score below it fails the build; set it where the score already is
}- Mutation testing breaks the code on purpose. mutastic makes small changes: it flips a
>=, negates or removes a condition, swaps+for-, drops a call or returns a default. Then it reruns the tests that reach the change. A change no test fails on is missed, a line the tests run without checking. - A miss comes with its evidence. The inputs the tests used, a test that would catch it, and the
verifycommand that proves the fix. That’s the loop an agent can close on its own. - The score is caught ÷ mutants run. Code no test reaches counts against it. Report it next to coverage, never instead of it: coverage says what ran; mutation score says what would be caught.
- Too slow for every build at full size.
-Pmutastic.since=mainmutates only the lines a pull request changed, which is where a miss is worth acting on. Run the whole project nightly. - Not every miss is a gap. A removed
Connection.close()is missed because no test checks for a leak.mutasticIgnorerecords why in.mutastic/ignores.yaml, a file you commit and review. Logging,toString()and trivial accessors are skipped on their own. - Honourable mention: PIT. The long-standing Java mutation tester, with the widest adoption. Same idea, a report to read rather than a fix to apply.
- The smallest worked example, one
addand 100% coverage that catches nothing, is in Bytes, Java, Mutation Testing with mutastic, and under PIT in Bytes, Java, Mutation Testing with PIT.
Integration testing: Test the seams against the real thing
testImplementation("org.springframework.boot:spring-boot-testcontainers")
testImplementation("org.testcontainers:testcontainers-ollama") // version from Boot's BOM: 2.0.5@Container
@ServiceConnection
static OllamaContainer ollama = new OllamaContainer("ollama/ollama:0.31.2");- The bugs live at the seams.
- One contract test per port, run against every adapter. An abstract test class states what the port promises. Each adapter’s test extends it and runs it against the real technology: a real SQLite file, a real Ollama in a container.
- Mock only what you own. Fake your own ports in unit tests. Test third-party behaviour in integration tests, because a mock of someone else’s API only checks what you think it does.
- Pin the image like any other dependency. Use a tag at least, a digest for anything you measure.
@ServiceConnectionwires the container’s URL into Spring with no properties to set.
Test runner: Skipped is not passed
- A skipped test is green and has checked nothing. Tests that need a model, a container or a dataset skip when it’s missing, and the build still passes.
- Requiring everything is hostile. A first-time cloner shouldn’t need a 40 GB model to run
./gradlew test. - So run the same tests two ways. One system property decides whether a missing input skips the test or fails it.
1. A strict task sets the property. ./gradlew test leaves it unset; ./gradlew integrationTest,
which is what CI runs, sets it:
tasks.register<Test>("integrationTest") {
filter { includeTestsMatching("*IntegrationTest") }
systemProperty("integration.required", "true")
}2. One helper reads it. It skips when the property is unset and fails when it’s set:
static final boolean REQUIRED = Boolean.getBoolean("integration.required");
static void present(boolean there, String whatIsMissing) {
if (REQUIRED && !there) {
throw new AssertionError("no oven, no bread: " + whatIsMissing);
}
assumeTrue(there, whatIsMissing);
}3. Every test states its inputs through it, never with a bare assumeTrue:
@Test
void sourdoughRisesOvernight() {
Requires.present(
oven.isWarm(),
"the oven at " + OVEN_URL + " is cold: start it with docker compose up oven");
// …
}Security: Scan what you depend on
plugins { id("org.owasp.dependencycheck") version "13.0.0" }
dependencyCheck {
failBuildOnCVSS = 7.0f // high and critical
nvd { apiKey = providers.environmentVariable("NVD_API_KEY").orNull }
}./gradlew dependencyCheckAnalyzechecks every resolved dependency against the NVD.- Get an NVD API key. The first run downloads the whole NVD; without a key the API allows 5 requests per 30 seconds, with one 50. Cache the data directory in CI so only the first run pays.
- Suppress in a file, with a reason. A false positive goes in the suppression XML with a note, not in a lower threshold.
Supply chain: Pin dependencies the way you pin tools
dependencyLocking { lockAllConfigurations() }./gradlew dependencies --write-lockswritesgradle.lockfile. Here that’s about 175 entries from a dozen declared dependencies. Commit it.- A BOM isn’t a lock. The Spring AI BOM chooses versions. The lockfile records what actually resolved, so a change shows up as a diff in review.
- Then checksums.
./gradlew --write-verification-metadata sha256 helprecords a hash per artifact, so a swapped jar fails the build.
Supply chain: Say what’s in the box
plugins { id("org.cyclonedx.bom") version "3.5.1" }./gradlew cyclonedxBomwritesbuild/reports/cyclonedx/application.cdx.json: every component, its version and its license.- Publish one with every release, so consumers can check a new CVE without rebuilding.
Gate: Nothing counts until CI runs it
on: [push, pull_request]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/setup-java@v6
with: { distribution: temurin, java-version: 25 }
- uses: gradle/actions/setup-gradle@v6
- run: ./gradlew checkcheckruns it all: the compiler, Spotless, Error Prone with NullAway, Checkstyle, the tests and coverage. Run the full mutation run and the container tests on their own schedule if they’re too slow for every push.- Mutation-test the change on pull requests. Add
./gradlew mutasticRun -Pmutastic.since=main, and check out withfetch-depth: 0:actions/checkoutfetches one commit by default, which leaves nomainto compare against. - Make it a required status check. A red build that can still be merged is only advice.
- Pin the actions. A major tag at least. A commit SHA is safer, because tags can move.
Too slow for every push, so run it nightly:
on:
schedule: [{ cron: "0 3 * * *" }] # 03:00 UTC
workflow_dispatch: # and on demand
jobs:
mutation:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/setup-java@v6
with: { distribution: temurin, java-version: 25 }
- uses: gradle/actions/setup-gradle@v6
- run: ./gradlew mutasticRun
- uses: actions/upload-artifact@v7
if: always() # the misses are the point
with: { name: mutastic-report, path: build/mutastic }
integration:
runs-on: [self-hosted, gpu] # the oven is a 40 GB model
steps:
- uses: actions/checkout@v7
- uses: actions/setup-java@v6
with: { distribution: temurin, java-version: 25 }
- uses: gradle/actions/setup-gradle@v6
- run: ./gradlew integrationTest # strict: a cold oven fails, not skips- Run the strict task, not
test. Unattended, a skip looks like a pass. - Container tests run on hosted runners. GitHub’s Linux runners have Docker, so Testcontainers works there. Only tests that need a GPU-sized model need a self-hosted runner.
- Upload the mutastic report even when the job passes. A score above the threshold can still hide the one miss that matters.
- Someone has to see a failure. GitHub notifies only the person who last edited the
cronline. Scheduled runs use the default branch, and in a public repository they’re turned off after 60 days without activity.
The enforcement floor holds
- When a gate breaks, fix the gate. If a new JDK or tool release breaks a check, bump it, pin a known-good version or fix its config. Don’t switch it off.
- Record every weakening. A disabled rule, a lowered threshold or a suppression gets its reason beside it in the config.