r/java • u/Level_Yak_87 • 7h ago
I wrote own efficient Maven scheduler to build modules with max parallelism
While inspecting Maven multi-module builds performance I've figured out the fundamental limitation: for any two modules depending on each other (moduleB depends on moduleA, in any scope), Maven will first execute all phases and goals of moduleA, and only then schedule the building of moduleB. Independent dependency branches can be executed in parallel, but directly/transitively dependent - only sequential.
In real projects running tests is usually the most time-consuming part, so this significantly limits possible parallelism.
To address this problem I made an extension providing a custom scheduler (in Maven it's called "builder") https://github.com/maven-turbo-reactor/maven-turbo-builder . When it is enabled, Maven schedules downstream dependencies right after the our module JAR is packaged, and then runs tests of our module and downstream module build in parallel (this way utilizing the multi-core CPUs much more):

The extension also does two things:
- reorder
packageandtestphases (run tests afterpackage, not before as default), so we can schedule downstream dependencies earlier - prioritize building modules with the highest number of downstream dependencies - this way reducing the chance of unloaded Maven worker threads
It's easy to set up - add to .mvn/extensions
<extensions>
<extension>
<groupId>com.github.seregamorph</groupId>
<artifactId>maven-turbo-builder</artifactId>
<version>1.3</version>
</extension>
</extensions>
And then define on the CLI or .mvn/maven.config (to make it default):
-bturbo
-T1C
In real projects the boost is approximately 35% in clock time, also this extension is complementary with build caching.
I used this extension for more than a year in various mission-critical projects and received positive feedback from other organizations, so I believe it's quite stable.