ChatGPT解决这个技术问题 Extra ChatGPT

Maven: how to do parallel builds?

When you build with maven on a multicore / multi-CPU machine it would often be possible to build different subprojects in parallel. Is there a way to do this with maven? Is there a plugin for this / whatever?


S
Sergey Vyacheslavovich Brunov

Maven 3 (as of beta 1) now supports parallel builds as an experimental feature.

For example,

mvn -T 4 clean install # Builds with 4 threads
mvn -T 1C clean install # 1 thread per cpu core
mvn -T 1.5C clean install # 1.5 thread per cpu core

Full documentation can be found on the Maven wiki: Parallel builds in Maven 3 - Apache Maven - Apache Software Foundation.


What does 1.5 threads mean? If I have a single core system, does that mean 1 thread, but a dual core system results in 3 threads?
@SaadMalik Yes, exactly, 3 threads on dual core, 6 threads on quad core systems, etc.
Is there a argument we can use for the JVM, so we can use this in eclipse?
What the point in using more that 1 thread per CPU?
@Enbugger some tests may cause threads to partially idle during tests, for example when waiting for endpoint or database responses.
S
Sergey Vyacheslavovich Brunov

The suggested solutions are great, but I wanted to add something to the answers here regarding the test stability during parallel builds.

So, when Maven parallel build is used:

mvn -T 4 clean install # Builds with 4 threads
mvn -T 1C clean install # 1 thread per cpu core
mvn -T 1.5C clean install # 1.5 thread per cpu core

Some issues with tests can appear. Note any behavior in tests which is different between serial and parallel test execution. Most of the times it happens do to improper test isolation resource-wise.

For example, test1 manipulate db entry with key 12345, which is hard-coded and test2 uses the same entry! It can't be good…

It's a situation that should be considered in the first place, but sometime it's forgotten and could lead to different problems once the switch to parallel maven build is made.

In case that happens, and you still want to use the parallel execution at least in some of the occasions, you can (of course, besides trying to fix the test and make them properly isolated) to disable Maven test runs using -DskipTests argument:

mvn clean install -T 4 -DskipTests

This is not the point of the question and it isn't good advice. Tests that break in parallel execution are bad tests, They may also be signaling bad code design. It is better to listen and act on it. Skipping tests to solve this is just sweeping something under the rug.
How it's related? When someone introduces parallel execution in his tests, it can happen. It's just pragmatism.
When running parallel builds, their test phases are also parallel, therefore make sure the debugging is disabled, as the allocation of the debug port breaks silently the parallel builds.
j
jor

Some of the CI build applications (e.g. hudson) can build multiple maven projects at the same time (and even on multiple machines).

Support for this in maven 'standalone' would also be nice, a quick look through the maven issue tracker gave me: http://jira.codehaus.org/browse/MNG-3004


I think the Hudson functionality for multiple maven projects at the same time is broken as is indicated in the MNG-3004 JIRA comments.
J
JonnyRaa

If you came to this question looking to sort out your build server and you're not using one that deals with maven natively the magic flag you are looking for is this:

-Dmaven.repo.local=someNoneGlobalDir

do that for each one of your builds and you can let them run all at the same time rather than having everything that uses maven in a queue!