ChatGPT解决这个技术问题 Extra ChatGPT

How can I run multiple npm scripts in parallel?

In my package.json I have these two scripts:

  "scripts": {
    "start-watch": "nodemon run-babel index.js",
    "wp-server": "webpack-dev-server",
  }

I have to run these 2 scripts in parallel everytime I start developing in Node.js. The first thing I thought of was adding a third script like this:

"dev": "npm run start-watch && npm run wp-server"

... but that will wait for start-watch to finish before running wp-server.

How can I run these in parallel? Please keep in mind that I need to see the output of these commands. Also, if your solution involves a build tool, I'd rather use gulp instead of grunt because I already use it in another project.

&& will run your scripts sequentially while & will run them in parallel.
A quick way of doing it is npm run start-watch & npm run wp-server. This will run the first command as a background thread. This works really well when one of the commands is not long running and does not need to be manually exited later. Something like concurrently allows you to kill all the threads at the same time with CTRL-C.
@vsync Does that apply to Windows?
@vsync Are you sure? Other comments are saying that's not how it works, and it didn't work in practice for me.
@Clonkex, yes BUT it's unreliable and I use concurrently npm package instead, which works well, and I only use Windows

S
Soviut

Use a package called concurrently.

npm i concurrently --save-dev

Then setup your npm run dev task as so:

"dev": "concurrently --kill-others \"npm run start-watch\" \"npm run wp-server\""

node ./node_modules/concurrently/src/main.js is not needed. concurrent will work just fine in scripts because the module installs a bin to ./node_modules/.bin/concurrent
There is also parallelshell. I actually recommend that one as concurrently uses multiple streams that mess with console output (colors may go weird, cursor gone) whereas parallelshell doesn't have that issue.
The bugs in concurrently mentioned by @StijndeWitt have now been fixed in 2.0.0 release. You can use --raw mode to preserve colors in output.
@StijndeWitt parallelshell has been deprecated in favor of npm-run-all github.com/keithamus/…
There has to be a better way for us to manage Javascript build/run scripts. Everything for this platform seems tacked together. quotes with escaped quotes and npm builds to call other 'npm run' builds.. This is getting pretty painful.
D
Diogo Cardoso

If you're using an UNIX-like environment, just use & as the separator:

"dev": "npm run start-watch & npm run wp-server"

Otherwise if you're interested on a cross-platform solution, you could use npm-run-all module:

"dev": "npm-run-all --parallel start-watch wp-server"

I do this - from time to time when I "ctrl-c" npm, the command keeps hanging on in background... Any ideas?
a && b starts b after a finished successfully, but nodemon never stops without errors, so that can't work. a & b starts a, moves it to the background and starts b right away. Win! a | b pipes the stdout of a to the stdin of b which requires both running simultaneously. Although this might seem to have the desired effect, you shouldn't use it here.
@KamilTomšík & is a really bad idea as it detaches the process. It means that npm will not be the parent process anymore. You'll end up with a zombie npm run start-watch that won't be killed with ctrl-c.
Just add wait to mitigate problem with hanging processes: "dev": "npm run start-watch & npm run wp-server & wait"
It’s not a zombie. But & on unix prevents the command from responding to C-c/C-z and also prevents its return code from propagating in case of a failure.
O
Oleg

From windows cmd you can use start:

"dev": "start npm run start-watch && start npm run wp-server"

Every command launched this way starts in its own window.


Perfect solution! I love that it launches the new window. Great for VS2015 package.json needs
This does not work if you have watcher tasks because && waits for the first command to finish before starting the second command and a watcher task will never finish.
@BennyNeugebauer The commands are preceded with the "start" command which opens up a new command line for each of the commands. I was confused at first as well because I thought "using the && operator will not work". This solution is very simple and requires no additional packages/work from the developer.
This is wrong. Command will be run sequentially. On Windows you have to use a plugin in order to run commands simultaneously.
it also now means i have to use windows work on your projects.
R
RyanZim

You should use npm-run-all (or concurrently, parallelshell), because it has more control over starting and killing commands. The operators &, | are bad ideas because you'll need to manually stop it after all tests are finished.

This is an example for protractor testing through npm:

scripts: {
  "webdriver-start": "./node_modules/protractor/bin/webdriver-manager update && ./node_modules/protractor/bin/webdriver-manager start",
  "protractor": "./node_modules/protractor/bin/protractor ./tests/protractor.conf.js",
  "http-server": "./node_modules/http-server/bin/http-server -a localhost -p 8000",
  "test": "npm-run-all -p -r webdriver-start http-server protractor"
}

-p = Run commands in parallel.

-r = Kill all commands when one of them finishes with an exit code of zero.

Running npm run test will start Selenium driver, start http server (to serve you files) and run protractor tests. Once all tests are finished, it will close the http server and the selenium driver.


I wonder how this works properly for running the tests, though. Whilst webdriver-start and http-server can run in parallel, the protractor task should only run after the first two.
@asenovm for order dependant tasks, why not just use gulp and gulp-sync?
J
Joe Chung

You can use one & for parallel run script

"dev": "npm run start-watch & npm run wp-server"

Reference link


Will this also work in Windows? Sorry, I am pretty new to node and I don't know how to verify this!
@BenisonSam I tried on my Windows pc, it doesn't run the 2nd command even with single "&"
The same answer got posted 4 years ago and has less upvotes than this. Also it's discussed enough why this approach is a bad idea, already. Uhm, why this has so many upvotes, again?
@MartinBraun quick and easy
tnx - This is the answer - all the other mentioned solutions are overkill
C
Corey

A better solution is to use &

"dev": "npm run start-watch & npm run wp-server"

No, it's not better because it does not work on all platforms.
I did not know that. What platforms doesn't it work on? @Corey - update your answer with the warning on inter-op and I'll upvote you
& works on Windows, but it works differently. On OSX, it will run both commands concurrently, but on Windows, it will run the first command, and after the first command exists, it will run the second command.
No it's not as it detaches the process, you won't be able to kill it in a simple fashion.
@ngryman That's what I expected too. However, I tried this and it does kill all three processes (dev, start-watch, and wp-server) when you hit Ctrl+C.
D
Darkowic

I've checked almost all solutions from above and only with npm-run-all I was able to solve all problems. Main advantage over all other solution is an ability to run script with arguments.

{
  "test:static-server": "cross-env NODE_ENV=test node server/testsServer.js",
  "test:jest": "cross-env NODE_ENV=test jest",
  "test": "run-p test:static-server \"test:jest -- {*}\" --",
  "test:coverage": "npm run test -- --coverage",
  "test:watch": "npm run test -- --watchAll",
}

Note run-p is shortcut for npm-run-all --parallel

This allows me to run command with arguments like npm run test:watch -- Something.

EDIT:

There is one more useful option for npm-run-all:

 -r, --race   - - - - - - - Set the flag to kill all tasks when a task
                            finished with zero. This option is valid only
                            with 'parallel' option.

Add -r to your npm-run-all script to kill all processes when one finished with code 0. This is especially useful when you run a HTTP server and another script that use the server.

  "test": "run-p -r test:static-server \"test:jest -- {*}\" --",

Another useful option is -l or --print-labels - it prints the task name as a prefix on each line of output, so you can tell them apart. Nicely colored also.
E
Entity Black

I have a crossplatform solution without any additional modules. I was looking for something like a try catch block I could use both in the cmd.exe and in the bash.

The solution is command1 || command2 which seems to work in both enviroments same. So the solution for the OP is:

"scripts": {
  "start-watch": "nodemon run-babel index.js",
  "wp-server": "webpack-dev-server",
  // first command is for the cmd.exe, second one is for the bash
  "dev": "(start npm run start-watch && start npm run wp-server) || (npm run start-watch & npm run wp-server)",
  "start": "npm run dev"
}

Then simple npm start (and npm run dev) will work on all platforms!


Double || didn't seem to work on my Windows 10 PowerShell, however, a single | seems to do fine even on PowerShell. I tried it with just two commands and could only see the output of 2nd part and not 1st one.
@HarshitGupta || might not be implemented in Windows Ppowershell. Apparently it was introduced in PowerShell [Core] 7.0 but might not be backported into Windows Powershell. Sadly my solution isn't bulletproof.
N
Neil Girardi

If you replace the double ampersand with a single ampersand, the scripts will run concurrently.


Exactly, it's simple and elegant, no need for dependencies or other magic.
@Ginzburg Because don't works the same for all platforms, like you can see in other answers.
B
Boaz

How about forking

Another option to run multiple Node scripts is with a single Node script, which can fork many others. Forking is supported natively in Node, so it adds no dependencies and is cross-platform.

Minimal example

This would just run the scripts as-is and assume they're located in the parent script's directory.

// fork-minimal.js - run with: node fork-minimal.js

const childProcess = require('child_process');

let scripts = ['some-script.js', 'some-other-script.js'];
scripts.forEach(script => childProcess.fork(script));

Verbose example

This would run the scripts with arguments and configured by the many available options.

// fork-verbose.js - run with: node fork-verbose.js

const childProcess = require('child_process');

let scripts = [
    {
        path: 'some-script.js',
        args: ['-some_arg', '/some_other_arg'],
        options: {cwd: './', env: {NODE_ENV: 'development'}}
    },    
    {
        path: 'some-other-script.js',
        args: ['-another_arg', '/yet_other_arg'],
        options: {cwd: '/some/where/else', env: {NODE_ENV: 'development'}}
    }
];

let runningScripts= [];

scripts.forEach(script => {
    let runningScript = childProcess.fork(script.path, script.args, script.options);

   // Optionally attach event listeners to the script
   runningScript.on('close', () => console.log('Time to die...'))

    runningScripts.push(runningScript); // Keep a reference to the script for later use
});

Communicating with forked scripts

Forking also has the added benefit that the parent script can receive events from the forked child processes as well as send back. A common example is for the parent script to kill its forked children.

 runningScripts.forEach(runningScript => runningScript.kill());

For more available events and methods see the ChildProcess documentation


e
eaorak
npm-run-all --parallel task1 task2

edit:

You need to have npm-run-all installed beforehand. Also check this page for other usage scenarios.


j
james_womack

Quick Solution

In this case, I'd say the best bet If this script is for a private module intended to run only on *nix-based machines, you can use the control operator for forking processes, which looks like this: &

An example of doing this in a partial package.json file:

{
  "name": "npm-scripts-forking-example",
  "scripts": {
    "bundle": "watchify -vd -p browserify-hmr index.js -o bundle.js",
    "serve":  "http-server -c 1 -a localhost",
    "serve-bundle": "npm run bundle & npm run serve &"
  }

You'd then execute them both in parallel via npm run serve-bundle. You can enhance the scripts to output the pids of the forked process to a file like so:

"serve-bundle": "npm run bundle & echo \"$!\" > build/bundle.pid && npm run serve & echo \"$!\" > build/serve.pid && npm run open-browser",

Google something like bash control operator for forking to learn more on how it works. I've also provided some further context regarding leveraging Unix techniques in Node projects below:

Further Context RE: Unix Tools & Node.js

If you're not on Windows, Unix tools/techniques often work well to achieve something with Node scripts because:

Much of Node.js lovingly imitates Unix principles You're on *nix (incl. OS X) and NPM is using a shell anyway

Modules for system tasks in Nodeland are also often abstractions or approximations of Unix tools, from fs to streams.


Nope, as & operator is not supported on Windows.
@StijndeWitt my post says "If you're not on Windows...". 0% of the folks I work with, at one of the largest tech companies in the world, run Node on Windows. So clearly my post is still valuable to many developers.
It's kind of a circular way of reasoning isn't it though? If you write your npm scripts like this you will not be able to use Windows because it won't work. So no one uses Windows, so it doesn't matter that it does not work... You end up with platform dependent software. Now if the thing that needs to be done is very hard to do cross-platform, than that might be a good trade-off to make. But this problem right here is very easy to do with standard npm scripts such as concurrently and parallelshell.
@StijndeWitt None of my reasoning was circular. I made a statement of fact sans reasoning. We're posting techniques common to Node developers, many of whom build & deploy on Linux servers. Yes, it should work on Windows if it's a userland script, but the majority of npm scripts are for development and deployment—mostly on *nix machines. Regarding the modules you mentioned a) it's an enormous stretch to call concurrently and parallelshell "standard" (~1500 downloads a day is far from standard in NPMland) and b) if you need additional software for a parallel process, you might as well use Gulp.
@StijndeWitt I appreciate being made aware of those modules though—thank you
A
Artem Belik
npm install npm-run-all --save-dev

package.json:

"scripts": {
  "start-watch": "...",
  "wp-server": "...",
  "dev": "npm-run-all --parallel start-watch wp-server"
}

More info: https://github.com/mysticatea/npm-run-all/blob/master/docs/npm-run-all.md


No it does not.
g
gmspacex

Just add this npm script to the package.json file in the root folder.

{
  ...
  "scripts": {
    ...
    "start": "react-scripts start", // or whatever else depends on your project
    "dev": "(cd server && npm run start) & (cd ../client && npm run start)"
  }
}

M
Mosia Thabo

... but that will wait for start-watch to finish before running wp-server.

For that to work, you will have to use start on your command. Others have already illustrated but this is how it will work, your code below:

"dev": "npm run start-watch && npm run wp-server"

Should be :

"dev": " start npm run start-watch && start npm run wp-server"

What this will do is, it will open a separate instance for each command and process them concurrently, which shouldn't be an issue as far as your initial issue is concerned. Why do I say so? It's because these instances both open automatically while you run only 1 statement, which is your initial goal.


I got this error. Where from start command. Error: spawn start ENOENT at Process.ChildProcess._handle.onexit (internal/child_process.js:269:19) at onErrorNT (internal/child_process.js:467:16) at processTicksAndRejections (internal/process/task_queues.js:82:21)
M
Muhammad Muzamil

step by step guide to run multiple parallel scripts with npm. install npm-run-all package globally

npm i -g npm-run-all

Now install and save this package within project where your package.json exists

npm i npm-run-all --save-dev

Now modify scripts in package.json file this way

"scripts": {
    "server": "live-server index.html",
    "watch": "node-sass scss/style.scss --watch",
    "all": "npm-run-all --parallel server watch"
},

now run this command

npm run all

more detail about this package in given link npm-run-all


i
ian

I ran into problems with & and |, which exit statuses and error throwing, respectively.

Other solutions want to run any task with a given name, like npm-run-all, which wasn't my use case.

So I created npm-run-parallel that runs npm scripts asynchronously and reports back when they're done.

So, for your scripts, it'd be:

npm-run-parallel wp-server start-watch


佚名

My solution is similar to Piittis', though I had some problems using Windows. So I had to validate for win32.

const { spawn } = require("child_process");

function logData(data) {
    console.info(`stdout: ${data}`);
}

function runProcess(target) {
    let command = "npm";
    if (process.platform === "win32") {
        command = "npm.cmd"; // I shit you not
    }
    const myProcess = spawn(command, ["run", target]); // npm run server

    myProcess.stdout.on("data", logData);
    myProcess.stderr.on("data", logData);
}

(() => {
    runProcess("server"); // package json script
    runProcess("client");
})();

S
SB3NDER

In a package.json in the parent folder:

"dev": "(cd api && start npm run start) & (cd ../client && start npm run start)"

this work in windows


n
nick

This worked for me

{
"start-express": "tsc && nodemon dist/server/server.js",
"start-react": "react-scripts start",
"start-both": "npm -p -r run start-react && -p -r npm run start-express"
}

Both client and server are written in typescript.

The React app is created with create-react-app with the typescript template and is in the default src directory.

Express is in the server directory and the entry file is server.js

typescript code and transpiled into js and is put in the dist directory .

checkout my project for more info: https://github.com/nickjohngray/staticbackeditor

UPDATE: calling npm run dev, to start things off

{"server": "tsc-watch --onSuccess \"node ./dist/server/index.js\"",
"start-server-dev": "npm run build-server-dev && node src/server/index.js",
"client": "webpack-dev-server --mode development --devtool inline-source-map --hot",
"dev": "concurrently \"npm run build-server-dev\"  \"npm run server\" \"npm run client\""}

I have updated my script, i thought that was working , I posted my update above
what is -p and -r for on npm?
C
Callat

In my case I have two projects, one was UI and the other was API, and both have their own script in their respective package.json files.

So, here is what I did.

npm run --prefix react start&  npm run --prefix express start&

Like your solution. Also have UI (node app) and API (Angular in a subfolder src, guess is cd src/ng serve), only the first part works. For example node app& cd src& ng serve.
P
Piittis

Simple node script to get you going without too much hassle. Using readline to combine outputs so the lines don't get mangled.

const { spawn } = require('child_process');
const readline = require('readline');

[
  spawn('npm', ['run', 'start-watch']),
  spawn('npm', ['run', 'wp-server'])
].forEach(child => {
    readline.createInterface({
        input: child.stdout
    }).on('line', console.log);

    readline.createInterface({
        input: child.stderr,
    }).on('line', console.log);
});

T
Thomas Ebert

I have been using npm-run-all for some time, but I never got along with it, because the output of the command in watch mode doesn't work well together. For example, if I start create-react-app and jest in watch mode, I will only be able to see the output from the last command I ran. So most of the time, I was running all my commands manually...

This is why, I implement my own lib, run-screen. It still very young project (from yesterday :p ) but it might be worth to look at it, in your case it would be:

run-screen "npm run start-watch" "npm run wp-server"

Then you press the numeric key 1 to see the output of wp-server and press 0 to see the output of start-watch.


I use npm-run-all and I get the output of both processes in the terminal.
Right, I think they did some update on the way to handle output, I lately use npm-run-all and seem to work pretty well so far.
w
woutvdd

You can also use pre and post as prefixes on your specific script.

  "scripts": {
    "predev": "nodemon run-babel index.js &",
    "dev": "webpack-dev-server"
  }

And then run: npm run dev


Y
Yilmaz

with installing npm install concurrently

"scripts": {
    "start:build": "tsc -w",
    "start:run": "nodemon build/index.js",
    "start": "concurrently  npm:start:*"
  },

A
AlexRMU

A simple and native way for Windows CMD

"start /b npm run bg-task1 && start /b npm run bg-task2 && npm run main-task"

(start /b means start in the background)


Great this works, this works in Windows as well.
J
Jamal

I think the best way is to use npm-run-all as below:

1- npm install -g npm-run-all <--- will be installed globally
2- npm-run-all --parallel server client


E
Edward Brey

Use concurrently to run the commands in parallel with a shared output stream. To make it easy to tell which output is from which process, use the shortened command form, such as npm:wp-server. This causes concurrently to prefix each output line with its command name.

In package.json, your scripts section will look like this:

 "scripts": {
    "start": "concurrently \"npm:start-watch\" \"npm:wp-server\"",
    "start-watch": "nodemon run-babel index.js",
    "wp-server": "webpack-dev-server"
  }

P
Peter M. Elias

Using just shell scripting, on Linux.

"scripts": {
  "cmd": "{ trap 'trap \" \" TERM; kill 0; wait' INT TERM; } && blocking1 & blocking2 & wait"
}

npm run cmd and then ^C will kill children and wait for clean exit.


Z
ZiiMakc

As you may need to add more and more to this scripts it will become messy and harder to use. What if you need some conditions to check, variables to use? So I suggest you to look at google/zx that allows to use js to create scripts.

Simple usage:

install zx: npm i -g zx add package.json commands (optional, you can move everything to scripts):

  "scripts": {
    "dev": "zx ./scripts/dev.mjs", // run script
    "build:dev": "tsc -w", // compile in watch mode
    "build": "tsc", // compile
    "start": "node dist/index.js", // run
    "start:dev": "nodemon dist/index.js", // run in watch mode
  },

create dev.mjs script file:

#!/usr/bin/env zx

await $`yarn build`; // prebuild if dist is empty
await Promise.all([$`yarn start:dev`, $`yarn build:dev`]); // run in parallel

Now every time you want to start a dev server you just run yarn dev or npm run dev.

It will first compile ts->js and then run typescrpt compiler and server in watch mode in parallel. When you change your ts file->it's will be recompiled by tsc->nodemon will restart the server.

Advanced programmatic usage

Load env variables, compile ts in watch mode and rerun server from dist on changes (dev.mjs):

#!/usr/bin/env zx
import nodemon from "nodemon";
import dotenv from "dotenv";
import path from "path";
import { fileURLToPath } from "url";

// load env variables
loadEnvVariables("../env/.env");

await Promise.all([
  // compile in watch mode (will recompile on changes in .ts files)
  $`tsc -w`,
  // wait for tsc to compile for first time and rerun server on any changes (tsc emited .js files)
  sleep(4000).then(() =>
    nodemon({
      script: "dist/index.js",
    })
  ),
]);

function sleep(ms) {
  return new Promise((resolve) => {
    setTimeout(resolve, ms);
  });
}

function getDirname() {
  return path.dirname(fileURLToPath(import.meta.url));
}

function loadEnvVariables(relativePath) {
  const { error, parsed } = dotenv.config({
    path: path.join(getDirname(), relativePath),
  });

  if (error) {
    throw error;
  }

  return parsed;
}