Done Parallel child-process execution and wait

May 19, 2016
5
0
In my Windows batch scripts I have often had the need to do parallel execution of a number of lengthy child-programs and then (in the same script) wait for them to finish before continue with the next steps in my batch. Especially on modern machines with lots of CPU cores. The main reason for me to do this is to speed up my build-scripts by utilizing multiple CPU cores in parallel. The problem, though, has been that there is no easy and simple way to achieve this in standard Windows batch scripts.

I was hoping to find something I could use to do it with TCC, but as far as I can see there is no such support. So I spent some time googling around until I found a nice little alternative Windows shell called "Yori". Yori is open source and can be downloaded from here. It makes it easy to implement parallel process execution and wait for them to finish. In essence, what you need to do is to start the different sub-processes as so-called "jobs" and then use the internal command "wait" to wait for them to finish. You can start any process as a job by adding the special characters "&!" at the end of the command. Something like this:

ping localhost &!
ping someotherhost &!
wait

I suggest you add something similar (and probably much better and more flexible) to the set of internal commands of TCC. Some kind of "job management and registry" and a set of internal commands to work with it. Any chance to see it in the "near" future?

BTW, there are many threads on Stack Overflow about the need for this thing. For example this one. I wish I could have mentioned TCC/Take Command in my answer there ;-)
 
Last edited:

rconn

Administrator
Staff member
May 14, 2008
11,925
133
You can do this already in TCC with the START command. START will set an internal variable with the process ID of the start'd process (%_STARTPID). You can check to see if that process is still running with something like "IF isprocess %_startpid ...".

Note that START will overwrite %_STARTPID each time, so if you're START'ing several jobs you should save %_STARTPID to another variable each time.

Also, take a look at the TCC JOBS command.
 
May 19, 2016
5
0
Thank you for your reply.

You can do this already in TCC with the START command. START will set an internal variable with the process ID of the start'd process (%_STARTPID). You can check to see if that process is still running with something like "IF isprocess %_startpid ...".
Thats true, but it is complicated. I will then have to write lots of script code to store the PIDs in a dynamic array (number of jobs will vary) and loop to wait for all of them to finish. What I which for is a way to do it in a clean, flexible and simple way that keeps my batch-script clean and simple. I don't care if the underlying code in TCC is complicated - you see what I mean ;-)

Please take a look at that Yori-shell and get some good ideas for TCC! I think there are lots of people needing this without knowing it yet. I mean, - a feature like this - if implemented the right way - will be a killer feature of TCC for many use cases.

Also, take a look at the TCC JOBS command.
That is to start Windows jobs (if I understand it right). This is not what I want.
 
Aug 23, 2010
586
7
That is to group windows processes into one execution group. Which is called a job.
And it does exactly what you want.
 
May 19, 2016
5
0
That is to group windows processes into one execution group. Which is called a job.
And it does exactly what you want.
Maybe you are right. I will look deeper into JOBS, JOBMONITOR and START /JOB and see. I was not aware of Windows Jobs in this regard.

From what I have seen so far it looks relatively complicated to use these commands.

What is the simplest way of starting process A, B and C as a job and then wait for all processes in that job to finish?