Using Perl 6 on Windows I want to programatically call a .bat file. Both the path to the .bat file and one of its arguments have a space in it. I can not call the .bat file directly for reasons given at the end of the question. The call I want to make is:
"C:\data\p6 repos\rakudo\perl6-m.bat" --target=mbc "--output=C:/data/p6 repos/rakudo/lib/.precomp/EA6F1A9F9A17B17B899C4C7C4A775A883DAF537E/33/33A52796DB3EBB40BEF94B7696A1B0AB7A29B5C5.bc" "C:/data/p6 repos/rakudo/lib/CompUnit/Repository/Staging.pm6"
I need to capture its STDIN and STDOUT and retrieve the exit code.
How do I do this?
According to this superuser answer one has to use the following command to run this (notice the extra quotes at the very start and end of my command):
cmd.exe /C ""C:\data\p6 repos\rakudo\perl6-m.bat" --target=mbc "--output=C:/data/p6 repos/rakudo/lib/.precomp/EA6F1A9F9A17B17B899C4C7C4A775A883DAF537E/33/33A52796DB3EBB40BEF94B7696A1B0AB7A29B5C5.bc" "C:/data/p6 repos/rakudo/lib/CompUnit/Repository/Staging.pm6""
That works when pasted into a cmd command window.
Doing this in Perl 6 errors out. Doing the same in Python gives the same errors. This is my code.
my $perl6 = 'C:\\data\\p6 repos\\rakudo\\perl6-m.bat';
my $bc = 'C:/data/p6 repos/rakudo/lib/.precomp/EA6F1A9F9A17B17B899C4C7C4A775A883DAF537E/33/33A52796DB3EBB40BEF94B7696A1B0AB7A29B5C5.bc';
my $path = 'C:/data/p6 repos/rakudo/lib/CompUnit/Repository/Staging.pm6';
react {
my $proc = Proc::Async.new(
'C:\WINDOWS\system32\cmd.exe', '/C', "\"\"$perl6\" --target=mbc \"--output=$bc\" \"$path\"\""
);
whenever $proc.stdout {
say "Out: $_"
}
whenever $proc.stderr {
say "Err: $_"
}
whenever $proc.start(ENV => %(RAKUDO_MODULE_DEBUG => 1)) {
say "Status: {$_.exitcode}"
}
}
This results in:
Err: The network path was not found.
Using forward slashes in the $perl6 command results in:
Err: The filename, directory name, or volume label syntax is incorrect.
Leaving the surrounding quotation marks of the command off
'C:\WINDOWS\system32\cmd.exe', '/C', "\"$perl6\" --target=mbc \"--output=$bc\" \"$path\""
results in:
Err: '\"C:\data\p6 repos\rakudo\perl6-m.bat\"' is not recognized as an internal or external command,
operable program or batch file
Err: .
Passing the arguments separately in the command
'C:\WINDOWS\system32\cmd.exe', '/C', $perl6, "--target=mbc", "--output=$bc", $path
results in:
Err: 'C:\data\p6' is not recognized as an internal or external command,
operable program or batch file.
Quoting the executable but leaving the arguments separately
'C:\WINDOWS\system32\cmd.exe', '/C', "\"$perl6\"", "--target=mbc", "--output=$bc", $path
results in:
Err: The filename, directory name, or volume label syntax is incorrect.
Also quoting the other arguments separately
'C:\WINDOWS\system32\cmd.exe', '/C', "\"$perl6\"", "\"--target=mbc\"", "\"--output=$bc\"", "\"$path\""
results in:
Err: '\"C:\data\p6 repos\rakudo\perl6-m.bat\"" "\"--target=mbc\"" "\"--output=C:/data/p6 repos/rakudo/lib/.precomp/EA6F1A9F9A17B17B8
Err: 99C4C7C4A775A883DAF537E/33/33A52796DB3EBB40BEF94B7696A1B0AB7A29B5C5.bc\"" "\"C:/data/p6 repos/rakudo/lib/CompUnit/Repository/Staging.pm6\"' is not recognized as an internal or external command,
operable program or batch file.
Keeping the arguments separate but adding quotationmarks at the beginning and end
'C:\WINDOWS\system32\cmd.exe', '/C', "\"\"$perl6\"", "\"--target=mbc\"", "\"--output=$bc\"", "\"$path\"\""
results in:
Err: The network path was not found.
The errors I get are largely the same when using comparable code on Python 3 and nodejs. So I'm pretty sure this is a problem further down.
So I think I tried everything one can possibly think of. I start to suspect that's a bug in cmd.exe and it's impossible to do.
On node.js the following works:
complete = '""C:\\data\\p6 repos\\rakudo\\perl6-m.bat" --target=mbc "--output=C:/data/p6 repos/rakudo/lib/.precomp/EA6F1A9F9A17B17B899C4C7C4A775A883DAF537E/33/33A52796DB3EBB40BEF94B7696A1B0AB7A29B5C5.bc" "C:/data/p6 repos/rakudo/lib/CompUnit/Repository/Staging.pm6""'
const { spawn } = require('child_process');
const bat = spawn('cmd.exe', ['/c', complete], { shell: true });
bat.stdout.on('data', (data) => {
console.log(data.toString());
});
bat.stderr.on('data', (data) => {
console.log(data.toString());
});
bat.on('exit', (code) => {
console.log(`Child exited with code ${code}`);
});
The trick is, that the above command results in two wrapped cmd.exe calls. That in turn gets rid of the limitation described above.
Sadly just copying this trick in Perl 6 still doesn't succeed
'C:\WINDOWS\system32\cmd.exe', '/d', '/s', '/c', "\"cmd.exe /c \"\"$perl6\" --target=mbc \"--output=$bc\" \"$path\"\""
That results in:
Err: The network path was not found.
The reason is, that libuv messes with the arguments it gets passed and puts a backslash before every quotation mark. There is a flag to turn that off called UV_PROCESS_WINDOWS_VERBATIM_ARGUMENTS.
The reason this works on node.js is, that it has some special handling that is triggered when 1. on windows, 2. the shell argument is passed to spawn and 3. the command one executes is cmd.exe again. In this very specific situation node passes the options /d /s /c to its cmd.exe and activates the UV_PROCESS_WINDOWS_VERBATIM_ARGUMENTS flag.
When adding that flag to moarvms implementation of spawnprocasync it works in Perl 6 also. Though I have no idea what other side effects that will have.
When calling the .bat file directly the respective line in the Perl 6 code above becomes
my $proc = Proc::Async.new( $perl6, "--target=mbc", "--output=$bc", $path );
The code errors out with:
Err: 'C:/data/p6' is not recognized as an internal or external command, operable program or batch file.
Here is some similar code in Python 3.
import subprocess
perl6 = 'C:/data/p6 repos/rakudo/perl6-m.bat'
bc = 'C:/data/p6 repos/rakudo/lib/.precomp/EA6F1A9F9A17B17B899C4C7C4A775A883DAF537E/33/33A52796DB3EBB40BEF94B7696A1B0AB7A29B5C5.bc'
path = 'C:/data/p6 repos/rakudo/lib/CompUnit/Repository/Staging.pm6'
p = subprocess.Popen([perl6, "--target=mbc", f"--output={bc}", path], stdout=subprocess.PIPE)
while p.poll() is None:
l = p.stdout.readline()
print(l)
print(p.stdout.read())
On Python this works.
The reason is, that Python calls the Windows API function CreateProcessW without specifying the first argument lpApplicationName. This is the place the actual call happens. application_name is left empty as long as one doesn't pass the executable argument to the subprocesss.Popen call above. When one passes the executable argument to subprocess.Popen the error is the same I get using Perl 6.
p = subprocess.Popen([perl6, "--target=mbc", f"--output={bc}", path], executable="\"C:/data/p6 repos/rakudo/perl6-m.bat\"", stdout=subprocess.PIPE)
Perl 6 uses libuv under the hood. In libuv it is currently not possible to not pass the first argument to CreateProcessW. Thus I can not call the .bat file directly as is possible using Python. The node.js docs (node.js also uses libuv) tell the same story.
@ugexe: Can you elaborate a bit more on the quotemeta solution? I think I might misunderstand your idea.
If I understand that idea correctly you propose to just never quote commands in a meaningful way on Windows and leave it completely up to the user. That would have the following effect: Multiple arguments can be passed to Proc::Async, giving the impression they are actually somehow processed separately. But in reality they are just joined together with a space. This counter intuitive behavior would bite everyone trying to do usecase 1. which is by far the largest usecase.
I do like the idea of implementing the quoting in rakudo though. All backends can then just call
CreateProcessWdirectly in the spawnprocasync OP implementation.