Q: Unquoting vars and arrays

x13

Nov 9, 2016
72
1
Netherlands
Hi all.

I have two things...

1) Unquoting a variable.

You would think this is easy with %@UNQUOTE ...

Example:

A variable containing this (extracted form a website) :

coderes=" <small>Version : <code>2.20.2.101</code> / January 12th 2020</small>"

Normally, one would simply use %@UNQUOTE[%coderes], but then I get this:

> echo %@unquote[%coderes]
TCC: (Sys) The process cannot access the file because it is being used by another process.

I assume this has to do with a temporary file.
Other times, I get a syntax error doing echo %coderes because it contains double quotes.

So I'm looking for an easy (and painless) way to remove all double quotes.

2) By extension, how would I go about removing all double quotes from an array containing HTML ?

Example:

SETARRAY code[100]
ECHO %@EXECARRAY[code,TYPE Free Software Downloads and Reviews - Filepuma.com] >& NUL
ECHO %code[50]
TCC: (Sys) The system cannot find the file specified.
""

> ECHO "%code[50]"
" <li><a class="addthis_button_facebook_like" fb:like:layout="button_count"></a></li>"

It would be great if I could get rid of all the double quotes.
As far as I know, TPIPE has no such filter (and it works on files, not arrays)...
Any ideas?
 
Apr 18, 2014
240
8
The problem is the redirection symbols within the variable, try switching off redirection with setdos (remember to switch it back on afterwards!).
Code:
d:\>set coderes
" <small>Version : <code>2.20.2.101</code> / January 12th 2020</small>"

d:\>setdos /x-6

d:\>echo %@unquote[%coderes]
 <small>Version : <code>2.20.2.101</code> / January 12th 2020</small>

d:\>setdos /x+6
 

x13

Nov 9, 2016
72
1
Netherlands
Damnit.

I never thought of SETDOS !
But still, seems to me that redirection characters within quotes should not be executed in the first place.

Any idea about removing quotes (") throughout an array (without a loop) ?

M.
 

Charles Dye

Super Moderator
Staff member
May 20, 2008
4,193
72
Albuquerque, NM
prospero.unm.edu
But still, seems to me that redirection characters within quotes should not be executed in the first place.
But once your remove the quotes, the redirection symbols are no longer between them, right?

If you're processing HTML, you'll also need to watch out for ampersands, the default command separator.
 

x13

Nov 9, 2016
72
1
Netherlands
True. The hiccup comes from the quotes, not <> which I can remove.

I use "^" as command separator, so no problem there. ;)
 
May 20, 2008
10,555
78
Syracuse, NY, USA
True. The hiccup comes from the quotes, not <> which I can remove.

I use "^" as command separator, so no problem there. ;)
Once you have removed the other nasty stuff, you can get rid of double quotes (line by line) with @STRIP["^"",%line] (as long as there's an even number of quotes in %line).

If you have a version of TR.EXE (or use the WSL tr on Windows10) you can get rid of quotes like this.

Code:
v:\> echo "foo" | d:\gnu\tr -d \"
foo

v:\> echo "foo" | wsl tr -d \"
foo
TPIPE (/replace) should be able to do it but I haven't stumbled of the right syntax.
 
May 20, 2008
10,555
78
Syracuse, NY, USA
TPIPE can double percent signs.

Code:
v:\> type percent.txt
%
%%
%%%
%%%%

v:\> tpipe /input=percent.txt /replace=4,0,0,0,0,0,0,0,0,"%%","%%%%"
%%
%%%%
%%%%%%
%%%%%%%%
 
May 20, 2008
10,555
78
Syracuse, NY, USA
It took a while to come up with this one. TPIPE can get rid of the quotes like this.

Code:
v:\> type quotes.txt
"foo
"bar"
"garbage""
no quotes

v:\> tpipe /input=quotes.txt /replace=4,0,0,0,0,0,0,0,0,"\042",""

foo
bar
garbage
no quotes
And TPIPE /simple=16 will remove HTML. You might be able to do the whole thing with one TPIPE (which would be pretty efficient).

[/code]TPIPE /input=infile /output=outfile /simple=16 /replace=4,0,0,0,0,0,0,0,0,"\042","" /replace=4,0,0,0,0,0,0,0,0,"%%","%%%%"[/code]

That's (1) get rid of HTML, (2) get rid of quotes, (3) double percents
 
May 20, 2008
10,555
78
Syracuse, NY, USA
Yeah, I tried using TPIPE / SIMPLE=16, but t completely removes the A HREF tag,

Example:



So that won't work in this case.

M.
I don't know what you ultimately want to do, but with TPIPE and some regular expression wizardry you might be able to do it. For example, before removing HTML, you could do something to this effect.

Code:
v:\> echo "<a href="https://site.com/files/file.exe" class="btn btn-success">Download</a>" | tpipe /replace=4,0,0,0,0,0,0,0,0,"<a (href=\".+?\").*>",$1
"href="https://site.com/files/file.exe" class="btn btn-success"Download</a>"
 
May 20, 2008
10,555
78
Syracuse, NY, USA
Maybe a better example.

Code:
v:\> echo "<a href="https://site.com/files/file.exe" class="btn btn-success">Download</a>" | tpipe /replace=4,0,0,0,0,0,0,0,0,"<a (href=\"[^"]+?\").+?>",$1
"href="https://site.com/files/file.exe""
 

x13

Nov 9, 2016
72
1
Netherlands
Yeah well..

I knew someone would eventually bring regex into the equation.
I'm worthless at it. I know basic vim commands like :%s// ... but for the rest, it never sunk in.

Could you explain the working of the last two arguments of your TPIPE ?

M.
 
May 20, 2008
10,555
78
Syracuse, NY, USA
I'll try.

"<a (href=\"[^"]+?\").+?>"

is matched by: <a href="

followed by [^"]+? which is 1 or more non-quote characters (? = greedy, as many as possible)

followed by a \" (a quote)

folllowed by .+? (any 1 or more characters (greedily))

followed by >

in general . is any character, .* means 0 or more, .+ means 1 or more ... add ? and it gets greedy (real Perl is greedy by default; TPIPE's Perl is not)

[...] is like TCC, any characters in the set

[^...] means any character not in the set

" is special (delineating the regex itself) so it has to be escaped as \" except when inside [].

The () are ignored in matching. In the next argument (the replacement string), $1 means whatever was matched inside (). $1 is TPIPE lingo; in real Perl, it would be \1 (or in general, \n (inside the nth parens))
 

x13

Nov 9, 2016
72
1
Netherlands
Wow.
I'm a technical guy, but that syntax is crazy.
That's why I haven't learned it.

Thanks though. With SETDOS /X- and some non-regex manipulation I'll figure it out.

M.
 
May 20, 2008
10,555
78
Syracuse, NY, USA
Wow.
I'm a technical guy, but that syntax is crazy.
That's why I haven't learned it.

Thanks though. With SETDOS /X- and some non-regex manipulation I'll figure it out.

M.
I've been using (trying to use) regexes for close to 30 years. I still don't feel very good about them. For me, it involves a lot of triel and error. I know the basics. Beyond the basics, and when it comes to the differences between regex languages (Perl, Gnu, PCRE, ...), IT'S GOOGLE TIME.
 

x13

Nov 9, 2016
72
1
Netherlands
You're right Vefatica!

It's very powerful stuff, for sure... but one typo and it doesn't work...

Reminds me of when I used to write %@array_name[%x] and I was surprised to get a syntax error.
:)
 
May 20, 2008
10,555
78
Syracuse, NY, USA
You're right Vefatica!

It's very powerful stuff, for sure... but one typo and it doesn't work...

Reminds me of when I used to write %@array_name[%x] and I was surprised to get a syntax error.
:)
To be fair, most things don't work with a typo. In the case of regexes, a typo is worth a half an hour of trying to figure out the mistake, and then not being sure how to fix it. :-)
 

x13

Nov 9, 2016
72
1
Netherlands
So true.

I even tried an online regex expression builder... didn't get very far.
The easiest I found is to copy someone else's example online.

M.
 
Apr 18, 2014
240
8
To make a long story short... RogerB gets the cigar because SETDOS /X-6 is exactly what I needed!

Thanks again.
M.
Thanks for the cigar :smile:

As for regexes, I'm with Vince on the trial and error approach. Whenever I use them I'm reminded of this quote:

Some people, when confronted with a problem, think “I know, I'll use regular expressions.” Now they have two problems.
In that form the quote is usually attributed to Jamie Zawinski, although it goes back further and is one of those expressions where you can insert your own choice into the "I know, I'll use..." bit.
 

x13

Nov 9, 2016
72
1
Netherlands
Yes I understand that.

But it's not like you could do a TPIPE and apply it to a whole array in one line (/INPUT and /OUTPUT have to be files)
 
May 20, 2008
10,555
78
Syracuse, NY, USA
Yes I understand that.

But it's not like you could do a TPIPE and apply it to a whole array in one line (/INPUT and /OUTPUT have to be files)
I'm not sure what you mean. With /output unspecified, you get stdout and %@execarray[array_name,TPIPE ...] works fine.
 
May 20, 2008
10,555
78
Syracuse, NY, USA
You don't need /input either which makes TPIPE work in pipes.

Code:
v:\> echo foo^r^nbar | tpipe /grep=3,0,0,0,0,0,0,0,foo
foo