Q: Unquoting vars and arrays

[x13]

Hi all.

I have two things...

1) Unquoting a variable.

You would think this is easy with %@UNQUOTE ...

Example:

A variable containing this (extracted form a website) :

coderes=" <small>Version : <code>2.20.2.101</code> / January 12th 2020</small>"

Normally, one would simply use %@UNQUOTE[%coderes], but then I get this:

> echo %@unquote[%coderes]
TCC: (Sys) The process cannot access the file because it is being used by another process.

I assume this has to do with a temporary file.
Other times, I get a syntax error doing echo %coderes because it contains double quotes.

So I'm looking for an easy (and painless) way to remove all double quotes.

2) By extension, how would I go about removing all double quotes from an array containing HTML ?

Example:

SETARRAY code[100]
ECHO %@EXECARRAY[code,TYPE Free Software Downloads and Reviews - Filepuma.com] >& NUL
ECHO %code[50]
TCC: (Sys) The system cannot find the file specified.
""

> ECHO "%code[50]"
" <li><a class="addthis_button_facebook_like" fb:like:layout="button_count"></a></li>"

It would be great if I could get rid of all the double quotes.
As far as I know, TPIPE has no such filter (and it works on files, not arrays)...

Any ideas?

 

[RogerB]

The problem is the redirection symbols within the variable, try switching off redirection with setdos (remember to switch it back on afterwards!).

d:\>set coderes
" <small>Version : <code>2.20.2.101</code> / January 12th 2020</small>"

d:\>setdos /x-6

d:\>echo %@unquote[%coderes]
 <small>Version : <code>2.20.2.101</code> / January 12th 2020</small>

d:\>setdos /x+6

 

[x13]

Damnit.

I never thought of SETDOS !
But still, seems to me that redirection characters within quotes should not be executed in the first place.

Any idea about removing quotes (") throughout an array (without a loop) ?

M.

 

[Charles Dye]

But still, seems to me that redirection characters within quotes should not be executed in the first place.

But once your remove the quotes, the redirection symbols are no longer between them, right?

If you're processing HTML, you'll also need to watch out for ampersands, the default command separator.

 

[x13]

True. The hiccup comes from the quotes, not <> which I can remove.

I use "^" as command separator, so no problem there. ;)

 

[vefatica]

If you're processing HTML, you'll also need to watch out for ampersands, the default command separator.

Percent signs too. Single ones will get lost when used in a command, even if quoted.

 

[x13]

Good point.

 

[vefatica]

True. The hiccup comes from the quotes, not <> which I can remove.

I use "^" as command separator, so no problem there. ;)

Once you have removed the other nasty stuff, you can get rid of double quotes (line by line) with @STRIP["^"",%line] (as long as there's an even number of quotes in %line).

If you have a version of TR.EXE (or use the WSL tr on Windows10) you can get rid of quotes like this.

v:\> echo "foo" | d:\gnu\tr -d \"
foo

v:\> echo "foo" | wsl tr -d \"
foo

TPIPE (/replace) should be able to do it but I haven't stumbled of the right syntax.

 

[vefatica]

TPIPE can double percent signs.

v:\> type percent.txt
%
%%
%%%
%%%%

v:\> tpipe /input=percent.txt /replace=4,0,0,0,0,0,0,0,0,"%%","%%%%"
%%
%%%%
%%%%%%
%%%%%%%%

 

[vefatica]

It took a while to come up with this one. TPIPE can get rid of the quotes like this.

v:\> type quotes.txt
"foo
"bar"
"garbage""
no quotes

v:\> tpipe /input=quotes.txt /replace=4,0,0,0,0,0,0,0,0,"\042",""

foo
bar
garbage
no quotes

And TPIPE /simple=16 will remove HTML. You might be able to do the whole thing with one TPIPE (which would be pretty efficient).

[/code]TPIPE /input=infile /output=outfile /simple=16 /replace=4,0,0,0,0,0,0,0,0,"\042","" /replace=4,0,0,0,0,0,0,0,0,"%%","%%%%"[/code]

That's (1) get rid of HTML, (2) get rid of quotes, (3) double percents

 

[x13]

Yeah, I tried using TPIPE / SIMPLE=16, but t completely removes the A HREF tag,

Example:

> echo "<a href="https://site.com/files/file.exe" class="btn btn-success">Download</a>" | TPIPE /SIMPLE=16

"Download"

So that won't work in this case.

M.

 

[vefatica]

Yeah, I tried using TPIPE / SIMPLE=16, but t completely removes the A HREF tag,

Example:



So that won't work in this case.

M.

I don't know what you ultimately want to do, but with TPIPE and some regular expression wizardry you might be able to do it. For example, before removing HTML, you could do something to this effect.

v:\> echo "<a href="https://site.com/files/file.exe" class="btn btn-success">Download</a>" | tpipe /replace=4,0,0,0,0,0,0,0,0,"<a (href=\".+?\").*>",$1
"href="https://site.com/files/file.exe" class="btn btn-success"Download</a>"

 

[vefatica]

Maybe a better example.

v:\> echo "<a href="https://site.com/files/file.exe" class="btn btn-success">Download</a>" | tpipe /replace=4,0,0,0,0,0,0,0,0,"<a (href=\"[^"]+?\").+?>",$1
"href="https://site.com/files/file.exe""

 

[x13]

Yeah well..

I knew someone would eventually bring regex into the equation.
I'm worthless at it. I know basic vim commands like :%s// ... but for the rest, it never sunk in.

Could you explain the working of the last two arguments of your TPIPE ?

M.

 

[vefatica]

I'll try.

"<a (href=\"[^"]+?\").+?>"

is matched by: <a href="

followed by [^"]+? which is 1 or more non-quote characters (? = greedy, as many as possible)

followed by a \" (a quote)

folllowed by .+? (any 1 or more characters (greedily))

followed by >

in general . is any character, .* means 0 or more, .+ means 1 or more ... add ? and it gets greedy (real Perl is greedy by default; TPIPE's Perl is not)

[...] is like TCC, any characters in the set

[^...] means any character not in the set

" is special (delineating the regex itself) so it has to be escaped as \" except when inside [].

The () are ignored in matching. In the next argument (the replacement string), $1 means whatever was matched inside (). $1 is TPIPE lingo; in real Perl, it would be \1 (or in general, \n (inside the nth parens))

 

[x13]

Wow.
I'm a technical guy, but that syntax is crazy.
That's why I haven't learned it.

Thanks though. With SETDOS /X- and some non-regex manipulation I'll figure it out.

M.

 

[vefatica]

In short, match

<a href="URL" ... >

and replace it with

href="URL"

 

[vefatica]

Wow.
I'm a technical guy, but that syntax is crazy.
That's why I haven't learned it.

Thanks though. With SETDOS /X- and some non-regex manipulation I'll figure it out.

M.

I've been using (trying to use) regexes for close to 30 years. I still don't feel very good about them. For me, it involves a lot of triel and error. I know the basics. Beyond the basics, and when it comes to the differences between regex languages (Perl, Gnu, PCRE, ...), IT'S GOOGLE TIME.

 

[x13]

You're right Vefatica!

It's very powerful stuff, for sure... but one typo and it doesn't work...

Reminds me of when I used to write %@array_name[%x] and I was surprised to get a syntax error.
:)

 

[x13]

To make a long story short... RogerB gets the cigar because SETDOS /X-6 is exactly what I needed!

Thanks again.
M.

 

[vefatica]

You're right Vefatica!

It's very powerful stuff, for sure... but one typo and it doesn't work...

Reminds me of when I used to write %@array_name[%x] and I was surprised to get a syntax error.
:)

To be fair, most things don't work with a typo. In the case of regexes, a typo is worth a half an hour of trying to figure out the mistake, and then not being sure how to fix it. :-)

 

[x13]

So true.

I even tried an online regex expression builder... didn't get very far.
The easiest I found is to copy someone else's example online.

M.

 

[RogerB]

To make a long story short... RogerB gets the cigar because SETDOS /X-6 is exactly what I needed!

Thanks again.
M.

Thanks for the cigar

As for regexes, I'm with Vince on the trial and error approach. Whenever I use them I'm reminded of this quote:

Some people, when confronted with a problem, think “I know, I'll use regular expressions.” Now they have two problems.

In that form the quote is usually attributed to Jamie Zawinski, although it goes back further and is one of those expressions where you can insert your own choice into the "I know, I'll use..." bit.

 

[x13]

So true!

PS to development team :

Would it be feasible to allow arrays for TPIPE as /INPUT and /OUTPUT ?

Just my 2¢.

M.

 

[vefatica]

As for using an array as output, we already have the more generic @EXECARRAY. I have often used it with TPIPE.

 

[x13]

Yes I understand that.

But it's not like you could do a TPIPE and apply it to a whole array in one line (/INPUT and /OUTPUT have to be files)

 

[vefatica]

Yes I understand that.

But it's not like you could do a TPIPE and apply it to a whole array in one line (/INPUT and /OUTPUT have to be files)

I'm not sure what you mean. With /output unspecified, you get stdout and %@execarray[array_name,TPIPE ...] works fine.

 

[x13]

Ah soooo!!

I thought both /INPUT and /OUTPUT were required in TPIPE.

Nice to learn something new. Thanks! :)

M.

 

[vefatica]

You don't need /input either which makes TPIPE work in pipes.

v:\> echo foo^r^nbar | tpipe /grep=3,0,0,0,0,0,0,0,foo
foo

 

[x13]

Top!