Change percent-sign to anything-else in filename

  • This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.
Aug 8, 2010
30
0
#1
Change percent-sign to anything-else in filename passed as a parameter.

This started out with "%20" in filenames. I KNOW that 0x20 is the hex code for a space, but I want to know how to change any percent-sign into any standard/printable character(s) so I can analyze the file's contents within TCC or a third-party application. Basically, I want to

change filename
before %in after.png
to
before anything after.png
or
anything.png

within the BTM file, or copy the pre-filename to the post-filename so I can analyze that in lieu of the actual filename. There were two cases that I tried:

Case 1 - the filename is passed directly as a parameter like
Analyze.btm "PATH\before %in after.png"

Case 2 - the string "before %in after.png" (without the "s) is the first line (line 0) in a file, e.g., "Analyze.txt", and the command is
Analyze.btm "PATH\Analyze.txt"

All I can get out of Analyze.btm is an analysis of

"PATH\before after.png"

which doesn't exist.

I've tried
=================
setdos /x-3 and -4

%@replace[SeeNote1,SeeNote2,SeeNote3] -and- "%@replace[SeeNote1,SeeNote2,SeeNote3]"

Note1: %% ^% \%

Note2: %%%% ^% ^%% ^%%%
using_a_text_string_still_has_the_same_results

Note3: %1 "%1" %@line[%1,0] "%@line[%1,0]"
%@line["%1",0] "%@line["%1",0]"

copy "%@line["%1",0]" anything.png
=================

Again, all I can get out of Analyze.btm is an analysis of

"PATH\before after.png"

which doesn't exist. This happens even when the "to" in the 'replace' function (Note2) is a simple lower-case alpha like
asdfg

I played with ffind and sed for awhile, but they didn't work either.

Well?
 
Jan 19, 2011
559
7
Norman, OK
#3
So much interest; so few replies. Almost as if I were being ostracized.
No ostracism, just befuddlement.
Code:
[C:\temp]echo %@URLDECODE[file%20name.txt]
file.txt

[C:\temp]echo %@URLDECODE[`file%20name.txt`]
file?name.txt
 

Charles Dye

Super Moderator
Staff member
May 20, 2008
3,383
39
Albuquerque, NM
prospero.unm.edu
#4
So much interest; so few replies. Almost as if I were being ostracized.
I doubt if anybody here knows you well enough, or cares enough, to 'ostracize' you, especially as you're using a pseudonym. If a post doesn't generate any responses, then people tend to forget it pretty quickly; there's no need to read anything personal into it.

Filenames containing percent signs are a terrible idea (yes, I'm aware that it's a web browser doing it) and will always be difficult to handle through the command line. You have to double the percent signs -- I don't think there's any other way to handle them. Worse, you have to double the percent signs once for every time the filename will get passed through the parser, e.g. quadruple every percent sign if the filename will get parsed twice. Other special characters can be handled more easily via SETDOS or quoting or whatever, but percent signs are always going to be a major pain in the butt.

I'd would strongly recommend just renaming these files. If there are only a few, it may be easiest to do it through Explorer, or Take Command's GUI 'List View' pane. Or you can fix them one-at-a-time through the command line:

Code:
ren "My%%20File.txt" "My File.txt"
The option labelled "Double %% in filenames" on the Command Line tab in the OPTIONS dialog may be helpful if you want to do it this way.

If you have a large number of files like this, another possible approach would be to use sed or the like to massage a directory listing into a batch file containing x number of REN lines.
 

samintz

Scott Mintz
May 20, 2008
1,203
11
Solon, OH, USA
#5
This is actually a very difficult problem
to solve because of the way the parser works. I believe Charles has
a plugin called SAFECHARS that could be of use. I know Steve F. uses it
all the time.

Code:
echo %@replace[%%,_,before %%in after.png]
before _in after.png

function foo=`%@replace[%%,_,%$]`

echo %@foo[before %%in after.png]
before  after.png

echo %@foo[before %%%in after.png]
before  after.png

echo %@foo[before %%%%in after.png]
before _in after.png

ren /n be* %%@foo[*]
before %in after.png -> before  after.png
     1 file would be
renamed
If it is a single file that you know
the name of, then just double the percents:
Code:
ren "before %%in after.png"
"before _in after.png"
-Scott

newbie <> wrote on 06/07/2011
09:16:41 AM:


>
> So much interest; so few replies. Almost as if I were being ostracized.
>
>
 

Charles Dye

Super Moderator
Staff member
May 20, 2008
3,383
39
Albuquerque, NM
prospero.unm.edu
#6
Change percent-sign to anything-else in filename passed as a parameter.

This started out with "%20" in filenames. I KNOW that 0x20 is the hex code for a space, but I want to know how to change any percent-sign into any standard/printable character(s) so I can analyze the file's contents within TCC or a third-party application. Basically, I want to
How often do people here have to deal with such filenames? Is this a common issue? Would it be worthwhile to create a plugin command to rename problematic filenames?
 
#7
How often do people here have to deal with such filenames? Is this a common issue? Would it be worthwhile to create a plugin command to rename problematic filenames?
Charles, I have run into the same problem with some application doing the same thing (I don't remember anymore exactly which application because I've been avoiding using it for a fairly long time (somehow I'm surviving!) both because of this problem and my very bad memory). This is, unfortunately to me, one of the very rare but also very substantive limitations of TCMD/TCC, so a work-around (plugin or whatever) would be very much appreciated!
 
#8
On Thu, 09 Jun 2011 00:51:48 -0400, mathewsdw <> wrote:

|---Quote (Originally by Charles Dye)---
|How often do people here have to deal with such filenames? Is this a common issue? Would it be worthwhile to create a plugin command to rename problematic filenames?
|---End Quote---
|Charles, I have run into the same problem with *some* application doing the same thing (I don't remember anymore exactly *which* application because I've been *avoiding* using it for a fairly long time (somehow I'm surviving!) both because of this problem and my very bad memory). This is, unfortunately to me, one of the *very rare* but also *very substantive* limitations of TCMD/TCC, so a work-around (plugin or whatever) would be *very much* appreciated!

Something like this ought to work.

v:\foo> dir /b
a%b.btm
c%d.txt

v:\foo> setdos /x-4 & for %file in (*%%*) ren %file %@replace[%%,foo,%file] &
setdos /x+4
V:\foo\a%b.btm -> V:\foo\afoob.btm
1 file renamed
V:\foo\c%d.txt -> V:\foo\cfood.txt
1 file renamed

v:\foo> dir /b
afoob.btm
cfood.txt
 
#9
On Thu, 09 Jun 2011 00:51:48 -0400, mathewsdw <> wrote:

|---Quote (Originally by Charles Dye)---
|How often do people here have to deal with such filenames? Is this a common issue? Would it be worthwhile to create a plugin command to rename problematic filenames?
|---End Quote---
|Charles, I have run into the same problem with *some* application doing the same thing (I don't remember anymore exactly *which* application because I've been *avoiding* using it for a fairly long time (somehow I'm surviving!) both because of this problem and my very bad memory). This is, unfortunately to me, one of the *very rare* but also *very substantive* limitations of TCMD/TCC, so a work-around (plugin or whatever) would be *very much* appreciated!

Something like this ought to work.

Code:
v:\foo> dir /b
a%b.btm
c%d.txt

v:\foo> setdos /x-4 & for %file in (*%%*) ren %file %@replace[%%,foo,%file] &
setdos /x+4
V:\foo\a%b.btm -> V:\foo\afoob.btm
     1 file renamed
V:\foo\c%d.txt -> V:\foo\cfood.txt
     1 file renamed

v:\foo> dir /b
afoob.btm
cfood.txt
 
#10
On Thu, 09 Jun 2011 00:51:48 -0400, mathewsdw <> wrote:
Code:
v:\foo> dir /b
a%b.btm
c%d.txt

v:\foo> setdos /x-4 & for %file in (*%%*) ren %file %@replace[%%,foo,%file] &
setdos /x+4
V:\foo\a%b.btm -> V:\foo\afoob.btm
     1 file renamed
V:\foo\c%d.txt -> V:\foo\cfood.txt
     1 file renamed

v:\foo> dir /b
afoob.btm
cfood.txt
Vince, while you are, as always, correct, it kind of misses the point (at least for me). I do not want to generally go around finding and renaming these files, I would prefer that .btm files be able to handle them no matter what characters are in their names! (Or, to put it another way, programs should be written to accommodate what the user wants to do, not force the user to accommodate the program's limitations.) This and closely related-issues are why I am often effectively forced to write C++ programs rather than .btm files, and I would much rather be able to write .btm files than C++ programs. I absolutely do not generally write C++ programs because I am "in love" with that language, I write C++ programs because it is the best tool that I have access to for doing many of the things I want to do on a regular basis. (There are other languages out there that may do the job such as Pearl or C#, for instance, but I found Pearl's somewhat non-procedural paradigm (as I remember it - I haven't actually used it for a decade or more) to be a pain for me to remember unless I was using it on an almost-daily basis, and I do not know C# and its similarity to C++ would just probably confuse me more than anything else at this point in time. (My bad memory is not a joke or funny or at all an exaggeration. It is becoming more and more of a real limiting factor in my life.) I am also (or was) quite familiar with Visual Basic, but I did not find it to be a language that was particularly easy to use when your primary purpose was manipulating file-system "objects" in various ways.) As I said earlier, I do consider this to be a substantial limitation of TCMD/TCC. (Although it is generally a perfectly tolerable one in light of TCMD/TCC's probably literally hundreds of other benefits! I would not be recommending it to people in my circle of acquaintances on such a regular basis (although, so far, to no good end as far as I know) if I did not consider that to be the case.)
 

Charles Dye

Super Moderator
Staff member
May 20, 2008
3,383
39
Albuquerque, NM
prospero.unm.edu
#11
Vince, while you are, as always, correct, it kind of misses the point (at least for me). I do not want to generally go around finding and renaming these files, I would prefer that .btm files be able to handle them no matter what characters are in their names!
Unfortunately, that would require a major rewrite of the parser, utterly destroying all backward compatibility with everything, even CMD.EXE. I can't do that in a plugin. Only Rex could do it, and I expect he'd sooner remove his own appendix with a pickle fork.

The best I can produce is a command to find and rename the more awful filenames.
 
#12
On Thu, 09 Jun 2011 05:01:06 -0400, mathewsdw <> wrote:

|Vince, while you are, as always, correct, it kind of misses the point (at least for me). I *do not* want to generally go around finding and renaming these files, I would prefer that .btm files be able to handle them *no matter what* characters are in their names!

I haven't followed this thread closely so I don't know what's been proposed. But
a command interpreter must have special characters and a command script author
must work around them. It's an unfortunate fact of life that "%" is used for
environment variables and is allowed in file names. I doubt TCC **could**
handle this without extra effort on the part of the user. If it could be made
to do so, we might not like the result.
 
#14
vefatica wrote:
| On Thu, 09 Jun 2011 05:01:06 -0400, mathewsdw <> wrote:
|
|| Vince, while you are, as always, correct, it kind of misses the
|| point (at least for me). I *do not* want to generally go around
|| finding and renaming these files, I would prefer that .btm files be
|| able to handle them *no matter what* characters are in their names!
|
| I haven't followed this thread closely so I don't know what's been
| proposed. But
| a command interpreter must have special characters and a command
| script author
| must work around them. It's an unfortunate fact of life that "%" is
| used for environment variables and is allowed in file names. I doubt
| TCC **could**
| handle this without extra effort on the part of the user. If it could
| be made
| to do so, we might not like the result.

The underlying problem is that the language used by COMMAND.COM and all its
descendants, including TCC and CMD, never made a clear distinction between
keywords, variables, and data. It used "delimiter" characters which were at
the time rejected by the file system. Those same delimiter characters are
now valid in filenames, and are commonly used in files received from other
filesystems. Without creating a new mechanism, e.g., a new class of
variables, the issue cannot be solved. The closest one can get in the
current form of TCC is the use of arrays or binary buffers.
--
Steve
 
#15
Unfortunately, that would require a major rewrite of the parser, utterly destroying all backward compatibility with everything, even CMD.EXE. I can't do that in a plugin. Only Rex could do it, and I expect he'd sooner remove his own appendix with a pickle fork.

The best I can produce is a command to find and rename the more awful filenames.
Charles, I am not at all surprised to hear that. (If this could have been done easily, I'm sure it would have been done by somebody (even Rex himself!) a long time ago!) I guess I'm stuck to writing C++ programs... (As I have probably indicated multiple times before, I do not under any circumstances whatsoever manually create file names of this nature. These file names at least used to since I try very hard to not use applications where this is the case come 100% of the time from external applications that I either bought or downloaded from the web.) But thank you anyway...
 

rconn

Administrator
Staff member
May 14, 2008
10,100
85
#16
The underlying problem is that the language used by COMMAND.COM and all its descendants, including TCC and CMD, never made a clear distinction between keywords, variables, and data. It used "delimiter" characters which were at the time rejected by the file system.
That's the nature of ALL interpreted languages -- it is not possible to overload all of those special/delimiter characters. TCC does handle a few special cases, like the square brackets, which can be either wildcard delimiters or filename characters. But variable expansion has to be done long before command argument processing is done.

Those same delimiter characters are now valid in filenames, and are commonly used in files received from other filesystems.
They're not (technically) valid filename characters, nor are they commonly used in other filesystems. They *are*, unfortunately, commonly created by browsers.
 
#17
I am loathe to suggest this, partly because I'm not sure that it is a good idea and partly because I suspect that Rex would end up with more support requests if it existed, but...

How about adding an option that turns off all variable expansion other than the %[varname] format?? This seems like it ought to be relatively simple to implement (depending on how many different places %something is handled) without too many knock-on effects. Obviously it completely blows CMD compatibility out of the water, but in this instance that is almost certainly going to be a necessity.
 
Aug 8, 2010
30
0
#18
Just for the record, I was debating whether to use "ostracized" or "ostrich egged", and the former won out. But ya gotta admit it worked.

In answer to JohnQSmith's June 9th question: "Yes, and more". I had seen forum threads with similar questions, but they just seemed to fizzle out. Meanwhile, I kept getting Unix "%20" (aka 'space') files interspersed with regular ones and I finally got tired of it. I never doubted the tenacity of the percent-sign as an Escape Character, and I believe something of that ilk is necessary. But, TCC is the decedent of 4dos.com which was presented as a command-interpreter replacement for command.com, and I believed that it was the leading candidate to make an exception to the percent-sign's sovereignty. The "Windows Picture and Fax Viewer" can handle percent-signs, so I know it can be done. But, my naiveté notwithstanding, I'm sure it circumvents command.com to do it.

I also tried a Short-File-Name approach to get around the percent-sign, but I couldn't integrate it into any of the TCC functions. I considered contracting a Perl or PHP developer to help me, but then I'd need help integrating that into a btm, and that clearly IS beyond the scope of this forum. So, I thought I'd try the forum with an earnest and detailed description of what I tried to see if it might wrangle out a final solution or a clearly defined dead-end. I got an unexpected collection of both along with my answer to JohnQSmith's question. This is not a "problem" per se, it is a state that was arrived at intentionally. There is interest in dealing with percent-signs, but the way to do it isn't easy and getting there may be more troublesome than it's worth. The "issue" isn't "resolved", but it isn't fuzzy anymore either - and that's a welcome outcome.

Regarding my anonymity, I thought "newbie" would better establish my status without attendant explanations than would my Anglo-Saxon name. I'm a former Analog (Electronics) Design Engineer with a BA in Psychology along with my BSEE. Those are the credentials I bring to backup my "newbie" status, and I can only hope they serve me well. Did I mention I'm also modest?

I am humbled by and grateful for the dedicated and knowledgeable replies to my initial input and to the ensuing inputs by your peers in what turned out to be a lively and enlightening discussion. My thanks to everyone for your kind attention and hearty deliberation. Really, I mean that. Thank You.
 

Charles Dye

Super Moderator
Staff member
May 20, 2008
3,383
39
Albuquerque, NM
prospero.unm.edu
#19
In answer to JohnQSmith's June 9th question: "Yes, and more". I had seen forum threads with similar questions, but they just seemed to fizzle out. Meanwhile, I kept getting Unix "%20" (aka 'space') files interspersed with regular ones and I finally got tired of it. I never doubted the tenacity of the percent-sign as an Escape Character, and I believe something of that ilk is necessary.
I don't know renaming these files would be an acceptable solution for you. If so, I've thrown together a plugin which can rename filenames containing percent signs and a few other problem characters (grave accents, carets). Consider this very beta -- test carefully, on copies of files, in a scratch directory.

Docs and download: unm.edu/~cdye/plugins/fixnames.html
 
#20
There is interest in dealing with percent-signs, but the way to do it isn't easy and getting there may be more troublesome than it's worth. The "issue" isn't "resolved", but it isn't fuzzy anymore either - and that's a welcome outcome.
Here is how I do it:

Code:
[c:\bat]> ver

TCC LE  12.01.45   Windows 7 [Version 6.1.7601]

[c:\bat]> dir *%%* /b
zz%20xx%20yy.txt
zz%20yy%41%42C.txt

[c:\bat]> dir *%%* /b | sed -f decode_URL.sed
ren "zz%%20xx%%20yy.txt" "zz xx yy.txt"
ren "zz%%20yy%%41%%42C.txt" "zz yyABC.txt"

[c:\bat]> type decode_URL.sed
# GNU sed script to rename files with URL encoding
#  input:      some%20name%20her%65.xyz
# output: ren "some%%20name%%20her%%65.xyz" "some name here.xyz"
#
v4;                 # require GNU sed v4.0+ for /e switch later
s/.*/\x22&\x22/g;   # wrap filename in "double quotes"
h;                  # copy "filename" to hold space

# TCC backticks prevent premature quote interpolation
# Separate batch file interprets standard percent-encoding
s/.*/echo `&` | decode-URL-encoding.btm /e

x;                  # swap pattern space and hold space
s/^/ren /;          # put "ren " to the front of the line
s/%/&&/g;           # double each percent symbol in filename
G;                  # append \n plus hold space to pattern space
s/\n/ /;            # replace \n with " "
#---done---
The basic solution at this point is to redirect the output of the sed or ssed command to >a_new_script.btm, which will do all the renaming.

If interested, I can post the contents of my decode-URL-encoding.btm, but it's not complex. It takes whatever is sent standard input, and decodes the URL-encoding for what is needed.

See http://en.wikipedia.org/wiki/Percent-encoding for more info.

Hope this helps.
 
Aug 8, 2010
30
0
#21
I don't know renaming these files would be an acceptable solution for you. If so, I've thrown together a plugin which can rename filenames containing percent signs and a few other problem characters (grave accents, carets). Consider this very beta -- test carefully, on copies of files, in a scratch directory.

Docs and download: unm.edu/~cdye/plugins/fixnames.html

On June 16th I thought this thread was exhausted and made my entry accordingly. The "issue" persisted (at least for me) and so I returned to the Internet in search of a solution. My searching brought up the title of this thread, and I decided to see what updates might have been made after mine.

Surprise! Charles Dye's "thrown together" FIXNAMES plugin not only does what I need to get done, it does it exactly how I think it should be done. FIXNAMES also has features and options that make it a powerful and versatile tool for what seemed an insurmountable problem. To me, it's unconscionable that there weren't numerous hints, tricks or utilities to handle the percent-sign in filenames. I suspect that anyone with a critical need for such a feature would probably also have the skills to write a utility to take care of them. I'm grateful for the time and effort spent in addressing this infrequent but vexing problem, and for the extra features and documentation. The precautionary comments are well taken, but FIXNAMES has worked flawlessly in all of the tasks I need it for - plus some.
 
#22
To me, it's unconscionable that there weren't numerous hints, tricks or utilities to handle the percent-sign in filenames.
Well, filenames with embedded % signs used to be rare enough to deal with them one at a time. It's only because some demented person designed a system for representing characters in filenames that are not legal in some filesystems by using a method invented for the purpose while ignoring the special significance of the "escape character" the method uses in the most frequently used command processors, instead of using one of the several methods already built into ASCII to "shift-out" or "escape". Now a few of us are inundated with such misnamed files, while most of us virtually never see the problem. Referring to the lack of effort to deal with a problem only a handful of minority ever experienced as "unconscionable" as "unconscionable" is a grossly unfair characterization.
 
#24
Well, filenames with embedded % signs used to be rare enough to deal with them one at a time.
For some. I was just handed a 2TB backup drive that's starting to fail. After installing the new backup drive and doing the backup, I wrote a BTM script to make sure nothing's been missed in the transition. What the script does is recursively descend through the file system of the old backup drive, and delete anything on it that's been backed up on the new drive. After the script runs, any files still on the old backup drive mean that the files aren't backed up.

At least in theory. Of course, things like temporary directories, cache directories, firewall logs, will not be current, but ignoring those, any files that still reside on the old backup that didn't make it to the new backup and are a concern. Outside of descript.ion and Thumbs.db, the real problem was that there were tens of thousands of files with pain-in-the-ass embedded characters like `, ', ^, &, one friggin file with "&&", and of course %20, %3A, etc. The files could on the new drive - maybe - but the script missed them.

Given that there were something like 18,000 of these files, spread over hundreds of directories, manual changes aren't feasible. I found the same problem others have referenced, and thought of using perl, but that brings in other problems. What I found did work was using GNU grep in a script as follows:

============================================================
@echo off
setlocal
set TEMPFILE=%TEMP%\%0.TMP

dir /b | grep %% > %TEMPFILE%
setdos /x-4
for file in (@%TEMPFILE%) do gosub PARSEFILE "%FILE%"
setdos /x0
quit

:PARSEFILE [line]
set NEWFILENAME=%@replace[%%20, ,%LINE%]
ren /eq %LINE% %NEWFILENAME%
return
============================================================

I originally tried "for /a-d file in (*) do...", but found that it stripped out the %20 in the files. By using dir /b, the raw file name gets saved to the text file. I had to use grep to seperate files with % and those without, because I simply could not get Take Command to differentiate (and that's not a criticism, I fully understand why).

I ran the above script on the source drives, did an incremental backup, ran the script on the old backup drive, and then ran the compare script, at which point the 18,000+ mismatched files were now only 14.

If anyone wants to use the above, the grep that I used was the Cygwin version, but any command line grep should work.