How to? How to enable UTF-8 output in tcmd.ini

Feb 23, 2012
240
3
Tcmd.ini allows for an option "UnicodeOutput=Yes", to set redirection from the command line to be written as unicode (from what I can see, the output is saved as UTF-16 Little Endian). I understand that there is also a theoretical option to save the output as UTF-8, but I do not see how to set that option within tcmd.ini. Is it possible to do so?
 

Charles Dye

Super Moderator
Staff member
May 20, 2008
3,966
53
Albuquerque, NM
prospero.unm.edu
I don't believe there is such an option, but I'll bet it would take Rex about two minutes to implement if there was sufficient interest in the feedback forum.
 

rconn

Administrator
Staff member
May 14, 2008
11,446
110
There's actually a couple of ways to do this; one in the current build and another in a test build that will be uploaded in another day or two.

The current solution is to choose a UTF-8 codepage -- TCC will then automatically convert the internal Unicode to UTF-8 when redirecting output.

The second solution (currently in beta with some other users who only want UTF-8 output sometimes) is with the TCC /U8 startup option (or "OPTION //UTF8Output=Yes" from the TCC command line).

Neither approach is currently documented, as they were deemed too obscure and too experimental for 99.99% of the users. They will be documented when the beta version is released (hopefully by the end of this week).
 
Feb 23, 2012
240
3
Hi Rex,
When you say to choose a UTF-8 code page, I gather (from other forum postings) that you mean to run the command "chcp 65001".
However, I find that this code page does not operate properly at all. I have a bunch of files with Hebrew filenames, which display correctly with my standard code page (862), as well as with the Windows Hebrew code page (1255). However, when I shift into 65001, Hebrew characters are no longer displayed at all; instead, I just receive blank spaces and black lines when performing a "dir".
Is this is an internal problem, or do you think it is a font problem?
 

rconn

Administrator
Staff member
May 14, 2008
11,446
110
The UTF-8 code page option in TCC only affects redirected file output; it doesn't have anything to do with normal text display in the console.

Are you running TCC in a Take Command tab window, or in a stand-alone Windows console? If it doesn't display properly in a stand-alone Windows console, it's a Windows problem. If it displays in the Windows console but not in a Take Command tab window, it's either a Take Command problem or it's because you're using different fonts in the console and in Take Command.

(Windows has very limited UTF-8 support; it may be that they just don't support your environment.)
 
Feb 23, 2012
240
3
Hi Rex,
I'm using it in the TC tab window, and that's where I see the problem.
In contrast, in a standalone TCC window, running "chcp 65001" has no negative effect, and I can still see my Hebrew filenames.
Both windows are using the "CourierNew" font.
However, I notice that in the standalone TCC window, even after I run "chcp 65001", redirection of a directory listing results in a UTF-16 file, not a UTF-8 file.
- Avi
 

rconn

Administrator
Staff member
May 14, 2008
11,446
110
However, I notice that in the standalone TCC window, even after I run "chcp 65001", redirection of a directory listing results in a UTF-16 file, not a UTF-8 file.
Not reproducible here. Are you sure you didn't have UnicodeOutput (/U startup option) active? TCC will look for that before it looks for a UTF-8 codepage.
 
Feb 23, 2012
240
3
Hi Rex,
Yes, you solved it, that was it - I had UnicodeOutput set to Yes in my .ini file. Now that I changed that, I can successfully switch to "chcp 65001" and redirect as UTF-8. I didn't realize that the UnicodeOutput option overrode the code page selection.
We still have the issue of the non-appearing letters in the tab view when in code page 65001, but it's not at all a critical item, since I can always use the standalone terminal. So I think we can close this issue for now.
Thanks,
Avi


Not reproducible here. Are you sure you didn't have UnicodeOutput (/U startup option) active? TCC will look for that before it looks for a Unicode codepage.