- May
- 13,123
- 180
I doubt anything is messing with timers. That thread uses an awful lot of CPU for what it does. If I don't use any monitor functions can I safely suspend that thread?
Does TCC know I'm visiting the forums?
- Thread starter vefatica
- Start date
I doubt anything is messing with timers. That thread uses an awful lot of CPU for what it does. If I don't use any monitor functions can I safely suspend that thread?
Rex, build 17 doesn't seem to be showing this behavior anymore. Did you do something between builds 15 and 17 to address it? Is anyone else still seeing this issue?
rconn
Administrator
- May
- 13,166
- 205
Staff member
I changed the API that TCC uses to wait before processing the next loop pass.
- May
- 13,123
- 180
From what to what? ... I'm curious. I'm really curious about why browsing affected that thread's CPU usage.
Now, here, that thread is below 0.01% CPU and isn't affected by any browsing.
rconn
Administrator
- May
- 13,166
- 205
Staff member
Yes, and yes.
rconn
Administrator
- May
- 13,166
- 205
Staff member
No.
Can you tell me what that thread is/does? Every conhost seems to have it. Why does it use more CPU when TCC is running under TCMD than when TCC isn't?
rconn
Administrator
- May
- 13,166
- 205
Staff member
I have no idea. Microsoft hasn't supplied source code for conhost.
When TCC is running under TCMD, it's doing some interprocess communication between tcc.exe and tcmd.exe.
- May
- 13,123
- 180
Here, when TCC is in TCMD, that thread is down around 0.14% CPU. (@2.67 GHz)
Well, obviously I didn't expect you to know about the internals of conhost, but I figured you might be able to duplicate that thread using CPU like I'm seeing and find some way to use a different API or some other optimization between TCC and TCMD to make it more efficient. I mean you didn't think you could fix the other problem and it has been around for years so I'm not giving up hope.
I know from experience that IPC is never cheap unfortunately.
- May
- 13,123
- 180
One thing is for sure, TCMD has to (somehow) keep track of what's happening in its tabbed consoles. That must involve a bit of CPU (and some cooperation from conhost).
rconn
Administrator
- May
- 13,166
- 205
Staff member
TCMD has a lot of threads; I can't tell from that info which one you want. (And it would be a really, really bad idea to randomly turn off TCMD's housekeeping threads.)
- May
- 13,123
- 180
The only thread that ProcessExplorer shows as using CPU is, I imagine, the one that monitors consoles. Here, it uses about 0.35% CPU per tab, which I figure ain't bad.
- May
- 239
- 2
Great, the new build stopped using so much CPU for _that_ TCC thread in Windows 10 as well. :)
There is still some smaller use of CPU for various threads (mostly for the TCMD main/console thread which I guess is unavoidable with the need to scan the consoles for changed data), but also (much smaller) for TCC itself. Would be nice if that could be reduced to 0 (or close to it). Ideally, a console application should not do anything while waiting for input.
cmd.exe and its associated conhost.exe process (on Windows 10) does not take any CPU at all when not active. Is there any reason the same could not apply for a stand-alone (not running inside TCMD) TCC.EXE?
Nevertheless, great to get this bigger issue fixed! :)
There is still some smaller use of CPU for various threads (mostly for the TCMD main/console thread which I guess is unavoidable with the need to scan the consoles for changed data), but also (much smaller) for TCC itself. Would be nice if that could be reduced to 0 (or close to it). Ideally, a console application should not do anything while waiting for input.
cmd.exe and its associated conhost.exe process (on Windows 10) does not take any CPU at all when not active. Is there any reason the same could not apply for a stand-alone (not running inside TCMD) TCC.EXE?
Nevertheless, great to get this bigger issue fixed! :)
I meant I turned off the display of the CPU usage on the TCMD status line not the thread. Does this stack trace tell you about the thread in question?
0 ntkrnlpa.exe!KeSetEvent+0x2a1
1 ntkrnlpa.exe!KeDelayExecutionThread+0x5cc
2 ntkrnlpa.exe!KeWaitForMutexObject+0x393
3 ntkrnlpa.exe!KiDeliverApc+0x664
4 ntkrnlpa.exe!KeRemoveQueueEx+0x990
5 ntkrnlpa.exe!KeDelayExecutionThread+0x5cc
6 ntkrnlpa.exe!KeWaitForMultipleObjects+0x538
7 ntkrnlpa.exe!IoSetIoCompletionEx+0x928
8 ntkrnlpa.exe!IoSetIoCompletionEx+0x695
9 ntkrnlpa.exe!ZwYieldExecution+0xb92
10 ntdll.dll!KiFastSystemCallRet
11 kernel32.dll!WaitForMultipleObjectsEx+0x8e
12 USER32.dll!GetWindowLongW+0x141
13 DUser.dll+0x1717
14 DUser.dll+0x1848
15 DUser.dll+0x14e9
16 USER32.dll!KillTimer+0x58
17 ntdll.dll!KiUserCallbackDispatcher+0x2e
18 tcmd.exe!CPUUsage+0x3c700
19 tcmd.exe!CPUUsage+0x2eda94
20 tcmd.exe!CPUUsage+0x175798
21 kernel32.dll!BaseThreadInitThunk+0x12
22 ntdll.dll!RtlInitializeExceptionChain+0xef
23 ntdll.dll!RtlInitializeExceptionChain+0xc2
rconn
Administrator
- May
- 13,166
- 205
Staff member
TCC is sitting inside a Windows API waiting for keyboard (& optionally mouse) input. I can't do any better than that without replacing the OS.
CMD is in a line input API; TCC is in a character (& mouse) input API. I could get down to the CMD level if I removed all of the TCC command line editing functionality. I'm not sure that's worthwhile to most users in order to save <0.01% of CPU usage (on a single core).
rconn
Administrator
- May
- 13,166
- 205
Staff member
The only thread that calls CPUUsage regularly is the status bar update thread. Turning off the status bar display doesn't stop the thread from retrieving the various info, which is used by other threads in TCMD.
If I run 5 instances of TCC under TCMD then the CPU usage of this threagoes up to about 2.5% CPU. The one thing I can say for this thread in TCMD, the thread in conhost that uses about .05% CPU per instance of TCC under TCMD and the thread that you reduced the CPU usage for in TCC in build 17 is that the CPU usage seems to be constant. If I run it on my 4.2 GHZ 4770K or a 2 GHZ E4400 the percentage remains about the same. I only bring it up in case you can think of a different API to use or a different way to do IPC or perhaps reducing the amount of time that you wake up to do the IPC between TCC and TCMD to reduce this. Sure it isn't a lot of CPU, but all of the optimizations you've done for memory usage probably weren't a big deal to most users with 8/16/32/etc GB machines and yet I'm glad you invested the time to do them. As I said you've said for years when the TCC CPU usage came up that there wasn't anything you could do, but you thought of a way to do what you needed to do with a different API. Here's hoping that you can think of something to reduce the CPU impact of TCC. Currently my system has been running for 25 hours, many of them idle, and here is what I see for TCMD and the conhost associated with TCC.
Process information for machine:
Name Pid Pri Thd Hnd Priv CPU Time Elapsed Time
tcmd 7888 8 9 590 36064 0:34:34.579 25:18:18.792
Process information for machine:
Name Pid Pri Thd Hnd Priv CPU Time Elapsed Time
conhost 6852 8 4 110 4444 0:17:05.316 25:18:26.398
They are the highest two processes using CPU Time in Task Manager on my system below System Idle Process.
- May
- 239
- 2
I get it down to about 0.12% for the tcc.exe and conhost.exe combined. (0.08 for tcc and 0.04 for conhost). Which I guess is acceptable even if zero would be better.
Still don't understand why tcc must use polling here (I guess?) while cmd can do without. Sure, tcc has better command line editing functionality, but it's not that different to cmd which also offers editing functionality even if more primitive. Hmm, never used the mouse to edit the command line, might be I am missing some features of tcc.
Rex, can you refresh my memory? Doesn't TCMD scan regularly to know if console processes have been created or destroyed? perhaps this is what is using a lot of the CPU shown in my prior posts. If that is the case why can't you hook CBT or CALLWNDPROC to know about creates and destroys or use a kernel driver to so if the hooks won't work? It seems like being notified of creates and destroys instead of having to scan would be much faster.
rconn
Administrator
- May
- 13,166
- 205
Staff member
TCMD scans for processes once a second; you won't even be able to detect the impact of that. (It only checks for new ones if you've specified auto attach, and it only checks for destruction of attached consoles.)
I tried CBT calls a long time ago; they were highly unreliable. Using global hooks slows things down more than the once a second scan. And requiring a device driver is a definite no-no for a lot of corporate customers.
So here are snapshots of the expensive thread in TCMD with 1, 5, and 10 TCC processes running. All of these processes are just setting idle while this is happening. There has to be a way to speed this up. Rex, can you describe what is happening in the idle case? Does TCMD scan for the creation and destruction of console processes? Is it scanning the TCC processes for input/output changes? Could shared memory be used, at least in the TCC case, instead of IPC to speed anything up?
TID CPU Cycles Delta Start Address
7752 0.61 184,475,692 tcmd.exe!CPUUsage+0x1757f4
7752 2.01 620,155,670 tcmd.exe!CPUUsage+0x1757f4
7752 2.83 875,521,718 tcmd.exe!CPUUsage+0x1757f4
TID CPU Cycles Delta Start Address
7752 0.61 184,475,692 tcmd.exe!CPUUsage+0x1757f4
7752 2.01 620,155,670 tcmd.exe!CPUUsage+0x1757f4
7752 2.83 875,521,718 tcmd.exe!CPUUsage+0x1757f4
rconn
Administrator
- May
- 13,166
- 205
Staff member
That's probably the thread that's scanning the hidden consoles for changes. TCMD doesn't use IPC (or shared memory) for this, it reads the console's memory directly.
On my system, having 10 tab windows open in TCMD uses around 1% of a single core. That seems entirely reasonable to me.
On my system, having 10 tab windows open in TCMD uses around 1% of a single core. That seems entirely reasonable to me.
My numbers were for entire CPU so that is between .6% and 2.6% of all 8 cores for 1 to 10 tabs. My system had the issue with the thread in TCC that slept for 25MS, and then woke up and did it's processing. Are you also sleeping for xMS in this thread and then waking up and scanning the console's memory? If so could you change to the API you changed for in the TCC thread? Is it possible that the thread isn't really waiting as long as it should on my system between memory scans? TCMD uses over a minute of CPU per hour just scanning consoles and I'd just like to reduce that as it doesn't seem like it uses nearly as much CPU on your system as does mine.
- May
- 13,123
- 180
It seems to depend in where and when you look. I have a TCMD running with 10 tabs, everything idling. TaskMgr shows a solid 0% CPU for TCMD. ProcessExplorer shows a solid 2.5% CPU for TCMD. One of them is wrong!
I think the best measurement is total_cpu/total_running; that will be the average. That will be high at first because TCMD does a lot at startup and in starting 10 instances of TCC. The longer I let things idle, the lower that gets. Right now,
18 minutes later (after a smoke with my boss)
It's probably pretty close to its limit of the actual momentary idle usage. I can only conclude TCMD is pretty efficient at monitoring the consoles.
I think the best measurement is total_cpu/total_running; that will be the average. That will be high at first because TCMD does a lot at startup and in starting 10 instances of TCC. The longer I let things idle, the lower that gets. Right now,
Code:
v:\> pstat 3284 | egrep "Module|Running|Total"
Module: D:\TC20\tcmd.exe
Running: 16 minutes 18 seconds
Total CPU: 2.0436131 sec (0.21%)18 minutes later (after a smoke with my boss)
Code:
v:\> pstat 3284 | egrep "Module|Running|Total"
Module: D:\TC20\tcmd.exe
Running: 36 minutes 15 seconds
Total CPU: 2.1684139 sec (0.10%)You are correct Task Manager seems to show a lot less CPU for TCMD than Process Explorer, but it still seems that it can be made faster. These are the top CPU users on my system for CPU time since boot below system idle process.
Process information for machine:
Name Pid Pri Thd Hnd Priv CPU Time Elapsed Time
tcmd 7748 8 12 622 40764 0:34:00.883 26:37:49.303
conhost 7896 8 4 110 4456 0:16:20.934 26:37:54.154
- May
- 13,123
- 180
That's 2.1% for TCMD. That's reasonable if you're using it; unreasonable if you're not using it or if it was just started.
I'd expect TCMD to be the biggest CPU user.