HISTORY /R1 performance

May 30, 2008
235
2
I've noticed that loading the saved history at TCMD init takes about 2 seconds on my machine (which has quite a fast i7 processor and plenty of memory).

I use "HISTORY /R1 history.sav" to load it. (Using /R option instead takes about 10 seconds).

My history size is set to the maximum value; 270,336. The history.sav file produced by SHRALIAS on unload is 558,758 bytes. (It saves it in UTF-16 encoding).

It seems to me that loading such a (relatively) small file into a memory buffer on a modern machine (with SSD) should hardly take so long. (Running WC on the file takes 0.01 seconds).

Could there be something sub-optimal with my config or is the problem with HISTORY? Perhaps it does extra processing when reading the lines or have to frequently reallocate memory when more lines are read.

(I use a shared history for all TCC instances with SHRALIAS)
 
May 20, 2008
11,378
98
Syracuse, NY, USA
It has always been slow. I didn't know about the /R1 option and it does make it significantly faster.
Code:
h:\shrdump> d h*
2013-05-21  05:00        325,208  history.sav
 
h:\shrdump> timer history /r1 history.sav
Timer 1 on: 12:33:53
Timer 1 off: 12:33:56  Elapsed: 0:00:02.73
That still seems slow (you're doing better than me, Niklas). I can TYPE that file in half the time.
 

rconn

Administrator
Staff member
May 14, 2008
12,340
149
I've noticed that loading the saved history at TCMD init takes about 2 seconds on my machine (which has quite a fast i7 processor and plenty of memory).

There are several things that can make a HISTORY /R slow:

1) Reading a history file that's bigger than the max history list size. HISTORY /R reads a line at a time, and when it gets to the max size it has to delete the first line, move the rest of the lines down, and append the next line. If you have a history file that's significantly larger than the max size, that takes a while.

2) Having Copy to End selected.

3) Having Move to End selected.

4) Having Duplicates "Save First" selected.

5) Having Duplicates "Save Last" selected.

6) Setting HistoryExclude.

If you don't do any of those things, HISTORY /R is fast. If you do one or more, you've got to pay the price.
 
May 30, 2008
235
2
There are several things that can make a HISTORY /R slow:

1) Reading a history file that's bigger than the max history list size. HISTORY /R reads a line at a time, and when it gets to the max size it has to delete the first line, move the rest of the lines down, and append the next line. If you have a history file that's significantly larger than the max size, that takes a while.

2) Having Copy to End selected.

3) Having Move to End selected.

4) Having Duplicates "Save First" selected.

5) Having Duplicates "Save Last" selected.

6) Setting HistoryExclude.

If you don't do any of those things, HISTORY /R is fast. If you do one or more, you've got to pay the price.

1. Yes, I can see that. But since the file read with HISTORY /R1 is produced by SHRALIAS (via SHRALIAS_SAVE_PATH and SHRALIAS /U) from the currently loaded history entries this should never be the case, or? Some "gotchas" to consider maybe?

2-6. Right, but none of these should matter when using the /R1 option as I understand it. It should read a line at a time, just adding it to the history buffer without performing any duplicate or other checks.

(Just to be sure I unchecked all history check-boxes and turned off duplicate checking when doing the read with R1. Made no difference)
 
Similar threads
Thread starter Title Forum Replies Date
J History file load hangs if file has complete garbage in it... Support 1
R History window wrong size, can't remember size change Support 12
T How to? retain command history across reboot? Support 6
nikbackm Speeding up the history dialog Support 6
D ParentDirectory does not update directory history Support 6
vefatica Command history search is off Support 7
D Directory History Window with wildcard search Support 3
R Long commands retrieved from command history have blank line when wrapped Support 20
Joe Caverly @HISTORY issues Support 4
vefatica Deleting entries in the command history popup? Support 3
H History and TPipe Oddness Support 3
D History and control-C Support 1
G Console exe up / down history Support 8
D What does the file "..\Everything\Run history.csv" store? Support 5
rps Cmd history recall Support 4
David McClelland Command history loosing entries Support 18
R history features. Support 6
rps How to? Shralias and producing history.sav, alias.sav .. Support 6
M Ctrl-C when using command line history does nothing Support 2
S Debugger Bat file call history Support 9
Charles Dye Ctrl-Shift-Up doesn't update directory history Support 1
vefatica ESC after recalling a command from history Support 3
T TCHelp executable history? Support 2
thorntonpg @blahblah added to history. Support 0
M Call History popup-window? Support 0
M HISTORY lessons ... Support 0
vefatica The help and IE's history Support 16
D Towards shared (dir-)history lists Support 3
vefatica Ctrl-Enter in command history window? Support 8
vefatica History recall turns off Quick-Edit Support 4
C history is getting deleted under unknown circumstances Support 9
R V19 - At startup, TCC save directory in the directory history list Support 2
T [TCC 18] Can't read nor write history Support 1
I Possible Bug: History /n Returns Nothing Support 3
jbanaszczyk Lost history Support 4
T history.sav Support 4
T How to? sort history in tcexit.btm Support 22
vefatica OT: Windows console history? Support 7
S directory history locks up TCC Support 3
R In addition to ALIAS /Z and FUNCTION /Z, HISTORY /Z? Support 1
E TCC History popups too small Support 0
C How to? Keeping all history lists Support 4
E Command History not showing on up arrow button Support 8
T Selecting line from popup history without executing Support 8
A How to? Filter history list with unicode chars Support 0
vefatica WAD HISTORY /R, No closing quote? Support 10
vefatica WAD DO /S ... all dirs wind up in the history! Support 12
D Dir History file check Support 18
ehab aboudaya How to? Command Input history for each tab? Support 0
A Ctrl-1 and the command history Support 5

Similar threads