HISTORY /R1 performance

[nikbackm]

I've noticed that loading the saved history at TCMD init takes about 2 seconds on my machine (which has quite a fast i7 processor and plenty of memory).

I use "HISTORY /R1 history.sav" to load it. (Using /R option instead takes about 10 seconds).

My history size is set to the maximum value; 270,336. The history.sav file produced by SHRALIAS on unload is 558,758 bytes. (It saves it in UTF-16 encoding).

It seems to me that loading such a (relatively) small file into a memory buffer on a modern machine (with SSD) should hardly take so long. (Running WC on the file takes 0.01 seconds).

Could there be something sub-optimal with my config or is the problem with HISTORY? Perhaps it does extra processing when reading the lines or have to frequently reallocate memory when more lines are read.

(I use a shared history for all TCC instances with SHRALIAS)

 

[vefatica]

It has always been slow. I didn't know about the /R1 option and it does make it significantly faster.

h:\shrdump> d h*
2013-05-21  05:00        325,208  history.sav
 
h:\shrdump> timer history /r1 history.sav
Timer 1 on: 12:33:53
Timer 1 off: 12:33:56  Elapsed: 0:00:02.73

That still seems slow (you're doing better than me, Niklas). I can TYPE that file in half the time.

 

[rconn]

I've noticed that loading the saved history at TCMD init takes about 2 seconds on my machine (which has quite a fast i7 processor and plenty of memory).

There are several things that can make a HISTORY /R slow:

1) Reading a history file that's bigger than the max history list size. HISTORY /R reads a line at a time, and when it gets to the max size it has to delete the first line, move the rest of the lines down, and append the next line. If you have a history file that's significantly larger than the max size, that takes a while.

2) Having Copy to End selected.

3) Having Move to End selected.

4) Having Duplicates "Save First" selected.

5) Having Duplicates "Save Last" selected.

6) Setting HistoryExclude.

If you don't do any of those things, HISTORY /R is fast. If you do one or more, you've got to pay the price.

 

[nikbackm]

There are several things that can make a HISTORY /R slow:

1) Reading a history file that's bigger than the max history list size. HISTORY /R reads a line at a time, and when it gets to the max size it has to delete the first line, move the rest of the lines down, and append the next line. If you have a history file that's significantly larger than the max size, that takes a while.

2) Having Copy to End selected.

3) Having Move to End selected.

4) Having Duplicates "Save First" selected.

5) Having Duplicates "Save Last" selected.

6) Setting HistoryExclude.

If you don't do any of those things, HISTORY /R is fast. If you do one or more, you've got to pay the price.

1. Yes, I can see that. But since the file read with HISTORY /R1 is produced by SHRALIAS (via SHRALIAS_SAVE_PATH and SHRALIAS /U) from the currently loaded history entries this should never be the case, or? Some "gotchas" to consider maybe?

2-6. Right, but none of these should matter when using the /R1 option as I understand it. It should read a line at a time, just adding it to the history buffer without performing any duplicate or other checks.

(Just to be sure I unchecked all history check-boxes and turned off duplicate checking when doing the read with R1. Made no difference)