1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Very long startup times - solved for me

Discussion in 'Support' started by Lippolt, Mar 24, 2009.

  1. Lippolt

    Joined:
    Mar 22, 2009
    Messages:
    10
    Likes Received:
    0
    Hello !

    I know there is alreay a thread about that theme, but my report has a slightly different direction. I also had increasing startup times (both for TCC and 4NT, version 10/61), with more than 5 minutes to wait, and no obvious extended disk activitiy. So I began to look in another places than loading big directory trees.

    For me disabling the history files for command history (and directory history) solved the problem - I have now an instant start again.

    Is there a known problem in loading back the history files into memory? Can other users also reduce their startup times by turning history files off?

    Best regards from Hannover in Germany,

    Peter
     
  2. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,732
    Likes Received:
    81
    Lippolt wrote:

    This should only be a problem if (1) you have REALLY big history files,
    and (2) you have small history lists, so that TCC has to continually
    delete the old entries and shuffle the list.

    Rex Conn
    JP Software
     
  3. dim

    dim Dimitry Andric

    Joined:
    May 31, 2008
    Messages:
    202
    Likes Received:
    0
    On 2009-03-25 01:29, Lippolt wrote:

    Yes, it has also been excruciatingly slow for me. I have turned off
    history loading a long time ago, since it didn't seem to get any better
    with subsequent releases...
     
  4. Lippolt

    Joined:
    Mar 22, 2009
    Messages:
    10
    Likes Received:
    0
    The history file was about 450 kbyte. Where can I set the size of the history list? Where can I limit the size of the history file?

    What is your suggestion for reasonable settings? And where have they to be set?

    Thanks for your fast reply,

    Peter
     
  5. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,732
    Likes Received:
    81
    Lippolt wrote:


    Depends on whether you're using global or local lists.

    If they're local:

    OPTION / Command Line / History Buffer Sizes.

    If they're global, they're set to the value of the History and
    DirHistory directives in the first TCC instance (default of 128K
    characters for the history list and 4K for the directory history).

    If you're doing a HISTORY /R, it adds the lines to the history one at a
    time, which means it has to check HistoryExclude for each line, and (if
    you have the HistDups option set) checking the entire history list for
    matching lines. *Plus* deleting all of the old entries and shuffling
    the list when you have a larger history file than history buffer.
    (Plus, it has to lock & unlock the list for each line to prevent any
    other process from accessing it while it's being updated.)

    Rex Conn
    JP Software
     
  6. vefatica

    Joined:
    May 20, 2008
    Messages:
    7,792
    Likes Received:
    29
    On Tue, 24 Mar 2009 21:01:27 -0500, rconn <> wrote:

    |If they're global, they're set to the value of the History and
    |DirHistory directives in the first TCC instance (default of 128K
    |characters for the history list and 4K for the directory history).

    That's what I thought, and I have "History=131072" in my INI file. But OPTION
    shows 133120. Another user reported a history file of over 400KB (over 200
    characters). What's the limit? I couldn't find it in the help.
    --
    - Vince
     
  7. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,732
    Likes Received:
    81
    vefatica wrote:

    The history list sizes are always padded slightly in order to store
    internal info. The maximum command history size is 500K, and the
    maximum directory history size is 50K. (But IMO anybody who uses a
    history that size is nuts!)

    Rex Conn
    JP Software
     
  8. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,523
    Likes Received:
    4
    rconn wrote:
    | Lippolt wrote:
    |
    |
    |
    | ---Quote---
    || The history file was about 450 kbyte. Where can I set the size of
    || the history list? Where can I limit the size of the history file?
    ||
    || What is your suggestion for reasonable settings? And where have they
    || to be set?
    | ---End Quote---
    | Depends on whether you're using global or local lists.
    |
    | If they're local:
    |
    | OPTION / Command Line / History Buffer Sizes.
    |
    | If they're global, they're set to the value of the History and
    | DirHistory directives in the first TCC instance (default of 128K
    | characters for the history list and 4K for the directory history).
    |
    | If you're doing a HISTORY /R, it adds the lines to the history one at
    | a time, which means it has to check HistoryExclude for each line, and
    | (if
    | you have the HistDups option set) checking the entire history list for
    | matching lines. *Plus* deleting all of the old entries and shuffling
    | the list when you have a larger history file than history buffer.
    | (Plus, it has to lock & unlock the list for each line to prevent any
    | other process from accessing it while it's being updated.)

    I suggest suboption for HISTORY /R to load the specified file without
    checking for duplicates. This being an option it would be up to the user
    whether or not eliminating duplicates from a history file are significant.

    Another possible subpotion would be to keep the list locked continuously
    until the HISTORY /R command is completed. This would certainly be harmless
    when local lists are used. For those who use a global list this would only
    affect those concurrent TCC instances where 1/ you manually execute commands
    while another instance executes HISTORY /R or 2/ you concurrently execute
    HISTORY/R. In both these situations the much reduced delay of not unlocking
    and relocking for each data line would be generally worth the slightly
    increased total delay.

    In my own use, HISTORY/R is executed by a special, transient instance of
    TCC, invoked when I log in as a user, and thereafter only manually. Each of
    the suggested suboptions would speed up the login process, without any
    aftereffects. Any further use of HISTORY /R is strictly manual.
    --
    Steve
     
  9. dim

    dim Dimitry Andric

    Joined:
    May 31, 2008
    Messages:
    202
    Likes Received:
    0
    On 2009-03-25 03:43, rconn wrote:

    Come on, 500 kiB is nothing these days; the average "soccer mom"
    consumer now buys a computer at Best Buy with 1 or 2 GiB of RAM. You
    are thinking in DOS-day terms. :)

    Also, if you type commands for a few days, the accumulated history is
    easily a few hundred kilobytes. Why should it throw away this old
    history?

    I have always logged all commands I typed, and this is just a few
    megabytes per year, completely irrelevant with today's memory and
    harddisk sizes.
     
  10. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,732
    Likes Received:
    81
    dim wrote:

    The amount of RAM available is irrelevant; the problem is when somebody
    wants to have a large history and also have TCC do extensive processing
    on it. (And then complain that processing their large list takes time!)


    Why keep all of it when you'll never look at 99% of it again?

    Rex Conn
    JP Software
     
  11. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,732
    Likes Received:
    81
    Steve Fábián wrote:


    Definitely NITV, as it would require a complete rewrite of HISTORY and
    DIRHISTORY. (It would also result in a lot of code duplication, as
    HISTORY and DIRHISTORY wouldn't be able to use the common routines for
    history saves.)


    This also requires a rewrite.

    Rex Conn
    JP Software
     
  12. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,523
    Likes Received:
    4
    rconn wrote:
    | Steve Fábián wrote:
    |
    |
    |
    | ---Quote---
    || I suggest suboption for HISTORY /R to load the specified file without
    || checking for duplicates. This being an option it would be up to the
    || user whether or not eliminating duplicates from a history file are
    || significant.
    | ---End Quote---
    | Definitely NITV, as it would require a complete rewrite of HISTORY and
    | DIRHISTORY. (It would also result in a lot of code duplication, as
    | HISTORY and DIRHISTORY wouldn't be able to use the common routines for
    | history saves.)

    Why would this be a complete rewrite? This would just temporarily imitate
    the HistDups=OFF setting.


    | ---Quote---
    || Another possible subpotion would be to keep the list locked
    || continuously until the HISTORY /R command is completed.
    | ---End Quote---
    | This also requires a rewrite.

    I definitely would not expect such changes before a new version. OTOH,
    wouldn't the same rules be applicable for both HISTORY/R and DIRHISTORY/R so
    that code sharing could continue?
    --
    Steve
     
  13. Jay Sage

    Joined:
    Jun 2, 2008
    Messages:
    284
    Likes Received:
    1
    I used to have code in TCEXIT.BTM that saved the entire command history
    and directory history to files. Code in TCSTART.BTM would load them
    using the /R option. Now that I understand how this slows down the
    launch of TCMD, I now save only a specified small number of lines from
    the ends of the histories. TCMD now starts up much faster.

    It would be nice though (albeit in a future version) if there were a way
    to load files directly into the command history and directory history
    with no processing at all. My saved history files, after all, do not
    need any additional processing; they were saved right out of the
    existing history.

    Alternatively, perhaps there could be separate facilities for saving
    histories on exit from TCC and reloading them for new TCC tasks. It is
    nice to be able to shut down TCMD or TCC tasks and later have it/them
    come up with some of the state restored.

    -- Jay
     
  14. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,523
    Likes Received:
    4
    Jay Sage wrote:
    | I used to have code in TCEXIT.BTM that saved the entire command
    | history
    | and directory history to files. Code in TCSTART.BTM would load them
    | using the /R option. Now that I understand how this slows down the
    | launch of TCMD, I now save only a specified small number of lines from
    | the ends of the histories. TCMD now starts up much faster.
    |
    | It would be nice though (albeit in a future version) if there were a
    | way
    | to load files directly into the command history and directory history
    | with no processing at all. My saved history files, after all, do not
    | need any additional processing; they were saved right out of the
    | existing history.
    |
    | Alternatively, perhaps there could be separate facilities for saving
    | histories on exit from TCC and reloading them for new TCC tasks. It is
    | nice to be able to shut down TCMD or TCC tasks and later have it/them
    | come up with some of the state restored.

    I save command and directory history only through the automatic mechanism
    built into SHRALIAS.EXE in recent versions, i.e., when SHRALIAS.EXE is
    terminated, all global tables are saved. I do use all four tables as
    globals. Whenever I log in into a WinXP session, a shortcut in
    %allusers\start menu\programs\startup starts a transient TCC instance, which
    loads all four global tables from the files where they were saved. Once
    that's done, SHRALIAS.EXE keeps them alive, so no new TCC instances ever
    need to load them again. In this manner the slow behavior of the HISTORY/R
    command occurs only on rare occasions (averaging less than once a day). Note
    that the same instance of TCC could instead be retained until logout, and
    used to run the various MONITOR commands (that is, be the "master thread"
    for them). It could also serve to dispatch other tasks either based on time
    of day, or on the occurrence of other events.
    --
    Steve
     
  15. Jay Sage

    Joined:
    Jun 2, 2008
    Messages:
    284
    Likes Received:
    1
    > I save command and directory history only through the
    > automatic mechanism built into SHRALIAS.EXE

    Steve, thank you so much for that information! Perhaps I was once aware
    of that SHRALIAS feature (I have defined a SHRALIAS_SAVE_PATH variable),
    but I had forgotten. I've now reworked my TCSTART and TCEXIT scripts to
    omit the loading and saving if SHRALIAS is running.

    -- Jay
     
  16. Rod Savard

    Joined:
    May 26, 2008
    Messages:
    481
    Likes Received:
    3
    Wild... 5 minutes or more to start TCC? My history is 32K and TCC starts instantly (with loading the history in TCSTART.BTM). I can't imagine it taking 5+ minutes even if your history is 15 times the size of mine.
     

Share This Page