1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Help with some parsing

Discussion in 'Support' started by samintz, Jan 19, 2010.

  1. samintz

    samintz Scott Mintz

    Joined:
    May 20, 2008
    Messages:
    1,203
    Likes Received:
    11
    I have a linker output file that is huge. I want to parse its output to show me all the external references for the modules contained within a certain library.

    The file format is:
    Loading member global.o from SQLite3.lib.
    definition: sqlite3Config
    definition: sqlite3GlobalFunctions
    definition: sqlite3CtypeMap
    definition: sqlite3UpperToLower
    definition: sqlite3PendingByte
    definition: sqlite3_Initialize()
    reference : __aeabi_memcpy4
    Loading member legacy.o from SQLite3.lib.

    etc.

    I want to display all the reference lines for the members in that library. It's relatively easy to locate the "Loading member..." line, however I essentially want to grab that block of text between the "Loading member" lines and then further search for only the "reference :..." lines.

    -Scott
     
  2. vefatica

    Joined:
    May 20, 2008
    Messages:
    8,129
    Likes Received:
    33
    On Tue, 19 Jan 2010 15:23:32 -0500, samintz <> wrote:

    |I have a linker output file that is huge. I want to parse its output to show me all the external references for the modules contained within a certain library.
    |
    |The file format is:
    |Loading member global.o from SQLite3.lib.
    | definition: sqlite3Config
    | definition: sqlite3GlobalFunctions
    | definition: sqlite3CtypeMap
    | definition: sqlite3UpperToLower
    | definition: sqlite3PendingByte
    | definition: sqlite3_Initialize()
    | reference : __aeabi_memcpy4
    |Loading member legacy.o from SQLite3.lib.
    |
    |etc.
    |
    |I want to display all the reference lines for the members in that library. It's relatively easy to locate the "Loading member..." line, however I essentially want to grab that block of text between the "Loading member" lines and then further search for only the "reference :..." lines.

    ... ever use egrep (perhaps gnu)?

    For example,

    egrep "Loading|reference" file
    --
    - Vince
     
  3. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,520
    Likes Received:
    4
    samintz wrote:
    | I have a linker output file that is huge. I want to parse its
    | output to show me all the external references for the modules
    | contained within a certain library.
    |
    | The file format is:
    | Loading member global.o from SQLite3.lib.
    | definition: sqlite3Config
    | definition: sqlite3GlobalFunctions
    ...
    | definition: sqlite3_Initialize()
    | reference : __aeabi_memcpy4
    | Loading member legacy.o from SQLite3.lib.
    ...
    | I want to display all the reference lines for the members in that
    | library. It's relatively easy to locate the "Loading member..."
    | line, however I essentially want to grab that block of text between
    | the "Loading member" lines and then further search for only the
    | "reference :..." lines.

    If I understand correctly, the library member above is global.o, which
    defines sqlite3Config, etc. It includes an external reference to
    __aeabi_memcpy4 (which would require the loading of some other module that
    defines __aeabi_memcpy4). From some work along similar lines I have done, I
    presume you want to create a table of two columns, one column the object
    module / library member name, and the other column the external references
    from that module, e.g.:

    Reference table
    module reference
    ------------------------
    global.o __aeabi_memcpy4
    ...

    Something like this would do the job (NOT TESTED):

    set xrh=%@fileopen[extref.lst,w,t]
    set ht=%@char[9]
    do line in @link.lst
    switch %@word[0,%line]
    case Loading
    set module=%@word[2,%line]
    case reference
    set x=%@filewrite[%xrh,%[module]%[ht]%@word[-0,%line]]
    endswitch
    enddo
    set x=%@fileclose[%xrh]

    Naturally, you may want to add error checking, break handling (I hate those
    leftover open files), possibly the library name for every line, and more
    substantial parsing in the action selection.

    You may also choose to create a second file, e.g., extdef.lst, which
    enumerates the definitions made available by each module (in each library).

    BTW, I have in decades past written Fortran programs for this kind of
    parsing of compiler, librarian and linker listings (in the VAX VMS
    environment).
    --
    HTH, Steve
     
  4. samintz

    samintz Scott Mintz

    Joined:
    May 20, 2008
    Messages:
    1,203
    Likes Received:
    11
    The problem is the file is over 17MB. And there are many thousands of files & libraries being linked together. I am only interested in the external references for those modules being pulled in from the SQLite3.lib library.

    This works, but takes forever to run:
    Code:
    set inner=0
    do ln in @out.txt
        iff %inner==1 then
            iff %@REGEX["Loading member .* from ","%ln"] == 1 then
                set inner=0
            else
                iff %@REGEX["reference : .*","%ln"] == 1 then
                    echo %ln
                endiff
                iterate
            endiff
        endiff    
        iff %@REGEX["Loading member .* from SQLite3\.lib","%ln"] == 1 then
            echo %ln
            set inner=1
        endiff
    enddo
    
    -Scott
     
  5. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,520
    Likes Received:
    4
    Assuming that @left or @right combined with case sensitive comparison is
    significantly faster than @REGEX followed by numeric test, the variant
    below should be faster than your original. It does have a tiny bit
    simplified logic, too.


    Code:
    ---------
    set inner=0
    do ln in @out.txt
    iff "%@left[14,%ln]" EQC "Loading member" then
    set inner=%@if[ "from SQLite3.lib." EQC %@right[16,%ln],1,0]
    elseiff %inner == 1 .and. %@left[27,%ln] EQC " reference :
    " then
    echo %ln
    endiff
    enddo


    You could use the egrep suggested by Vince to reduce the file size first,
    and than apply the filter above.
    --
    HTH, Steve
     
  6. drrob1

    drrob1 Guest

    I use powergrep (at powergrep.com) which is a native windows ap. It's
    not the cheapest but it works very well.

    vefatica wrote:

     
  7. NetMage

    Joined:
    Nov 18, 2009
    Messages:
    17
    Likes Received:
    0
    Sometimes Perl is the right tool for a job... or maybe Icon, if you're really old school :)

    Code:
    use feature ":5.10";
    
    open(MODFILE,$ARGV[0]) || die "Couldn't open $ARGV[0]";
    
    my %LibDefs, %LibRefs, $CurLib, $CurMember;
    
    while ($line = readline(MODFILE)) {
        given ($line) {
            when (/^Loading member (.+) from (.+)\.$/) {
                $CurMember = $1;
                $CurLib = $2;
            }
            when (/^definition: (.+)$/) {
                push @{$LibDefs{$CurLib}->{$CurMember}},$1;
            }
            when (/^reference : (.+)$/) {
                push @{$LibRefs{$CurLib}->{$CurMember}},$1;
            }
        }
    }
    
    
    foreach $libname (keys %LibRefs) {
        if ((not defined($ARGV[1])) || ($libname eq $ARGV[1])) {
        say "In Library $libname";
        say "Definitions:";
        foreach $membername (keys %{$LibDefs{$libname}}) {
            foreach $defname (@{$LibDefs{$libname}->{$membername}}) {
                say "$membername, $defname";
            }
        }
        say "References:";
        foreach $membername (keys %{$LibRefs{$libname}}) {
            foreach $refname (@{$LibRefs{$libname}->{$membername}}) {
                say "$membername, $refname";
            }
        }
        }
    }
    
    close(MODFILE);
    
     

Share This Page