Help with some parsing

samintz

Scott Mintz
May 20, 2008
1,294
11
Solon, OH, USA
#1
I have a linker output file that is huge. I want to parse its output to show me all the external references for the modules contained within a certain library.

The file format is:
Loading member global.o from SQLite3.lib.
definition: sqlite3Config
definition: sqlite3GlobalFunctions
definition: sqlite3CtypeMap
definition: sqlite3UpperToLower
definition: sqlite3PendingByte
definition: sqlite3_Initialize()
reference : __aeabi_memcpy4
Loading member legacy.o from SQLite3.lib.

etc.

I want to display all the reference lines for the members in that library. It's relatively easy to locate the "Loading member..." line, however I essentially want to grab that block of text between the "Loading member" lines and then further search for only the "reference :..." lines.

-Scott
 
#2
On Tue, 19 Jan 2010 15:23:32 -0500, samintz <> wrote:

|I have a linker output file that is huge. I want to parse its output to show me all the external references for the modules contained within a certain library.
|
|The file format is:
|Loading member global.o from SQLite3.lib.
| definition: sqlite3Config
| definition: sqlite3GlobalFunctions
| definition: sqlite3CtypeMap
| definition: sqlite3UpperToLower
| definition: sqlite3PendingByte
| definition: sqlite3_Initialize()
| reference : __aeabi_memcpy4
|Loading member legacy.o from SQLite3.lib.
|
|etc.
|
|I want to display all the reference lines for the members in that library. It's relatively easy to locate the "Loading member..." line, however I essentially want to grab that block of text between the "Loading member" lines and then further search for only the "reference :..." lines.

... ever use egrep (perhaps gnu)?

For example,

egrep "Loading|reference" file
--
- Vince
 
#3
samintz wrote:
| I have a linker output file that is huge. I want to parse its
| output to show me all the external references for the modules
| contained within a certain library.
|
| The file format is:
| Loading member global.o from SQLite3.lib.
| definition: sqlite3Config
| definition: sqlite3GlobalFunctions
...
| definition: sqlite3_Initialize()
| reference : __aeabi_memcpy4
| Loading member legacy.o from SQLite3.lib.
...
| I want to display all the reference lines for the members in that
| library. It's relatively easy to locate the "Loading member..."
| line, however I essentially want to grab that block of text between
| the "Loading member" lines and then further search for only the
| "reference :..." lines.

If I understand correctly, the library member above is global.o, which
defines sqlite3Config, etc. It includes an external reference to
__aeabi_memcpy4 (which would require the loading of some other module that
defines __aeabi_memcpy4). From some work along similar lines I have done, I
presume you want to create a table of two columns, one column the object
module / library member name, and the other column the external references
from that module, e.g.:

Reference table
module reference
------------------------
global.o __aeabi_memcpy4
...

Something like this would do the job (NOT TESTED):

set xrh=%@fileopen[extref.lst,w,t]
set ht=%@char[9]
do line in @link.lst
switch %@word[0,%line]
case Loading
set module=%@word[2,%line]
case reference
set x=%@filewrite[%xrh,%[module]%[ht]%@word[-0,%line]]
endswitch
enddo
set x=%@fileclose[%xrh]

Naturally, you may want to add error checking, break handling (I hate those
leftover open files), possibly the library name for every line, and more
substantial parsing in the action selection.

You may also choose to create a second file, e.g., extdef.lst, which
enumerates the definitions made available by each module (in each library).

BTW, I have in decades past written Fortran programs for this kind of
parsing of compiler, librarian and linker listings (in the VAX VMS
environment).
--
HTH, Steve
 

samintz

Scott Mintz
May 20, 2008
1,294
11
Solon, OH, USA
#4
The problem is the file is over 17MB. And there are many thousands of files & libraries being linked together. I am only interested in the external references for those modules being pulled in from the SQLite3.lib library.

This works, but takes forever to run:
Code:
set inner=0
do ln in @out.txt
    iff %inner==1 then
        iff %@REGEX["Loading member .* from ","%ln"] == 1 then
            set inner=0
        else
            iff %@REGEX["reference : .*","%ln"] == 1 then
                echo %ln
            endiff
            iterate
        endiff
    endiff    
    iff %@REGEX["Loading member .* from SQLite3\.lib","%ln"] == 1 then
        echo %ln
        set inner=1
    endiff
enddo
-Scott
 
#5
Assuming that @left or @right combined with case sensitive comparison is
significantly faster than @REGEX followed by numeric test, the variant
below should be faster than your original. It does have a tiny bit
simplified logic, too.


Code:
---------
set inner=0
do ln in @out.txt
iff "%@left[14,%ln]" EQC "Loading member" then
set inner=%@if[ "from SQLite3.lib." EQC %@right[16,%ln],1,0]
elseiff %inner == 1 .and. %@left[27,%ln] EQC " reference :
" then
echo %ln
endiff
enddo


You could use the egrep suggested by Vince to reduce the file size first,
and than apply the filter above.
--
HTH, Steve
 
D

drrob1

Guest
#6
I use powergrep (at powergrep.com) which is a native windows ap. It's
not the cheapest but it works very well.

vefatica wrote:

> On Tue, 19 Jan 2010 15:23:32 -0500, samintz <> wrote:
>
> |I have a linker output file that is huge. I want to parse its output to show me all the external references for the modules contained within a certain library.
> |
> |The file format is:
> |Loading member global.o from SQLite3.lib.
> | definition: sqlite3Config
> | definition: sqlite3GlobalFunctions
> | definition: sqlite3CtypeMap
> | definition: sqlite3UpperToLower
> | definition: sqlite3PendingByte
> | definition: sqlite3_Initialize()
> | reference : __aeabi_memcpy4
> |Loading member legacy.o from SQLite3.lib.
> |
> |etc.
> |
> |I want to display all the reference lines for the members in that library. It's relatively easy to locate the "Loading member..." line, however I essentially want to grab that block of text between the "Loading member" lines and then further search for only the "reference :..." lines.
>
> ... ever use egrep (perhaps gnu)?
>
> For example,
>
> egrep "Loading|reference" file
 
Nov 18, 2009
19
0
#7
Sometimes Perl is the right tool for a job... or maybe Icon, if you're really old school :)

Code:
use feature ":5.10";

open(MODFILE,$ARGV[0]) || die "Couldn't open $ARGV[0]";

my %LibDefs, %LibRefs, $CurLib, $CurMember;

while ($line = readline(MODFILE)) {
    given ($line) {
        when (/^Loading member (.+) from (.+)\.$/) {
            $CurMember = $1;
            $CurLib = $2;
        }
        when (/^definition: (.+)$/) {
            push @{$LibDefs{$CurLib}->{$CurMember}},$1;
        }
        when (/^reference : (.+)$/) {
            push @{$LibRefs{$CurLib}->{$CurMember}},$1;
        }
    }
}


foreach $libname (keys %LibRefs) {
    if ((not defined($ARGV[1])) || ($libname eq $ARGV[1])) {
    say "In Library $libname";
    say "Definitions:";
    foreach $membername (keys %{$LibDefs{$libname}}) {
        foreach $defname (@{$LibDefs{$libname}->{$membername}}) {
            say "$membername, $defname";
        }
    }
    say "References:";
    foreach $membername (keys %{$LibRefs{$libname}}) {
        foreach $refname (@{$LibRefs{$libname}->{$membername}}) {
            say "$membername, $refname";
        }
    }
    }
}

close(MODFILE);