Help with some parsing

samintz

Scott Mintz
May 20, 2008
1,488
15
Solon, OH, USA
I have a linker output file that is huge. I want to parse its output to show me all the external references for the modules contained within a certain library.

The file format is:
Loading member global.o from SQLite3.lib.
definition: sqlite3Config
definition: sqlite3GlobalFunctions
definition: sqlite3CtypeMap
definition: sqlite3UpperToLower
definition: sqlite3PendingByte
definition: sqlite3_Initialize()
reference : __aeabi_memcpy4
Loading member legacy.o from SQLite3.lib.

etc.

I want to display all the reference lines for the members in that library. It's relatively easy to locate the "Loading member..." line, however I essentially want to grab that block of text between the "Loading member" lines and then further search for only the "reference :..." lines.

-Scott
 
May 20, 2008
11,285
95
Syracuse, NY, USA
On Tue, 19 Jan 2010 15:23:32 -0500, samintz <> wrote:

|I have a linker output file that is huge. I want to parse its output to show me all the external references for the modules contained within a certain library.
|
|The file format is:
|Loading member global.o from SQLite3.lib.
| definition: sqlite3Config
| definition: sqlite3GlobalFunctions
| definition: sqlite3CtypeMap
| definition: sqlite3UpperToLower
| definition: sqlite3PendingByte
| definition: sqlite3_Initialize()
| reference : __aeabi_memcpy4
|Loading member legacy.o from SQLite3.lib.
|
|etc.
|
|I want to display all the reference lines for the members in that library. It's relatively easy to locate the "Loading member..." line, however I essentially want to grab that block of text between the "Loading member" lines and then further search for only the "reference :..." lines.

... ever use egrep (perhaps gnu)?

For example,

egrep "Loading|reference" file
--
- Vince
 
May 20, 2008
3,515
4
Elkridge, MD, USA
samintz wrote:
| I have a linker output file that is huge. I want to parse its
| output to show me all the external references for the modules
| contained within a certain library.
|
| The file format is:
| Loading member global.o from SQLite3.lib.
| definition: sqlite3Config
| definition: sqlite3GlobalFunctions
...
| definition: sqlite3_Initialize()
| reference : __aeabi_memcpy4
| Loading member legacy.o from SQLite3.lib.
...
| I want to display all the reference lines for the members in that
| library. It's relatively easy to locate the "Loading member..."
| line, however I essentially want to grab that block of text between
| the "Loading member" lines and then further search for only the
| "reference :..." lines.

If I understand correctly, the library member above is global.o, which
defines sqlite3Config, etc. It includes an external reference to
__aeabi_memcpy4 (which would require the loading of some other module that
defines __aeabi_memcpy4). From some work along similar lines I have done, I
presume you want to create a table of two columns, one column the object
module / library member name, and the other column the external references
from that module, e.g.:

Reference table
module reference
------------------------
global.o __aeabi_memcpy4
...

Something like this would do the job (NOT TESTED):

set xrh=%@fileopen[extref.lst,w,t]
set ht=%@char[9]
do line in @link.lst
switch %@word[0,%line]
case Loading
set module=%@word[2,%line]
case reference
set x=%@filewrite[%xrh,%[module]%[ht]%@word[-0,%line]]
endswitch
enddo
set x=%@fileclose[%xrh]

Naturally, you may want to add error checking, break handling (I hate those
leftover open files), possibly the library name for every line, and more
substantial parsing in the action selection.

You may also choose to create a second file, e.g., extdef.lst, which
enumerates the definitions made available by each module (in each library).

BTW, I have in decades past written Fortran programs for this kind of
parsing of compiler, librarian and linker listings (in the VAX VMS
environment).
--
HTH, Steve
 

samintz

Scott Mintz
May 20, 2008
1,488
15
Solon, OH, USA
The problem is the file is over 17MB. And there are many thousands of files & libraries being linked together. I am only interested in the external references for those modules being pulled in from the SQLite3.lib library.

This works, but takes forever to run:
Code:
set inner=0
do ln in @out.txt
    iff %inner==1 then
        iff %@REGEX["Loading member .* from ","%ln"] == 1 then
            set inner=0
        else
            iff %@REGEX["reference : .*","%ln"] == 1 then
                echo %ln
            endiff
            iterate
        endiff
    endiff    
    iff %@REGEX["Loading member .* from SQLite3\.lib","%ln"] == 1 then
        echo %ln
        set inner=1
    endiff
enddo

-Scott
 
May 20, 2008
3,515
4
Elkridge, MD, USA
Assuming that @left or @right combined with case sensitive comparison is
significantly faster than @REGEX followed by numeric test, the variant
below should be faster than your original. It does have a tiny bit
simplified logic, too.


Code:
---------
set inner=0
do ln in @out.txt
iff "%@left[14,%ln]" EQC "Loading member" then
set inner=%@if[ "from SQLite3.lib." EQC %@right[16,%ln],1,0]
elseiff %inner == 1 .and. %@left[27,%ln] EQC " reference :
" then
echo %ln
endiff
enddo


You could use the egrep suggested by Vince to reduce the file size first,
and than apply the filter above.
--
HTH, Steve
 
D

drrob1

Guest
I use powergrep (at powergrep.com) which is a native windows ap. It's
not the cheapest but it works very well.

vefatica wrote:

> On Tue, 19 Jan 2010 15:23:32 -0500, samintz <> wrote:
>
> |I have a linker output file that is huge. I want to parse its output to show me all the external references for the modules contained within a certain library.
> |
> |The file format is:
> |Loading member global.o from SQLite3.lib.
> | definition: sqlite3Config
> | definition: sqlite3GlobalFunctions
> | definition: sqlite3CtypeMap
> | definition: sqlite3UpperToLower
> | definition: sqlite3PendingByte
> | definition: sqlite3_Initialize()
> | reference : __aeabi_memcpy4
> |Loading member legacy.o from SQLite3.lib.
> |
> |etc.
> |
> |I want to display all the reference lines for the members in that library. It's relatively easy to locate the "Loading member..." line, however I essentially want to grab that block of text between the "Loading member" lines and then further search for only the "reference :..." lines.
>
> ... ever use egrep (perhaps gnu)?
>
> For example,
>
> egrep "Loading|reference" file
 
Nov 18, 2009
22
0
Sometimes Perl is the right tool for a job... or maybe Icon, if you're really old school :)

Code:
use feature ":5.10";

open(MODFILE,$ARGV[0]) || die "Couldn't open $ARGV[0]";

my %LibDefs, %LibRefs, $CurLib, $CurMember;

while ($line = readline(MODFILE)) {
    given ($line) {
        when (/^Loading member (.+) from (.+)\.$/) {
            $CurMember = $1;
            $CurLib = $2;
        }
        when (/^definition: (.+)$/) {
            push @{$LibDefs{$CurLib}->{$CurMember}},$1;
        }
        when (/^reference : (.+)$/) {
            push @{$LibRefs{$CurLib}->{$CurMember}},$1;
        }
    }
}


foreach $libname (keys %LibRefs) {
    if ((not defined($ARGV[1])) || ($libname eq $ARGV[1])) {
    say "In Library $libname";
    say "Definitions:";
    foreach $membername (keys %{$LibDefs{$libname}}) {
        foreach $defname (@{$LibDefs{$libname}->{$membername}}) {
            say "$membername, $defname";
        }
    }
    say "References:";
    foreach $membername (keys %{$LibRefs{$libname}}) {
        foreach $refname (@{$LibRefs{$libname}->{$membername}}) {
            say "$membername, $refname";
        }
    }
    }
}

close(MODFILE);
 
Similar threads
Thread starter Title Forum Replies Date
vefatica Online help? Support 2
fishman@panix.com HELP!! I just updated to Version 28 and as usual I cannot make extended directory search work. Support 2
JohnQSmith New online help file wrong logo Support 7
Jay Sage Documentation Function @DRIVE Missing in Help Lists Support 0
Charles Dye More help nits Support 0
D Documentation Typo in COPY help Support 0
FreezerBurnt Help making a CMD and TCC compatible batch file Support 7
Joe Caverly Documentation COMMANDS in Version 27 help Support 8
K_Meinhard How to? Help window position Support 7
U Command help on file names Support 5
D Documentation Help "Startup" page does not describe global lists Support 0
vefatica Help disappears? Support 6
rchapmanitt Help Purchasing Support 2
Charles Dye Documentation Help nits: @ZIPFILECRC Support 0
C version help Support 3
C Documentation Help Nit. REN / RENAME Support 0
Dmitry L. Kobyakov Documentation Error in the Help: the WINDOW command Support 0
vefatica Help for @PID Support 2
vefatica Need help with XML Support 3
Alpengreis Documentation Minor issue in help file for view /E Support 1
R Documentation Advanced Directives Help Support 0
R Documentation Help Nit with SETP Support 0
R wmic command help Support 9
R Documentation Help Nit > bdebugger View Menu Support 0
R Documentation A couple of Help nits Support 0
R Documentation Help typo Support 1
MikeBaas How to? I need help with RD Support 5
R Documentation v25 Help Comments Support 1
Joe Caverly DIRectory Colorization in v25 Help File Support 0
vefatica Help nit Support 0
D error in help documentation? Support 0
Joe Caverly _EXPANSION in help file Support 0
K_Meinhard Help file error Support 1
Joe Caverly JOBMONITOR in Help File Support 1
A Documentation [Help file] OPTION "//" synopsis formatting issue Support 0
D The local HELP viewer... Support 0
Chen Touboul Overtyping issue - i press the Insert key, help till press enter for new line in TCC Support 2
C Documentation Help nix ZIP /M Support 0
Joe Caverly v24 Help File - DATEMONITOR Support 1
Joe Caverly 7zip command links to zip help Support 0
K_Meinhard Help problems Support 3
Charles Dye Documentation Help Nits Forever: Checksum functions Support 0
Joe Caverly SAVECONSOLE in Help File Support 0
Joe Caverly CMDebug v23 Help: /FEEDBACK /UPDATE Support 0
Joe Caverly CMDebug v23 Help: Link to MOUNTVHD goes to MOUNTISO in UNMOUNTVHD Support 0
Joe Caverly Help not launching in CMDebug v23.00.14 x64 Support 1
Joe Caverly TASKDIALOG Help link error Support 1
C Help Nit Support 0
J Need activation help. Support 1
Joe Caverly Documentation SETP correction in help file Support 0

Similar threads