How to? Pull out multiple matching lines

samintz

Scott Mintz
May 20, 2008
1,503
16
Solon, OH, USA
I have a file filled with a bunch of debug output. I want to filter the file and only pull out the lines corresponding to the values that are transmitted.

For example:
Code:
Message transmitted: Thu Feb 02 16:09:57.552 2012  Thread ID 0x00001cc8
ASA Unconnected message: Get Attributes All Device
T: 01 02 20 01 24 01
           received: Thu Feb 02 16:09:57.559 2012
R: 81 00 00 00 01 00 0e 00 60 00 15 01 70 30 4b 4c 69 00 1c 44 42 5f 31 37 35
   36 2d 4c 37 35 2f 41 20 49 4e 54 56 49 45 57 45 5f 32 31 5f 32 30
Vendor                                  0001h
Product Type                            000eh
Product Code                            0060h

or
Code:
Message transmitted: Thu Feb 02 16:10:08.310 2012  Thread ID 0x00001c5c
ASA Connected message:   Connection ID 0c81ed10    Multiple Service Packet MessageRouter
T: 0a 02 20 02 24 01 02 00 06 00 21 00 08 03 21 00 49 03 24 04 01 00 01 00 0d
   00 41 73 73 6f 63 69 61 74 65 64 54 61 67 08 03 21 00 49 03 24 01 01 00 01
   00 07 00 43 6f 6d 6d 65 6e 74
           received: Thu Feb 02 16:10:08.330 2012
R: 8a 00 00 00 02 00 06 00 0e 00 88 00 00 00 04 00 00 00 88 00 00 00 01 00 00
   00
I want to pull out all the T: lines. In the first example, it's pretty easy because it's a single line. In the second example, it's spread over three lines.
This matches all the lines that start with T:
Code:
ffind /v /e"^T:" Debug.Log
And this matches all the multi-line output but only displays the first line:
Code:
ffind /v /e"^T:.+(\h+\s)*\n(\s+\h+)+" Debug.Log
How do I get all three (or more) lines of a multiline match?
-Scott
 

samintz

Scott Mintz
May 20, 2008
1,503
16
Solon, OH, USA
This also works to match all the lines but still only displays the first line and not the entire match.

Code:
ffind /v /e"(?m)^T:.+received:" Debug.Log
 

samintz

Scott Mintz
May 20, 2008
1,503
16
Solon, OH, USA
I wrote a batch file to do it.
Code:
setlocal
set st=0
set cnt=0
do ln in @Debug.Log
    iff %@regex["^T:",%ln ] then
        echos %ln
        set st=1
        set /a cnt+=1
    elseiff %@regex["\s+received:",%ln ] then
        set st=0
        echo.
    elseiff %st == 1 then
        echos. %@trim[%ln]
    endiff
enddo
echo found %cnt matches
endlocal
 
Aug 2, 2011
258
4
Berlin, Germany
Hello Scott,

for me that's a nice example of using regular expressions.
I would have solved it with @word and a similar logic, but indeed regular expression are very powerful.
I think I will have to go in for it.

But what I didn't see before is the usage of an incomplete condition "%@regex["^T:",%ln]". Shouldn't it be "%@regex["^T:",%ln] = 1"?

regards
frank
 
May 20, 2008
3,515
4
Elkridge, MD, USA
Scott:
Neat solution, had I been free I'd have suggested a logical equivalent, but with slightly different tests:

setlocal
unset st
set cnt=0
do ln in @Debug.Log
set w1=%@word[0,%ln]
switch %w1
case T:
echos %ln
set st=1
set /a cnt+=1
case received:
unset st
echo.
default
if defined st echos. %@trim[%ln]
endswitch
enddo
echo found %cnt matches
endlocal

REX:
While I know speed is only rarely significant, please tell us which of the methods below is faster so it would make a difference when dealing with huge amounts of data / thousands of test repetitions.

1/ processing a file line by line using @file in a loop vs. reading the file into an array and processing the array elements in a loop, and unsetting the array., or using @fileopen/@fileread/@fileclose.

2/ using @REGEX[] vs. extracting the data element using %@left[] / %@instr[] / %@right[] / %@word[] and checking it with == or other tests

3/ when a variable is used as a logical (Boolean), as ST above, setting and checking its value vs. setting, unsetting and testing "defined"

4/ the iff/elseiff/else method or SWITCH.
 

rconn

Administrator
Staff member
May 14, 2008
12,345
150
While I know speed is only rarely significant, please tell us which of the methods below is faster so it would make a difference when dealing with huge amounts of data / thousands of test repetitions.

I doubt you'll find much variation among any of the methods. You'll certainly spend vastly more time trying to find out than you'll save over a few hundred million repetitions. (You probably already have just by asking the question!)
 
Similar threads
Thread starter Title Forum Replies Date
MickeyF TCC crashing when copying multiple files (now resolved) Support 6
N Unable to bind multiple keys to the same key mapping directive Support 9
vefatica Console not redrawn in BTM or with multiple commands Support 4
C duplicate files in multiple plugin sites Support 9
kb6ojs ALIAS with multiple commands? Support 9
vefatica How do I do this with a multiple-line DO? Support 7
Joe Caverly Multiple Text Searches at once using FFIND or TPIPE Support 4
R How to? Append files in multiple subfolders using copy? Support 8
N multiple command alias and redirection Support 0
B Can copy command concatenate multiple mp3 files Support 2
C INSTALLED returnes same program multiple times Support 14
M Delete multiple Alternate Datastreams Support 5
D What are the rules for detecting multiple versions of TC on a machine? Support 7
C FOLDERMONITOR inactive with multiple /I Support 1
C WAD VIEW print multiple files - switches fail Support 10
vefatica Multiple instances of help file Support 3
J How to? Array sort according to multiple keys Support 3
Alexander WAD The number of files and dirs are multiplied when using multiple wildcards Support 6
Jay Sage Version 17 Installer Leaves Multiple Copies of ShrAlias Support 48
R Fixed Internal ZIP command fails to process multiple files Support 3
M How to? PDir on multiple wildcard patterns... Support 2
A WAD Dir daterange + multiple path wildcards crashes tcc Support 2
T How to? Using FOLDERMONITOR 1 event causes multiple trigger(ing)s Support 8
D WAD Multiple TEE causese weird variable scoping? Support 7
L "for" in multiple lines Support 2
S TAR with multiple files and /G Support 2
Frank VIEWing multiple files possible? Support 4
J syntax problems multiple commands (command & command) Support 4
J How do I - ffind - with multiple items Support 10
R multiple monitor 'help' Support 7
H Another multiple FTP password prompt issue Support 2
J tcmd height with multiple monitors Support 2
T Open & Configure multiple TCC windows on STARTUP Support 4
R FUNCTION to count NUMBER of files matching a pattern and specific length/not length Support 4
A List windows with titles matching given pattern? Support 2
rps @FILESIZE[....,a] allocated size not matching Dir results Support 8
C Touch command — I Need to touch matching files in another directory. Support 5
David McClelland TPIPE matching problem Support 3

Similar threads