How to? Pull out multiple matching lines

  • This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

samintz

Scott Mintz
May 20, 2008
1,203
11
Solon, OH, USA
#1
I have a file filled with a bunch of debug output. I want to filter the file and only pull out the lines corresponding to the values that are transmitted.

For example:
Code:
Message transmitted: Thu Feb 02 16:09:57.552 2012  Thread ID 0x00001cc8
ASA Unconnected message: Get Attributes All Device
T: 01 02 20 01 24 01
           received: Thu Feb 02 16:09:57.559 2012
R: 81 00 00 00 01 00 0e 00 60 00 15 01 70 30 4b 4c 69 00 1c 44 42 5f 31 37 35
   36 2d 4c 37 35 2f 41 20 49 4e 54 56 49 45 57 45 5f 32 31 5f 32 30
Vendor                                  0001h
Product Type                            000eh
Product Code                            0060h
or
Code:
Message transmitted: Thu Feb 02 16:10:08.310 2012  Thread ID 0x00001c5c
ASA Connected message:   Connection ID 0c81ed10    Multiple Service Packet MessageRouter
T: 0a 02 20 02 24 01 02 00 06 00 21 00 08 03 21 00 49 03 24 04 01 00 01 00 0d
   00 41 73 73 6f 63 69 61 74 65 64 54 61 67 08 03 21 00 49 03 24 01 01 00 01
   00 07 00 43 6f 6d 6d 65 6e 74
           received: Thu Feb 02 16:10:08.330 2012
R: 8a 00 00 00 02 00 06 00 0e 00 88 00 00 00 04 00 00 00 88 00 00 00 01 00 00
   00
I want to pull out all the T: lines. In the first example, it's pretty easy because it's a single line. In the second example, it's spread over three lines.
This matches all the lines that start with T:
Code:
ffind /v /e"^T:" Debug.Log
And this matches all the multi-line output but only displays the first line:
Code:
ffind /v /e"^T:.+(\h+\s)*\n(\s+\h+)+" Debug.Log
How do I get all three (or more) lines of a multiline match?
-Scott
 

samintz

Scott Mintz
May 20, 2008
1,203
11
Solon, OH, USA
#2
This also works to match all the lines but still only displays the first line and not the entire match.

Code:
ffind /v /e"(?m)^T:.+received:" Debug.Log
 

samintz

Scott Mintz
May 20, 2008
1,203
11
Solon, OH, USA
#3
I wrote a batch file to do it.
Code:
setlocal
set st=0
set cnt=0
do ln in @Debug.Log
    iff %@regex["^T:",%ln ] then
        echos %ln
        set st=1
        set /a cnt+=1
    elseiff %@regex["\s+received:",%ln ] then
        set st=0
        echo.
    elseiff %st == 1 then
        echos. %@trim[%ln]
    endiff
enddo
echo found %cnt matches
endlocal
 
#4
Hello Scott,

for me that's a nice example of using regular expressions.
I would have solved it with @word and a similar logic, but indeed regular expression are very powerful.
I think I will have to go in for it.

But what I didn't see before is the usage of an incomplete condition "%@regex["^T:",%ln]". Shouldn't it be "%@regex["^T:",%ln] = 1"?

regards
frank
 
#5
Scott:
Neat solution, had I been free I'd have suggested a logical equivalent, but with slightly different tests:

setlocal
unset st
set cnt=0
do ln in @Debug.Log
set w1=%@word[0,%ln]
switch %w1
case T:
echos %ln
set st=1
set /a cnt+=1
case received:
unset st
echo.
default
if defined st echos. %@trim[%ln]
endswitch
enddo
echo found %cnt matches
endlocal

REX:
While I know speed is only rarely significant, please tell us which of the methods below is faster so it would make a difference when dealing with huge amounts of data / thousands of test repetitions.

1/ processing a file line by line using @file in a loop vs. reading the file into an array and processing the array elements in a loop, and unsetting the array., or using @fileopen/@fileread/@fileclose.

2/ using @REGEX[] vs. extracting the data element using %@left[] / %@instr[] / %@right[] / %@word[] and checking it with == or other tests

3/ when a variable is used as a logical (Boolean), as ST above, setting and checking its value vs. setting, unsetting and testing "defined"

4/ the iff/elseiff/else method or SWITCH.
 

rconn

Administrator
Staff member
May 14, 2008
10,100
85
#6
While I know speed is only rarely significant, please tell us which of the methods below is faster so it would make a difference when dealing with huge amounts of data / thousands of test repetitions.
I doubt you'll find much variation among any of the methods. You'll certainly spend vastly more time trying to find out than you'll save over a few hundred million repetitions. (You probably already have just by asking the question!)