How to? Pull out multiple matching lines

[samintz]

I have a file filled with a bunch of debug output. I want to filter the file and only pull out the lines corresponding to the values that are transmitted.

For example:

Message transmitted: Thu Feb 02 16:09:57.552 2012  Thread ID 0x00001cc8
ASA Unconnected message: Get Attributes All Device
T: 01 02 20 01 24 01
           received: Thu Feb 02 16:09:57.559 2012
R: 81 00 00 00 01 00 0e 00 60 00 15 01 70 30 4b 4c 69 00 1c 44 42 5f 31 37 35
   36 2d 4c 37 35 2f 41 20 49 4e 54 56 49 45 57 45 5f 32 31 5f 32 30
Vendor                                  0001h
Product Type                            000eh
Product Code                            0060h

or

Message transmitted: Thu Feb 02 16:10:08.310 2012  Thread ID 0x00001c5c
ASA Connected message:   Connection ID 0c81ed10    Multiple Service Packet MessageRouter
T: 0a 02 20 02 24 01 02 00 06 00 21 00 08 03 21 00 49 03 24 04 01 00 01 00 0d
   00 41 73 73 6f 63 69 61 74 65 64 54 61 67 08 03 21 00 49 03 24 01 01 00 01
   00 07 00 43 6f 6d 6d 65 6e 74
           received: Thu Feb 02 16:10:08.330 2012
R: 8a 00 00 00 02 00 06 00 0e 00 88 00 00 00 04 00 00 00 88 00 00 00 01 00 00
   00

I want to pull out all the T: lines. In the first example, it's pretty easy because it's a single line. In the second example, it's spread over three lines.
This matches all the lines that start with T:

ffind /v /e"^T:" Debug.Log

And this matches all the multi-line output but only displays the first line:

ffind /v /e"^T:.+(\h+\s)*\n(\s+\h+)+" Debug.Log

How do I get all three (or more) lines of a multiline match?
-Scott

 

[samintz]

This also works to match all the lines but still only displays the first line and not the entire match.

ffind /v /e"(?m)^T:.+received:" Debug.Log

 

[samintz]

I wrote a batch file to do it.

setlocal
set st=0
set cnt=0
do ln in @Debug.Log
    iff %@regex["^T:",%ln ] then
        echos %ln
        set st=1
        set /a cnt+=1
    elseiff %@regex["\s+received:",%ln ] then
        set st=0
        echo.
    elseiff %st == 1 then
        echos. %@trim[%ln]
    endiff
enddo
echo found %cnt matches
endlocal

 

[Frank]

Hello Scott,

for me that's a nice example of using regular expressions.
I would have solved it with @word and a similar logic, but indeed regular expression are very powerful.
I think I will have to go in for it.

But what I didn't see before is the usage of an incomplete condition "%@regex["^T:",%ln]". Shouldn't it be "%@regex["^T:",%ln] = 1"?

regards
frank

 

[Steve Fabian]

Scott:
Neat solution, had I been free I'd have suggested a logical equivalent, but with slightly different tests:

setlocal
unset st
set cnt=0
do ln in @Debug.Log
set w1=%@word[0,%ln]
switch %w1
case T:
echos %ln
set st=1
set /a cnt+=1
case received:
unset st
echo.
default
if defined st echos. %@trim[%ln]
endswitch
enddo
echo found %cnt matches
endlocal

REX:
While I know speed is only rarely significant, please tell us which of the methods below is faster so it would make a difference when dealing with huge amounts of data / thousands of test repetitions.

1/ processing a file line by line using @file in a loop vs. reading the file into an array and processing the array elements in a loop, and unsetting the array., or using @fileopen/@fileread/@fileclose.

2/ using @REGEX[] vs. extracting the data element using %@left[] / %@instr[] / %@right[] / %@word[] and checking it with == or other tests

3/ when a variable is used as a logical (Boolean), as ST above, setting and checking its value vs. setting, unsetting and testing "defined"

4/ the iff/elseiff/else method or SWITCH.

 

[rconn]

While I know speed is only rarely significant, please tell us which of the methods below is faster so it would make a difference when dealing with huge amounts of data / thousands of test repetitions.

I doubt you'll find much variation among any of the methods. You'll certainly spend vastly more time trying to find out than you'll save over a few hundred million repetitions. (You probably already have just by asking the question!)