Extract fields with TPIPE?

[vefatica]

Using TPIPE, how do I turn lines like

string1 string2 string3 string4 string5

into

string3 string4

It's pretty easy with the UNIX CUT ...

... | cut -d " " -f3,4

 

[JohnQSmith]

How about

tpipe /selection=7...

 

[vefatica]

How about

tpipe /selection=7...

Thanks JohnQSmith. That works but requires naming the fields to be removed ... in two groups. Since I wanted 4 and 5, I had to remove 6-9 and 1-3 separately.

I had tried /selection=4 (restrict to delimited fields) but kept getting errors saying that type 4 required subfilters. I have no idea how to use subfilters.

I was hoping TPIPE would be faster than other methods, but ... this took 32.5 seconds.

v:\ipreg> timer & (tpipe /input=delegated-arin-latest /grep=3,0,0,0,0,0,0,0,ipv4 /replace=0,0,0,0,0,0,0,0,0,"|"," " /selection=7,0,6,9,1,4,"",0,0,0,0 /selection=7,0,1,3,1,4,"",0,0,0,0) & timer

while this equivalent took 21.1 seconds (and was a bit easier to compose).

timer & (grep ipv4 delegated-arin-latest | tr "|" " " | cut -d " " -f4,5) & timer

 

[JohnQSmith]

Thanks JohnQSmith.

No problem.

I struggled with it for about 30 minutes and then left it at /selection=7... after I kept getting bad parameters messages. I tend to migrate toward Cygwin first (with it's friendly *ix commands) before trying other stuff.

I'm a problem solver, not a problem struggler. Looking at your /selection= parameters, I can see why I was struggling. Apparently I was leaving some of them blank that I should have been filling in. I guess I'll need to fight (struggle) with it some more.

 

[JohnQSmith]

I had tried /selection=4 (restrict to delimited fields) but kept getting errors saying that type 4 required subfilters.

From the TextPipe (not TPIPE) documentation,

These filters restrict the amount of text that is affected by sub filters i.e. without affecting the main text.

which means you are restricting any sub-filters to only being applied to the delimited fields.

Also, you must be parsing that file over a network or on a slow computer, because it only took me 6 seconds (Timer 1 off: 13:40:11 Elapsed: 0:00:05.99) on my computer reading the file from an internal hard drive (disc). Assuming you're using pretty much the same file (ftp://ftp.arin.net/pub/stats/arin/delegated-arin-extended-latest 28-Apr-2017 01:45 8.3M).

 

[vefatica]

From the TextPipe (not TPIPE) documentation,

which means you are restricting any sub-filters to only being applied to the delimited fields.

Also, you must be parsing that file over a network or on a slow computer, because it only took me 6 seconds (Timer 1 off: 13:40:11 Elapsed: 0:00:05.99) on my computer reading the file from an internal hard drive (disc). Assuming you're using pretty much the same file (ftp://ftp.arin.net/pub/stats/arin/delegated-arin-extended-latest 28-Apr-2017 01:45 8.3M).

My file is a day older, but they're essentially the same, and it's local. You get 6 seconds with console output?! If so, wow! That's almost 60,000 lines. I'm getting 17 seconds now with this.

v:\ipreg> timer & (tpipe /input=delegated-arin-latest /grep=3,0,0,0,0,0,0,0,ipv4 /replace=0,0,0,0,0,0,0,0,0,"|"," " /selection=7,0,6,9,1,4,"",0,0,0,0 /selection=7,0,1,3,1,4,"",0,0,0,0) & timer

It's 2 seconds if I output to a file.

Do you know how to use subfilters?

 

[JohnQSmith]

Haven't messed with subfilters yet. Currently looking at the /replace filter with extract enabled.

Edit:

You get 6 seconds with console output?! If so, wow! That's almost 60,000 lines. I'm getting 17 seconds now with this.

Sorry for the confusion. The 6 seconds was grep|cut|tr. Takes 14.4 with your command line posted above.

 

[vefatica]

Haven't messed with subfilters yet. Currently looking at the /replace filter with extract enabled.

Edit:


Sorry for the confusion. The 6 seconds was grep|cut|tr. Takes 14.4 with your command line posted above.

I'm now getting 12 seconds with that line (equal to the grep time). I can get it down to 7 seconds if I use a smaller console with a raster font; in that case, it's better than the grep time.

 

[JohnQSmith]

Here's 5.5 seconds. 0.3 seconds with a redirect into a file.

gawk -F"|" '{ if ($3 == "ipv4") print $4 " " $5 }' delegated-arin-extended-latest.txt

 

[JohnQSmith]

Tinkering with subfilters. Here's a working example.

11:35:10 $ echo this:is:a:test:of:the:emergency:broadcast:system > clip: & tpipe /clipboard /selection=4,0,4,7,0,5,":",0 /startsubfilters /simple=5 /endsubfilters & type clip:
this:is:a:TEST:OF:THE:EMERGENCY:broadcast:system

• Feed input string into clipboard.
• TPIPE
  • Read from and write back to clipboard
  • Create a selection filter
    • Using delimited field restriction type (4)
    • Define locate (0), start field (4), end field (7)
    • No MoveTo (0)
    • Set CustomDelimiter (5) to colon (":")
    • No headers on data (0)
  • Start sub filter on restricted range defined by selection filter
    • Convert restricted range to uppercase
  • End sub filter on restricted range
• Show clipboard

 

[vefatica]

I'll try that when I get home. I was more curious about the help's "/startsubfilters" and "/endsubfilters".