Extract fields with TPIPE?

Jan 19, 2011
614
15
Norman, OK
How about
Code:
tpipe /selection=7...
 
Last edited:
May 20, 2008
12,173
133
Syracuse, NY, USA
How about
Code:
tpipe /selection=7...
Thanks JohnQSmith. That works but requires naming the fields to be removed ... in two groups. Since I wanted 4 and 5, I had to remove 6-9 and 1-3 separately.

I had tried /selection=4 (restrict to delimited fields) but kept getting errors saying that type 4 required subfilters. I have no idea how to use subfilters.

I was hoping TPIPE would be faster than other methods, but ... this took 32.5 seconds.
Code:
v:\ipreg> timer & (tpipe /input=delegated-arin-latest /grep=3,0,0,0,0,0,0,0,ipv4 /replace=0,0,0,0,0,0,0,0,0,"|"," " /selection=7,0,6,9,1,4,"",0,0,0,0 /selection=7,0,1,3,1,4,"",0,0,0,0) & timer
while this equivalent took 21.1 seconds (and was a bit easier to compose).
Code:
timer & (grep ipv4 delegated-arin-latest | tr "|" " " | cut -d " " -f4,5) & timer
 
Jan 19, 2011
614
15
Norman, OK
Thanks JohnQSmith.
No problem.

I struggled with it for about 30 minutes and then left it at /selection=7... after I kept getting bad parameters messages. I tend to migrate toward Cygwin first (with it's friendly *ix commands) before trying other stuff.

I'm a problem solver, not a problem struggler. Looking at your /selection= parameters, I can see why I was struggling. Apparently I was leaving some of them blank that I should have been filling in. I guess I'll need to fight (struggle) with it some more.
 
Jan 19, 2011
614
15
Norman, OK
I had tried /selection=4 (restrict to delimited fields) but kept getting errors saying that type 4 required subfilters.
From the TextPipe (not TPIPE) documentation,
These filters restrict the amount of text that is affected by sub filters i.e. without affecting the main text.
which means you are restricting any sub-filters to only being applied to the delimited fields.

Also, you must be parsing that file over a network or on a slow computer, because it only took me 6 seconds (Timer 1 off: 13:40:11 Elapsed: 0:00:05.99) on my computer reading the file from an internal hard drive (disc). Assuming you're using pretty much the same file (ftp://ftp.arin.net/pub/stats/arin/delegated-arin-extended-latest 28-Apr-2017 01:45 8.3M).
 
May 20, 2008
12,173
133
Syracuse, NY, USA
From the TextPipe (not TPIPE) documentation,

which means you are restricting any sub-filters to only being applied to the delimited fields.

Also, you must be parsing that file over a network or on a slow computer, because it only took me 6 seconds (Timer 1 off: 13:40:11 Elapsed: 0:00:05.99) on my computer reading the file from an internal hard drive (disc). Assuming you're using pretty much the same file (ftp://ftp.arin.net/pub/stats/arin/delegated-arin-extended-latest 28-Apr-2017 01:45 8.3M).
My file is a day older, but they're essentially the same, and it's local. You get 6 seconds with console output?! If so, wow! That's almost 60,000 lines. I'm getting 17 seconds now with this.
Code:
v:\ipreg> timer & (tpipe /input=delegated-arin-latest /grep=3,0,0,0,0,0,0,0,ipv4 /replace=0,0,0,0,0,0,0,0,0,"|"," " /selection=7,0,6,9,1,4,"",0,0,0,0 /selection=7,0,1,3,1,4,"",0,0,0,0) & timer

It's 2 seconds if I output to a file.

Do you know how to use subfilters?
 
Jan 19, 2011
614
15
Norman, OK
Haven't messed with subfilters yet. Currently looking at the /replace filter with extract enabled.

Edit:

You get 6 seconds with console output?! If so, wow! That's almost 60,000 lines. I'm getting 17 seconds now with this.
Sorry for the confusion. The 6 seconds was grep|cut|tr. Takes 14.4 with your command line posted above.
 
May 20, 2008
12,173
133
Syracuse, NY, USA
Haven't messed with subfilters yet. Currently looking at the /replace filter with extract enabled.

Edit:


Sorry for the confusion. The 6 seconds was grep|cut|tr. Takes 14.4 with your command line posted above.
I'm now getting 12 seconds with that line (equal to the grep time). I can get it down to 7 seconds if I use a smaller console with a raster font; in that case, it's better than the grep time.
 
Jan 19, 2011
614
15
Norman, OK
Here's 5.5 seconds. 0.3 seconds with a redirect into a file.
Code:
gawk -F"|" '{ if ($3 == "ipv4") print $4 " " $5 }' delegated-arin-extended-latest.txt
 
Jan 19, 2011
614
15
Norman, OK
Tinkering with subfilters. Here's a working example.
Code:
11:35:10 $ echo this:is:a:test:of:the:emergency:broadcast:system > clip: & tpipe /clipboard /selection=4,0,4,7,0,5,":",0 /startsubfilters /simple=5 /endsubfilters & type clip:
this:is:a:TEST:OF:THE:EMERGENCY:broadcast:system
  • Feed input string into clipboard.
  • TPIPE
    • Read from and write back to clipboard
    • Create a selection filter
      • Using delimited field restriction type (4)
      • Define locate (0), start field (4), end field (7)
      • No MoveTo (0)
      • Set CustomDelimiter (5) to colon (":")
      • No headers on data (0)
    • Start sub filter on restricted range defined by selection filter
      • Convert restricted range to uppercase
    • End sub filter on restricted range
  • Show clipboard
 
May 20, 2008
12,173
133
Syracuse, NY, USA
I'll try that when I get home. I was more curious about the help's "/startsubfilters" and "/endsubfilters".
 

Similar threads