Extract fields with TPIPE?

Jan 19, 2011
604
14
Norman, OK
How about
Code:
tpipe /selection=7...
 
Last edited:
May 20, 2008
11,400
99
Syracuse, NY, USA
How about
Code:
tpipe /selection=7...
Thanks JohnQSmith. That works but requires naming the fields to be removed ... in two groups. Since I wanted 4 and 5, I had to remove 6-9 and 1-3 separately.

I had tried /selection=4 (restrict to delimited fields) but kept getting errors saying that type 4 required subfilters. I have no idea how to use subfilters.

I was hoping TPIPE would be faster than other methods, but ... this took 32.5 seconds.
Code:
v:\ipreg> timer & (tpipe /input=delegated-arin-latest /grep=3,0,0,0,0,0,0,0,ipv4 /replace=0,0,0,0,0,0,0,0,0,"|"," " /selection=7,0,6,9,1,4,"",0,0,0,0 /selection=7,0,1,3,1,4,"",0,0,0,0) & timer
while this equivalent took 21.1 seconds (and was a bit easier to compose).
Code:
timer & (grep ipv4 delegated-arin-latest | tr "|" " " | cut -d " " -f4,5) & timer
 
Jan 19, 2011
604
14
Norman, OK
Thanks JohnQSmith.
No problem.

I struggled with it for about 30 minutes and then left it at /selection=7... after I kept getting bad parameters messages. I tend to migrate toward Cygwin first (with it's friendly *ix commands) before trying other stuff.

I'm a problem solver, not a problem struggler. Looking at your /selection= parameters, I can see why I was struggling. Apparently I was leaving some of them blank that I should have been filling in. I guess I'll need to fight (struggle) with it some more.
 
Jan 19, 2011
604
14
Norman, OK
I had tried /selection=4 (restrict to delimited fields) but kept getting errors saying that type 4 required subfilters.
From the TextPipe (not TPIPE) documentation,
These filters restrict the amount of text that is affected by sub filters i.e. without affecting the main text.
which means you are restricting any sub-filters to only being applied to the delimited fields.

Also, you must be parsing that file over a network or on a slow computer, because it only took me 6 seconds (Timer 1 off: 13:40:11 Elapsed: 0:00:05.99) on my computer reading the file from an internal hard drive (disc). Assuming you're using pretty much the same file (ftp://ftp.arin.net/pub/stats/arin/delegated-arin-extended-latest 28-Apr-2017 01:45 8.3M).
 
May 20, 2008
11,400
99
Syracuse, NY, USA
From the TextPipe (not TPIPE) documentation,

which means you are restricting any sub-filters to only being applied to the delimited fields.

Also, you must be parsing that file over a network or on a slow computer, because it only took me 6 seconds (Timer 1 off: 13:40:11 Elapsed: 0:00:05.99) on my computer reading the file from an internal hard drive (disc). Assuming you're using pretty much the same file (ftp://ftp.arin.net/pub/stats/arin/delegated-arin-extended-latest 28-Apr-2017 01:45 8.3M).
My file is a day older, but they're essentially the same, and it's local. You get 6 seconds with console output?! If so, wow! That's almost 60,000 lines. I'm getting 17 seconds now with this.
Code:
v:\ipreg> timer & (tpipe /input=delegated-arin-latest /grep=3,0,0,0,0,0,0,0,ipv4 /replace=0,0,0,0,0,0,0,0,0,"|"," " /selection=7,0,6,9,1,4,"",0,0,0,0 /selection=7,0,1,3,1,4,"",0,0,0,0) & timer

It's 2 seconds if I output to a file.

Do you know how to use subfilters?
 
Jan 19, 2011
604
14
Norman, OK
Haven't messed with subfilters yet. Currently looking at the /replace filter with extract enabled.

Edit:

You get 6 seconds with console output?! If so, wow! That's almost 60,000 lines. I'm getting 17 seconds now with this.
Sorry for the confusion. The 6 seconds was grep|cut|tr. Takes 14.4 with your command line posted above.
 
May 20, 2008
11,400
99
Syracuse, NY, USA
Haven't messed with subfilters yet. Currently looking at the /replace filter with extract enabled.

Edit:


Sorry for the confusion. The 6 seconds was grep|cut|tr. Takes 14.4 with your command line posted above.
I'm now getting 12 seconds with that line (equal to the grep time). I can get it down to 7 seconds if I use a smaller console with a raster font; in that case, it's better than the grep time.
 
Jan 19, 2011
604
14
Norman, OK
Here's 5.5 seconds. 0.3 seconds with a redirect into a file.
Code:
gawk -F"|" '{ if ($3 == "ipv4") print $4 " " $5 }' delegated-arin-extended-latest.txt
 
Jan 19, 2011
604
14
Norman, OK
Tinkering with subfilters. Here's a working example.
Code:
11:35:10 $ echo this:is:a:test:of:the:emergency:broadcast:system > clip: & tpipe /clipboard /selection=4,0,4,7,0,5,":",0 /startsubfilters /simple=5 /endsubfilters & type clip:
this:is:a:TEST:OF:THE:EMERGENCY:broadcast:system
  • Feed input string into clipboard.
  • TPIPE
    • Read from and write back to clipboard
    • Create a selection filter
      • Using delimited field restriction type (4)
      • Define locate (0), start field (4), end field (7)
      • No MoveTo (0)
      • Set CustomDelimiter (5) to colon (":")
      • No headers on data (0)
    • Start sub filter on restricted range defined by selection filter
      • Convert restricted range to uppercase
    • End sub filter on restricted range
  • Show clipboard
 
May 20, 2008
11,400
99
Syracuse, NY, USA
I'll try that when I get home. I was more curious about the help's "/startsubfilters" and "/endsubfilters".
 
Similar threads
Thread starter Title Forum Replies Date
MikeBaas How to...extract a string from a .json-file? Support 10
A Installer /extract: extracts into current directory if target not exists Support 12
D /extract: option for the installer Support 4
C tpipe and extract email addresses Support 1
Phileosophos Unzip doesn't extract to specified output folder Support 5
R TC13 popup window edit fields Support 8
vefatica Question about @FIELDS Support 3
samintz How to? TPIPE /simple=61 Support 6
R Bug TPIPE's pdf to text conversions don't work Support 2
Kachupp How to? tpipe Support 4
H TPIPE /simple=4 Support 12
D TPIPE produces unwanted logs. Support 1
Kachupp How to? tpipe Support 8
vefatica Documentation TPIPE, /perl, DotMatchesNewLines Support 1
vefatica Stop TPIPE /EOL removing bad EOLs (and TPIPE changine EOLs)? Support 2
vefatica TPIPE in an alias? Support 2
C How to? TPIPE ?'s Support 2
H History and TPipe Oddness Support 3
Jay Sage Baffling Behavior With TPIPE Support 5
vefatica TPIPE: unbalanced escaped quotes in a regex? Support 5
Joe Caverly Multiple Text Searches at once using FFIND or TPIPE Support 4
Kachupp FFIND TPIPE Support 9
vefatica TPIPE and avarice? Support 5
Peter Murschall TPIPE generate unicode on Piping or redirecting Support 3
x13 TPIPE /REPLACE - Remove line (including EOL) Support 3
Charles Dye Documentation TPIPE /REPLACELIST Support 0
Joe Caverly TPIPE and /inputstring Support 3
vefatica How to? Add real <ESC> characters with TPIPE Support 2
vefatica How to? TPIPE ... see output right away? Support 4
vefatica TPIPE.EXE crashes on Ctrl-C Support 9
O TPIPE: Error -Can´t access textpipeengine.dll Support 24
vefatica TPIPE, crash and question Support 1
MickeyF Another TPIPE question Support 6
MickeyF Can't get my TPIPE /replace to work Support 4
Joe Caverly Printing TPIPE documentation Support 3
C WAD tpipe /simple=28 Support 8
vefatica Remove double-quotes with TPIPE Support 4
vefatica TPIPE crashes on Ctrl-C Support 1
C How to? Add date/time to custom tpipe based BTM Support 1
C tpipe /simple=28 error Support 5
C TPipe /dup Support 5
T TPipe: Replacing String with Double Quotes Support 3
vefatica How do I do this with TPIPE? Support 1
vefatica TPIPE's /perl option? Support 1
D TPIPE help typo Support 0
vefatica TPIPE access violations are back Support 2
Joe Caverly Word Count using TPIPE Support 10
Dan Glynhampton TPIPE in Build 41 Support 4
Phileosophos TPIPE doesn't work at all with v16 Support 3
vefatica TPIPE access violation again Support 1

Similar threads