Regular Expression Syntax

Onigmo Regular Expressions Version 6.2.0

 

This section covers the Ruby regular expression syntax. For information on Perl regular expression syntax, see your Perl documentation or https://perldoc.perl.org/perlre.html.

 

hmtoggle_plus1Syntax elements

 

\ escape (enable or disable meta character meaning)
|alternation
(...)group
[...]character class  

 

hmtoggle_plus1Characters

 

\thorizontal tab (0x09)
\vvertical tab   (0x0B)
\nnewline        (0x0A)
\rreturn         (0x0D)
\bback space     (0x08)
\fform feed      (0x0C)
\abell           (0x07)
\eescape         (0x1B)
\nnnoctal char            (encoded byte value)
\xHHhexadecimal char      (encoded byte value)

\x{7HHHHHHH} wide hexadecimal char (character code point value)

\cxcontrol char          (character code point value)
\C-xcontrol char          (character code point value)
\M-xmeta  (x|0x80)        (character code point value)
\M-\C-xmeta control char     (character code point value)

 

(* \b is effective in character class [...] only)

 

hmtoggle_plus1Character types

 

.any character (except newline)

 

\w word character
 
Not Unicode:
                 alphanumeric, "_" and multibyte char.
 
Unicode:
       General_Category -- (Letter|Mark|Number|Connector_Punctuation)

 

\W       non word char

 

\swhitespace char
 
Not Unicode:
       \t, \n, \v, \f, \r, \x20
 
Unicode:
       0009, 000A, 000B, 000C, 000D, 0085(NEL),
       General_Category -- Line_Separator

                    -- Paragraph_Separator

                     -- Space_Separator

 

\Snon whitespace char

 

\ddecimal digit char
 
Unicode: General_Category -- Decimal_Number

 

\Dnon decimal digit char

 

\hhexadecimal digit char   [0-9a-fA-F]

 

\Hnon hexadecimal digit char

 

Character Property

 

* \p{property-name}

* \p{^property-name}    (negative)

* \P{property-name}     (negative)

 

property-name:

 

+ works on all encodings

Alnum, Alpha, Blank, Cntrl, Digit, Graph, Lower, Print, Punct, Space, Upper, XDigit, Word, ASCII,

 

    + works on UTF8, UTF16, UTF32

 

 \R       Linebreak

 

          Unicode:

            (?>\x0D\x0A|[\x0A-\x0D\x{85}\x{2028}\x{2029}])

 

          Not Unicode:

            (?>\x0D\x0A|[\x0A-\x0D])

 

 \X       eXtended grapheme cluster

 

          Unicode:

            (?>\P{M}\p{M}*)

 

          Not Unicode:

            (?m:.)

 

hmtoggle_plus1Quantifier

 

greedy

 
?       1 or 0 times
*       0 or more times
+       1 or more times
{n,m}   at least n but not more than m times
{n,}    at least n times
{,n}    at least 0 but not more than n times ({0,n})
{n}     n times

 

reluctant

 
??      1 or 0 times
*?      0 or more times
+?      1 or more times
{n,m}?  at least n but not more than m times  
{n,}?   at least n times
{,n}?   at least 0 but not more than n times (== {0,n}?)

 

possessive (greedy and does not backtrack after repeated)

 
?+      1 or 0 times
*+      0 or more times
++      1 or more times
 
({n,m}+, {n,}+, {n}+ are possessive op. in ONIG_SYNTAX_JAVA only)
 
ex. /a*+/ === /(?>a*)/

 

hmtoggle_plus1Anchors

 

^beginning of the line
$end of the line
\bword boundary
\Bnot word boundary
\Abeginning of string
\Zend of string, or before newline at the end
\zend of string
\Gmatching start position (*)

 

hmtoggle_plus1Character class

 

^...negative class (lowest precedence operator)
x-yrange from x to y
[...]set (character class in character class)
..&&..intersection (low precedence at the next of ^)

         

ex. [a-w&&[^c-g]z] ==> ([a-w] AND ([^c-g] OR z)) ==> [abh-w]

 

* If you want to use '[', '-', ']' as a normal character in a character class, you should escape these characters by '\'.

 

POSIX bracket ([:xxxxx:], negate [:^xxxxx:])

 

Not Unicode Case:

 

alnum        alphabet or digit char

alpha        alphabet

ascii        code value: [0 - 127]

blank        \t, \x20

cntrl

digit        0-9

graph        include all of multibyte encoded characters

lower

print        include all of multibyte encoded characters

punct

space        \t, \n, \v, \f, \r, \x20

upper

word        alphanumeric, "_" and multibyte characters

xdigit        0-9, a-f, A-F

 

Unicode Case:

 

alnum        Letter | Mark | Decimal_Number

alpha        Letter | Mark

ascii        0000 - 007F

blank        Space_Separator | 0009

cntrl        Control | Format | Unassigned | Private_Use | Surrogate

digit        Decimal_Number

graph        [[:^space:]] && ^Control && ^Unassigned && ^Surrogate

lower        Lowercase_Letter

print        [[:graph:]] | [[:space:]]

punct        Connector_Punctuation | Dash_Punctuation | Close_Punctuation | Final_Punctuation | Initial_Punctuation | Other_Punctuation | Open_Punctuation

space        Space_Separator | Line_Separator | Paragraph_Separator | 0009 | 000A | 000B | 000C | 000D | 0085

upper        Uppercase_Letter

word        Letter | Mark | Decimal_Number | Connector_Punctuation

xdigit        0030 - 0039 | 0041 - 0046 | 0061 - 0066  (0-9, a-f, A-F)

 

hmtoggle_plus1Extended groups

 

(?#...)comment

 

(?imxdau-imx)      option on/off

i: ignore case

m: multi-line (dot(.) match newline)

x: extended form

 

character set option (character range option)

d: Default (compatible with Ruby 1.9.3)

\w, \d and \s doesn't match non-ASCII characters.

\b, \B and POSIX brackets use the each encoding's rules.

a: ASCII

ONIG_OPTION_ASCII_RANGE option is turned on.

\w, \d, \s and POSIX brackets doesn't match non-ASCII characters.

\b and \B use the ASCII rules.

u: Unicode

ONIG_OPTION_ASCII_RANGE option is turned off.

\w (\W), \d (\D), \s (\S), \b (\B) and POSIX brackets use the each encoding's rules.

 

(?imxdau-imx:subexp)option on/off for subexp

 

(?:subexp)not captured group
(subexp)captured group

 

(?=subexp)look-ahead
(?!subexp)negative look-ahead
(?<=subexp)look-behind
(?<!subexp)negative look-behind
 
Subexp of look-behind must be fixed character length. But different character length is allowed in top level alternatives only.
ex. (?<=a|bc) is OK. (?<=aaa(?:b|cd)) is not allowed.
 
In negative-look-behind, captured group isn't allowed, but shy group(?:) is allowed.

 

\Kkeep

Another expression of look-behind. Keep the stuff left of the \K, don't include it in the result.

 

(?>subexp)atomic group
don't backtrack in subexp.

 

(?<name>subexp)define named group
(All characters of the name must be a word character. And first character must not be a digit or upper case)
Not only a name but a number is assigned like a captured group.
 
Assigning the same name as two or more subexps is allowed. In this case, a subexp call can not be performed although the back reference is possible.

 

 (?(cond)yes-subexp), (?(cond)yes-subexp|no-subexp)

                   conditional expression

                   Matches yes-subexp if (cond) yields a true value, matches no-subexp otherwise.

                   Following (cond) can be used:

 

                   (n)  (n >= 1)

                       Checks if the numbered capturing group has matched something.

 

                   (<name>), ('name')

                       Checks if a group with the given name has matched something.

 

hmtoggle_plus1Back reference

 

\nback reference by group number (n >= 1)
\k<n>back reference by group number (n >= 1)
\k'n'back reference by group number (n >= 1)
\k<-n>back reference by relative group number (n >= 1)
\k'-n'back reference by relative group number (n >= 1)
\k<name>back reference by group name
\k'name'back reference by group name

 

In the back reference by the multiplex definition name, a subexp with a large number is referred to preferentially. (When not matched, a group of the small number is referred to.)

 

* Back reference by group number is forbidden if named group is defined in the pattern and ONIG_OPTION_CAPTURE_GROUP is not setted.

 

Back reference with nest level

 

level: 0, 1, 2, ...

 

\k<n+level>     (n >= 1)

\k<n-level>     (n >= 1)

\k'n+level'     (n >= 1)

\k'n-level'     (n >= 1)

\k<-n+level>    (n >= 1)

\k<-n-level>    (n >= 1)

\k'-n+level'    (n >= 1)

\k'-n-level'    (n >= 1)

 

\k<name+level>

\k<name-level>

\k'name+level'

\k'name-level'

 

Destinate relative nest level from back reference position.  

 

example 1.

 

/\A(?<a>|.|(?:(?<b>.)\g<a>\k<b+0>))\z/.match("reer")

 

example 2.

 

r = Regexp.compile(<<'__REGEXP__'.strip, Regexp::EXTENDED)

(?<element> \g<stag> \g<content>* \g<etag> ){0}

(?<stag> < \g<name> \s* > ){0}

(?<name> [a-zA-Z_:]+ ){0}

(?<content> [^<&]+ (\g<element> | [^<&]+)* ){0}

(?<etag> </ \k<name+1> >){0}

\g<element>

__REGEXP__

 

p r.match('<foo>f<bar>bbb</bar>f</foo>').captures

 

hmtoggle_plus1Subexp call ("Tanaka Akira special")

 

\g<name>call by group name
\g'name'call by group name
\g<n>call by group number (n >= 1)
\g'n'call by group number (n >= 1)

\g<0>       call the whole pattern recursively

\g'0'       call the whole pattern recursively

\g<-n>      call by relative group number (n >= 1)

\g'-n'      call by relative group number (n >= 1)

\g<+n>      call by relative group number (n >= 1)

\g'+n'      call by relative group number (n >= 1)

 

* left-most recursive call is not allowed.

ex.(?<name>a|\g<name>b)   => error
(?<name>a|b\g<name>c)  => OK

 

* Call by group number is forbidden if named group is defined in the pattern and ONIG_OPTION_CAPTURE_GROUP is not set.

 

* If the option status of called group is different from calling position then the group's option is effective.

 

ex. (?-i:\g<name>)(?i:(?<name>a)){0}  match to "A"

 

Perl syntax:: use (?&name), (?n), (?-n), (?+n), (?R) or (?0) instead.

 

hmtoggle_plus1Captured group

 

Behavior of the no-named group (...) changes with the following conditions. (But named group is not changed.)

 

case 1. /.../    (named group is not used, no option)

 

 (...) is treated as a captured group.

 

case 2. /.../g    (named group is not used, 'g' option)

 

 (...) is treated as a no-captured group (?:...).

 

case 3. /..(?<name>..)../    (named group is used, no option)

 

 (...) is treated as a no-captured group (?:...).

 numbered-backref/call is not allowed.

 

case 4. /..(?<name>..)../G    (named group is used, 'G' option)

 

 (...) is treated as a captured group.

 numbered-backref/call is allowed.

 

 where

 g:  ONIG_OPTION_DONT_CAPTURE_GROUP

 G:  ONIG_OPTION_CAPTURE_GROUP

 

hmtoggle_plus1Syntax dependent options

 

+ RUBY

 (?m): dot(.) match newline

 

+ PERL, JAVA, and Python

 (?s): dot(.) match newline

 (?m): ^ match after newline, $ match before newline

 

+ PERL

(?d), (?l): same as (?u)

 

hmtoggle_plus1Original extensions

 

+ hexadecimal digit char type  \h, \H

+ named group              (?<name>...)

+ named backref            \k<name>

+ subexp call                  \g<name>, \g<group-num>

 

hmtoggle_plus1Missing features compared with Perl 5.14.0

 

+ \N{name}, \N{U+xxxx}, \N

+ \l,\u,\L,\U, \C

+ \v, \V, \h, \H, \o{xxx}

+ (?{code})

+ (??{code})

+ (?|...)

+ (*VERB:ARG)

 

* \Q...\E

    This is effective in PERL and JAVA.

 

hmtoggle_plus1Disabled functions by default syntax

 

+ capture history

 

    (?@...) and (?@<name>...)

 

    ex. /(?@a)*/.match("aaa") ==> [<0-1>, <1-2>, <2-3>]

 

hmtoggle_plus1Problems

 

+ Invalid encoding byte sequence is not checked.

 

    ex. UTF-8

 

    * Invalid first byte is treated as a character.

      /./u =~ "\xa3"

 

    * Incomplete byte sequence is not checked.

      /\w+/ =~ "a\xf3\x8ec"