Everything is working as it should be. There are no culprits.
The recogniser uses these rules:
1. Capturing. Number each parenthesised subexpression by the position of its first open parenthesis symbol, counting from 1 from the left, ignoring any nesting.
2. Leftmost match. For the expression as a whole and for each subexpression, consider only leftmost matches.
3. Alternation. Of several alternatives that match, always choose either (a) the leftmost or (b) the longest. Perl 5's and TCC's recgnisers both choose (a).
%@regexsub[1,(1|123),1234]
Number the whole parenthesised expression 1.
Consider only the leftmost matches, that is, both of the alternatives.
Of the matching alternatives, choose the leftmost: 1.
Return what subexpression 1 matched: 1.
%@regexsub[1,(2|123),1234]
Number the whole parenthesised expression 1.
Consider only the leftmost match, that is, the second alternative.
Choose the only matching alternative: 123.
Return what subexpression 1 matched: 123.
%@regexsub[1,(1)|(123),1234]
Number the first alternative 1, the second alternative 2.
Consider only the leftmost matches, that is, both of the alternatives.
Of the matching alternatives, choose the leftmost: (1).
Return what subexpression 1 matched: 1.
%@regexsub[1,(123)|(1),1234]
Number the first alternative 1, the second alternative 2.
Consider only the leftmost matches, that is, both of the alternatives.
Of the matching alternatives, choose the leftmost: (123).
Return what subexpression 1 matched: 123.
%@regexsub[1,(234)|(1),1234]
Number the first alternative 1, the second alternative 2.
Consider only the leftmost match, that is, the second alternative.
Choose the only matching alternative: (1).
Return what subexpression 1 matched: [nothing].
%@regexsub[1,(2)|(123),1234]
Number the first alternative 1, the second alternative 2.
Consider only the leftmost match, that is, the second alternative.
Choose the only matching alternative: (123).
Return what subexpression 1 matched: [nothing].
%@regexsub[2,(2)|(123),1234]
Number the first alternative 1, the second alternative 2.
Consider only the leftmost match, that is, the second alternative.
Choose the only matching alternative: (123).
Return what subexpression 2 matched: 123.
%@regexsub[1,(234)|(1),1234]
Number the first alternative 1, the second alternative 2.
Consider only the leftmost match, that is, the second alternative.
Choose the only matching alternative: (1).
Return what subexpression 1 matched: [nothing].
%@regexsub[2,(234)|(1),1234]
Number the first alternative 1, the second alternative 2.
Consider only the leftmost match, that is, the second alternative.
Choose the only matching alternative: (1).
Return what subexpression 2 matched: 1.