Update: A beta version of XRegExp 0.3 is now available as part of the RegexPal download package.
JavaScript's regular expression flavor doesn't support named capture. Well, says who? XRegExp 0.2 brings named capture support, along with several other new features. But first of all, if you haven't seen the previous version, make sure to check out my post on XRegExp 0.1, because not all of the documentation is repeated below.
Highlights
- Comprehensive named capture support (New)
- Supports regex literals through the
<font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">addFlags</font>method (New) - Free-spacing and comments mode (
<font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">x</font>) - Dot matches all mode (
<font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">s</font>) - Several other minor improvements over v0.1
Named capture
There are several different syntaxes in the wild for named capture. I've compiled the following table based on my understanding of the regex support of the libraries in question. XRegExp's syntax is included at the top.
| Library | Capture | Back |
In replacement | Stored at |
|---|---|---|---|---|
| XRegExp | <font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">(<name>…)</name></font> |
<font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">\k<name></name></font> |
<font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">${name}</font> |
<font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">result.name</font> |
| .NET |
<font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">(?<name>…)</name></font>
<font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">(?'name'…)</font> |
<font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">\k<name></name></font>
<font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">\k'name'</font> |
<font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">${name}</font> |
<font size="2"><font style="BACKGROUND-COLOR: #f5f5ff" color="#000088">Matcher.<wbr>Groups('name')</wbr></font></font> |
| Perl 5.10 (beta) |
<font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">(?<name>…)</name></font>
<font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">(?'name'…)</font> |
<font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">\k<name></name></font>
<font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">\k'name'</font>
<font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">\g{name}</font> |
<font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">$+{name}</font> |
?? |
| Python | <nobr><font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">(?P<name>…)</name></font></nobr> |
<font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">(?P=name)</font> |
<font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">\g<name></name></font> |
<font size="2"><font style="BACKGROUND-COLOR: #f5f5ff" color="#000088">result.<wbr>group('name')</wbr></font></font> |
| PHP preg (PCRE) | (.NET, Perl, and Python styles) | <font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">$regs['name']</font> |
<font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">$result['name']</font> |
|
No other major regex library currently supports named capture, although the JGsoft engine (used by products like RegexBuddy) supports both .NET and Python syntax. XRegExp does not use a question mark at the beginning of a named capturing group because that would prevent it from being used in regex literals (JavaScript would immediately throw an "invalid quantifier" error).
XRegExp supports named capture on an on-request basis. You can add named capture support to any regex though the use of the new "<font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">k</font>" flag. This is done for compatibility reasons and to ensure that regex compilation time remains as fast as possible in all situations.
Following are several examples of using named capture:
<SPAN class=comment><FONT color=#238b23>// Add named capture support using the XRegExp constructor</FONT></SPAN>
var repeatedWords = new XRegExp("\\b (<word> \\w+ ) \\s+ \\k<word> \\b", "gixk");
<SPAN class=comment><FONT color=#238b23>// Add named capture support using RegExp, after overriding the native constructor</FONT></SPAN>
XRegExp.overrideNative();
var repeatedWords = new RegExp("\\b (<word> \\w+ ) \\s+ \\k<word> \\b", "gixk");
<SPAN class=comment><FONT color=#238b23>// Add named capture support to a regex literal</FONT></SPAN>
var repeatedWords = /\b (<word> \w+ ) \s+ \k<word> \b/.addFlags("gixk");
var data = "The the test data.";
<SPAN class=comment><FONT color=#238b23>// Check if data contains repeated words</FONT></SPAN>
var hasDuplicates = repeatedWords.test(data);
<SPAN class=comment><FONT color=#238b23>// hasDuplicates: true</FONT></SPAN>
<SPAN class=comment><FONT color=#238b23>// Use the regex to remove repeated words</FONT></SPAN>
var output = data.replace(repeatedWords, "${word}");
<SPAN class=comment><FONT color=#238b23>// output: "The test data."</FONT></SPAN>
In the above code, I've also used the <FONT style="BACKGROUND-COLOR: #f5f5ff" color=#000088 size=2>x</FONT> flag provided by XRegExp, to improve readability. Note that the <FONT style="BACKGROUND-COLOR: #f5f5ff" color=#000088 size=2>addFlags</FONT> method can be called multiple times on the same regex (e.g., <FONT style="BACKGROUND-COLOR: #f5f5ff" color=#000088 size=2>/pattern/g.addFlags("k").addFlags("s")</FONT>), but I'd recommend adding all flags in one shot, for efficiency.
Here are a few more examples of using named capture, with an overly simplistic URL-matching regex (for comprehensive URL parsing, see parseUri):
var url = "http://microsoft.com/path/to/file?q=1";
var urlParser = new XRegExp("^(<protocol>[^:/?]+)://(<host>[^/?]*)(<path>[^?]*)\\?(<query>.*)", "k");
var parts = urlParser.exec(url);
<SPAN class=comment><FONT color=#238b23>/* The result:
parts.protocol: "http"
parts.host: "microsoft.com"
parts.path: "/path/to/file"
parts.query: "q=1" */</FONT></SPAN>
<SPAN class=comment><FONT color=#238b23>// Named backreferences are also available in replace() callback functions as properties of the first argument</FONT></SPAN>
var newUrl = url.replace(urlParser, function(match){
return match.replace(match.host, "yahoo.com");
});
<SPAN class=comment><FONT color=#238b23>// newUrl: "http://yahoo.com/path/to/file?q=1"</FONT></SPAN>
Note that XRegExp's named capture functionality does not support deprecated JavaScript features including the <FONT style="BACKGROUND-COLOR: #f5f5ff" color=#000088 size=2>lastMatch</FONT> property of the global <FONT style="BACKGROUND-COLOR: #f5f5ff" color=#000088 size=2>RegExp</FONT> object and the <FONT style="BACKGROUND-COLOR: #f5f5ff" color=#000088 size=2>RegExp.prototype.compile()</FONT> method.
Singleline (s) and extended (x) modes
The other non-native flags XRegExp supports are <FONT style="BACKGROUND-COLOR: #f5f5ff" color=#000088 size=2>s</FONT> (singleline) for "dot matches all" mode, and <FONT style="BACKGROUND-COLOR: #f5f5ff" color=#000088 size=2>x</FONT> (extended) for "free-spacing and comments" mode. For full details about these modifiers, see the FAQ in my XRegExp 0.1 post. However, one difference from the previous version is that XRegExp 0.2, when using the <FONT style="BACKGROUND-COLOR: #f5f5ff" color=#000088 size=2>x</FONT> flag, now allows whitespace between a regex token and its quantifier (quantifiers are, e.g., <FONT style="BACKGROUND-COLOR: #f5f5ff" color=#000088 size=2>+</FONT>, <FONT style="BACKGROUND-COLOR: #f5f5ff" color=#000088 size=2>*?</FONT>, or <FONT style="BACKGROUND-COLOR: #f5f5ff" color=#000088 size=2>{1,3}</FONT>). Although the previous version's handling/limitation in this regard was documented, it was atypical compared to other regex libraries. This has been fixed.
The code
<SPAN class=comment><FONT color=#238b23>/* XRegExp 0.2.2; MIT License
By Steven Levithan <http://stevenlevithan.com>
----------
Adds support for the following regular expression features:
- Free-spacing and comments ("x" flag)
- Dot matches all ("s" flag)
- Named capture ("k" flag)
- Capture: (<name>...)
- Backreference: \k<name>
- In replacement: ${name}
- Stored at: result.name
*/</FONT></SPAN>
<SPAN class=comment><FONT color=#238b23>/* Protect this from running more than once, which would break its references to native functions */</FONT></SPAN>
if (window.XRegExp === undefined) {
var XRegExp;
(function () {
var native = {
RegExp: RegExp,
exec: RegExp.prototype.exec,
match: String.prototype.match,
replace: String.prototype.replace
};
XRegExp = function (pattern, flags) {
return native.RegExp(pattern).addFlags(flags);
};
RegExp.prototype.addFlags = function (flags) {
var pattern = this.source,
useNamedCapture = false,
re = XRegExp._re;
flags = (flags || "") + native.replace.call(this.toString(), /^[\S\s]+\//, "");
if (flags.indexOf("x") > -1) {
pattern = native.replace.call(pattern, re.extended, function ($0, $1, $2) {
return $1 ? ($2 ? $2 : "(?:)") : $0;
});
}
if (flags.indexOf("k") > -1) {
var captureNames = [];
pattern = native.replace.call(pattern, re.capturingGroup, function ($0, $1) {
if (/^\((?!\?)/.test($0)) {
if ($1) useNamedCapture = true;
captureNames.push($1 || null);
return "(";
} else {
return $0;
}
});
if (useNamedCapture) {
<SPAN class=comment><FONT color=#238b23>/* Replace named with numbered backreferences */</FONT></SPAN>
pattern = native.replace.call(pattern, re.namedBackreference, function ($0, $1, $2) {
var index = $1 ? captureNames.indexOf($1) : -1;
return index > -1 ? "\\" + (index + 1).toString() + ($2 ? "(?:)" + $2 : "") : $0;
});
}
}
<SPAN class=comment><FONT color=#238b23>/* If "]" is the leading character in a character class, replace it with "\]" for consistent
cross-browser handling. This is needed to maintain correctness without the aid of browser sniffing
when constructing the regexes which deal with character classes. They treat a leading "]" within a
character class as a non-terminating, literal character, which is consistent with IE, .NET, Perl,
PCRE, Python, Ruby, JGsoft, and most other regex engines. */</FONT></SPAN>
pattern = native.replace.call(pattern, re.characterClass, function ($0, $1) {
<SPAN class=comment><FONT color=#238b23>/* This second regex is only run when a leading "]" exists in the character class */</FONT></SPAN>
return $1 ? native.replace.call($0, /^(\[\^?)]/, "$1\\]") : $0;
});
if (flags.indexOf("s") > -1) {
pattern = native.replace.call(pattern, re.singleline, function ($0) {
return $0 === "." ? "[\\S\\s]" : $0;
});
}
var regex = native.RegExp(pattern, native.replace.call(flags, /[sxk]+/g, ""));
if (useNamedCapture) {
regex._captureNames = captureNames;
<SPAN class=comment><FONT color=#238b23>/* Preserve capture names if adding flags to a regex which has already run through addFlags("k") */</FONT></SPAN>
} else if (this._captureNames) {
regex._captureNames = this._captureNames.valueOf();
}
return regex;
};
String.prototype.replace = function (search, replacement) {
<SPAN class=comment><FONT color=#238b23>/* If search is not a regex which uses named capturing groups, just run the native replace method */</FONT></SPAN>
if (!(search instanceof native.RegExp && search._captureNames)) {
return native.replace.apply(this, arguments);
}
if (typeof replacement === "function") {
return native.replace.call(this, search, function () {
<SPAN class=comment><FONT color=#238b23>/* Convert arguments[0] from a string primitive to a string object which can store properties */</FONT></SPAN>
arguments[0] = new String(arguments[0]);
<SPAN class=comment><FONT color=#238b23>/* Store named backreferences on the first argument before calling replacement */</FONT></SPAN>
for (var i = 0; i < search._captureNames.length; i++) {
if (search._captureNames[i]) arguments[0][search._captureNames[i]] = arguments[i + 1];
}
return replacement.apply(window, arguments);
});
} else {
return native.replace.call(this, search, function () {
var args = arguments;
return native.replace.call(replacement, XRegExp._re.replacementVariable, function ($0, $1, $2) {
<SPAN class=comment><FONT color=#238b23>/* Numbered backreference or special variable */</FONT></SPAN>
if ($1) {
switch ($1) {
case "$": return "$";
case "&": return args[0];
case "`": return args[args.length - 1].substring(0, args[args.length - 2]);
case "'": return args[args.length - 1].substring(args[args.length - 2] + args[0].length);
<SPAN class=comment><FONT color=#238b23>/* Numbered backreference */</FONT></SPAN>
default:
<SPAN class=comment><FONT color=#238b23>/* What does "$10" mean?
- Backreference 10, if at least 10 capturing groups exist
- Backreference 1 followed by "0", if at least one capturing group exists
- Else, it's the string "$10" */</FONT></SPAN>
var literalNumbers = "";
$1 = +$1; <SPAN class=comment><FONT color=#238b23>/* Cheap type-conversion */</FONT></SPAN>
while ($1 > search._captureNames.length) {
literalNumbers = $1.toString().match(/\d$/)[0] + literalNumbers;
$1 = Math.floor($1 / 10); <SPAN class=comment><FONT color=#238b23>/* Drop the last digit */</FONT></SPAN>
}
return ($1 ? args[$1] : "$") + literalNumbers;
}
<SPAN class=comment><FONT color=#238b23>/* Named backreference */</FONT></SPAN>
} else if ($2) {
<SPAN class=comment><FONT color=#238b23>/* What does "${name}" mean?
- Backreference to named capture "name", if it exists
- Else, it's the string "${name}" */</FONT></SPAN>
var index = search._captureNames.indexOf($2);
return index > -1 ? args[index + 1] : $0;
} else {
return $0;
}
});
});
}
};
RegExp.prototype.exec = function (str) {
var result = native.exec.call(this, str);
if (!(this._captureNames && result && result.length > 1)) return result;
for (var i = 1; i < result.length; i++) {
var name = this._captureNames[i - 1];
if (name) result[name] = result[i];
}
return result;
};
String.prototype.match = function (regexp) {
if (!regexp._captureNames || regexp.global) return native.match.call(this, regexp);
return regexp.exec(this);
};
})();
}
<SPAN class=comment><FONT color=#238b23>/* Regex syntax parsing with support for escapings, character classes, and various other context and cross-browser issues */</FONT></SPAN>
XRegExp._re = {
extended: /(?:[^[#\s\\]+|\\(?:[\S\s]|$)|\[\^?]?(?:[^\\\]]+|\\(?:[\S\s]|$))*]?)+|(\s*#[^\n\r]*\s*|\s+)([?*+]|{\d+(?:,\d*)?})?/g,
singleline: /(?:[^[\\.]+|\\(?:[\S\s]|$)|\[\^?]?(?:[^\\\]]+|\\(?:[\S\s]|$))*]?)+|\./g,
characterClass: /(?:[^\\[]+|\\(?:[\S\s]|$))+|\[\^?(]?)(?:[^\\\]]+|\\(?:[\S\s]|$))*]?/g,
capturingGroup: /(?:[^[(\\]+|\\(?:[\S\s]|$)|\[\^?]?(?:[^\\\]]+|\\(?:[\S\s]|$))*]?|\((?=\?))+|\((?:<([$\w]+)>)?/g,
namedBackreference: /(?:[^\\[]+|\\(?:[^k]|$)|\[\^?]?(?:[^\\\]]+|\\(?:[\S\s]|$))*]?|\\k(?!<[$\w]+>))+|\\k<([$\w]+)>(\d*)/g,
replacementVariable: /(?:[^$]+|\$(?![1-9$&`']|{[$\w]+}))+|\$(?:([1-9]\d*|[$&`'])|{([$\w]+)})/g
};
XRegExp.overrideNative = function () {
<SPAN class=comment><FONT color=#238b23>/* Override the global RegExp constructor/object with the XRegExp constructor. This precludes accessing
properties of the last match via the global RegExp object. However, those properties are deprecated as
of JavaScript 1.5, and the values are available on RegExp instances or via RegExp/String methods. It also
affects the result of (/x/.constructor == RegExp) and (/x/ instanceof RegExp), so use with caution. */</FONT></SPAN>
RegExp = XRegExp;
};
<SPAN class=comment><FONT color=#238b23>/* indexOf method from Mootools 1.11; MIT License */</FONT></SPAN>
Array.prototype.indexOf = Array.prototype.indexOf || function (item, from) {
var len = this.length;
for (var i = (from < 0) ? Math.max(0, len + from) : from || 0; i < len; i++) {
if (this[i] === item) return i;
}
return -1;
};
You can download it, or get the packed version (2.7 KB).
XRegExp has been tested in IE 5.5–7, Firefox 2.0.0.4, Opera 9.21, Safari 3.0.2 beta for Windows, and Swift 0.2.
Finally, note that the <font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">XRE</font> object from v0.1 has been removed. XRegExp now only creates one global variable: <font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">XRegExp</font>. To permanently override the native <font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">RegExp</font> constructor/object, you can now run <font style="BACKGROUND-COLOR: #f5f5ff" color="#000088" size="2">XRegExp.overrideNative();</font>
