How to Write and Test a Regular Expression Without Losing Your Mind

The first thing to understand about regular expressions is that they are not search. When you type a word into a search box, the engine looks for that word. When you write a regex, you describe the shape of the text you want, and the engine finds everything that fits the shape.

That distinction sounds academic and it is not. Almost every regex bug comes from writing a pattern that matches the examples you had in mind and also matches a dozen strings you never considered.

The greedy trap

This is the single most common regex bug, and it has a one-character fix.

.* is greedy — it matches as much as it possibly can, then gives back only what it must for the rest of the pattern to succeed. .*? is lazy — it matches as little as possible.

Given the string <b>bold</b> and <i>italic</i>:

• <.*> matches <b>bold</b> and <i>italic</i> — the whole thing, because greedy takes the last > available.
• <.*?> matches <b> — lazy stops at the first >.

Neither is “right”. The point is that you have to decide, and the default is greedy whether you meant it or not. If you have ever extracted an HTML tag and gotten three tags back, this was why.

Character classes and the escapes people get wrong

. does not match any character. It matches any character except a line break. A pattern relying on . to cross lines silently fails, and the fix is the s flag (dotAll), which makes . match newlines too.

\d is broader than you think. In JavaScript with the u flag, and in most modern engines, \d matches any Unicode decimal digit — including Arabic-Indic and full-width digits. If you need exactly 0-9, write [0-9].

Inside a character class, position changes meaning. In [a-z] the hyphen is a range. To match a literal hyphen, put it first or last: [-a-z]. Similarly, ] must be escaped or placed first inside the class.

^ means two different things. Inside a character class it negates: [^0-9] is “anything that is not a digit”. Outside, it anchors to the start of the string. Same symbol, opposite jobs.

Anchors, and the multiline surprise

^ and $ anchor to the start and end of the entire string by default, not to each line. That is why a validation pattern like ^\d{5}$ correctly rejects 12345\n67890 — good for validating a single value, wrong for scanning a document.

Add the m (multiline) flag and they start matching at line boundaries instead. This is a real source of confusion, because the same pattern behaves differently depending on a flag that is easy to forget.

A related trap: in many engines $ matches just before a trailing newline, so ^\d{5}$ may accept 12345\n. If the difference matters, be explicit about it.

Groups: capture, name, or skip

Parentheses do three different jobs, and choosing the right one matters for both clarity and speed.

• (...) is a capturing group. It records what matched, and it appears in your match results.
• (?:...) is a non-capturing group. It groups without recording — use it when you need alternation or a quantifier over several tokens but do not need the value.
• (?<name>...) is a named group. It is still a capturing group, but you retrieve it by name instead of by index.

Prefer named groups in anything that will be maintained. match.groups.year survives an edit that inserts a new group earlier in the pattern; match[3] does not.

The performance trap: catastrophic backtracking

This one does not produce a wrong answer. It produces a hung process.

Certain patterns, applied to input that almost matches, make the engine try an exponential number of combinations. The classic shape is nested quantifiers over overlapping alternatives:

(a+)+b

Against a long string of as with no b at the end, this takes time proportional to 2ⁿ. Twenty-five characters is enough to hang a process; forty is enough to hang it for years.

Three defences:

Avoid nested quantifiers. (a+)+ is almost always rewritable as a+.

Be specific about what you are matching. \d+ instead of .* dramatically reduces the search space.

Test with non-matching input. This is the whole point of a tester. A pattern that runs instantly on the text you meant to match can hang forever on text you did not.

When not to use a regular expression

Regular expressions describe regular languages. Some things are not regular, and forcing them into a pattern produces something unmaintainable that is also wrong.

HTML. Nested tags cannot be matched by a regular expression, because the nesting depth is unbounded. Use an HTML parser. The famous Stack Overflow answer on this is famous for a reason.

Email addresses. The full specification permits quoted strings, comments, and IP-literal domains. Every “complete” email regex you have seen is either wrong or unreadable. Validate the shape loosely, then send a confirmation email — that is the only real verification.

JSON and other structured data. Same problem as HTML: nesting. Use a parser.

The rule of thumb: if the thing you are matching can contain an arbitrary number of nested copies of itself, regex is the wrong tool.

Testing properly

A tester is not for confirming that your pattern works. It is for finding where it does not.

1. Open the regex tester.
2. Paste your pattern and select the flags you intend to use in production — testing without g and shipping with it is a common mistake.
3. Test with input that should not match. This is the step people skip, and it is the one that finds the bugs.
4. Check the capture groups, not just the match. A pattern can match correctly and capture the wrong substring.
5. For anything going into a hot path, test with a long non-matching string to rule out backtracking.

Everything runs in your browser, which matters more than it sounds: regex patterns in production code frequently contain internal hostnames, parameter names, or fragments of proprietary formats.

Frequently asked questions

Why does my regex match too much? Almost always greediness. Add ? after the quantifier — .* becomes .*? — to make it stop at the first opportunity instead of the last.

What is the difference between . and [\s\S]? . excludes line breaks unless the s flag is set. [\s\S] matches everything including newlines, and is the traditional way to express that without depending on a flag.

What is catastrophic backtracking? Exponential search behaviour caused by nested quantifiers. The pattern is correct but the runtime explodes on input that nearly matches. Fix it by removing nested quantifiers and making character classes specific.

Which regex flavour is this? JavaScript (ECMAScript), because matching runs in the browser. Most syntax is identical to PCRE and Python; lookbehind and named groups are supported.

Can I parse HTML with a regex? No. Nested structures have unbounded depth, which is outside what regular expressions can describe. Use a parser.

How should I validate an email address? Loosely — check for an @, a dot, and a plausible length — then send a confirmation email. A regex that fully implements the email specification is unreadable and still does not prove the address exists.