Getting Started with Regular Expressions

cjx is currently working on a crawler project and urgently needs to capture the content he wants from the page, but obtaining it through logical judgment is too complicated. Fortunately, we have the powerful tool of regular expressions. Many things can be done easily. Cjx also had some knowledge about regular expressions before. However, I have always been in a state of incomplete understanding, and it is difficult to effectively write a satisfactory regular expression by myself. Recently I found a book on the Internet called Mastering Regular Expressions written by Jeffrey E.F. Fried. After reading the first chapter, I suddenly found that I could write a few regular rules, haha~~~ cjx suddenly felt like he had upgraded from a loser to a tall, rich and handsome man... The following is a summary of the first chapter of the book~

Start and end of line

Perhaps the easiest to understand metacharacters are the caret symbol ^ and the dollar sign $. When checking a line of text, ^ represents the beginning of a line and $ represents the end.

Readers should develop the habit of understanding regular expressions according to characters. For example, don’t do this:

　^cat matches lines starting with cat

but should be understood like this:

　^cat matches text that starts with c as the first character of a line, followed by an a, followed by a t .

There is no difference in the results of these two interpretations, but it is easier to understand the internal logic of newly encountered regular expressions by interpreting them by characters.

Match one of several characters

If we need to search for the word "grey" and are not sure whether it is written as "gray", we can use the regular expression structure [...]. It allows the user to list characters that are expected to match somewhere, usually called a character group.

So, gr[ea]y means: find g first, followed by an r, then an a or e, and finally a y.

Within the character group, the character metacharacter '-' represents a range: is exactly the same as . We can also combine character ranges with normal text as we like:

[0-9A-Z_!.?] can match a number, uppercase letter, underscore, exclamation point, period or question mark.

Exclusive character group

Replace [...] with [^...]. This character group will match any unlisted characters. For example: [^1-6] matches any character except 1 to 6. The ^ at the beginning of this group means exclusion, so what is listed here is not the characters you want to match, but the characters you don't want to match.

Use dot to match any character

metacharacter. It is a simple way to write a character group used to match any character. If we need to use a "match any character" placeholder in an expression, using the dot is convenient.

Match any subexpression

The metacharacter | is a very concise metacharacter, which means "or". Relying on it, we can combine different sub-expressions into a total expression, and this total expression can match any sub-expression.

Optional elements

Now let’s look at the matching of color and colour. The difference between them is that the following word has one more u than the previous one. We can use coloru?r to solve this problem. The metacharacter ? (that is, the question mark) represents optional options. Adding it after a character means that the character is allowed to appear here, but its occurrence is not a necessary condition for successful matching.

Other quantifiers: Repeated

+ (plus sign) and * (asterisk) function similarly to question marks. The metacharacter + means that the immediately preceding element appears one or more times, while * means that the immediately preceding element appears any number of times, or does not appear at all.
Next, look at a TAG like

---

, which represents a horizontal line with a height of 14. In addition, there must be a space between HR and SIZE, but there can be no spaces on both sides of the equal sign. So if you want to find such HR in a web page, the expression is written as

---

. Sometimes the SIZE value of HR is often variable, and the SIZE attribute may not exist. Then the expression can be rewritten as . Wow, that's amazing.

Brackets and backreferences

So far, we have seen two uses of brackets: 1. Limiting the scope of multiple options; 2. Combining several characters into a unit, which is used by quantifiers such as question marks or asterisks . Now I want to introduce another use of brackets, which is backreference. Although it is not common in egrep (although the popular GNU version does support this feature), it is very common in other tools.
In tool software that supports backreferences, brackets can "memorize" the text matched by the subexpressions in them. No matter what these texts are, the metacharacter sequence 1 can remember them.

Of course, we can use multiple parentheses in an expression. Then use 1, 2, 3, etc. to represent the text matched by the first, second, and third sets of brackets. Brackets are performed in the order of appearance of the open brackets '(' from left to right, so 1 in ([a-z])([0-9])12 represents the content of [a-z] matching, while 2 represents [0- 9] Matching content.

Magic escape
Sometimes, we may need to match some symbols such as .+*?, but at the same time we find that they are meta symbols, so we can add an escape in front of them. Symbols are used to match these special meta-symbols

.

Some useful abbreviations
t Tab character
n Line feed character
r Carriage return character
s Any whitespace character, such as space, newline, tab indentation, etc. All whitespace characters
S Any character except s
w [a-zA-Z0-9] is very useful in w+ and can be used to match a word
W any character except w
d [0-9], that is, the number
D any character except d, that is [^0-9]