How does Node in the Controller layer perform data verification?

[Video tutorial recommendation: node js tutorial]

Humorous back-end programmers usually laugh at themselves as CURD Boy. CURD, which is the addition, deletion, modification and query of a certain storage resource, is completely data-oriented programming.

It’s great. Data-oriented programming often leads to a more thorough understanding of the business, thereby writing higher-quality code and creating fewer bugs. Since it is data-oriented programming, it is even more necessary to avoid the appearance of dirty data and strengthen data verification. Otherwise, should we trust the front-end data verification? After all, the front-end data verification goes directly to the user for more user-friendly feedback at the UI layer.

Data verification layer

The backend is divided into various levels due to its emphasis on business logic and various data to be processed. There are many backend projects I have experienced. There are various named layers for Controller, Service, Model, Helper, Entity, etc. But there must be a layer here called Controller, which stands at the top layer of the backend and directly receives the data transmitted by the client.

Since the Controller layer is the top layer that interacts with client data on the server side, it adheres to the principle of Fail Fast and is responsible for the function of data filter. Illegal data will be sent back directly, as majestic as the door gods Qin Qiong and Yuchi Gong.

Data verification also derives a semi-documented by-product. You only need to take a look at the data verification layer to know which fields are to be transmitted and what formats they are in.

The following are common data verifications. This article describes how to verify them:

1. required/optional
2. Basic data verification, such as number , string, timestamp and the conditions that the value needs to meet
3. Complex data verification, such as IP, mobile phone number, email and domain name

const body = {
  id,
  name,
  mobilePhone,
  email
}

The author has come into contact with a system without data verification layer Back-end projects, if/else are flooded at various levels, which is extremely painful and requires refactoring in minutes.

JSON Schema

JSON Schema Based on JSON for data verification format, with a specification json-schema.org, currently (2020 -08) The latest version is 7.0. Various server programming languages ​​have implemented the specifications, such as go, java, php, etc. Of course, there are also great javascripts, such as tepid ajv.

The following is a Schema for verifying user information. It can be seen that the syntax is complex and cumbersome:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "title": "User",
  "description": "用户信息",
  "type": "object",
  "properties": {
    "id": {
      "description": "用户 ID",
      "type": "integer"
    },
    "name": {
      "description": "用户姓名",
      "type": "string"
    },
    "email": {
      "description": "用户邮箱",
      "type": "string",
      "format": "email",
      "maxLength": 20
    },
    "mobilePhone": {
      "description": "用户手机号",
      "type": "string",
      "pattern": "^(?:(?:\+|00)86)?1[3-9]\d{9}$",
      "maxLength": 15
    }
  },
  "required": ["id", "name"]
}

For complex data type verification, JSON Schema has the following Format built-in for convenient and quick verification

• Dates and times
• Email addresses
• Hostnames
• IP Addresses
• Resource identifiers
• URI template
• JSON Pointer
• Regular Expressions

For mobile phone numbers that are not in the built-in Format, use ajv.addFormat to manually add the Format

ajv.addFormat('mobilePhone', (str) => /^(?:(?:\+|00)86)?1[3-9]\d{9}$/.test(str));

Joi

#joi claims to be the most powerful JS verification library and has also gained 16,000 stars on github. Compared with JSON Schema, its syntax is more concise and powerful.

The most powerful data validation library for JS

Complete the same validation, only requires less code, and can complete more powerful validation. The following are only examples, please go to the documentation for more examples.

const schema = Joi.object({
  id: Joi.number().required(),
  name: Joi.number().required(),
  email: Joi.string().email({ minDomainSegments: 2, tlds: { allow: ['com', 'net'] } }),
  mobilePhone: Joi.string().pattern(/^(?:(?:\+|00)86)?1[3-9]\d{9}$/),

  password: Joi.string().pattern(/^[a-zA-Z0-9]{3,30}$/),
  // 与 password 相同的校验
  repeatPassword: Joi.ref('password'),
})
  // 密码与重复密码需要同时发送
  .with('password', 'repeat_password');
  // 邮箱与手机号提供一个即可
  .xor('email', 'mobilePhone')

Data verification and routing layer integration

Since the data is passed directly from the routing, koajs officially implemented a based on joi joi-router, the pre-data is verified to the routing layer, and the query, body and params passed from the front end are verified.

joi-router also parses and limits various content-type transmitted by the front end based on co-body. If it is limited to application/json, CSRF attacks can also be prevented to a certain extent.

const router = require('koa-joi-router');
const public = router();

public.route({
  method: 'post',
  path: '/signup',
  validate: {
    header: joiObject,
    query: joiObject,
    params: joiObject,
    body: joiObject,
    maxBody: '64kb',
    output: { '400-600': { body: joiObject } },
    type: 'json',
    failure: 400,
    continueOnError: false
  },
  pre: async (ctx, next) => {
    await checkAuth(ctx);
    return next();
  },
  handler: async (ctx) => {
    await createUser(ctx.request.body);
    ctx.status = 201;
  },
});

Regular expressions and secure regular expressions

When the author was troubleshooting a performance problem, he discovered that an API actually took too long in the data verification layer, which I had never thought of. The root of the problem lies in unsafe regular expressions. So what are unsafe regular expressions?

For example, the regular expression below that can hang up the CPU is a time bomb, and the number of backtracking has increased exponentially.

可以参考文章 浅析 ReDos 原理与实践

const safe = require('safe-regex')
const re = /(x+x+)+y/

// 能跑死 CPU 的一个正则
re.test('xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx')

// 使用 safe-regex 判断正则是否安全
safe(re)   // false

数据校验，针对的大多是字符串校验，也会充斥着各种各样的正则表达式，保证正则表达式的安全相当紧要。safe-regex 能够发现哪些不安全的正则表达式。

总结

1. Controller 层需要进行统一的数据校验，可以采用 JSON Schema (Node 实现 ajv) 与 Joi
2. JSON Schema 有官方规范及各个语言的实现，但语法繁琐，可使用校验功能更为强大的 Joi
3. 进行字符串校验时，注意不安全的正则引起的性能问题

更多编程相关知识，可访问：编程教学！！

The above is the detailed content of How does Node in the Controller layer perform data verification?. For more information, please follow other related articles on the PHP Chinese website!