Crawler parsing method 1: JOSN parsing

Many languages ​​​​can crawl, but crawlers based on python are more concise and convenient. Crawlers have also become an essential part of the python language. There are many ways for crawlers to parse data. In the previous article, I introduced to you what kind of data the crawler can obtain and the specific parsing methods. ,This This article brings you JSON parsing.

JSON

Json is simply an object and an array in JavaScript, so these two structures are objects and arrays. Structure, through these two structures various complex structures can be represented.

Object:

The object is represented in js as the content enclosed by { }, and the data structure is { key: value, key: value, ... } The structure of key-value pairs. In object-oriented languages, key is the attribute of the object, and value is the corresponding attribute value, so it is easy to understand. The value method is object.key to obtain the attribute value. The type of this attribute value can be a number. , strings, arrays, objects.

Array:

The array in js is the content enclosed by square brackets [ ], and the data structure is ["Python", "javascript", "C ", ...], the value acquisition method is the same as in all languages, using index to obtain, the type of field value can be numbers, strings, arrays, objects.

JSON (JavaScript Object Notation) is a lightweight data exchange format that makes it easy for people to read and write. It also facilitates machine analysis and generation. Suitable for data interaction scenarios, such as data interaction between the front and backend of a website.

import json
def resolveJson(path):
    file = open(path, "rb")
    fileJson = json.load(file)
    field = fileJson["field"]
    futures = fileJson["futures"]
    type = fileJson["type"]
    name = fileJson["name"]
    time = fileJson["time"]
    return (field, futures, type, name, time)
def output():
    result = resolveJson(path)
    print(result)
    for x in result:
        for y in x:
            print(y)
path = r"C:\Users\dell\Desktop\kt\test.json"
output()

Note that when the function returns multiple values, it returns a tuple;

When performing a for loop on a string, each character will be traversed

Python JSON

In this chapter we will introduce how to use Python language to encode and decode JSON objects.

JSON (JavaScript Object Notation) is a lightweight data exchange format that is easy for humans to read and write.

JSON function

Using the JSON function requires importing the json library: import json.

Function                                                                                                                                                                                                                                     Encoded JSON string decoded into Python object

json.dumps

##json.dumps Used to encode Python objects into JSON strings.

Syntax

json.dumps(obj, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, cls=None, indent=None, separators =None, encoding="utf-8", default=None, sort_keys=False, **kw)

Example

The following example encodes the array into JSON format Data:

#!/usr/bin/python
import json
data = [ { 'a' : 1, 'b' : 2, 'c' : 3, 'd' : 4, 'e' : 5 } ]
json = json.dumps(data)
print json

The execution result of the above code is:

[{"a": 1, "c": 3, "b": 2, "e": 5, "d": 4}]

Use parameters to format JSON data for output:

>>> import json
>>> print json.dumps({'a': 'Runoob', 'b': 7}, sort_keys=True, indent=4, separators=(',', ': '))
{
    "a": "Runoob",
    "b": 7
}

Conversion control of python original type to json type Table:

Python

#list, tuple                                     array

str, unicode                                                     ’s

int, long, float                                                                                                                                                   false

None null

json.loads

json.loads 用于解码 JSON 数据。该函数返回 Python 字段的数据类型。

语法

json.loads(s[, encoding[, cls[, object_hook[, parse_float[, parse_int[, parse_constant[, object_pairs_hook[, **kw]]]]]]]])

实例

以下实例展示了Python 如何解码 JSON 对象：

<pre class="brush:php;toolbar:false">
#!/usr/bin/python
import json
jsonData = &#39;{"a":1,"b":2,"c":3,"d":4,"e":5}&#39;;
text = json.loads(jsonData)
print text

以上代码执行结果为：

{u'a': 1, u'c': 3, u'b': 2, u'e': 5, u'd': 4}

json 类型转换到 python 的类型对照表：

JSON Python

object dict

array list

string unicode

number (int) int, long

number (real) float

true True

false False

null None

使用第三方库：Demjson

Demjson 是 python 的第三方模块库，可用于编码和解码 JSON 数据，包含了 JSONLint 的格式化及校验功能。

Github 地址：https://github.com/dmeranda/demjson

官方地址：http://deron.meranda.us/python/demjson/

环境配置

在使用 Demjson 编码或解码 JSON 数据前，我们需要先安装 Demjson 模块。本教程我们会下载 Demjson 并安装：

$ tar -xvzf demjson-2.2.3.tar.gz
$ cd demjson-2.2.3
$ python setup.py install

JSON 函数

函数 描述

encode 将 Python 对象编码成 JSON 字符串

decode 将已编码的 JSON 字符串解码为 Python 对象

encode

Python encode() 函数用于将 Python 对象编码成 JSON 字符串。

语法

demjson.encode(self, obj, nest_level=0)

实例

以下实例将数组编码为 JSON 格式数据：

<pre class="brush:php;toolbar:false">
#!/usr/bin/python
import demjson
data = [ { &#39;a&#39; : 1, &#39;b&#39; : 2, &#39;c&#39; : 3, &#39;d&#39; : 4, &#39;e&#39; : 5 } ]
json = demjson.encode(data)
print json

以上代码执行结果为：

[{"a":1,"b":2,"c":3,"d":4,"e":5}]

decode

Python 可以使用 demjson.decode() 函数解码 JSON 数据。该函数返回 Python 字段的数据类型。

语法

demjson.decode(self, txt)

实例

以下实例展示了Python 如何解码 JSON 对象：

<pre class="brush:php;toolbar:false">
#!/usr/bin/python
import demjson
json = &#39;{"a":1,"b":2,"c":3,"d":4,"e":5}&#39;;
text = demjson.decode(json)
print  text

以上代码执行结果为：

{u'a': 1, u'c': 3, u'b': 2, u'e': 5, u'd': 4}

The above is the detailed content of Crawler parsing method 1: JOSN parsing. For more information, please follow other related articles on the PHP Chinese website!