python logging module examples and improvements

Many applications will have a log module, which is used to record some key information during the operation of the system to facilitate tracking of the system's operating status. In python, we don't need third-party logging components, because it already provides us with a simple, easy-to-use, and powerful logging module: logging.

python prints all attribute values ​​​​of the object:

def prn_obj(obj):
  print '\n'.join(['%s:%s' % item for item in obj.__dict__.items()])

Python logger object attributes (obtained by the above function)

name:get_data
parent:<logging.RootLogger instance at 0x1d8bd88>
handlers:[<logging.FileHandler instance at 0x21bcc68>]
level:10
disabled:1   #当前的logger是否有效，默认为0
manager:<logging.Manager instance at 0x1d8bea8>
propagate:0   #是否将本级日志
filters:[]

Some logs cannot be output

File: logger.conf

 [formatters]
keys=default
 
[formatter_default]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
class=logging.Formatter
 
[handlers]
keys=console, error_file
 
[handler_console]
class=logging.StreamHandler
formatter=default
args=tuple()
 
[handler_error_file]
class=logging.FileHandler
level=INFO
formatter=default
args=("logger.log", "a")
 
[loggers]
keys=root
 
[logger_root]
level=DEBUG
formatter=default
handlers=console,error_file

File: logger.py

 #!/bin/env python
 
import logging
from logging.config import logging
 
class Test(object):
  """docstring for Test"""
  def __init__(self):
    logging.config.fileConfig("logger.conf")
    self.logger = logging.getLogger(__name__)
 
  def test_func(self):
    self.logger.error('test_func function')
 
class Worker(object):
  """docstring for Worker"""
  def __init__(self):
    logging.config.fileConfig("logger.conf")
    self.logger = logging.getLogger(__name__)
 
    data_logger = logging.getLogger('data')
    handler = logging.FileHandler('./data.log')
    fmt = logging.Formatter('%(asctime)s|%(message)s')
    handler.setFormatter(fmt)
    data_logger.addHandler(handler)
    data_logger.setLevel(logging.DEBUG)
    self.data_logger = data_logger
 
  def test_logger(self):
    self.data_logger.error("test_logger function")
    instance = Test()
    self.data_logger.error("test_logger output")
    instance.test_func()
 
 
def main():
  worker = Worker()
  worker.test_logger()
 
if __name__ == '__main__':
  main()

Question 1: During the test process, only one statement of test_logger function can be printed out
Question 2: Obviously only statements are printed in data_logger, but related logs also appear in the logger's log.

Solution to Problem 1:

Use the python -m pdb logger.py statement to debug the script and find that after executing the instance = Test() statement, print '\n '.join(['%s:%s' % item for item in self.data_logger.__dict__.items()]) debugging statement shows that the disable attribute value of data_logger has changed from 0 to True. At this time, the corresponding attribute of logger The same changes have occurred. This change causes the logger object to stop logging. Referring to the relevant manual of the python logging module, I found that "The fileConfig() function takes a default parameter, disable_existing_loggers, which defaults to True for reasons of backward compatibility. This may or may not be what you want, since it will cause any loggers existing before the fileConfig() call to be disabled unless they (or an ancestor) are explicitly named in the configuration.”, that is, calling the fileconfig() function will disable all previously existing loggers. In the Python 2.7 version, the fileConfig() function adds a parameter, logging.config.fileConfig(fname, defaults=None, disable_existing_loggers=True), which can explicitly set disable_existing_loggers to FALSE to avoid disabling the original logger. Changing the logging.config.fileConfig function in the Test class in the above code to logging.config.fileConfig("./logger.conf", disable_existing_loggers=0) can solve the problem. However, since this code is located in the same program, you can directly use the logging.getLogger(LOGGOR_NAME) function to reference the same logger, without calling the logging.config.fileConfig function to reload it.

Solution to Problem 2:

The logger object has an attribute propagate. If this attribute is True, the information to be output will be pushed to all superior loggers of the logger. These superiors The handlers corresponding to the logger will print the received information to the associated log. The related root logger attributes are configured in the logger.conf configuration file. This root logger is the default logger log.

The modified ones are as follows:

File：logger.conf

 [formatters]
keys=default, data
 
[formatter_default]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
class=logging.Formatter
 
[formatter_data]
format=%(asctime)s|%(message)s
class=logging.Formatter
 
[handlers]
keys=console, error_file, data_file
 
[handler_console]
class=logging.StreamHandler
formatter=default
args=tuple()
 
[handler_error_file]
class=logging.FileHandler
level=INFO
formatter=default
args=("logger.log", "a")
 
[handler_data_file]
class=logging.FileHandler
level=INFO
formatter=data
args=("data_new.log", "a")
 
[loggers]
keys=root, data
 
[logger_root]
level=DEBUG
handlers=console,error_file
 
[logger_data]
level=DEBUG
handlers=data_file
qualname=data
propagate=0

File：logger.py

 #!/bin/env python
 
import logging
from logging.config import logging
 
class Test(object):
  """docstring for Test"""
  def __init__(self):
    self.logger = logging.getLogger(__name__)
 
  def test_func(self):
    self.logger.error('test_func function')
 
class Worker(object):
  """docstring for Worker"""
  def __init__(self):
    logging.config.fileConfig("logger.conf")
    self.logger = logging.getLogger(__name__)
    self.data_logger = logging.getLogger('data')
 
  def test_logger(self):
    self.data_logger.error("test_logger function")
    instance = Test()
    self.data_logger.error("test_logger output")
    instance.test_func()
 
 
def main():
  worker = Worker()
  worker.test_logger()
 
if __name__ == '__main__':
  main()

For more python logging module examples and improvements related articles, please pay attention to the PHP Chinese website!