Debian上Python日志管理有哪些最佳实践

在Debian上使用Python进行日志管理时，可以遵循以下最佳实践：

1. 初始化logging模块并创建基本日志器：

   import logging
   
   logger = logging.getLogger("my_logger")
   logger.setLevel(logging.DEBUG)
   handler = logging.StreamHandler()
   formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
   handler.setFormatter(formatter)
   logger.addHandler(handler)
   logger.info("Hello, logging world!")
   

2. 设置日志级别，精细化控制信息输出：

   logger.debug("Debugging info...")
   logger.info("Important notice.")
   logger.warning("Watch out!")
   logger.error("Oops, something went wrong.")
   logger.critical("Emergency alert!")
   
   logger.setLevel(logging.WARNING)
   

3. 定制日志格式，让日志信息一目了然：

   formatter = logging.Formatter("%(asctime)s [%(levelname)s] %(module)s.%(funcName)s(): %(message)s", datefmt="%Y-%m-%d %H:%M:%S",)
   handler.setFormatter(formatter)
   logger.info("Now with a custom format!")
   

4. 添加日志处理器，灵活处理日志输出目的地：

   file_handler = logging.FileHandler("app.log")
   file_handler.setFormatter(formatter)
   logger.addHandler(file_handler)
   logger.info("This message goes both to console and file!")
   

5. 使用多个日志处理器：

   file_handler = logging.FileHandler('app.log')
   file_handler.setLevel(logging.DEBUG)
   file_formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
   file_handler.setFormatter(file_formatter)
   
   console_handler = logging.StreamHandler()
   console_handler.setLevel(logging.INFO)
   console_formatter = logging.Formatter('%(levelname)s - %(message)s')
   console_handler.setFormatter(console_formatter)
   
   logger.addHandler(file_handler)
   logger.addHandler(console_handler)
   

6. 使用日志轮转工具： 安装 logrotate：

   sudo apt-get install logrotate
   

   配置 logrotate： 创建或编辑 /etc/logrotate.d/spider 文件：

   /path/to/spider/*.log {
       daily rotate 7
       compress
       delaycompress
       missingok
       notifempty
       create 0640 spider spider
       postrotate
           /path/to/spider/restart_spider.sh
       endscript
   }
   

   创建重启脚本 /path/to/spider/restart_spider.sh：

   #!/bin/bash
   sudo systemctl stop spider
   sleep 5
   sudo systemctl start spider
   

7. 使用ELK Stack（Elasticsearch, Logstash, Kibana）： 安装和配置ELK Stack：

   • Elasticsearch：

     wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
     echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list
     sudo apt-get update && sudo apt-get install elasticsearch
     

   • Logstash：

     wget -qO - https://artifacts.elastic.co/GPG-KEY-logstash | sudo apt-key add -
     echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/logstash-7.x.list
     sudo apt-get update && sudo apt-get install logstash
     

   • Kibana：

     wget -qO - https://artifacts.elastic.co/GPG-KEY-kibana | sudo apt-key add -
     echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/kibana-7.x.list
     sudo apt-get update && sudo apt-get install kibana
     

   配置Logstash： 创建 /etc/logstash/conf.d/spider.conf 文件：

   input {
       file {
           path "/path/to/spider/*.log"
           start_position "beginning"
       }
   }
   
   filter {
       # 添加过滤器配置
   }
   
   output {
       elasticsearch {
           hosts ["localhost:9200"]
           index "spider-logs"
       }
       stdout {
       }
   }
   

   启动Logstash：

   sudo systemctl start logstash
   

   配置Kibana： 访问 http://localhost:5601，使用默认的用户名和密码（kibana_system / changeme）登录，然后配置索引模式以匹配你的日志数据。

通过这些最佳实践，可以有效地管理Python日志，确保系统的高效运行和问题的快速排查。