Home | Mirror | Search

4. webalizer

What is Webalizer?

The Webalizer is a fast, free web server log file analysis program. It produces highly detailed, easily configurable usage reports in HTML format, for viewing with a standard web browser

  1. install webalizer

    sudo apt-get install webalizer
    				
  2. config

    vim /etc/webalizer/webalizer.conf
    
    LogFile /home/netkiller/logs/access.log
    OutputDir /home/netkiller/public_html/webalizer
    				

    rotate log

    Incremental yes
    				
  3. crontab

    /etc/cron.daily/webalizer

    
    netkiller@shenzhen:~$ cat /etc/cron.daily/webalizer
    #!/bin/sh
    # /etc/cron.daily/webalizer: Webalizer daily maintenance script
    # This script was originally written by
    # Remco van de Meent <remco@debian.org>
    # and now, all rewrited by Jose Carlos Medeiros <jose@psabs.com.br>
    
    # This script just run webalizer agains all .conf files in /etc/webalizer directory
    
    WEBALIZER=/usr/bin/webalizer
    WEBALIZER_CONFDIR=/etc/webalizer
    
    [ -x ${WEBALIZER} ] || exit 0;
    [ -d ${WEBALIZER_CONFDIR} ] || exit 0;
    
    for i in ${WEBALIZER_CONFDIR}/*.conf; do
      # run agains a rotated or normal logfile
      LOGFILE=`awk '$1 ~ /^LogFile$/ {print $2}' $i`;
    
      # empty ?
      [ -s "${LOGFILE}" ] || continue;
      # readable ?
      [ -r "${LOGFILE}" ] || continue;
    
      # there was a output ?
      OUTDIR=`awk '$1 ~ /^OutputDir$/ {print $2}' $i`;
      #  exists something ?
      [ "${OUTDIR}" != "" ] || continue;
      # its a directory ?
      [ -d ${OUTDIR} ] || continue;
      # its writable ?
      [ -w ${OUTDIR} ] || continue;
    
      # Run Really quietly, exit with status code if !0
      ${WEBALIZER} -c ${i} -Q || continue;
      RET=$?;
    
      # Non rotated log file
      NLOGFILE=`awk '$1 ~ /^LogFile$/ {gsub(/\.[0-9]+(\.gz)?/,""); print $2}' $i`;
    
      # check current log, if last log is a rotated logfile
      if [ "${LOGFILE}" != "${NLOGFILE}" ]; then
        # empty ?
        [ -s "${NLOGFILE}" ] || continue;
        # readable ?
        [ -r "${NLOGFILE}" ] || continue;
    
        ${WEBALIZER} -c ${i} -Q ${NLOGFILE};
        RET=$?;
      fi;
    done;
    
    # exit with webalizer's exit code
    exit $RET;
    
    				
  4. initialization

    sudo /usr/bin/webalizer
    				
  5. http://netkiller.8800.org/webalizer/

最後附上Webalizer的參數表:
可以執行webalizer –h得到所有命令行參數:
Usage: webalizer [options] [log file]
-h = 打印幫助信息
-v -V = 打印版本信息
-d = 打印附加調試信息
-F type = 日誌格式類型. type= (clf | ftp | squid)
-i = 忽略歷史檔案
-p = 保留狀態 (遞增模式)
-q = 忽略消息信息
-Q = 忽略所有信息
-Y = 忽略國家圖形
-G = 忽略小時統計圖形
-H = 忽略小時統計信息
-L = 忽略彩色圖例
-l num = 在圖形中使用數字背景綫
-m num = 訪問超時 (seconds)
-T = 打印時間信息
-c file = 指定配置檔案
-n name = 使用的主機名
-o dir = 結果輸出目錄
-t name = 指定報告題目上的主機名
-a name = 隱藏用戶代理名稱
-r name = 隱藏訪問連結
-s name = 隱藏客戶
-u name = 隱藏URL
-x name = 使用檔案副檔名
-P name = 頁面類型副檔名
-I name = index別名
-A num = 顯示前幾名客戶類型
-C num = 顯示前幾名國家
-R num = 顯示前幾名連結
-S num = 顯示前幾名客戶
-U num = 顯示前幾名URLs
-e num = 顯示前幾名訪問頁面
-E num = 顯示前幾名不存在的頁面
-X = 隱藏個別用戶
-D name = 使用dns緩存檔案
-N num = DNS 進程數 (0=禁用dns)
		

4.1. 手工生成

$ sudo webalizer -c /etc/webalizer/webalizer.conf -o /var/www/webalizer/web2 /opt/logs/web2/www/access_log
			

分析多個檔案

# find ./ -exec sudo webalizer -p -c /etc/webalizer/webalizer.conf -o /var/www/webalizer/my /mnt/logs/www/{} \;
			

4.2. 批量處理歷史數據

下面腳本可以批量處理歷史日誌,等這個腳本運行完後在crontab中加入另一個腳本。

for f in /mnt/logs/cdn/*.gz ; do webalizer -c /etc/webalizer/webalizer.conf -o /var/www/webalizer/cdn/ $f ; done
			

crontab

webalizer -c /etc/webalizer/webalizer.conf -o /var/www/webalizer/cdn/ /mnt/logs/cdn/$(date -d '-1 day' +'%Y-%m-%d').log.gz
			

多域名批量處理

for d in /mnt/cdn/* ; do
    htmldir=/var/www/webalizer/$(basename $d)
    mkdir -p $htmldir
    for f in $d/*.log.gz ; do webalizer -c /etc/webalizer/webalizer.conf -o $htmldir $f ; done
done
			

crontab

#!/bin/bash
for d in /mnt/cdn/*;
do
    htmldir=/var/www/webalizer/$(basename $d)
    mkdir -p $htmldir
    webalizer -c /etc/webalizer/webalizer.conf -o $htmldir $d/$(date -d '-1 day' +'%Y_%m_%d').log.gz
done
			

4.3. crontab

sudo webalizer  -F clf -p -t www.example.com -Q -c /etc/webalizer/webalizer.conf -o /var/www/webalizer/example /mnt/logs/www/access.$(date -d '-1 day' +'%Y-%m-%d').log
			
comments powered by Disqus