[yaala] Re: Re: Problems .....
Federico Lombardo
yaala@verplant.org
Wed, 30 Apr 2003 15:30:59 +0200
This is a multi-part message in MIME format.
------=_NextPart_000_0211_01C30F2D.803CF790
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
Here we are... sorry for the late response.
actually my squid logs are about 400 - 600 Megs large.
The Problem happens also with a test log on 60M.
(rember I'm using squid 2.5Stable2).
So I attach you the three files
combined output problem, classic output problem and my config,
Another problem debugging is:
when I use combined mode, the program finish to parse the file, and then
"die" (cfr. Combined.txt).
Using classic output will use my CPU at 99% FOR EVER!!!! and the debug is
always that rows contained in the file called "classic.txt".
Best Regards,
Federico
------=_NextPart_000_0211_01C30F2D.803CF790
Content-Type: text/plain;
name="combined.txt"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
filename="combined.txt"
ISS}{crazyxxx3dworld.com} +=3D 190
lib/Data/Core.pm:=20
$DATA->{ARRAY(0x819df58)}{*UNRESOLVED*}{2003-04-01}{DIRECT}{17}{200}{GET}=
{image/gif}{*UNRESOLVED*}{HTTP}{TCP_MISS}{crazyxxx3dworld.com}=20
+=3D 4475
lib/Data/Core.pm:=20
$DATA->{ARRAY(0x819dcac)}{*UNRESOLVED*}{2003-04-01}{DIRECT}{17}{200}{GET}=
{image/gif}{*UNRESOLVED*}{HTTP}{TCP_MISS}{crazyxxx3dworld.com}=20
+=3D 1
lib/Data/Core.pm:=20
$DATA->{ARRAY(0x819ddf0)}{*UNRESOLVED*}{2003-04-01}{DIRECT}{17}{200}{GET}=
{image/gif}{*UNRESOLVED*}{HTTP}{TCP_MISS}{crazyxxx3dworld.com}=20
+=3D 193
lib/Data/Core.pm:=20
$DATA->{ARRAY(0x819df58)}{*UNRESOLVED*}{2003-04-01}{DIRECT}{17}{200}{GET}=
{image/gif}{*UNRESOLVED*}{HTTP}{TCP_MISS}{crazyxxx3dworld.com}=20
+=3D 1661
lib/Data/Core.pm:=20
$DATA->{ARRAY(0x819dcac)}{*UNRESOLVED*}{2003-04-01}{DIRECT}{17}{200}{GET}=
{image/gif}{*UNRESOLVED*}{HTTP}{TCP_MISS}{crazyxxx3dworld.com}=20
+=3D 1
lib/Data/Core.pm:=20
$DATA->{ARRAY(0x819ddf0)}{*UNRESOLVED*}{2003-04-01}{DIRECT}{17}{200}{GET}=
{image/gif}{*UNRESOLVED*}{HTTP}{TCP_MISS}{crazyxxx3dworld.com}=20
+=3D 373
lib/Data/Core.pm:=20
$DATA->{ARRAY(0x819df58)}{*UNRESOLVED*}{2003-04-01}{DIRECT}{17}{200}{GET}=
{image/gif}{*UNRESOLVED*}{HTTP}{TCP_MISS}{crazyxxx3dworld.com}=20
+=3D 5098
lib/Data/Core.pm:=20
$DATA->{ARRAY(0x819dcac)}{*UNRESOLVED*}{2003-04-01}{DIRECT}{17}{200}{GET}=
{image/gif}{*UNRESOLVED*}{HTTP}{TCP_MISS}{crazyxxx3dworld.com}=20
+=3D 1
lib/Data/Core.pm:=20
$DATA->{ARRAY(0x819ddf0)}{*UNRESOLVED*}{2003-04-01}{DIRECT}{17}{200}{GET}=
{image/gif}{*UNRESOLVED*}{HTTP}{TCP_MISS}{crazyxxx3dworld.com}=20
+=3D 374
lib/Data/Core.pm:=20
$DATA->{ARRAY(0x819df58)}{*UNRESOLVED*}{2003-04-01}{DIRECT}{17}{200}{GET}=
{image/gif}{*UNRESOLVED*}{HTTP}{TCP_MISS}{crazyxxx3dworld.com}=20
+=3D 7010
lib/Data/Core.pm:=20
$DATA->{ARRAY(0x819dcac)}{*UNRESOLVED*}{2003-04-01}{DIRECT}{17}{200}{GET}=
{image/gif}{*UNRESOLVED*}{HTTP}{TCP_MISS}{crazyxxx3dworld.com}=20
+=3D 1
lib/Data/Core.pm:=20
$DATA->{ARRAY(0x819ddf0)}{*UNRESOLVED*}{2003-04-01}{DIRECT}{17}{200}{GET}=
{image/gif}{*UNRESOLVED*}{HTTP}{TCP_MISS}{crazyxxx3dworld.com}=20
+=3D 186
lib/Data/Core.pm:=20
$DATA->{ARRAY(0x819df58)}{*UNRESOLVED*}{2003-04-01}{DIRECT}{17}{200}{GET}=
{image/gif}{*UNRESOLVED*}{HTTP}{TCP_MISS}{crazyxxx3dworld.com}=20
+=3D 2358
lib/Data/Core.pm:=20
$DATA->{ARRAY(0x819dcac)}{*UNRESOLVED*}{2003-04-01}{DIRECT}{17}{200}{GET}=
{image/gif}{*UNRESOLVED*}{HTTP}{TCP_MISS}{crazyxxx3dworld.com}=20
+=3D 1
lib/Data/Core.pm:=20
$DATA->{ARRAY(0x819ddf0)}{*UNRESOLVED*}{2003-04-01}{DIRECT}{17}{200}{GET}=
{image/gif}{*UNRESOLVED*}{HTTP}{TCP_MISS}{crazyxxx3dworld.com}=20
+=3D 196
lib/Data/Core.pm:=20
$DATA->{ARRAY(0x819df58)}{*UNRESOLVED*}{2003-04-01}{DIRECT}{17}{200}{GET}=
{image/gif}{*UNRESOLVED*}{HTTP}{TCP_MISS}{crazyxxx3dworld.com}=20
+=3D 1733
lib/Data/Core.pm:=20
$DATA->{ARRAY(0x819dcac)}{*UNRESOLVED*}{2003-04-01}{DIRECT}{17}{200}{GET}=
{image/gif}{*UNRESOLVED*}{HTTP}{TCP_MISS}{crazyxxx3dworld.com}=20
+=3D 1
lib/Data/Core.pm:=20
$DATA->{ARRAY(0x819ddf0)}{*UNRESOLVED*}{2003-04-01}{DIRECT}{17}{200}{GET}=
{image/gif}{*UNRESOLVED*}{HTTP}{TCP_MISS}{crazyxxx3dworld.com}=20
+=3D 195
lib/Data/Core.pm:=20
$DATA->{ARRAY(0x819df58)}{*UNRESOLVED*}{2003-04-01}{DIRECT}{17}{200}{GET}=
{image/gif}{*UNRESOLVED*}{HTTP}{TCP_MISS}{crazyxxx3dworld.com}=20
+=3D 2830
lib/Data/Core.pm:=20
$DATA->{ARRAY(0x819dcac)}{*UNRESOLVED*}{2003-04-01}{DIRECT}{17}{200}{GET}=
{image/gif}{*UNRESOLVED*}{HTTP}{TCP_MISS}{crazyxxx3dworld.com}=20
+=3D 1
lib/Data/Core.pm:=20
$DATA->{ARRAY(0x819ddf0)}{*UNRESOLVED*}{2003-04-01}{DIRECT}{17}{200}{GET}=
{image/gif}{*UNRESOLVED*}{HTTP}{TCP_MISS}{crazyxxx3dworld.com}=20
+=3D 193
lib/Data/Core.pm:=20
$DATA->{ARRAY(0x819df58)}{*UNRESOLVED*}{2003-04-01}{DIRECT}{17}{200}{GET}=
{image/gif}{*UNRESOLVED*}{HTTP}{TCP_MISS}{crazyxxx3dworld.com}=20
+=3D 789
./yaala: Generating pages..Died at lib/Report/Combined.pm line 41.
------=_NextPart_000_0211_01C30F2D.803CF790
Content-Type: text/plain;
name="classic.txt"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
filename="classic.txt"
lib/Data/Core.pm: Returning, $sum =3D 0
lib/Data/Core.pm: Query not unique. Performing subqueries for 8 values =
of field 'method'.
lib/Data/Core.pm: Unavailable field requested. Returning 0.
lib/Data/Core.pm: Unavailable field requested. Returning 0.
lib/Data/Core.pm: Unavailable field requested. Returning 0.
lib/Data/Core.pm: Unavailable field requested. Returning 0.
lib/Data/Core.pm: Unavailable field requested. Returning 0.
lib/Data/Core.pm: Unavailable field requested. Returning 0.
lib/Data/Core.pm: Unavailable field requested. Returning 0.
lib/Data/Core.pm: Unavailable field requested. Returning 0.
lib/Data/Core.pm: Returning, $sum =3D 0
lib/Data/Core.pm: Query not unique. Performing subqueries for 8 values =
of field 'method'.
lib/Data/Core.pm: Unavailable field requested. Returning 0.
lib/Data/Core.pm: Unavailable field requested. Returning 0.
lib/Data/Core.pm: Unavailable field requested. Returning 0.
lib/Data/Core.pm: Unavailable field requested. Returning 0.
lib/Data/Core.pm: Unavailable field requested. Returning 0.
lib/Data/Core.pm: Unavailable field requested. Returning 0.
lib/Data/Core.pm: Unavailable field requested. Returning 0.
lib/Data/Core.pm: Unavailable field requested. Returning 0.
lib/Data/Core.pm: Returning, $sum =3D 0
------=_NextPart_000_0211_01C30F2D.803CF790
Content-Type: application/octet-stream;
name="config"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
filename="config"
##################################################=0A=
# yaala 0.6.2 config 2003-03-11 #=0A=
#---=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D---------=
----------------#=0A=
# For exact instructions please see the README #=0A=
# and the notes above each entry. #=0A=
##################################################=0A=
# $Id: config,v 1.3 2003/03/11 22:17:53 octo Exp $=0A=
=0A=
# Tells yaala the directory to save the html pages in.=0A=
# You should manually copy .gif and .css there from html=0A=
# directory.=0A=
# Default is 'reports'=0A=
directory: 'reports';=0A=
=0A=
# Here you can choose between the ``new'' Combined-output module and the=0A=
# Classic-output which emulates 0.4.x behaviour.=0A=
# Default is to use 'Combined'=0A=
#report: 'Combined';=0A=
report: 'Classic'; =0A=
# The module used for parsing the logfile(s)=0A=
# The modules coming with this package are:=0A=
# - Bind9=0A=
# - Common=0A=
# - Ncsa=0A=
# - Wnserver=0A=
# - Squid=0A=
# - Xferlog=0A=
# Default: 'Common'=0A=
logtype: 'Squid';=0A=
=0A=
#########################################################################=0A=
# Output #=0A=
#---=3D=3D=3D=3D=3D=3D=3D=3D---------------------------------------------=
---------------#=0A=
# The directive 'select' selects data to be printed in the report. #=0A=
# #=0A=
# First you have to know that there are two types of fields: normal #=0A=
# fields (sometimes also called 'key') and aggregations. An aggregation #=0A=
# is basically everything you can sum up. In a webserver logfile this #=0A=
# would be the amount of bytes transferred and the number of requests. #=0A=
# The keyfields is everything else, e.g. the status code, because it #=0A=
# doesn't make sense to sum it up. #=0A=
# #=0A=
# The syntax for select-statements is a bit like SQL. A basic select #=0A=
# looks as follows: #=0A=
# select: aggregation BY field; #=0A=
# #=0A=
# This displays, for example, the amount of bytes transferred on each #=0A=
# day. For more detailed output you can select more than one #=0A=
# (key)field. Right now up to three are supported. The fields have to #=0A=
# be comma-seperated: #=0A=
# select: aggregation BY field0, field1, field2; #=0A=
# #=0A=
# Ok, now you might only be interested in a part of all the requests. #=0A=
# For example you might wonder, how many times google has visited each #=0A=
# file. You can do this like this: #=0A=
# select: requests BY file WHERE host =3D~ google; #=0A=
# #=0A=
# Or, more general like this: #=0A=
# select: aggregation BY field[, field ..] WHERE field <CMP> value; #=0A=
# #=0A=
# '<CMP>' is the rule how to match the values. Methods implemented are: #=0A=
# - '=3D=3D' equal #=0A=
# - '!=3D' not equal #=0A=
# - '=3D~' regular expression (non-numeric only) #=0A=
# =3D '!~' negated regular expression (non-numeric only) #=0A=
# - '<', '>' lesser/greater than #=0A=
# - '<=3D', '>=3D' lesser/greater or equal #=0A=
# #=0A=
#########################################################################=0A=
=0A=
=0A=
select: "requests BY client, date, hierarchycode, hour, httpstatus, =
method, mime, peer, protocol, resultcode, server";=0A=
=0A=
select: "elapsed BY client, date, hierarchycode, hour, httpstatus, =
method, mime, peer, protocol, resultcode, server";=0A=
=0A=
select: "bytes BY client, date, hierarchycode, hour, httpstatus, method, =
mime, peer, protocol, resultcode, server";=0A=
=0A=
=0A=
# Fields provided by the 'Bind9' parser:=0A=
# Aggregations:=0A=
# - requests=0A=
# Keyfields:=0A=
# - date=0A=
# - hour=0A=
# - client=0A=
# - query=0A=
# - class=0A=
# - type=0A=
# - severity=0A=
# - category=0A=
=0A=
# Fields provided by the 'Common' parser:=0A=
# Aggregations:=0A=
# - bytes=0A=
# - requests=0A=
# Keyfields:=0A=
# - date=0A=
# - file=0A=
# - host=0A=
# - hour=0A=
# - status=0A=
# - tld=0A=
=0A=
# Fields provided by the 'Ncsa' parser:=0A=
# Aggregations:=0A=
# - bytes=0A=
# - requests=0A=
# Keyfields:=0A=
# - browser=0A=
# - date=0A=
# - file=0A=
# - host=0A=
# - hour=0A=
# - os=0A=
# - referer=0A=
# - status=0A=
# - tld=0A=
# - user=0A=
# - virtualhost=0A=
=0A=
# Fields provided by the 'Squid' parser:=0A=
# Aggregations:=0A=
# - bytes=0A=
# - elapsed=0A=
# - requests=0A=
# Keyfields:=0A=
# - client=0A=
# - date=0A=
# - hierarchycode=0A=
# - hour=0A=
# - httpstatus=0A=
# - method=0A=
# - mime=0A=
# - peer=0A=
# - protocol=0A=
# - resultcode=0A=
# - server=0A=
=0A=
# Fields provided by the 'Xferlog' parser:=0A=
# Aggregations:=0A=
# - bytes=0A=
# - count=0A=
# Keyfields:=0A=
# - host=0A=
# - user=0A=
# - access_mode=0A=
# - date=0A=
# - hour=0A=
# - file=0A=
# - completion_status=0A=
# - direction=0A=
# - transfer_type=0A=
# - transfer_time=0A=
# - special_action=0A=
=0A=
# Fields provided by the 'Netacct' parser:=0A=
# Please check/edit netacct.config, too!!=0A=
# (EXPERIMENTAL!)=0A=
# Aggregations:=0A=
# - bytes=0A=
# - packetcount=0A=
# - connections=0A=
# Keyfields:=0A=
# - date=0A=
# - destination=0A=
# - destinationport=0A=
# - hour=0A=
# - interface=0A=
# - month=0A=
# - protocol=0A=
# - source=0A=
# - sourceport=0A=
=0A=
##################################################=0A=
# filtering #=0A=
#---=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D----------------------------------#=0A=
# These options adjust filtering data which #=0A=
# appear in reports. #=0A=
##################################################=0A=
=0A=
# Wether or not yaala shall try to lookup domain names of ip adresses.=0A=
# Set to 'true' or 'false'. Default is not to.=0A=
#reverse_lookup: 'true';=0A=
=0A=
# Sets how many subdomains of a host should be displayed. "1" means only=0A=
# the domain (plus the top-level domain), e.g. "example.com", "2" would =
be=0A=
# "subdomain.example.com". Set zero to get the full length of a hostname.=0A=
# Defaults to "1"=0A=
#host_width: 1;=0A=
=0A=
##################################################=0A=
# HTML #=0A=
#---=3D=3D=3D=3D=3D=3D---------------------------------------#=0A=
# These options affects html files generation, #=0A=
# mostly - the HEAD section. #=0A=
##################################################=0A=
=0A=
# If u're going to browse html pages from FILES=0A=
# rather then via http AND on OS with another=0A=
# default charset, specify charset of your html=0A=
# pages to put into META http-equiv tag.=0A=
# With webserver, proper charset SHOULD be passed =0A=
# in http header by server.=0A=
# Default is 'iso-8859-1'.=0A=
#html_charset: iso-8859-1;=0A=
=0A=
# URL to css file with style definition for=0A=
# report pages. Goes linked it from html head.=0A=
# You may put here an url or path to other css file,=0A=
# (maybe - site-wide or reports-wide)=0A=
# default is 'style.css' (should be copied where reports lie)=0A=
#html_stylesheet: '/default.css';=0A=
#html_stylesheet: '/yaala-reports/style.css';=0A=
html_stylesheet: 'style.css';=0A=
=0A=
# The following two options control the size of the graphs generated.=0A=
graph_height: 250;=0A=
graph_width: 500;=0A=
------=_NextPart_000_0211_01C30F2D.803CF790--