[yaala] Re: Re: Problems .....

[Florian Forster]

--PjJwPjbdn5RfZuWK
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Hi,

On Wed, Apr 30, 2003 at 03:30:59PM +0200, Federico Lombardo wrote:
> actually my squid logs are about 400 - 600 Megs large.
> The Problem happens also with a test log on 60M.

Well, ok, I never had the possibility of testing yaala with files that
big, so when you run into problems with them (yaala running longer than,
say, 10 Minutes, e.g.) try smaller files first.. If they work without
any problems then yaala is propably just not scaling very well..

> combined output problem, classic output problem and my config,

> when I use combined mode, the program finish to parse the file, and then
> "die" (cfr. Combined.txt).
Ok, the combined-output is rather easy to explain and it's more a
documentation bug I guess: With the combined output you can't combine
more than three fields (and one aggregation..) at once. The plugin
simply dies when it comes to generating the output, which is a bug. The
next version will check the number of fields before doing anything
else..

> Using classic output will use my CPU at 99% FOR EVER!!!! and the debug is
> always that rows contained in the file called "classic.txt".

Hm, it seems as if the storage-retrieval routines are very inefficent
when the number of values grows. A workaround is to remove the fields
you don't need that desperately and limit yourself to about 5 fields
(I'm guessing here) for now. I'll have a look at these routines as soon
as I can to make them faster.. Oh, by the way, using the classic report
with both, clients and servers, in one file will definitely result in
_huge_ files. The output might very well get bigger than the logfile(s)
with that configuration..

I hope this helps.. Please let me know about your experience with big
logfiles :)

Regards,
-octo

--=20
Florian octo Forster
Hacker in training
GnuPG: 0x91523C3D
http://verplant.org/

--PjJwPjbdn5RfZuWK
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE+sY+Qjvke9ZFSPD0RAtEcAJ43UTTxGz99YvcBbZsOo7HNSQ2BowCfYwIh
NIwhYQLQhCwzLJ6mQnzBZWo=
=p1yB
-----END PGP SIGNATURE-----

--PjJwPjbdn5RfZuWK--