[yaala] So, lets actually do something...

[Florian Forster]

--vkogqOf2sHV7VnPd
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Hi there ;)

ok, the computer I'm reading mails on is back online.

On Sat, Nov 30, 2002 at 09:09:06PM +0600, qmax wrote:
> Truely saying, my version of yaala is unconsistent:
> - very nonoptimal memory usage;

Ok, this is kind of a two sided blade: On the one hand, we _need_ to find
a good way to store data befor we actually get to work (anything else is
just impractical), on the other hand: some issues may not be expected to
show up or may rise with new modules being written.

In particular it would be very interesting to know wether
  $hash->{'a'}{'b'}
or
  $hash->{'a' . 'b'}
consumes more memory and which has the fater access-times.

> - complicated selection syntax;

One thing that would really make things easier was, if keyfield-selection
and selection of aggregations would be on one line. Something like:
  select: field0, field1 by sum(aggr);

Since, as soon as you have such operators the question of precedence
arises, I would suggest the following (rough overview; ignore spaces,
precedence is descending):
 ( )
 ,
 +
 /

With: ',' being the keyfield-seperator, '+' to combine fields and '/' to
relate the fields just as it was. Your meta-keys (keys filtered by their
value) something like this came to my mind:
  select: key[value] by sum(aggr);

'value' could be a regular expression and therefore being parsed by perl..

> - untractable and unenhanceable internal agregation storage;

probably related to point (1)..

> And i think, that we should chose your branch end as start point,
> redesigning it step-by step.

Sorry, what do you mean by "chose your branch end as start point"??

> The primary issues IMHO are:
> - making clean interface b/wean modules to allow them to be
>   modificable independantly;

Yes. That is mostly the parser modules and the report-generating part. All
the other modules shoudl have a well-defined interface as well, though.

> - redesigning data storage to allow further enhancement;
> - implementing another table description scheme;

Roger that. See above ;)

> - making yaala more configureable;

Yas. I wanted to move the config-parsing parts out of the main program and
change it so the config syntax would be far more flexible.

> Primary modules are:
> - setup: to parse config-options;

Erm, well, se above ;)

> - parser;
> - data: to store data;
> - report;

Ditto ;)

> The main module should allow to configure yaala from config file or
> command line.
> I thing, this should be done as --option=3Dvalue parameter.

I think it would be best to use Getopt::Long or something along this
lines. As far as I know is it part of all major perl distributions and
shouldn't nag the users..

> The setup module should then parse those options and create internal
> data structure, other then main::config.
> IMHO main::config should hold only string presentation,
> and other structure required to hold parsed parameters.
> This structure avoids need of alot of functions like
> setup::datalabels, setup::datatypes, etc (from yaala-qmax).

I think there should be a subroutine to access configuration options and
an interface for parser modules to register with that module. Other
modules (data-storage, report-generation) should then be able to query
that subroutine for configuration settings.

Also, there should be a public subroutine for parsing (i.e. "importing")
the config file, so we can easily parse a secondary config file, which
hold special settings for that particular module (parser modules mostly I
guess).

> The data module should be able to store tree-structured data pool -
> to allow specifyng aggregation for any key combination.
> I suggested [ \@keysequence, \%tree, \$exprdef ] and going to
> implement this soon.

Ok, cool :) I wonder how good this structure could be:
  $hash->{$aggr}{"$key1:$val1;$key2:$val2;...keyN:valN;"}
I put "$aggr" first because I assume that accessing a very small hash
first, and then a big one is more efficient than accessing a huge hash
first and then a (just as) very small one.

This structure makes data-retrieval very slow, but data-storage rather
fast. So the parsing-part is likely to speed up, but in turn the
page-generation process would slow down.=20

> Interface functions are
>     init() - should get all necessary from config structure
>     store(%parsed_hash)
>     retrieve(something) - to retrieve data from pool.
> This 'something' should represent reference to data in pool and be
> enough universal to avoid data-dependency b/wean report and data
> modules.

Clean, easy and functional. I like it ;)

> Later i'll post my suggestions about new selection expr.

For some random thoughts of mine see above. It's just some brainstorming
ideas that I got at 2:45 at night, so if it's all crap don't mind it
please ;)

> I guess u should also make yaala-discuss maillist - for users to
> post suggestions and bug-reports and announce it at yaala homepage.
> This is out of developement scope.

I will do that as soon as it seems to become a problem. With only two
people on the ML this is very unlikely and a second ML would be oversized
;)

Ok, good night, have fun hacking and I'll dig into a config-module
tomorrow ;)

~ octo

--=20
Florian Forster
Hacker in training
GnuPG: 0x91523C3D
http://verplant.org/
http://octo-pus.de/

#!/usr/bin/perl
$o=3Dfiihcgdjdghfdfhcggeh;
@o=3Dsplit//,q!:./eajkhn!.
q@o/rorhppt/ttveglnocto@
;for(c..o){$o=3D~s#$_#q&&.
shift@o#ge;}print$o."\n"

--vkogqOf2sHV7VnPd
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.0 (GNU/Linux)

iD8DBQE96Wohjvke9ZFSPD0RArTqAJ0Sivh7si0/8L10GwOfkSbidNX13gCfTIfB
2Iwbm6tVADm6crHJ33mB4wo=
=pkGA
-----END PGP SIGNATURE-----

--vkogqOf2sHV7VnPd--