[collectd] Transmitting data using unreliable wireless mesh networks

[Josef Liška]

Hi,
ideal solution for "noise" style loss would be some kind of FEC
(http://en.wikipedia.org/wiki/Forward_error_correction), such as
Reed–Solomon codes.

Also, check FSP (http://en.wikipedia.org/wiki/File_Service_Protocol) for
one solution of reliable file transfers across unreliable networks,
which was popular in nineties when people used to connect to the network
over dialup.

I personally think it is not a primary purpose of collectd and you will
have to solve link problem with other means.

I do not think TCP is a solution:
- it will add quite a big overhead (like 3-way handshake)
- AFAIK TCP timers are configured at OS level and not app level
- I think TCP itself does not have group ack (which is found on wifi
links to improve both reliability and throughput)


Possible easy ways:

- if you are happy with csv files on central server, you can configure
collectd to write csv files locally on openwrt and then use rsync
to periodically sync them to server. You can even use lsyncd to keep the
synchronisation process near-realtime. You will probably have to tweak
settings like timeouts, retries etc.

- openvpn has nice "keepalive" feature and will ping server every few
senconds and try to reconnect on no-reply and will be able to keep at
least few tens of packets in queue. So if your values can fit in 1
packet and interval is 10s, you can probably withstand periods of 3-5
minutes without connection.


Best regards
Josef Liška



On 05/06/2013 09:23 AM, Jan Lühr wrote:
> Hello folks,
> 
> we're using collectd to gather statistical data from OpenWRT Nodes. All data is transmitted and evaluated on a dedicated server (unicast).
> However, since we are using an unreliable wireless mesh network (freifunk) packets get lost and I'd cope with that.
> 
> By nature, losses are not distributed evenly - we'd like to cope with burst losses (no packets can be transmitted for some minutes) and noise (x% of all packets get lost no matter when).
> 
> Can you recommend strategies for doing so? 
> In theory I can think of:
> -> Using TCP for transmission  (high timeouts / retries for covering burst losses)
> -> Maintaining a backlog / slide window in collectd
> -> Transmitting data multiple times while not taking care of successful transmitted data.
> 
> What's the collectd way of doing this?
> 
> Thanks,
> Keep smiling
> yanosz
> _______________________________________________
> collectd mailing list
> collectd at verplant.org
> http://mailman.verplant.org/listinfo/collectd
>