[liboping-changes] octo's ping library: Changes to 'master'

Florian Forster octo at verplant.org
Tue May 9 13:32:50 CEST 2017


 .travis.yml                      |   11 
 AUTHORS                          |    4 
 bindings/perl/Oping.xs           |    8 
 configure.ac                     |    9 
 src/liboping.c                   |  902 +++++++++++++++++++++------------------
 src/mans/liboping.pod            |    6 
 src/mans/oping.pod               |   13 
 src/mans/ping_iterator_count.pod |   33 +
 src/oping.c                      |   63 ++
 src/oping.h                      |    1 
 10 files changed, 617 insertions(+), 433 deletions(-)

New commits:
commit 48bdd3100dd26d2c6b7e9c3683a2e1a98fc53f2f
Author: Florian Forster <ff at octo.it>
Date:   Tue May 9 13:27:08 2017 +0200

    liboping(3), oping(1): Correct licensing information.

commit 9a548e0a9f08fe0fae6b7e99deade227d1210c99
Author: Florian Forster <ff at octo.it>
Date:   Tue May 9 08:48:30 2017 +0200

    AUTHORS: Add Luke and Hamish.

commit 545d5b99b8fbcbd03b04b08035e862e8c74cceb9
Author: Florian Forster <ff at octo.it>
Date:   Tue May 9 08:45:48 2017 +0200

    Remove empty "ToDo" file.

commit 57d629220ecdc21449f78ef0585caf7d8e22e059
Author: Florian Forster <ff at octo.it>
Date:   Tue May 9 08:41:18 2017 +0200

    src/liboping.c: Declare a variable at the shortest possible scope.

commit 9bb97adae60f9fabc9ccfcbd5211491bc1128b9c
Author: Florian Forster <ff at octo.it>
Date:   Tue May 9 08:40:33 2017 +0200

    src/liboping.c: Use sizeof(*ptr) instead of sizeof(type_t).
    
    Also remove an unnecessary cast from void* to pingobj_t*.

commit fb632fa5f0abc4e44e757a892dd28339e904b8f1
Author: Florian Forster <ff at octo.it>
Date:   Tue May 9 08:39:44 2017 +0200

    src/liboping.c: Use composite literals when calling setsockopt().

commit fa2d261c69ed631380139732617753bcbb5263e7
Author: Florian Forster <ff at octo.it>
Date:   Tue May 9 08:38:08 2017 +0200

    src/liboping.c: Don't check for NULL when calling free().
    
    free(3) handles NULL pointers correctly, so these checks are not needed.

commit 214995c1a066ae00a1a4f369abb09f475221f5ff
Merge: e6c0277 9cc4dd5
Author: Florian Forster <ff at octo.it>
Date:   Tue May 9 08:19:57 2017 +0200

    Merge remote-tracking branch 'github/pr/11'

commit e6c02774a043ab4e5c605fb1a80428decca58d54
Author: Florian Forster <ff at octo.it>
Date:   Tue May 9 08:16:26 2017 +0200

    Bump ABI version.
    
    The new symbol ping_iterator_count() was introduced in
    7950f924afbd902b506bbaedfc5f7f1bce6eacc9.
    
    Pull-Request: #23

commit e67d7327869b9e7e91cdf1971670d29c8bd22e64
Author: Hamish Coleman <hamish at zot.org>
Date:   Tue May 9 09:32:04 2017 +0800

    Add perl binding and documentation for new ping_iterator_count function

commit 8fe50ddb4a18b033f8ea63ef36393ef8b1033f64
Author: Hamish Coleman <hamish at zot.org>
Date:   Tue May 9 09:16:29 2017 +0800

    Fix up code style

commit c6b844685b14443d6ba38f2869d6d09f570848a5
Author: Hamish Coleman <hamish at zot.org>
Date:   Fri Feb 17 15:52:16 2017 +0800

    Handle duplicate hosts without visual corruption

commit 7950f924afbd902b506bbaedfc5f7f1bce6eacc9
Author: Hamish Coleman <hamish at zot.org>
Date:   Fri Feb 17 15:51:35 2017 +0800

    Add a helper to count the size of an iterator

commit eb18372a3ff1cd42c57e256b5f0cc948d7abb7f0
Author: Hamish Coleman <hamish at zot.org>
Date:   Fri Feb 17 15:42:37 2017 +0800

    Clean up screen in the case where we are adding a new row

commit 1e43ebd101e03b69931c82606635ddfd82496e24
Merge: 5e8f666 2645b3e
Author: Florian Forster <ff at octo.it>
Date:   Mon May 8 16:23:52 2017 +0200

    Merge remote-tracking branch 'github/pr/20'

commit 5e8f666c2abfbf455a284b54750dad19f97fe5fe
Author: Florian Forster <ff at octo.it>
Date:   Fri May 5 14:43:29 2017 +0200

    configure: Check if PKG_INSTALLDIR is defined before using it.

commit c355b05d6acf9aaba1b7d1a0a5cc47d5d4934bf1
Author: Florian Forster <ff at octo.it>
Date:   Fri May 5 14:29:49 2017 +0200

    Add .travis.yml

commit 9cc4dd5515940662d13590ed2e68bcb2608accc9
Author: Florian Forster <ff at octo.it>
Date:   Fri May 5 11:11:55 2017 +0200

    src/liboping.c: Ensure ping_send() returns non-zero on success.
    
    This is the documented behavior.

commit a951a70fdab6b431ce20a2ff7f2a1a0916975d08
Author: Florian Forster <ff at octo.it>
Date:   Fri May 5 11:07:48 2017 +0200

    src/liboping.c: Fix debugging statement.
    
    The dprintf() macro referred to a variable which no longer exists,
    which only surfaced when compiling with --enable-debug.

commit 1eb6f8f2eff9d3f9024a6c9e70007a5684eaa462
Author: Florian Forster <ff at octo.it>
Date:   Fri May 5 10:42:35 2017 +0200

    src/liboping.c: Refactor ping_send() further.
    
    * Give more meaningful names to the central "ptr" and "pings" variables
      (now "host_to_ping" and "pings_in_flight").
    * Remove special case for (errno == EINTR); effectively we only printed a
      different debugging message in that case.
    * Remove err_fds. We were not checking it at all.
    * Simplify the logic so only one receive or send operation happens per
      loop iteration. Previously, one IPv4 and one IPv6 operation might happen
      in the same loop iteration. The new logic always receives all replies
      before starting to send out more requests.
    * Assign the write file descriptor to its own variable to make clear that
      only file descriptor is set in the write_fds bitmask.

commit 4ef7c73e336179f61a432f087090cc48ffce3478
Author: Florian Forster <ff at octo.it>
Date:   Fri May 5 10:07:19 2017 +0200

    src/liboping.c: Start refactoring ping_send().
    
    * Avoid the unnecessary copies of obj->head (ph), obj->fd4 (fd4) and
      obj->fd6 (fd6). Assigning these to local variables suggests that the
      decoupling is necessary, which is confusing when this is not really the
      case.
    * Only scan for IPv4 and IPv6 hosts when resetting their latency and TTL
      and make sure appropriate sockets are open outside of the loop. This
      makes it easier to read and understand under which circumstances which
      socket is opened.
    * Move some variables to inside the while loop.

commit 450f754967e543d3b19a9bc7ae796e26b5565ecc
Author: Florian Forster <ff at octo.it>
Date:   Fri May 5 09:17:20 2017 +0200

    src/liboping.c: Document ping_open_socket().
    
    Also make sure that obj->errmsg is set on all error paths and evaluate
    errno as soon as possible so that calls to close() and such don't reset it.

commit 1fa5bd2529ecbbb725002d8e409216ef7d2f0776
Author: Florian Forster <ff at octo.it>
Date:   Fri May 5 09:06:17 2017 +0200

    src/liboping.c: Improve coding style.
    
    Don't have the if-condition and if-body in the same line.

commit 0432695fbfa55bbdab77304f4f25636655e9ac06
Author: Florian Forster <ff at octo.it>
Date:   Fri May 5 08:50:50 2017 +0200

    src/liboping.c: Clean up untidy whitespace.

commit 40f1574f0b5f713e44c73115553977900c9f933f
Merge: d717d41 c56d49b
Author: Florian Forster <ff at octo.it>
Date:   Fri May 5 08:43:52 2017 +0200

    Merge remote-tracking branch 'github/master' into lh/performance

commit 2645b3e8439abd58bc17d14f39d822c52c13d7be
Author: Florian Forster <ff at octo.it>
Date:   Mon Mar 20 08:09:20 2017 +0100

    src/oping.c: Coding style fixes.
    
    This also removes a "for loop initial declaration" which GCC complains
    about when not in C99 mode. *sigh*

commit 514c433ab31adfbaa11f1a18c01e33b7fe4579ba
Author: Hamish Coleman <hamish at zot.org>
Date:   Mon Mar 20 10:19:47 2017 +0800

    Use the correct macro for hostname max length

commit fd1715441f92d3607749c37e01d1fcafe78e7d9b
Author: Hamish Coleman <hamish at zot.org>
Date:   Sun Mar 19 20:13:56 2017 +0800

    Whitespace changes
    * Ensure that all touched lines use tabs for indentation
    * Ensure operators have visible spaces around them

commit 0e123c831965baaf931bf76e7b199acb3520ce1d
Author: Hamish Coleman <hamish at zot.org>
Date:   Fri Feb 17 16:02:45 2017 +0800

    Document the live add key press

commit a03647c685031c4e3ddadcced520524cf4699b58
Author: Hamish Coleman <hamish at zot.org>
Date:   Fri Feb 17 13:18:58 2017 +0800

    Remove attempt to scroll old data in main window

commit 3c19ece1649ccc3b11f729dd4c93918df40275f3
Author: Hamish Coleman <hamish at zot.org>
Date:   Fri Feb 17 13:02:47 2017 +0800

    Allow adding a new host after noping has started

commit 8291dce0cd5f40fc849c0c492a42605ccd9246f8
Author: Hamish Coleman <hamish at zot.org>
Date:   Fri Feb 17 13:02:21 2017 +0800

    Ensure that impossibly latency values do not affect stats

commit c19bb6cbb0e9e4946b1bffdd1fec7e6250bc43e5
Author: Hamish Coleman <hamish at zot.org>
Date:   Fri Feb 17 13:01:39 2017 +0800

    Remove unused variable

commit d717d41928fab9354e88d91ef1996151565012d8
Author: Luke Heberling <collectd at c-ware.com>
Date:   Sat Aug 6 14:46:30 2016 +0200

    src/liboping.c: Performance improvements.
    
    I'm finding collectd's ping plugin inadequate for host lists larger than
    a few hundred. It consumes an entire 2.8Ghz CPU and reports random false
    packet loss (Latest collectd and liboping from git.verplant.org on
    debian squeeze and lenny). This seems to be because of some
    inefficiencies in liboping. For instance, a FD is created for every
    host. Since each raw socket receives all raw packets, only one FD would
    be needed for all hosts (of the same addrfamily). Besides consuming all
    those additional FD's, each reply packet is received and processed
    separately on each FD causing O(n^2) complexity. Also for each packet
    processed on each FD, a scan of all hosts is performed causing O(n^3).
    
    So, I implemented some new features for liboping:
       * use icmp ident as "hash" for table lookup on RX
       * allocate one socket FD per addressfamily (not per host)
       * multiplex socket writes in the same select loop as reads
          * This was necessary because sometimes the first reply
             is received before last request is sent.
    
    It's working well for me in testing so far. 1-3% CPU with 1000 hosts,
    compared to 100% and useless at 500 hosts before these changes. If
    there's interest I'll update this thread with any significant changes.
    As always, changes to address style/portability/other issues that
    inhibit integration upstream will be my pleasure.




More information about the liboping-changes mailing list