From jesset at gmail.com Sun Apr 16 12:55:24 2017 From: jesset at gmail.com (Jesse Thompson) Date: Sun, 16 Apr 2017 03:55:24 -0700 Subject: [liboping] Running into some limits in the vicinity of 1024 hosts (exactly 1021+, in fact) Message-ID: I have attached a testcase so that you can explore the same phenomena. Please bear in mind that I am both rusty and relatively inexperienced with C debugging strategies, this reached beyond my gdb depth for example and appears to crash within the ping_send function anyway. ;) I have tested this on two computers. On either, testing 1021 rounds works fine: # ./testcase 1021 Testing 1021 simultaneous hosts. Testing send...[ ]... send test did not crash. Done! On 1022 or above, the first computer I tested (Debian Linux) would terminate like this: >>>>>>>>>>>>>>> # ./testcase 1022 Testing 1022 simultaneous hosts. Testing send...[ *** buffer overflow detected ***: ./testcase terminated ======= Backtrace: ========= /lib/x86_64-linux-gnu/libc.so.6(+0x731af)[0x7f0c80eae1af] /lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x37)[0x7f0c80f33aa7] /lib/x86_64-linux-gnu/libc.so.6(+0xf6cc0)[0x7f0c80f31cc0] /lib/x86_64-linux-gnu/libc.so.6(+0xf8a17)[0x7f0c80f33a17] /usr/lib/x86_64-linux-gnu/liboping.so.0(+0x1a4d)[0x7f0c811e7a4d] ./testcase[0x400b66] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7f0c80e5cb45] ./testcase[0x4008c9] ======= Memory map: ======== 00400000-00401000 r-xp 00000000 ca:01 2506811 /root/dev/pingcore_v7/testcase 00601000-00602000 rw-p 00001000 ca:01 2506811 /root/dev/pingcore_v7/testcase 016be000-01742000 rw-p 00000000 00:00 0 [heap] 7f0c80c25000-7f0c80c3b000 r-xp 00000000 ca:01 450563 /lib/x86_64-linux-gnu/libgcc_s.so.1 7f0c80c3b000-7f0c80e3a000 ---p 00016000 ca:01 450563 /lib/x86_64-linux-gnu/libgcc_s.so.1 7f0c80e3a000-7f0c80e3b000 rw-p 00015000 ca:01 450563 /lib/x86_64-linux-gnu/libgcc_s.so.1 7f0c80e3b000-7f0c80fdc000 r-xp 00000000 ca:01 450699 /lib/x86_64-linux-gnu/libc-2.19.so 7f0c80fdc000-7f0c811dc000 ---p 001a1000 ca:01 450699 /lib/x86_64-linux-gnu/libc-2.19.so 7f0c811dc000-7f0c811e0000 r--p 001a1000 ca:01 450699 /lib/x86_64-linux-gnu/libc-2.19.so 7f0c811e0000-7f0c811e2000 rw-p 001a5000 ca:01 450699 /lib/x86_64-linux-gnu/libc-2.19.so 7f0c811e2000-7f0c811e6000 rw-p 00000000 00:00 0 7f0c811e6000-7f0c811ea000 r-xp 00000000 ca:01 3424331 /usr/lib/x86_64-linux-gnu/liboping.so.0.2.10 7f0c811ea000-7f0c813e9000 ---p 00004000 ca:01 3424331 /usr/lib/x86_64-linux-gnu/liboping.so.0.2.10 7f0c813e9000-7f0c813ea000 r--p 00003000 ca:01 3424331 /usr/lib/x86_64-linux-gnu/liboping.so.0.2.10 7f0c813ea000-7f0c813eb000 rw-p 00004000 ca:01 3424331 /usr/lib/x86_64-linux-gnu/liboping.so.0.2.10 7f0c813eb000-7f0c8140b000 r-xp 00000000 ca:01 450677 /lib/x86_64-linux-gnu/ld-2.19.so 7f0c815fc000-7f0c815ff000 rw-p 00000000 00:00 0 7f0c81607000-7f0c8160b000 rw-p 00000000 00:00 0 7f0c8160b000-7f0c8160c000 r--p 00020000 ca:01 450677 /lib/x86_64-linux-gnu/ld-2.19.so 7f0c8160c000-7f0c8160d000 rw-p 00021000 ca:01 450677 /lib/x86_64-linux-gnu/ld-2.19.so 7f0c8160d000-7f0c8160e000 rw-p 00000000 00:00 0 7fff40c1b000-7fff40c3c000 rw-p 00000000 00:00 0 [stack] 7fff40d13000-7fff40d14000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] Aborted <<<<<<<<<<<<<<< The second computer would complain about too many open files: $ sudo ./testcase [sudo] password for jesse: No apparent arguments offered. Testing 2000 simultaneous hosts. Received an error on iteration 1022.: -1 (Too many open files) Until I upped the ulimit, and then it also just buffer overflowed. (incidentally, my first computer simply already had a larger ulimit.) >>>>>>>>>>>>>>> $ ulimit -n 40960 $ ulimit -n 40960 $ sudo ./testcase 1022 Testing 1022 simultaneous hosts. Testing send...[ *** buffer overflow detected ***: ./testcase terminated ======= Backtrace: ========= /lib/i386-linux-gnu/libc.so.6(+0x67377)[0xb7604377] /lib/i386-linux-gnu/libc.so.6(__fortify_fail+0x68)[0xb7694548] /lib/i386-linux-gnu/libc.so.6(+0xf5738)[0xb7692738] /lib/i386-linux-gnu/libc.so.6(+0xf743a)[0xb769443a] /usr/lib/i386-linux-gnu/liboping.so.0(+0x15ce)[0xb77545ce] ./testcase[0x80489c8] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf7)[0xb75b5637] ./testcase[0x80486e1] ======= Memory map: ======== 08048000-08049000 r-xp 00000000 09:7f 2491072 /home/jesse/dev/pingcore_v7/testcase 08049000-0804a000 r--p 00000000 09:7f 2491072 /home/jesse/dev/pingcore_v7/testcase 0804a000-0804b000 rw-p 00001000 09:7f 2491072 /home/jesse/dev/pingcore_v7/testcase 08c11000-08c72000 rw-p 00000000 00:00 0 [heap] b759c000-b759d000 rw-p 00000000 00:00 0 b759d000-b774c000 r-xp 00000000 09:7f 12589310 /lib/i386-linux-gnu/ libc-2.23.so b774c000-b774d000 ---p 001af000 09:7f 12589310 /lib/i386-linux-gnu/ libc-2.23.so b774d000-b774f000 r--p 001af000 09:7f 12589310 /lib/i386-linux-gnu/ libc-2.23.so b774f000-b7750000 rw-p 001b1000 09:7f 12589310 /lib/i386-linux-gnu/ libc-2.23.so b7750000-b7753000 rw-p 00000000 00:00 0 b7753000-b7757000 r-xp 00000000 09:7f 10748919 /usr/lib/i386-linux-gnu/liboping.so.0.2.11 b7757000-b7758000 r--p 00003000 09:7f 10748919 /usr/lib/i386-linux-gnu/liboping.so.0.2.11 b7758000-b7759000 rw-p 00004000 09:7f 10748919 /usr/lib/i386-linux-gnu/liboping.so.0.2.11 b775a000-b7776000 r-xp 00000000 09:7f 12583781 /lib/i386-linux-gnu/libgcc_s.so.1 b7776000-b7777000 rw-p 0001b000 09:7f 12583781 /lib/i386-linux-gnu/libgcc_s.so.1 b7777000-b777a000 rw-p 00000000 00:00 0 b777a000-b777c000 r--p 00000000 00:00 0 [vvar] b777c000-b777d000 r-xp 00000000 00:00 0 [vdso] b777d000-b779f000 r-xp 00000000 09:7f 12589302 /lib/i386-linux-gnu/ ld-2.23.so b779f000-b77a0000 rw-p 00000000 00:00 0 b77a0000-b77a1000 r--p 00022000 09:7f 12589302 /lib/i386-linux-gnu/ ld-2.23.so b77a1000-b77a2000 rw-p 00023000 09:7f 12589302 /lib/i386-linux-gnu/ ld-2.23.so bf8b7000-bf8d8000 rw-p 00000000 00:00 0 [stack] Aborted (core dumped) <<<<<<<<<<<<<<< Does this represent a limitation in the library, or an instance where my environment needs more room built out somewhere, like a larger stack or heap or something analogous to the ulimit? Is this a limit that I might be able to raise if I simply try to hack the liboping source and rebuild, and/or what kind of scaleability tradeoffs will I encounter if I try? For my purposes it is not strictly out of the question to limit my broad ping swaths to batches of 1,000 or so, but on the other hand I am looking to target only about 3,000 hosts as it is, so batching things sounds relatively premature to me unless there are some important scaleability concerns I need to be keeping in mind. *shrugs* Let me know your thoughts, and thanks again for building a library that all of my testing suggests is working rather awesomely in the 1,000 host range at least. :D - - Jesse -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: testcase.c Type: text/x-csrc Size: 2506 bytes Desc: not available URL: