| <!DOCTYPE html> |
| |
| <html lang="en" data-content_root="./"> |
| <head> |
| <meta charset="utf-8" /> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" /> |
| |
| <title>iperf3 FAQ — iperf3 3.15 documentation</title> |
| <link rel="stylesheet" type="text/css" href="_static/pygments.css?v=fa44fd50" /> |
| <link rel="stylesheet" type="text/css" href="_static/bootstrap-sphinx.css?v=4468db6d" /> |
| <link rel="stylesheet" href="_static/bootstrap-3.3.6/css/bootstrap.min.css" type="text/css" /> |
| <link rel="stylesheet" href="_static/bootstrap-sphinx.css" type="text/css" /> |
| <link rel="stylesheet" href="_static/esnet/esnet.css" type="text/css" /> |
| <script src="_static/documentation_options.js?v=c35749de"></script> |
| <script src="_static/doctools.js?v=888ff710"></script> |
| <script src="_static/sphinx_highlight.js?v=dc90522c"></script> |
| <script src="_static/js/jquery-1.11.0.min.js"></script> |
| <script src="_static/js/jquery-fix.js"></script> |
| <script src="_static/bootstrap-3.3.6/js/bootstrap.min.js"></script> |
| <script src="_static/bootstrap-sphinx.js"></script> |
| <link rel="icon" href="_static/favicon.ico"/> |
| <link rel="index" title="Index" href="genindex.html" /> |
| <link rel="search" title="Search" href="search.html" /> |
| <link rel="prev" title="iperf3 Development" href="dev.html" /> |
| <meta charset='utf-8'> |
| <meta http-equiv='X-UA-Compatible' content='IE=edge,chrome=1'> |
| <meta name='viewport' content='width=device-width, initial-scale=1.0, maximum-scale=1'> |
| <meta name="apple-mobile-web-app-capable" content="yes"> |
| |
| </head><body> |
| |
| <div id="navbar" class="navbar navbar-default navbar-fixed-top"> |
| <div class="container"> |
| <div class="navbar-header"> |
| <!-- .btn-navbar is used as the toggle for collapsed navbar content --> |
| <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".nav-collapse"> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| <a class="navbar-brand" href="index.html"> |
| iperf3</a> |
| </div> |
| |
| <div class="collapse navbar-collapse nav-collapse"> |
| <ul class="nav navbar-nav"> |
| |
| <li class="dropdown globaltoc-container"> |
| <a role="button" |
| id="dLabelGlobalToc" |
| data-toggle="dropdown" |
| data-target="#" |
| href="index.html">Section <b class="caret"></b></a> |
| <ul class="dropdown-menu globaltoc" |
| role="menu" |
| aria-labelledby="dLabelGlobalToc"><ul class="current"> |
| <li class="toctree-l1"><a class="reference internal" href="news.html">iperf3 Project News</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="obtaining.html">Obtaining iperf3</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="building.html">Building iperf3</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="invoking.html">Invoking iperf3</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="dev.html">iperf3 Development</a></li> |
| <li class="toctree-l1 current"><a class="current reference internal" href="#">iperf3 FAQ</a></li> |
| </ul> |
| </ul> |
| </li> |
| |
| |
| |
| |
| |
| <li> |
| <a href="dev.html" title="Previous Chapter: iperf3 Development"> |
| <span class="glyphicon glyphicon-chevron-left"></span> |
| <span class="hidden-sm"> |
| iperf3 Development |
| </span> |
| </a> |
| </li> |
| |
| |
| <li class="divider-vertical"></li> |
| |
| <li><a href="genindex.html">Index</a></li> |
| <li><a href="https://www.es.net">ESnet</a></li> |
| |
| <li><a href="https://github.com/esnet/iperf"><img src="_static/icon-github.png" height="20" width="20"></a></li> |
| |
| |
| |
| <li class="hidden-sm"> |
| <div id="sourcelink"> |
| <a href="_sources/faq.rst.txt" |
| rel="nofollow">Source</a> |
| </div></li> |
| |
| </ul> |
| |
| |
| <form class="navbar-form navbar-right" action="search.html" method="get"> |
| <div class="form-group"> |
| <input type="text" name="q" class="form-control" placeholder="Search" /> |
| </div> |
| <input type="hidden" name="check_keywords" value="yes" /> |
| <input type="hidden" name="area" value="default" /> |
| </form> |
| |
| </div> |
| </div> |
| </div> |
| |
| <div class="container"> |
| <div class="row"> |
| <div class="col-md-3"> |
| <div id="sidebar" class="bs-sidenav" role="complementary"><ul> |
| <li><a class="reference internal" href="#">iperf3 FAQ</a></li> |
| </ul> |
| |
| </div> |
| </div> |
| <div class="col-md-9"> |
| |
| <section id="iperf3-faq"> |
| <span id="faq"></span><h1>iperf3 FAQ<a class="headerlink" href="#iperf3-faq" title="Link to this heading">¶</a></h1> |
| <dl> |
| <dt>What is the history of iperf3, and what is the difference between iperf2 and iperf3?</dt><dd><p>iperf2 was orphaned in the late 2000s at version 2.0.5, despite some |
| known bugs and issues. After spending some time trying to fix |
| iperf2’s problems, ESnet decided by 2010 that a new, simpler tool |
| was needed, and began development of iperf3. The goal was make the |
| tool as simple as possible, so others could contribute to the code |
| base. For this reason, it was decided to make the tool single |
| threaded, and not worry about backwards compatibility with |
| iperf2. Many of the feature requests for iperf3 came from the |
| perfSONAR project (<a class="reference external" href="http://www.perfsonar.net">http://www.perfsonar.net</a>).</p> |
| <p>Then in 2014, Bob (Robert) McMahon from Broadcom restarted |
| development of iperf2 (See |
| <a class="reference external" href="https://sourceforge.net/projects/iperf2/">https://sourceforge.net/projects/iperf2/</a>). He fixed many of the |
| problems with iperf2, and added a number of new features similar to |
| iperf3. iperf2.0.8, released in 2015, made iperf2 a useful tool. iperf2’s |
| current development is focused is on using UDP for latency testing, as well |
| as broad platform support.</p> |
| <p>In 2023, iperf3 was modified and restructured to support |
| multi-threading, so that it uses one thread per test stream. This |
| allows it to use multiple CPU cores during tests, which in turn |
| permit it to keep up with continually increasing network link and |
| path bandwidths across the backbones of ESnet and other network |
| providers.</p> |
| <p>As of this writing (2024), both iperf2 and iperf3 are being actively |
| (although independently) developed. We recommend being familiar with |
| both tools, and use whichever tool’s features best match your needs.</p> |
| <p>A feature comparison of iperf2, iperf3, and nuttcp is available at: |
| <a class="reference external" href="https://fasterdata.es.net/performance-testing/network-troubleshooting-tools/throughput-tool-comparision/">https://fasterdata.es.net/performance-testing/network-troubleshooting-tools/throughput-tool-comparision/</a></p> |
| </dd> |
| <dt>iperf3 parallel stream performance is much less than iperf2. Why?</dt><dd><p>Versions of iperf3 before version 3.16 were all single threaded, and |
| iperf2 is multi-threaded. This could result in a performance gap |
| because iperf3 was only able to use one CPU core on a host, which |
| turned into a bottleneck when trying to do high bitrate tests |
| (faster than about 25 Gbps).</p> |
| <p>Beginning with version 3.16, iperf3 is multi-threaded, which allows |
| it to take advantage of multiple CPU cores during a test (one thread |
| per stream). iperf3 has been observed to send and receive |
| approximately 160Gbps on a 200Gbps path in a test involving multiple |
| TCP flows, with little or no tuning.</p> |
| <p>Prior to multi-threading support in iperf3, one might need to use |
| the method described <a class="reference external" href="https://fasterdata.es.net/performance-testing/network-troubleshooting-tools/iperf/multi-stream-iperf3/">here</a> |
| to achieve faster speeds.</p> |
| </dd> |
| <dt>I’m trying to use iperf3 on Windows, but having trouble. What should I do?</dt><dd><p>iperf3 is not officially supported on Windows, but iperf2 is. We |
| recommend you use iperf2.</p> |
| <p>Some people are using Cygwin to run iperf3 in Windows, but not all |
| options will work. Some community-provided binaries of iperf3 for |
| Windows exist.</p> |
| </dd> |
| <dt>How can I build a statically-linked executable of iperf3?</dt><dd><p>There are a number of reasons for building an iperf3 executable with |
| no dependencies on any shared libraries. Unfortunately this isn’t |
| quite a straight-forward process.</p> |
| <p>The steps below have nominally been tested on CentOS 7.4, but |
| can probably be adapted for use with other Linux distributions:</p> |
| <ol class="arabic simple"> |
| <li><p>If necessary, install the static C libraries; for CentOS this is |
| the <code class="docutils literal notranslate"><span class="pre">glibc-static</span></code> package.</p></li> |
| <li><p>If OpenSSL is installed, be sure that its static libraries are |
| also installed, from the <code class="docutils literal notranslate"><span class="pre">openssl-static</span></code> package.</p></li> |
| <li><p>Be sure that <code class="docutils literal notranslate"><span class="pre">lksctp-*</span></code> packages are not installed, because |
| as of this writing, there do not appear to be any static |
| libraries available for SCTP.</p></li> |
| <li><p>Configure iperf3 thusly: <code class="docutils literal notranslate"><span class="pre">configure</span> <span class="pre">"LDFLAGS=--static"</span> |
| <span class="pre">--disable-shared</span></code> These options are necessary to disable the |
| generation of shared libraries and link the executable |
| statically. For iperf-3.8 or later, configuring as <code class="docutils literal notranslate"><span class="pre">configure</span> |
| <span class="pre">--enable-static-bin</span></code> is another, shorter way to accomplish |
| this. If SCTP is installed on the system it might also be |
| necessary to pass the <code class="docutils literal notranslate"><span class="pre">--without-sctp</span></code> flag at configure |
| time.</p></li> |
| <li><p>Compile as normal.</p></li> |
| </ol> |
| <p>It appears that for FreeBSD (tested on FreeBSD 11.1-RELEASE), only |
| the last two steps are needed to produce a static executable.</p> |
| </dd> |
| <dt>How can I build on a system that doesn’t support profiled executables?</dt><dd><p>This problem has been noted by users attempting to build iperf3 for |
| Android systems, as well as some recent versions of macOS. |
| There are several workarounds. In order from least |
| effort to most effort:</p> |
| <ol class="arabic"> |
| <li><p>Beginning with iperf-3.8, profiled executables are actually not |
| built by default, so this question becomes somewhat moot. Pass |
| the <code class="docutils literal notranslate"><span class="pre">--enable-profiling</span></code> flag to <code class="docutils literal notranslate"><span class="pre">configure</span></code> to build |
| profiled executables.</p></li> |
| <li><p>In iperf-3.6 and iperf-3.7, the <code class="docutils literal notranslate"><span class="pre">--disable-profiling</span></code> flag can be |
| passed to <code class="docutils literal notranslate"><span class="pre">configure</span></code> to disable the building of profiled |
| object files and the profiled executable.</p></li> |
| <li><p>At the time the linking of the iperf3 profiled executable fails, |
| the “normal” iperf3 executable is probably already created. So if |
| you are willing to accept the error exit from the make process |
| (and a little bit of wasted work on the build host), you might |
| not need to do anything.</p></li> |
| <li><p>After the configure step, there will be a definition in |
| <code class="docutils literal notranslate"><span class="pre">src/Makefile</span></code> that looks like this:</p> |
| <div class="highlight-default notranslate"><div class="highlight"><pre><span></span>noinst_PROGRAMS = t_timer$(EXEEXT) t_units$(EXEEXT) t_uuid$(EXEEXT) \ |
| iperf3_profile$(EXEEXT) |
| </pre></div> |
| </div> |
| <p>If you edit it to look like this, it will disable the build of the profiled iperf3:</p> |
| <div class="highlight-default notranslate"><div class="highlight"><pre><span></span>noinst_PROGRAMS = t_timer$(EXEEXT) t_units$(EXEEXT) t_uuid$(EXEEXT) |
| </pre></div> |
| </div> |
| </li> |
| <li><p>Similar to item 2 above, but more permanent…if you edit |
| <code class="docutils literal notranslate"><span class="pre">src/Makefile.am</span></code> and change the line reading like this:</p> |
| <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">noinst_PROGRAMS</span> <span class="o">=</span> <span class="n">t_timer</span> <span class="n">t_units</span> <span class="n">t_uuid</span> <span class="n">iperf3_profile</span> |
| </pre></div> |
| </div> |
| <p>To look like this:</p> |
| <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">noinst_PROGRAMS</span> <span class="o">=</span> <span class="n">t_timer</span> <span class="n">t_units</span> <span class="n">t_uuid</span> |
| </pre></div> |
| </div> |
| <p>And then run <code class="docutils literal notranslate"><span class="pre">./bootstrap.sh</span></code>, that will regenerate the project |
| Makefiles to make the exclusion of the profiled iperf3 executable |
| permanent (within that source tree).</p> |
| </li> |
| </ol> |
| </dd> |
| <dt>I’m seeing quite a bit of unexpected UDP loss. Why?</dt><dd><p>First, confirm you are using iperf 3.1.5 or higher. There was an |
| issue with the default UDP send size that was fixed in |
| 3.1.5. Second, try adding the flag <code class="docutils literal notranslate"><span class="pre">-w2M</span></code> to increase the socket |
| buffer sizes. That seems to make a big difference on some hosts.</p> |
| </dd> |
| <dt>iperf3 UDP does not seem to work at bandwidths less than 100Kbps. Why?</dt><dd><p>You’ll need to reduce the default packet length to get UDP rates of less that 100Kbps. Try <code class="docutils literal notranslate"><span class="pre">-l100</span></code>.</p> |
| </dd> |
| <dt>TCP throughput drops to (almost) zero during a test, what’s going on?</dt><dd><p>A drop in throughput to almost zero, except maybe for the first |
| reported interval(s), may be related to problems in NIC TCP Offload, |
| which is used to offload TCP functionality to the NIC (see |
| <a class="reference external" href="https://en.wikipedia.org/wiki/TCP_offload_engine">https://en.wikipedia.org/wiki/TCP_offload_engine</a>). The goal of TCP |
| Offload is to save main CPU performance, mainly in the areas of |
| segmentation and reassembly of large packets and checksum |
| computation.</p> |
| <p>When TCP packets are sent with the “Don’t Fragment” flag set, which |
| is the recommended setting, segmentation is done by the TCP stack |
| based on the reported next hop MSS in the ICMP Fragmentation Needed |
| message. With TCP Offload, active segmentation is done by the NIC on |
| the sending side, which is known as TCP Segmentation offload (TSO) |
| or in Windows as Large Send Offload (LSO). It seems that there are |
| TSO/LSO implementations which for some reason ignore the reported |
| MSS and therefore don’t perform segmentation. In these cases, when |
| large packets are sent, e.g. the default iperf3 128KB (131,072 |
| bytes), iperf3 will show that data was sent in the first interval, |
| but since the packets don’t get to the server, no ack is received |
| and therefore no data is sent in the following intervals. It may |
| happen that after certain timeout the main CPU will re-send the |
| packet by re-segmenting it, and in these cases data will get to the |
| server after a while. However, it seems that segmentation is not |
| automatically continued with the next packet, so the data transfer |
| rate be very low.</p> |
| <p>The recommended solution in such a case is to disable TSO/LSO, at |
| least on the relevant port. See for example: |
| <a class="reference external" href="https://atomicit.ca/kb/articles/slow-network-speed-windows-10/">https://atomicit.ca/kb/articles/slow-network-speed-windows-10/</a>. If |
| that doesn’t help then “Don’t Fragment” TCP flag may be |
| disabled. See for example: |
| <a class="reference external" href="https://support.microsoft.com/en-us/help/900926/recommended-tcp-ip-settings-for-wan-links-with-a-mtu-size-of-less-than">https://support.microsoft.com/en-us/help/900926/recommended-tcp-ip-settings-for-wan-links-with-a-mtu-size-of-less-than</a>. However, |
| note that disabling the “Don’t Fragment” flag may cause other |
| issues.</p> |
| <p>To test whether TSO/LSO may be the problem, do the following:</p> |
| <ul class="simple"> |
| <li><p>If different machine configurations are used for the client and |
| server, try the iperf3 reverse mode (<code class="docutils literal notranslate"><span class="pre">-R</span></code>). If TSO/LSO is only |
| enabled on the client machine, this test should succeed.</p></li> |
| <li><p>Reduce the sending length to a small value that should not require |
| segmentation, using the iperf3 <code class="docutils literal notranslate"><span class="pre">-l</span></code> option, e.g. <code class="docutils literal notranslate"><span class="pre">-l</span> <span class="pre">512</span></code>. It |
| may also help to reduce the MTU by using the iperf3 <code class="docutils literal notranslate"><span class="pre">-M</span></code> option, |
| e.g. <code class="docutils literal notranslate"><span class="pre">-M</span> <span class="pre">1460</span></code>.</p></li> |
| <li><p>Using tools like Wireshark, identify the required MSS in the ICMP |
| Fragmentation Needed messages (if reported). Run tests with the |
| <code class="docutils literal notranslate"><span class="pre">-l</span></code> value set to 2 times the MSS and then 4 times, 6 times, |
| etc. With TSO/LSO issue in each test the throughput should be |
| reduced more. It may help to increase the testing time beyond the |
| default 10 seconds to better see the behavior (iperf3 <code class="docutils literal notranslate"><span class="pre">-t</span></code> |
| option).</p></li> |
| </ul> |
| </dd> |
| <dt>What congestion control algorithms are supported?</dt><dd><p>On Linux, run this command to see the available congestion control |
| algorithms (note that some algorithms are packaged as kernel |
| modules, which must be loaded before they can be used):</p> |
| <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">/</span><span class="n">sbin</span><span class="o">/</span><span class="n">sysctl</span> <span class="n">net</span><span class="o">.</span><span class="n">ipv4</span><span class="o">.</span><span class="n">tcp_available_congestion_control</span> |
| </pre></div> |
| </div> |
| <p>On FreeBSD, the equivalent command is:</p> |
| <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">/</span><span class="n">sbin</span><span class="o">/</span><span class="n">sysctl</span> <span class="n">net</span><span class="o">.</span><span class="n">inet</span><span class="o">.</span><span class="n">tcp</span><span class="o">.</span><span class="n">cc</span><span class="o">.</span><span class="n">available</span> |
| </pre></div> |
| </div> |
| </dd> |
| <dt>I’m using the <code class="docutils literal notranslate"><span class="pre">--logfile</span></code> option. How do I see file output in real time?</dt><dd><p>Use the <code class="docutils literal notranslate"><span class="pre">--forceflush</span></code> flag.</p> |
| </dd> |
| <dt>I’m using the –fq-rate flag, but it does not seem to be working. Why?</dt><dd><p>You need to add ‘net.core.default_qdisc = fq’ to /etc/sysctl.conf for that option to work.</p> |
| </dd> |
| <dt>I’m having trouble getting iperf3 to work on Windows, Android, etc. Where can I get help?</dt><dd><p>iperf3 only supports Linux, FreeBSD, and OSX. For other platforms we recommend using iperf2.</p> |
| </dd> |
| <dt>I managed to get a Windows executable built, but why do I get a BSOD on Windows 7?</dt><dd><p>There seems to be a bug in Windows 7 where running iperf3 from a |
| network filesystem can cause a system crash (in other words Blue |
| Screen of Death, or BSOD). This is a Windows bug addressed in kb2839149:</p> |
| <p><a class="reference external" href="https://support.microsoft.com/en-us/help/2839149/stop-error-0x00000027-in-the-rdbss-sys-process-in-windows-7-or-windows">https://support.microsoft.com/en-us/help/2839149/stop-error-0x00000027-in-the-rdbss-sys-process-in-windows-7-or-windows</a></p> |
| <p>A hotfix is available under kb2732673:</p> |
| <p><a class="reference external" href="https://support.microsoft.com/en-us/help/2732673/-delayed-write-failed-error-message-when--pst-files-are-stored-on-a-ne">https://support.microsoft.com/en-us/help/2732673/-delayed-write-failed-error-message-when–pst-files-are-stored-on-a-ne</a></p> |
| </dd> |
| <dt>Why can’t I run a UDP client with no server?</dt><dd><p>This is potentially dangerous, and an attacker could use this for a |
| denial of service attack. We don’t want iperf3 to be an attack tool.</p> |
| </dd> |
| <dt>I’m trying to use iperf3 to test a 40G/100G link…What do I need to know?</dt><dd><p>See the following pages on fasterdata.es.net:</p> |
| <ul class="simple"> |
| <li><p><a class="reference external" href="https://fasterdata.es.net/host-tuning/100g-tuning/">https://fasterdata.es.net/host-tuning/100g-tuning/</a></p></li> |
| <li><p><a class="reference external" href="https://fasterdata.es.net/performance-testing/network-troubleshooting-tools/iperf/multi-stream-iperf3/">https://fasterdata.es.net/performance-testing/network-troubleshooting-tools/iperf/multi-stream-iperf3/</a></p></li> |
| </ul> |
| </dd> |
| <dt>My receiver didn’t get all the bytes that got sent but there was no loss. Huh?</dt><dd><p>iperf3 uses a control connection between the client and server to |
| manage the start and end of each test. Sometimes the commands on |
| the control connection can be received and acted upon before all of |
| the test data has been processed. Thus the test ends with data |
| still in flight. This effect can be significant for short (a few |
| seconds) tests, but is probably negligible for longer tests.</p> |
| </dd> |
| <dt>A file sent using the <code class="docutils literal notranslate"><span class="pre">-F</span></code> option got corrupted…what happened?</dt><dd><p>The <code class="docutils literal notranslate"><span class="pre">-F</span></code> option to iperf3 is not a file transfer utility. It’s a |
| way of testing the end-to-end performance of a file transfer, |
| including filesystem and disk overheads. So while the test will |
| mimic an actual file transfer, the data stored to disk may not be |
| the same as what was sent. In particular, the file size will be |
| rounded up to the next larger multiple of the transfer block size, |
| and for UDP tests, iperf’s metadata (containing timestamps and |
| sequence numbers) will overwrite the start of every UDP packet |
| payload.</p> |
| </dd> |
| <dt>I have a question regarding iperf3…what’s the best way to get help?</dt><dd><p>Searching on the Internet is a good first step. |
| <a class="reference external" href="http://stackoverflow.com/">http://stackoverflow.com/</a> has a number of iperf3-related questions |
| and answers, but a simple query into your favorite search engine can |
| also yield some results.</p> |
| <p>There is a mailing list nominally used for iperf3 development, |
| <a class="reference external" href="mailto:iperf-dev%40googlegroups.com">iperf-dev<span>@</span>googlegroups<span>.</span>com</a>.</p> |
| <p>We discourage the use of the iperf3 issue tracker on GitHub for |
| support questions. Actual bug reports, enhancement requests, or |
| pull requests are encouraged, however.</p> |
| </dd> |
| </dl> |
| </section> |
| |
| |
| </div> |
| |
| </div> |
| </div> |
| <footer class="footer"> |
| <div class="container"> |
| <div class="row"> |
| <div class="col-md-1"> |
| <a href="http://www.es.net/"><img src="_static/ESnet_Final_Logos_All_Blue_Circle_Stamp_RGB.png" alt="[ESnet]"></a> |
| </div> |
| <div class="col-md-11"> |
| <p class="pull-right"> |
| <a href="#">Back to top</a> |
| |
| </p> |
| <p> |
| © Copyright 2014-2023, ESnet.<br/> |
| Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 7.2.6.<br/> |
| </p> |
| </div> |
| </div> |
| </div> |
| </footer> |
| </body> |
| </html> |
