[ipxe-devel] TCP flow control window shrink on discard causes low throughput

[Steven Haber]

Hello!

As the subject states, I'm seeing a situation where the TCP window
shrink logic causes a connection to be throttled indefinitely. I am
downloading a ~600MB ISO image from an internal server using iPXE's
HTTP capabilities. Intermittently I will see the download slow to a
crawl. A packet capture shows the receive side decreasing the TCP
window drastically and permanently, from a reasonable size of ~150k
down to 1k. I can send the capture to you if it'd be useful. The
responsible code lives in src/net/tcp.c:tcp_discard. Below is a patch
that remedies the issue. Obviously this is hack. The correct solution
might involve capping the maximum window size decrease per discard,
and adding complimentary window size increase code to the main rx
path. This way slowdowns wouldn't be as severe and we could recover
from them.

What do you think? Thanks!

Steven Haber
Software Engineer
Qumulo, Inc.





diff --git a/src/net/tcp.c b/src/net/tcp.c
index 987cb63..3c59e46 100644
--- a/src/net/tcp.c
+++ b/src/net/tcp.c
@@ -1331,31 +1331,31 @@ static unsigned int tcp_discard ( void ) {
        struct tcp_rx_queued_header *tcpqhdr;
        uint32_t max_win;
        unsigned int discarded = 0;

        /* Try to drop one queued RX packet from each connection */
        list_for_each_entry ( tcp, &tcp_conns, list ) {
                list_for_each_entry_reverse ( iobuf, &tcp->rx_queue, list ) {

                        /* Limit window to prevent future discards */
                        tcpqhdr = iobuf->data;
                        max_win = ( tcpqhdr->seq - tcp->rcv_ack );
                        if ( max_win < tcp->max_rcv_win ) {
                                DBGC ( tcp, "TCP %p reducing maximum window "
                                       "from %d to %d\n",
                                       tcp, tcp->max_rcv_win, max_win );
-                               tcp->max_rcv_win = max_win;
+                               //tcp->max_rcv_win = max_win;
                        }

                        /* Remove packet from queue */
                        list_del ( &iobuf->list );
                        free_iob ( iobuf );

                        /* Report discard */
                        discarded++;
                        break;
                }
        }

        return discarded;
 }