[ros-dev] Loopback problem

[Ge van Geldorp]

I've been tracking down a problem with ibrowser being extremely slow for me
(it took 64 sec to load http://www.reactos.org). It turns out to be a
problem associated with the loopback interface. The attached test program
(gcc -o loop.exe loop.c -lws2_32) is the minimum test program to demonstrate
the problem. It occurs when you send a small amount of data over the
loopback interface when there is no pending recv (so you send, then start a
recv for the data, then another send, then another recv, no recv waiting
while you send).

Sequence of events:
- Start first send
- TCP/IP stack queues the data at the receiving end
- First send returns
- Start first recv
- Queued data is retrieved, but it is determined that there's plenty of
space in the TCP window left, so there is no ACK sent back
- First recv returns
- Start second send
- tcp_output determines that the connection is not idle (there is some
un-ACKed data) and queues the data at the sending end
- Second send returns
- Start second recv
- Since there is no data waiting at the receiving end, recv just sits there
- An internal timer with a period of 2.5 sec expires
- The timer proc notices that there is an ACK pending
- ACK is sent back to the sending end
- Sending end determines connection is idle now and sends the queued data to
the receiving end
- Timer proc terminates and reschedules itself
- Data has now arrived at the receiving end and can be retrieved by the
waiting recv. Again no ACK is sent back
- Second recv returns
- Third send starts, queues its data at sending end and returns
- Third recv starts, has to wait 2.5 sec for the internal timer to timeout
and then returns
- etc.

Although I understand what's wrong, I have a bit of difficulty trying to
figure out how to fix it. Attached is a proposed fix, which basically
attacks the problem at the sending end by removing the check if the
connection is idle or not. With that fix the loop.c program works as I
expect and http://www.reactos.org loads in a much more reasonable 2-3 sec in
ibrowser. The problem that I have with the fix is that it's a change in code
we borrowed from the BSD stack. I can hardly imagine that a piece of
software so heavily used as the BSD stack would have such a fundamental
problem.

Any thoughts?

GvG
-------------- next part --------------
#include <stdio.h>
#include <windows.h>

#define DATASIZE 1

int
main(int argc, char *argv[])
{
  WSADATA wsaData;
  SOCKET Listen, Send, Receive;
  struct sockaddr_in ListenAddr, TalkAddr;
  int AddrLen;
  char Data[DATASIZE];
  unsigned Count;
  DWORD Time1, Time2;
  int BytesToReceive, BytesReceived;

  if (0 != WSAStartup(MAKEWORD(1, 1), &wsaData))
    {
       fprintf(stderr, "WSAStartup failed\n");
       exit(1);
    }

  Listen = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
  if (INVALID_SOCKET == Listen)
    {
      fprintf(stderr, "socket() for Listen failed\n");
      exit(1);
    }
  ListenAddr.sin_family = AF_INET;
  ListenAddr.sin_port = 0;
  ListenAddr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
  if (0 != bind(Listen, (struct sockaddr *) &ListenAddr, sizeof(struct sockaddr_in)))
    {
      fprintf(stderr, "bind() for Listen failed\n");
      closesocket(Listen);
      exit(1);
    }
  AddrLen = sizeof(struct sockaddr_in);
  if (0 != getsockname(Listen, (struct sockaddr *) &ListenAddr, &AddrLen))
    {
      fprintf(stderr, "getsockname() failed\n");
      closesocket(Listen);
      exit(1);
    }
  if (0 != listen(Listen, 1))
    {
      fprintf(stderr, "listen() failed\n");
      closesocket(Listen);
      exit(1);
    }

  Send = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
  if (INVALID_SOCKET == Send)
    {
      fprintf(stderr, "socket() for Send failed\n");
      closesocket(Listen);
      exit(1);
    }
  TalkAddr.sin_family = AF_INET;
  TalkAddr.sin_port = 0;
  TalkAddr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
  if (0 != bind(Send, (struct sockaddr *) &TalkAddr, sizeof(struct sockaddr_in)))
    {
      fprintf(stderr, "bind() for Send failed\n");
      closesocket(Send);
      closesocket(Listen);
      exit(1);
    }

  if (0 != connect(Send, (struct sockaddr *) &ListenAddr, sizeof(struct sockaddr_in)))
    {
      fprintf(stderr, "connect() failed\n");
      closesocket(Send);
      closesocket(Listen);
      exit(1);
    }
  Receive = accept(Listen, NULL, NULL);
  if (INVALID_SOCKET == Receive)
    {
      fprintf(stderr, "accept() failed\n");
      closesocket(Send);
      closesocket(Listen);
      exit(1);
    }
  closesocket(Listen);

  for (Count = 0; Count < 5; Count++)
    {
      memset(Data, Count & 0xff, DATASIZE);
      Time1 = GetTickCount();
      if (DATASIZE != send(Send, Data, DATASIZE, 0))
        {
          fprintf(stderr, "send() failed\n");
          closesocket(Receive);
          closesocket(Send);
          exit(1);
        }
      Time2 = GetTickCount();
      printf("send took %u ms\n", (unsigned) (Time2 - Time1));
      BytesToReceive = DATASIZE;
      while (0 != BytesToReceive)
        {
          BytesReceived = recv(Receive, Data + DATASIZE - BytesToReceive, BytesToReceive, 0);
          if (BytesReceived <= 0)
            {
              fprintf(stderr, "recv() failed\n");
              closesocket(Receive);
              closesocket(Send);
              exit(1);
            }
          BytesToReceive -= BytesReceived;
        }
      printf("recv took %u ms\n\n", (unsigned) (GetTickCount() - Time2));
    }

  closesocket(Receive);
  closesocket(Send);

  return 0;
}
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tcp_output.diff
Type: application/octet-stream
Size: 508 bytes
Desc: not available
Url : http://www.reactos.org/pipermail/ros-dev/attachments/20051221/48d65e0d/tcp_output.obj