Difference between revisions of "Tcpip.sys"

From ReactOS Wiki
Jump to: navigation, search
m
 
(8 intermediate revisions by one other user not shown)
Line 1: Line 1:
 +
{{DISPLAYTITLE:tcpip.sys}}
 
It's been a long time since I edited this page.  Here's where I am:
 
It's been a long time since I edited this page.  Here's where I am:
  
== What needs work==
+
== What needs work ==
 
Paste From discussion with Arty in #ReactOS
 
Paste From discussion with Arty in #ReactOS
  
 
   1) fixing irp cancellation would make the situation much better
 
   1) fixing irp cancellation would make the situation much better
 
   also, afd could use refactoring in select
 
   also, afd could use refactoring in select
 
 
 
   in drivers/network/afd/afd/main.c, sort out DestroySocket
 
   in drivers/network/afd/afd/main.c, sort out DestroySocket
 
   DestroySocket is ungraceful because we could have a mid-flight receive
 
   DestroySocket is ungraceful because we could have a mid-flight receive
Line 13: Line 13:
 
   removing this pseudo-refcount code from afd
 
   removing this pseudo-refcount code from afd
 
   that would allow the completion code in select and read to be simplified
 
   that would allow the completion code in select and read to be simplified
 
+
 
 
   2) drivers/network/tcpip/tcpip/dispatch.c <-- here's what's needed:
 
   2) drivers/network/tcpip/tcpip/dispatch.c <-- here's what's needed:
 
   I put in IRPFinish but it's not that great after all
 
   I put in IRPFinish but it's not that great after all
Line 23: Line 23:
 
   use this fact to withdraw and complete cancelled reads immediately
 
   use this fact to withdraw and complete cancelled reads immediately
 
   make the tcp worker hard close cancel-closed sockets internally
 
   make the tcp worker hard close cancel-closed sockets internally
 
+
 
 
   3) blocking recv on the packet path doesn't cancel properly either
 
   3) blocking recv on the packet path doesn't cancel properly either
 
   but it's easier to fix
 
   but it's easier to fix
 
   the main offender in the packet case is DGRemoveIRP
 
   the main offender in the packet case is DGRemoveIRP
 
   in lib/drivers/ip/transport/datagram/datagram.c
 
   in lib/drivers/ip/transport/datagram/datagram.c
 
 
 
   probably the first one to hit is tcp.c (lib/drivers/ip/transport/tcp/event.c)
 
   probably the first one to hit is tcp.c (lib/drivers/ip/transport/tcp/event.c)
 
   since fixing it won't break anything else
 
   since fixing it won't break anything else
 
   specifically, TCPSleep needs to be implemented by passing a baton between worker threads
 
   specifically, TCPSleep needs to be implemented by passing a baton between worker threads
 
   there's still some of casper's refcount detritus in a few places, also remove that
 
   there's still some of casper's refcount detritus in a few places, also remove that
 
 
  
 
== Yet Newer News ==
 
== Yet Newer News ==
Line 92: Line 90:
 
* Building and dispensing ndis packets and buffers library.
 
* Building and dispensing ndis packets and buffers library.
  
http://svn.reactos.org/svn/reactos/trunk/reactos/drivers/network/tcpip/?view=log
+
== Changelog ==
 
+
{{Code history|drivers/network/tcpip}}
[[ChangeLog-0.2.0]]
 
  
* IP addresses are now registry-configured, per-adapter ([[Vizzini]])
+
[[Category:Development]]
* DisplayTCPPacket, TCPListen prototype implementations ([[Casper Hornstrup]])
+
[[Category:Drivers]]
* DispTdiAssociateAddress, DispTdiListen, DisplayTCPHeader, DisplayTCPPacket, TCPListen, TCPiReceive, TCPReceive implementations ([[Casper Hornstrup]])
 

Latest revision as of 07:45, 15 May 2014

It's been a long time since I edited this page. Here's where I am:

What needs work

Paste From discussion with Arty in #ReactOS

 1) fixing irp cancellation would make the situation much better
 also, afd could use refactoring in select
 in drivers/network/afd/afd/main.c, sort out DestroySocket
 DestroySocket is ungraceful because we could have a mid-flight receive
 that needs to be cancelled and completed by tcpip.sys
 we should wait for that, rather than pending and shutdown that way
 removing this pseudo-refcount code from afd
 that would allow the completion code in select and read to be simplified
 2) drivers/network/tcpip/tcpip/dispatch.c <-- here's what's needed:
 I put in IRPFinish but it's not that great after all
 get rid of it
 TCP itself doesn't need recmutex if the winding of different tasks can be sorted out
 so do that gracefull in tcp.c
 remove recmutex from tcpip
 make cancellation local by allowing a tcp request to exist without a backing file object when the file object has been closed
 use this fact to withdraw and complete cancelled reads immediately
 make the tcp worker hard close cancel-closed sockets internally
 3) blocking recv on the packet path doesn't cancel properly either
 but it's easier to fix
 the main offender in the packet case is DGRemoveIRP
 in lib/drivers/ip/transport/datagram/datagram.c
 probably the first one to hit is tcp.c (lib/drivers/ip/transport/tcp/event.c)
 since fixing it won't break anything else
 specifically, TCPSleep needs to be implemented by passing a baton between worker threads
 there's still some of casper's refcount detritus in a few places, also remove that

Yet Newer News

We're basically good on functionality at this point, plus or minus some details. The biggest problem right now is stability around socket close time. I've embarked on reading some stuff and doing some thinking about IRP cancellation and completion.

New News

Well I'm making slow progress on tcpip.sys. I feel more confident with it than before after cleaning up and clearing up many small allocations and much misuse of PC(Packet)->DLComplete and kin. Now each function which sends a packet takes a callback and context from the parent, and stores it someplace guaranteed not to overlap somebody else's callback.

There's no swapping of callback info into ndis packet context area any longer. It's used by datalink/lan.c to call the neighbor, and arp back *only*.

There were a ton of ways a packet could get out to the adapter. Some were in neighbor, arp, icmp, datagram, transmit. I changed this:

  • arp and neighbor may send a packet on the wire using the interface transmit function.
  • Everyone else uses IPSendDatagram, which uses the same code path for all datagrams.
  • Neighbor takes a small hit and uses the same code path for every datagram as well.

After doing that, I have not seen any bugchecks on output, and I'm confident that any remaining errors can be spotted.

I will do more. Currently, the system stops either on a deadlock that i haven't nailed down yet or on an exc14 from ne2000, WordMode receive. I'm not sure what the bugcheck is about but I need to talk to tamlin since he seemed to think it was a qemu (and not real hardware) phenomenon.

Old News

oskittcp is now available as a separate userland library, and has an application in apps/tests/oskittcp that uses it. Our goal is to make it work right in all cases. Royce Mitchell has volunteered to work on it in userland some. How to set up to have it communicate with the special qudp is in here: Oskit TCP QUDP Setup.

TCPIP.SYS is coming along. I've separated out LAN.SYS (not checked in yet) which encapsulates access to the ndis adapters of all flavors. It also isolates TCPIP.SYS from having to know very much about media types and other things that are a pain in the semprini. I'm rewriting lan.c and arp.c in tcpip to take advantage of it.

Next task: separate the files in network into one or two userland libraries (route.a and ip.a) which will also be linkable in userland apps. This will give us almost everything that can be separated out. We will just need to finish worrying about io completion, double free, and other sorts of bugs in tcpip which may not be sorted out yet.

I rewrite AFD.SYS in the intervening time. The rewrite is very simple. No timers, no callbacks. Everything is either a TDI request or an io completion. Almost the whole thing runs at PASSIVE_LEVEL.

I've started working with oskit's TCP part, integrating it into our tcpip.sys. This will finally enable TCP, without us having to finish it ourselves. Primarily, I am focused on getting addresses, packets and information into and out of TCP. As I go, I am finding and fixing lots of bugs in tcpip, afd and ndis. As these are fixed, we'll get better results.

My ultimate goal is to be able to update the reactos sources from svn on reactos, as well as use lynx, curl and wget.

Stuff to do with oskittcp and reactos (stream of consciousness):

There is a bit of confusion regarding what the Header member of IPPacket points to. In one, and as far as I can tell, only one instance, it's used to refer to a link layer header. I'd like to avoid that somehow. Ultimately, I'd like to remove all instances of link layer header from tcpip, but need to think before leaping. It works now, in that packets make their way to oskit and are recognized and handled.

I need the information in the struct socket and CONNECTION_ENDPOINT to match. These are really the same data structure. I'm going to migrate out all of the in_pcb functions to be handled by reactos, at least.

Last night, I got the first RST packet after a SYN,SYN/ACK. The RST was because the connection wasn't found in the in_pcb hash table. I don't like that hash table very much. At the very least, I can make reactos search for a CONNECTION_ENDPOINT, then return a fake inp, or alternately, store the inp in the CONNECTION_ENDPOINT.

I've been doing lots of work trying to isolate functionality in tcpip.sys:

  • Addresses and functions that work on them.
  • Prefixes.
  • Interfaces.
  • Info -> IOCTL_TCP_SET_INFORMATION_EX

Ongoing work:

  • Verifying more of ndis.
  • Isolating things that should be libraries to tcpip.sys (list, cache management)
  • Building and dispensing ndis packets and buffers library.

Changelog

Commit history (Source code can be found in: /reactos/drivers/network/tcpip)