Clients Stop Receiving Data Suddenly

I am working on a client/server application. Everything seems to go well for a while (a few hrs, a few mins its kind of random) then suddenly clients stop receiving data. maybe 1 or two, or all of them. in the end I end up having to restart the server piece to resolve it, but I am trying to understand why its doing this. I've added try catches around the send piece, but it doesnt error out. When the app fires, the listener is started on its own thread. When a new connection is made, that connection is passed to a class that handles all of my connection data(send/receive, responding to commands, etc). The class is started in its own thread, so each connection is running in its own memory space. This issue seems to happen with 1 client connected, or 20, it doesn't really seem to matter.

This causes other issues, like a customer who thinks he is disconnected, will shut down his app, which is supposed to send a command to close the connection on the server side which it does not., so now I have several "closed" connections still open on the server, which is obviously also bad. Another thing I dont get is that it worked fine when I was developing the pieces on my dev box, so I dont get why I have all of these "timeout" issues now. (I put that in quotes, because the timeout period is the default of 10 mins, and I xmit data, at LEAST every minute when the server pushes data updates to all clients)

Here is some of the relevant code
vb
'Command Object, this holds all of the command information being passed back and forth to tell the server what to do, and as well as hold the response.
'The strCMD is the command, strParms is the parameters of the command, obj is ANY object I want to send along with this information
<Serializable()> Public Structure CommandData
Public strCMD As String
[Code] .....