Non-Blocking UDP Socket Server causing huge packet lost

Question

I am trying to make Non-Blocking UDP Socket

If data comes, it really become slow and data packet lost a lot. (client doesn't have problem sending data)

Is there any solution?

socket.Bind(endpoint);
// bind any incoming connection to that socket
socket.Blocking = false;
// socket non-blocking 

IPEndPoint sender = new IPEndPoint(IPAddress.Any, Dispatcher.Invoke(() => Convert.ToInt32(Port_TextBox.Text)));
// wait for an incoming connection
EndPoint tmpRemote = (EndPoint)sender;
// as soon as it gets one, wire it up to this tmpRemote

while (true)
{
    try
    {
        recv = socket.ReceiveFrom(data, ref tmpRemote);
    }
    catch (SocketException e)
    {
        continue;
    }
}