I've just found there is a duplicate IP, on my DataCenter (not from my servers, it seems like someone over there is using one of my ips).
I turned off all the interfaces, and pinged all the ips on that server, one of them answered a ping (when the interface was down !).
I think it can cause the kernel panic error, when they are both answering tcp traffic (my server and the other machine) at the same time.
In the meantime, I turned off the interface on my server that uses that ip so all the rest could work, I will update the post when I will
be sure the problem was solved.