Or are you talking about RDP? If your machine is vulnerable to RDP hacking, you've done something horribly wrong in your network setup. They're (presumably) all going to be connected to the internet through a NAT anyway, so opening a firewall exception to RDP LAN connections is really not any additional risk.
My particular setup has them all on the same 1GB wired ethernet switch; for most uses it's the same responsiveness as directly using the machine on its own physical hardware.
... a KVM is hardware.
Or are you talking about RDP? If your machine is vulnerable to RDP hacking, you've done something horribly wrong in your network setup. They're (presumably) all going to be connected to the internet through a NAT anyway, so opening a firewall exception to RDP LAN connections is really not any additional risk.
My particular setup has them all on the same 1GB wired ethernet switch; for most uses it's the same responsiveness as directly using the machine on its own physical hardware.