Various networking issues might occur when pfSense is used as a "Virtual Server as a Gateway" (https://docs.onapp.com/display/53AG/Virtual+Server+as+a+Gateway). In one particular case we observed the following:
- pfSense was installed as a guest on a KVM hypervisor.
- pfSense was used to NAT traffic between the internal and external networks (default configuration).
- There were no TCP/UDP connections between a client VM and the outside world when both pfSense and client VM were running on the same HV.
- When client VM was running on the separate HV, TCP / UDP connections were slow.
- In both cases, ICMP traffic worked as expected.
It appears paravirtualized drives (VirtIO in KVM; PV in XEN) are causing these issues. In order to resolve the problem, some or all of steps below must be followed:
- Disable hardware checksum offload inside pfSense (System -> Advanced -> Networking -> "Disable hardware checksum offload"). The VM has to be rebooted in order to apply the change. In our case, this measure was sufficient to resolve a customer's problem.
- Disable tx offloading on the hypervisor side.