"pfSense as a gateway" issues

Issue

Various networking issues might occur when pfSense is used as a "Virtual Server as a Gateway" (https://docs.onapp.com/display/53AG/Virtual+Server+as+a+Gateway). In one particular case we observed the following:

1. pfSense was installed as a guest on a KVM hypervisor.
2. pfSense was used to NAT traffic between the internal and external networks (default configuration).
3. There were no TCP/UDP connections between a client VM and the outside world when both pfSense and client VM were running on the same HV.
4. When client VM was running on the separate HV, TCP / UDP connections were slow.
5. In both cases, ICMP traffic worked as expected.

Resolution

It appears that paravirtualized drives (VirtIO in KVM; PV in XEN) are causing these issues. In order to resolve the problem, some or all of steps below must be followed:

1. Disable hardware checksum offload inside pfSense (System -> Advanced -> Networking -> "Disable hardware checksum offload"). The VM has to be rebooted in order to apply the change. In our case, this measure was sufficient to resolve a customer's problem.
2. Disable tx offloading on the hypervisor side.