Thanks for the response, Ashvin.
I definitely agree with your first point about the SRX adjusting the MSS in the TCP SYN packets. Thanks for the reference to RFC879 as well. I've now had a read of that and I think I understand what the problem is likely to be.
Based on the reading that I've done so far, my understanding of the SRX1 behaviour when the df-bit setting on an IPsec VPN is left at default (ie. "clear) is that the SRX won't send the ICMP type 3, code 4 response to the sender, it will simply do the fragmentation needed before putting the encrypted packet into the VPN tunnel. This setting is identical on both SRX1 and SRX2, therefore I would be very suprised to see any ICMP type 3, code 4 packets being used.
What I now think is happening is that SRX2 is picking up the SYN-ACK packet being returned to the app client and adjusting the MSS in that packet to 1350, since that what the tcp-mss ipsec-vpn setting is on that SRX. This is why the SYN-ACK packet in the capture I've seen shows the adjusted MSS value, rather than the typical value of 1460.
When traffic is shifted to the internet-based VPN, the terminating device at the other end (??? Cisco) is not adjusting the MSS value on the SYN-ACK packet coming back from the app server, therefore the app client recieves an MSS of 1460 and thinks it can send larger packets. This fits with the behaviour and the packet captures that I've seen.
I could possibly affect this behaviour by changing the df-bit setting on the specific IPsec tunnel to "copy" to force SRX1 to send the ICMP type 3, code 4 response back to app client. That may be one solution that could work. However, I'm hoping to convince the other end to do the MSS adjust on the SYN-ACK packet.
Thanks again for your response - it provided me a new track to investigate.
Ben.
