I have checked the duplex and speed mode. Here are the following
ge-0/0/0: Full duplex, speed 100
ge-0/0/1: Full duplex, speed 1000
ge-3/0/0: Full duplex, speed 100
ge-3/0/1: Full duplex, speed 1000
There is no CRC error on the physcial interface. However, I found out one thing is that, once I turn off the UTM profile in the Fortinet Firewall. Everything seems to be working fine. From here I suspect it has something to do with the MTU size or Session size that cause the problem as even when the UTM profile was turned on in the Fortinet Firewall. I can ping 8.8.8.8 or www.google.com with no issue. The only issue is just browsing. ( Whether it is http or https ).