Hi,
Yes you can definitely go from 11.4 directly to 12.3 as both are EEOL junos version, but recommend to go to 12.1X44 and then move to 12.3 as there are few security checks implemented from 12.1X44 onwards to strengthen teh Junos code.
Regards,
Guru Prasad