The virtual irb interface require that at least one physical interface in that vlan be up for the virtual interface to come up. So likely you have all the vlans on a trunk port facing the switch. When the switch is disconnected there are no active physical interfaces so the irb interfaces all go up/down in status.
You can verify this by using
show interface terse