Dear anyone,
The master control plane switches from Node1 to Node0. The master data plane is on Node0. Node0 traffic abnormality occurred.
SRX650 [12.3X48-D30.7];
Node0 found some log,
Dec 30 00:26:48 2016 : %PFE-3: PFEMAN: mastership change is found. from secondary to primary. Dec 30 00:26:48 2016 : %PFE-3: PFEMAN: Master socket closed Dec 30 00:26:48 2016 : %PFE-3: PFEMAN disconnected; PFEMAN socket closed abruptly Dec 30 00:26:48 2016 : %PFE-3: CMLC: Master closed connection (errno=0) Dec 30 00:26:48 2016 : %PFE-0: CMLC: Going disconnected; Routing engine chassis socket closed abruptly Dec 30 00:26:48 2016 : %PFE-5: Routing engine PFEMAN reconnection succeeded after 1 tries Dec 30 00:26:48 2016 : %PFE-5: PFEMAN master RE reconnection made Dec 30 00:26:48 2016 : %PFE-3: CMLC: Retrying master connection, attempt 1 to 0x1300001 Dec 30 00:26:48 2016 : %PFE-3: CMLC: Lost contact with master routing engine Dec 30 00:26:48 2016 : %PFE-3: CMLC: Forwarding will cease in 4 minutes, 59 seconds -----> What does this mean, abort forwarding? Dec 30 00:26:48 2016 : %PFE-3: usp_trace_ipc_disconnect:Trace client disconnected. Attempting to reconnect Dec 30 00:26:48 2016 : %PFE-3: usp_trace_ipc_reconnect:USP trace client cannot reconnect to server Dec 30 00:26:48 2016 : %PFE-3: PFEMAN: unknown timer type 20
Dec 30 00:27:02 2016 /kernel: %KERN-3: if_pfe_peek_peer_info: Socket 0xc4be8000 error 64 Dec 30 00:27:02 2016 /kernel: %KERN-3: pfe_listener_common_task: Error. Socket 0xc4be8000 closed. Dec 30 00:27:07 2016 : %PFE-3: CMLC: Retrying master connection, attempt 50 to 0x1300001 Dec 30 00:27:13 2016 flowd_octeon_hm: %USER-3: flowd_srx_i2c_read: Reading i2c data, dev 0x77 group 0x3 (ret = 1) [Resource temporarily unavailable] retry attempt 2 Dec 30 00:27:19 2016 flowd_octeon_hm: %USER-3: flowd_srx_i2c_write: failed writing device at offset 0 (ret = -1) [Resource temporarily unavailable]
Dec 30 00:27:16 2016 eventd: %SYSLOG-3: sendto: No buffer space available ……………………6 Dec 30 00:27:46 2016 snmpd[92257]: %DAEMON-3-SNMPD_SEND_FAILURE: trap_io_send_trap_now: send to (10.0.19.37) failure: No buffer space available Dec 30 00:27:46 2016 snmpd[92257]: %DAEMON-3-SNMPD_SEND_FAILURE: trap_io_send_trap_now: send to (10.0.19.58) failure: No buffer space available
Dec 30 00:27:26 2016 dcd[92259]: %DAEMON-3: get_fpc_i2cid_from_ifd: Cannot determine FPC i2cidDec 30 00:27:38 2016 /kernel: %KERN-1-STP: STP IPC op 1 (ForwardingState) failed, err 1 (Unknown) ----->What does it mean ? Dec 30 00:27:38 2016 /kernel: %KERN-1-STP: STP IPC op 1 (ForwardingState) failed, err 1 (Unknown) Dec 30 00:27:38 2016 : %PFE-3: CMLC: Retrying master connection, attempt 100 to 0x1300001 Dec 30 00:27:43 2016 : %PFE-3: CMLC: Clearing disconnected; RE connection re-established Dec 30 00:43:58 2016 : %PFE-3: usp_ipc_server_send: failed to send message via ipc pipe Dec 30 00:43:59 2016 : %PFE-3: usp_ipc_server_msg_handler: failed to send eom message
Dec 30 00:27:20 2016 /kernel: %KERN-4: ge-0/0/0:tag-protocol-id not configurable on this interface Dec 30 00:27:20 2016 /kernel: %KERN-4: ge-0/0/1:tag-protocol-id not configurable on this interface ………… Dec 30 00:27:21 2016 /kernel: %KERN-5-KERN_LACP_INTF_STATE_CHANGE: lacp_update_state_userspace: cifd ge-1/0/4 - DETACHED state - will not carry traffic Dec 30 00:27:21 2016 : %PFE-3: JBCM(1/0):jbcm_sfp_eeprom_read: write to i2c switch failed Dec 30 00:27:21 2016 /kernel: %KERN-4: ge-1/0/6:tag-protocol-id not configurable on this interface Dec 30 00:27:21 2016 : %PFE-3: JBCM(1/0):jbcm_sfp_eeprom_read: write to i2c switch failed Dec 30 00:27:21 2016 : %PFE-3: JBCM(1/0):jbcm_sfp_eeprom_read: write to i2c switch failed Dec 30 00:27:21 2016 : %PFE-3: CMLC: Lost contact with master routing engine Dec 30 00:27:21 2016 : %PFE-3: CMLC: Forwarding will cease in 4 minutes, 26 seconds Dec 30 00:27:22 2016 /kernel: %KERN-1-RT_PFE: NH IPC op 11 (DELETE NEXTHOP) failed, err 7 (Doesn't Exist) peer_class 0, peer_index 0 peer_type 10 Dec 30 00:27:22 2016 /kernel: %KERN-1-RT_PFE: NH IPC op 11 (DELETE NEXTHOP) failed, err 7 (Doesn't Exist) peer_class 0, peer_index 0 peer_type 10
What is happening ? Where is the problem ? Software or hardware ?
Who encountered a similar problem?
thank you very much