Please enable javascript, or click here to visit my ecommerce web site powered by Shopify.

Community Forum > Reboot of QuantaStor CE results in FC links not initializing properly

Using 1 port QLogic ISP2432-based 4Gb FC cards on both ends with 2 cables connecting testing storage server (Dell 2950 with PERC6) to VM Host (Dell R610 using same FC cards) as a proof of concept.

Upon initial installation, could not initialize FC cards until I unplugged, waited for link down then plugged them in again, then storage would show up to VMware and all was well.

Tested doing restart of storage server, all comes up except the FC cards do not initialize, although according to /var/log/syslog it appears they do. Only way to get it to recognize and properly connect is to physically disconnect, wait for link down and then reconnect.

Contents of /var/log/syslog relevant to this post:

May 6 22:46:37 netstorage kernel: [ 114.140016] rport-3:0-0: blocked FC remote port time out: removing rport
May 6 22:46:38 netstorage kernel: [ 115.164013] rport-4:0-0: blocked FC remote port time out: removing rport

--- note that this is 2 minutes after bootup, and it appears something is timing out on the FC driver
-- waited a couple minutes longer, to no avail

May 6 22:50:05 netstorage kernel: [ 322.170903] qla2x00t(0): Disabling target mode
-- manually disabled target mode on fc card through web interface, reenabled in target mode, did not work

May 6 22:50:05 netstorage kernel: [ 322.170911] qla2x00t: Clearing targets DB for target ffff88083e03c000
May 6 22:50:05 netstorage kernel: [ 322.170926] qla2xxx 0000:08:00.0: Performing ISP abort - ha= ffff88084df886e8.
May 6 22:50:06 netstorage kernel: [ 322.940020] qla2xxx 0000:08:00.0: LIP reset occured (f801).
May 6 22:50:06 netstorage kernel: [ 322.940026] qla2x00t(0): LIP reset occured
May 6 22:50:06 netstorage kernel: [ 322.956014] qla2xxx 0000:08:00.0: LIP occured (f801).
May 6 22:50:06 netstorage kernel: [ 322.956018] qla2x00t(0): LIP occured
May 6 22:50:06 netstorage kernel: [ 322.972014] qla2xxx 0000:08:00.0: LOOP UP detected (4 Gbps).
May 6 22:50:06 netstorage kernel: [ 322.972018] qla2x00t(0): Loop up occured
May 6 22:51:01 netstorage CRON[4116]: (root) CMD (/opt/osnexus/quantastor/bin/qs_checkservice.sh)
May 6 22:51:13 netstorage kernel: [ 390.408150] qla2x00t(0): Enabling target mode
May 6 22:51:13 netstorage kernel: [ 390.408166] qla2x00t(0): session for wwn 21:00:00:1b:32:0e:22:25 (loop_id 0, s_id 0:0:1, confirmed completion supported) added
May 6 22:51:13 netstorage kernel: [ 390.408177] scst: Using security group "21:00:00:1b:32:0e:22:25" for initiator "21:00:00:1b:32:0e:22:25" (target 21:00:00:1b:32:86:cb:b5)
May 6 22:51:13 netstorage kernel: [ 390.408214] qla2xxx 0000:08:00.0: Performing ISP abort - ha= ffff88084df886e8.
May 6 22:51:13 netstorage kernel: [ 390.408647] qla2x00t(0): session for port 21:00:00:1b:32:0e:22:25 (loop ID 0) scheduled for deletion in 50 secs
May 6 22:51:14 netstorage kernel: [ 391.196017] qla2xxx 0000:08:00.0: LIP occured (f801).
May 6 22:51:14 netstorage kernel: [ 391.196024] qla2x00t(0): LIP occured
May 6 22:51:14 netstorage kernel: [ 391.212013] qla2xxx 0000:08:00.0: LOOP UP detected (4 Gbps).
May 6 22:51:14 netstorage kernel: [ 391.212016] qla2x00t(0): Loop up occured
May 6 22:52:01 netstorage CRON[4386]: (root) CMD (/opt/osnexus/quantastor/bin/qs_checkservice.sh)
May 6 22:52:03 netstorage kernel: [ 440.541503] qla2x00t(0): local session for loop_id 0 deleted
May 6 22:52:13 netstorage kernel: [ 450.524020] rport-3:0-1: blocked FC remote port time out: removing rport
-- More messing around attempting reinitializing did not help

May 6 22:52:53 netstorage kernel: [ 489.700573] qla2xxx 0000:0a:00.0: LOOP DOWN detected (2 5 0).
May 6 22:52:53 netstorage kernel: [ 489.700587] qla2x00t(1): Loop down occured
May 6 22:52:53 netstorage kernel: [ 489.703582] qla2x00t(1): LINK REINIT (loop 0xffff, subcode 0)
May 6 22:52:53 netstorage kernel: [ 489.703588] qla2x00t(1): Port logout (loop 0x0, subcode 6)
May 6 22:52:53 netstorage kernel: [ 489.703594] ***WARNING***: qla2x00t(1): Link failure detected
-- Here is where I physically disconnected

May 6 22:52:53 netstorage kernel: [ 489.703597] qla2x00t: Clearing targets DB for target ffff88083e03c1d0
May 6 22:53:01 netstorage CRON[4591]: (root) CMD (/opt/osnexus/quantastor/bin/qs_checkservice.sh)
May 6 22:53:16 netstorage kernel: [ 512.987636] qla2x00t(1): LINK REINIT (loop 0xffff, subcode 0)
-- Waited a moment then reconnected

May 6 22:53:16 netstorage kernel: [ 513.017238] qla2xxx 0000:0a:00.0: LIP occured (f801).
May 6 22:53:16 netstorage kernel: [ 513.017249] qla2x00t(1): LIP occured
May 6 22:53:16 netstorage kernel: [ 513.020802] qla2xxx 0000:0a:00.0: LOOP UP detected (4 Gbps).
May 6 22:53:16 netstorage kernel: [ 513.020816] qla2x00t(1): Loop up occured
May 6 22:53:17 netstorage kernel: [ 513.822509] qla2x00t(1): session for wwn 21:00:00:1b:32:06:e3:eb (loop_id 0, s_id 0:0:1, confirmed completion supported) added
May 6 22:53:17 netstorage kernel: [ 513.822522] scst: Using security group "21:00:00:1b:32:06:e3:eb" for initiator "21:00:00:1b:32:06:e3:eb" (target 21:00:00:1b:32:86:4b:b5)
-- Now this particular interface is working and shows up in vmware like nothing happened, the second one I did not touch and it continues to not work

Anyone have any idea what could be causing this? I know am confident the cards themselves are good, as other packaged solutions I have tried work with no problem, and am testing the waters with QuantaStor

May 6, 2015 | Registered CommenterJoel Jones

Update:

It appears that the Loop Initialization Primitive is not being called appropriately on startup or is called at the wrong time

Manually issuing lip appears to get it up and running
---------------------------------------------------------------------------------

qadmin@netstorage:~$ sudo scstadmin -issue_lip

Collecting current configuration: done.


-> Making requested changes.
-> Driver/target 'iscsi/iqn.2009-10.com.osnexus:1945e168-6f5d77c7696bc129:BronzeStorage' has no 'host' attribute, ignoring.
-> Driver/target 'iscsi/iqn.2009-10.com.osnexus:1945e168-ac73c2e6022ae71f:VeeamBackup' has no 'host' attribute, ignoring.
-> Issuing LIP on fibre channel driver/target 'qla2x00t/21:00:00:1b:32:86:4b:b5' (host4): done.
-> Issuing LIP on fibre channel driver/target 'qla2x00t/21:00:00:1b:32:86:cb:b5' (host3): done.
-> Done.

All done.
qadmin@netstorage:~$
------------------------------------------------------------------------------

I would expect that this should happen at startup. either it is failed being called from one of the init scripts or it is failing to run.

I have tested adding the command to /etc/rc.local but it does not appear to work. I even created a script in /etc/init.d and symlinked it to /etc/rc2.d with no luck either.

Any help would be appreciated, thanks!

May 7, 2015 | Registered CommenterJoel Jones

Hello Joel,

The Fiber Channel target is only supported in the Enterprise edition of QuantaStor with Platinum or Gold Licenses, we do not support the Fiber channel target feature in Community edition or Silver level licenses.

That being said, you may want to take a look at upgrading to the 3.15 QuantaStor release, we included a fix for a problem some Enterprise Edition customers were having with the FC Cards starting in Target mode after a reboot.

Thank You,
Chris Golden
OSNEXUS Support

May 7, 2015 | Registered CommenterChris Golden

Chris,

------Quote----------
The Fiber Channel target is only supported in the Enterprise edition of QuantaStor with Platinum or Gold Licenses, we do not support the Fiber channel target feature in Community edition or Silver level licenses.
------End Quote---------

Fair enough.

----Quote----------
That being said, you may want to take a look at upgrading to the 3.15 QuantaStor release, we included a fix for a problem some Enterprise Edition customers were having with the FC Cards starting in Target mode after a reboot.
-----End Quote----

I believe this is what you are referring to:

v3.15.0.7762 (May 1st 2015)
Upgrade Instructions
Click here for instructions on upgrading to QuantaStor v3.15.

ISO/DVD/USB Boot Install Image
ISO DVD image: osn_quantastor_v3.15.0.7762.iso
MD5 Hash: osn_quantastor_v3.15.0.7762.md5
Change Log
SAN / Storage Volumes

adds and updates accessTimeStamp on storage volume objects
adds CLI command to allow for updating the createByScheduleId field on storage volumes
fix to storage volume assignment by host in large grid configurations
fix for user level CHAP credentials multi-node auto-update logic
fix to T10 device descriptor format to use pool UUID rather than system UUID. Only applies to newly created storage volumes, existing volumes are unchanged.
fix to Storage Volume Utilization entries to show the volume name and date stamp in the 'name' field

***fix for FC port startup at boot time ***

fix to adding/removing a initiator IQN to/from a host to update ACLs when using host groups
fix to allow deleting storage volumes which are a member of a storage volume group


-- Just be aware that I am testing the latest version outlined in this changelog and it appears to still not work under certain circumstances. Perhaps something for your engineers to be aware of.

Thanks anyway!

May 7, 2015 | Registered CommenterJoel Jones