Hi guys,

I am a little curious about defragmentation. Probably someone can explain defragmentation.

First let me start what I have done so far:

1. Created a DG, Volume (1GB) etc (testing system only)

2. Created about 1000 1MB files

3. Deleted every 2nd file

And now I am curious. When I ran fsadm -E /local (/local is the mounted test volume of course) I get the following output:

...

    Free Extents By Size
           1:          2            2:          2            4:          3
           8:          7           16:          6           32:          6
          64:          5          128:          5          256:          6
         512:          3         1024:        326         2048:          2

...

It shows clearly 326 free 1024K extents. From my understanding the file system is fragmentated now? Because when I create a large file now it would use all the 1024K extents and not one 128M extent, two 64M extent etc? Am I wrong/right?

And how can I rearrange the extents (if it is possible). I have tried various fsadm options like -e, -d and -C but without success.

BTW:

pkginfo -l VRTSvxfs

...
    PSTAMP:  5.1.103.000-5.1SP1RP3_x86-2012-09-16-VERITAS-FS-142635-12
...

uname -a
SunOS sol10 5.10 Generic_142910-17 i86pc i386 i86pc

Any suggestions?

A lot of you know that I’ve been hanging around Health IT for over 20 years.  Some of you know that I’ve been involved in IT for nearly 30 years.  A few of you may know that well before I discovered programming and IT, I worked in the professional theatre.

What none of you know is that all I learned about resilient systems and business continuity and contingency planning I learned in the theatre.  Not in IT.  I say none of you knew that (until now) because I didn’t know it until last night. Well, you’ve either quit reading by now or are ready to have me explain. So, here goes . . .

Last night I went to the theatre.  It was 2 legendary Texas singer/songwriters - - you’d know them if I named names but I won’t.  Bare stage, acoustic guitars, minimal lighting, and 2 mics.  It was an historic theatrical venue, probably under 1,000 seats on 3 levels and any decent singer or actor could fill the place with no mic but I’m already looking around for extra lights, backup sound . . . My wife, whom I met when we worked in the theatre, reminds me that they really wouldn’t need that for this kind of show.  She had already done the risk analysis.

I was thinking back to other theatrical events.  I worked my way through undergraduate studies (in theatre) as a card-carrying stagehand in a roadhouse that presented touring shows, touring groups, regional and local rentals.  In show business, just like healthcare, it is all about the audience member.  Let’s call them the patient.  In healthcare, they are looking for care for a pain, injury or ideally just to protect their health.  In show business they want to be entertained - - another way being taken care of.

So, here was my introduction to business continuity:  A national Broadway tour of “Fiddler on the Roof” was coming to our roadhouse.  It was February in North Dakota and on this particular day that meant high winds and snow.  The cast had arrived on time but the trucks carrying the sets and costumes arrived at the theatre about the time the curtain was supposed to be going up.    The local presenter called the producer in NY who said  “Hell, yes, there will be a show.”  It took about 10 minutes to decide to do this:  Bring down the front curtain, have the principal (star) do his nightclub act on the stage in front of the curtain while we did the set up behind the curtain; we cut some of the scenic elements to get the set up down to about 2 hours.  The audience got a nightclub act; the actor got a “free” tryout. The show went on.  I’m sure much union overtime was paid but 3,000 “cultural patients” left the “hospital” healthy that night.  The company was so high on adrenaline that it was a stunning performance!  While no one locally thought of it, the NY producer knew exactly what to do and doing nothing was not an option.  Contingency planning!

Resiliency and high availability was a different story.  It was one of the first shows in that roadhouse and I was working the show as an electrician.  The show was The Four Seasons.  Yes, as in Frankie Valli and The . . .  This was before Jersey Boys . . . mid-70’s.  No sets but lots of costumes and lots and lots of sound equipment.  We unloaded what looked like a complete sound set up and I thought we were ready to go to sound checks when the road manager for The Four Seasons called us all over and said “Let’s finish unloading.”  We all went back to the loading dock and there was a complete, identical, fully redundant sound system (these mixing boards and set up probably cost ~$200,000 - - back then).  We unloaded it and set it up right next to the one we had just set up.  I had to ask: “Why?”  The answer is one I’ve not ever forgotten:  “We make our money making this sound; if we can’t make this sound we don’t make money.  If it’s a different sound, people won’t pay for it.  We can’t afford to miss one note, one word.”  Let me just say the answer made perfect sense to me then . . . and it still made sense 30 years later when I was asked to lead the selection and implementation of an EMR at the IDS where I was CIO.

Doing a nightclub act while the show was being set up?  A completely redundant sound system for The Four Seasons?  These were business decisions . . . the stage hands didn’t decide, the actors didn’t decide although they were all part of the decision and had to participate.  It took more money, it took more time and effort, but all the “patients” were assured of getting the show they paid for.  They got what they wanted, needed and expected.  No one said “the computer is down” or ”we can’t do that right now”. 

Healthcare has to be able to do availability and continuity at least as well as a theatrical road house.  I can’t even think of what it means if we can’t.  Clinicians depend on the data to treat and care for patients and that does not happen on the planned schedules of IT or the CFO or the doctor.  It needs to happen no matter what - - “. . . we can’t afford to miss one note, one word.”

Remind me to tell you about giving Ray Charles’ his 40th birthday cake backstage in the Green Room . . .

Hello,

I am new to veritas. We got this error on our server and the listener failed over to passive node. There was no network issue that time also the sudo was working perfectly. Not sure what went wrong. Any help on this error is much appreciated.

2013/01/13 03:25:23 VCS INFO V-16-20002-211 Netlsnr :monitor:Monitor procedure /opt/VRT

Sagents/ha/bin/Netlsnr/LsnrTest.pl returned the output: su: Unknown id: oracle

Jithu

As you may recall, I was pretty salty about being denied access to Symantec’s Worldwide Technical Symposium and Partner Tech Engage Conference last October.

Well, I might have overreacted a teeny bit.

I was cordially invited to attend the Gartner Data Center Conference during the week of Dec. 3, 2012. With my metal shards glimmering like a thousand nebulas, attendees simply couldn't resist stopping to see me as they moved between the conference and expo halls. I imagine this is what every day feels like for a Wheel of Fortune slot machine.

It was truly an event for the high rollers. As one of the world’s leading IT research and advisory companies, Gartner assembles the A-Team of analysts to present the latest and greatest in the technology space several times each year.

Our big buzz phrase? Business continuity. And who better to tell you about it than the data center who – with the help of my partner, of course – kept a whole host of systems online while more or less BASE jumping without a parachute?

When I wasn’t showing off for conference participants, I listened in on Symantec Vice President Doug Matthews’ sessions about ways to ensure that business IT infrastructure runs smooth like buffed chrome – and keeps running that way.

The solution in this situation seems as simple as statistical analysis to me. Symantec products cover the basics of high availability, storage management, service levels, security from threats and malware, disaster recovery, and then some. You, savvy IT superstar, have probably spent enough time on this website to have figured that out already.

But you’d be surprised by how many users spend time and money to the nth degree trying to integrate a plethora of disparate point solutions. It’s like popping a fish oil capsule, munching a few cloves of raw garlic and then knocking it all back with an Ensure shake, rather than taking a single multivitamin. It’s counterintuitive and cumbersome, the exact opposite of how you envision the tech side of your operation.

Creating and implementing the business continuity package that meets your needs shouldn't feel like traversing Middle Earth on foot. Symantec’s products can be customized across both physical and virtual platforms to protect your business against unexpected setbacks, often caused by application failures, operational errors and other hardware or environmental factors.

Now you might be wondering, who or what is Sauron in this scenario? Following my incident in San Jose, I was certain that the fools in the marketing department were the only external threats that my kind had to worry about. Results from the 2012 Data Breach Investigations Report showed that 94 percent of stolen data is plundered from servers. As Doug explained, mobile threats, malware attacks, expanded target attacks and data breaches are our very formidable opponents.

Running Veritas Cluster Server is like swaddling your business’ servers in cloaks of invisibility. Symantec Critical System Protection can shield all levels of the vSphere environment and lock down when faced with targeted attacks.

Your applications and your data are safe with Symantec, so your business will keep on shining. Like me.

Hello all, what is the recommended configuration for SSIS in a SQL cluster?  If we develop the packages on a different non-clustered instance with BIDS, do I even need to install SSIS within the cluster environment?  This is on a 6.0.1 cluster environment with SFWHA.

Happy Valentine’s Day everyone! Today, we are taking our relationship with Microsoft to the next level. Symantec is announcing the new Windows Server 2012 platform support release for Veritas Storage Foundation High Availability for Windows.

With the new Windows Server 2012 platform support release, Storage Foundation HA for Windows provides online storage management, full VM level automated Disaster Recovery, and Microsoft Virtual Fiber Channel support for Hyper-V in Windows Server 2012. This allows administrators to attach an array LUN directly into the Hyper-V VM and get a shared storage model within Hyper-V VM that is very similar to how they would share storage in a physical environment. Administrators can enable traditional clustering and high availability configuration and deployment in the Hyper-V guest.

Why wait? Check out what’s new in store here: http://www.symantec.com/storage-foundation-high-availability-for-windows

The next versions of Veritas Operations Manager (VOM) and VOM Advanced (available within this year), will have exciting new features, including: 

•  Multi-tenancy management: Test the ability to create and manage groups of servers/clusters/arrays and providing different levels of visibility and operational access to those groups.
• Server or cluster management: Evaluate the usefulness of the brand new overview dashboards, tree based multi-pane layout, access to in context Storage Foundation or Veritas Cluster Server (VCS) operations and a host of other user interface enhancements.
• Storage reporting: Test the new storage overview dashboard, tree based organization of the array and switch objects and ability to get appropriate server side visibility.
• Licensing: Check out the new format for license reports.
• Virtualization discovery: Evaluate the end-to-end visibility for all major virtualization technologies that provides you a mapping from virtual machine to the array disks.
• Disaster recovery planning: Test the Recovery Plan feature that uses Virtual Business Services (VBS) and an open framework for custom scripts to execute your Disaster Recovery workflows.
• SAN fabric discovery: View your SAN infrastructure and the correlations with the discovered servers and storage arrays.
• Enhanced dashboard: Create custom dashboards for individual users.
• Topology views: See relationships between different elements of the configuration.
• Web services API: A partial implementation to get configuration details out of the VOM database as well as set user definable attributes.

The VOM and VOM Advanced 6.0 Technology Previews are available now. Customers already running either the VOM Management Server or Enterprise Server are ideal candidates to participate and provide feedback. New customers who have shown interest and been waiting for some of these features are welcome as well. There are 2 options for customers to participate:

• Option -1: Install VOM in your lab and configure your servers/clusters/arrays or use one of our preconfigured virtual machine images. Minimal installation steps required but this will give you the flexibility of testing the product per your will and for a longer time period. Also, this will allow more folks from your organization to get their hands on the product.
• Option-2: Participate in a usability test where you will engage in a Webex call and walk through an area of the product that you are interested in.

To participate, please contact the Veritas Operations Manager team. Customers will have a great opportunity to test the new features and provide feedback in order to improve and address new product developments. Moreover, customers who participate and submit feedback will receive an incentive of up to $175. 

As we've mentioned many times on these blog pages, organizations are virtualizing more of their applications for good reason.  Virtualization means lower costs for power and cooling, with greater agility.  However, there are risks in terms of virtualizing business critical applications as it relates to high availability and disaster recovery.  While the VMware tools protect the VM and the physical infrastructure from failures, they are not aware of the applications running in the virtual environment.  This is the "application blind spot" that IT organizations need to consider as they move business critical applications to virtual environments.  The Symantec high availability and disaster recovery solution, aka Veritas Cluster Server, monitors and recovers applications removing this blind spot.  To learn more about these capabilities, please visit our "Application Blind Spot" website and download analyst reports from IDC and Taneja Group.   

Hi guys, please I need your help...

Our environment right now is summarized below:

Existing DFS configuration :

1. Two clustered physical servers used as DFS server with dfs namespace \\subdomain.domain.com\public

2. there are 14 folder links to different folders hosting departmental folders and files which are subdivided into userdata and groupdata

folders.

3. At the storage level, we have voulmes (LUNs) mapped to these DFS servers from a SanMelody controlled SAN storage hosting the users data.

New Environment.

1. Vmware powered virtualized environment

2. a new dfs server (VM) to host the dfs namespace

3. Storage volumes (LUNs) will be mapped to the new DFS server (VM) from a Sansymphony-V empowered SAN storage.

I have these ideas but I am not sure if any of them will work. here are the two proposals I am thinking:

A.

1. Create a new dfs namespace root hosted by the new VM server ... \subdom.domain-name.com\public\2

2. Create an folder link with same name as the existing ones but now being hosted in the new storage volumes mapped to the new DFS server

(VM)

3. create a DFS replication link  with each folder in the existing and new DFS namespaces

4. Allow the replication to complete and consistent with security

5. Then start cutting over each department to the new namespace (\\subdomain.dom-name.com\public\2) using very good communication plan

6. Please note that we a looking at a combine 10TB of data here...

questions:

1. is it possible to have a new namespace root as mentioned above?

2. can I setup DFS-R between folder of different namesspaces withing same domain?

3. what is the best way to approach and fix this issue?

4. please help DFS gurus.

B.

1. I am looking at setting up an 'pass-through' integration between the SanMelody and Sansymphony virtual storages by presenting the

SANmelody virtual disks to a symphony-v server and then pass that volumes through and back to the original client, with this done one can

then have the ability to mirror (& migrate) the data to disk pools on the other symphony-v server.

2. Then I will use just one down time using DFSUTILS migrate the DFS root to the new DFS Server (VM) from the existing physical DFS server

but with the dfs folder link now pointing to the folders in the VMware sansymphony-V powered storage volume, which by now is hosting exact

same copies above.

Questions:

1. is it possible to achieve the above?

2. any configuration ideas please?

Now my question is how do I seemlessly migrate DFS folders from his existing  environment to  the new one without distrupting the live

users?

I am looking for solutions that are practicable from experienced DFS/DFS-R gurus, please help me if you have come accross my scenario

before?

Enterprises are increasingly looking at Flash storage to help narrow the I/O performance gaps with traditional storage. However, multiple questions arise regarding reliability, performance and data efficiency management capabilities. Get answers to all these questions in this joint webcast by Symantec and Violin Memory.

Learn how Violin Memory flash arrays combined with storage management software from Symantec:

• Optimizes the ultra-fast storage without any compromise on security, reliability or data availability
• Drives high performance and ensures high availability
• Provides advanced data management features

Speakers:
Ryan Jancaitis, Symantec Product Management
Ranga Rajagopalan, Violin Memory Product Management

REGISTER NOW!

One of the requested operations that seems to be popular at the moment is how to migrate existing RDMs to VMDKs, especially more so since Veritas Cluster Server 6.0.x now supports VMDKs .  In this two part article we shall take a look on how to achieve this with the first part highlighting how to convert RDMs to VMDKs on a standalone VM with minimal downtime using Storage Foundation. The second part will take a look at what steps need to be taken when Veritas Cluster Server is in the mix.

Let’s take a look at a typical virtual machine that has a RDM attached and walk through the steps needed to migrate the data from RDM to VMDKs

In Fig 1 a virtual machine would typically have RDMs attached to them these would be in either virtual or physical mode.

Fig 1 – RDM configuration for a virtual machine

From within the virtual machine the RDM would be seen as a disk and within the disk there would typically be a number of volumes that would contain the data for the application, the example show is for SQL 2008. A number of mount points had been created within a drive letter for the SQL instance and it is these volumes that will be live migrated to the new VMDKs.

Fig 2 – Storage Layout for SQL2008 from within a virtual machine

On the virtual machine add new VMDKs that will be used for the new placement of the application data.

Fig 3 – Adding VMDKs to the configuration of a virtual machine

Inside the virtual machine notice that the new disks will be seen, perform a rescan or the SCSI bus if required. Once Disk Signatures have been written with the Veritas Enterprise Administration UI the new disks can be added to the Dynamic Disk group which currently holds the volumes for the SQL application, “PROD” in this case.

Fig 4 –Adding VMDK disks to the Dynamic Diskgroup configuration for a virtual machine

Once the new disks are added to the diskgroup then data can be started to live migrate from the RDM disk to the new VMDK disks. This Subdisk move process will leverage a feature of Storage Foundation called SmartMove, this will assist in copying over only the writen blocks on the RDM to the new disks instead of all blocks from the volumes which a traditional block copy method whould use, this SmartMove process will reduce the data migration process especially if the writen blocks are much less then the overall disk size that’s being moved.

Fig 5 – New VMDKs ready for volume migration

The process of moving subdisks with the Veritas Enterprise Administrator is possible to use a drag and drop method for the sub disks from one disk to another but with multiple operations and also for accuracy it’s much easier to do this via CLI and create a script to do this.

Fig 6 – Moving a volume subdisk by dragging and dropping

Finding the subdisks that need moving from the CLI can be achieved by using the command vxprint as highlighted below.

C:\Users\administrator.WINDOM>vxprint


Diskgroup = PROD

TY NAME         ASSOC        KSTATE   LENGTH(KB)   PLOFFS   STATE    TUTIL0  PUT

IL0

sd Disk3-01     PROD_MNT-01   ENABLED  204800   0        -        -       -

sd Disk3-02     PROD_DATA-01   ENABLED  2097152  0        -        -       -

sd Disk3-03     PROD_REG-01   ENABLED  1024000  0        -        -       -

sd Disk3-04     PROD_DB-01   ENABLED  2097152  0        -        -       -

sd Disk3-05     PROD_LOG-01   ENABLED  1843200  0        -        -       -

sd Disk3-06     PROD_MNT-01   ENABLED  818610   204800        -        -       -


dm Harddisk1    Disk1        -        4192193  -        -        -       -

dm Harddisk2    Disk2        -        4192193  -        -        -       -

dm Harddisk3    Disk3        -        62652349 -        -        -       -

dg PROD         PROD         -        -        -        -        -       -

v  PROD_DATA    -            ENABLED  2097152  -        ACTIVE   -       -

pl PROD_DATA-01 PROD_DATA    ENABLED  2097152  -        ACTIVE   -       -

v  PROD_DB      -            ENABLED  2097152  -        ACTIVE   -       -

pl PROD_DB-01   PROD_DB      ENABLED  2097152  -        ACTIVE   -       -

v  PROD_LOG     -            ENABLED  1843200  -        ACTIVE   -       -

pl PROD_LOG-01  PROD_LOG     ENABLED  1843200  -        ACTIVE   -       -

v  PROD_MNT     -            ENABLED  1023410  -        ACTIVE   -       -

pl PROD_MNT-01  PROD_MNT     ENABLED  1023410  -        ACTIVE   -       -

v  PROD_REG     -            ENABLED  1024000  -        ACTIVE   -       -

pl PROD_REG-01  PROD_REG     ENABLED  1024000  -        ACTIVE   -       -

The subdisks that need moving are subdisks Disk3-01 to Disk3-06, these will be moved to the new disks, the move operation is carried out with the command vxsd –g(DiskGroup) mv –b Disk03-01 Harddisk1, or vxsd -gPROD mv -b Disk3-01 Harddisk1 in the example shown below.

Fig 7 –Movement complete for the first subdisk using SmartMove.

Notice the –b in the command will carry out the command in the background so that multiple moves can be carried out in one go.

Fig 8 – Subdisk volumes being migrated with SmartMove.

Now that the volumes have been migrated the RDM can be safely removed from the VM, this can be carried out without powering down the VMs too using the hotplug features of vSphere.

Fig 9 – Final Storage migration complete

That’s all the steps needed to move the data from the RDM disks to VMDKs, in the next part of this article we shall take a look at the steps needed when Veritas Cluster Server is in the mix, 

Hi,

There are 3 LUNs (450GB each) for a file system (1.3TB). Now the file system used only 25GB, is it possible to remove some LUN on line?

OS: Solaris 10

Veritas Foundation Suite: 5.0

Fstyp: Vxfs

Thanks in advance.

Hi, please send me solutions on the following topics.

The vxdclid daemon core dumps on AIX hosts with Storage Foundation 4.0 MP4

This is the output:

LABEL:          CORE_DUMP

IDENTIFIER:     A924A5FC

Date/Time:       Tue Feb  5 06:38:24 USAST 2013

Sequence Number: 14290551

Machine Id:      00C8468E4C00

Node Id:         edmerpr2

Class:           S

Type:            PERM

WPAR:            Global

Resource Name:   SYSPROC

Description

SOFTWARE PROGRAM ABNORMALLY TERMINATED

Probable Causes

SOFTWARE PROGRAM

User Causes

USER GENERATED SIGNAL

        Recommended Actions

        CORRECT THEN RETRY

Failure Causes

SOFTWARE PROGRAM

        Recommended Actions

        RERUN THE APPLICATION PROGRAM

        IF PROBLEM PERSISTS THEN DO THE FOLLOWING

        CONTACT APPROPRIATE SERVICE REPRESENTATIVE

Detail Data

SIGNAL NUMBER

          11

USER'S PROCESS ID:

              34080660

FILE SYSTEM SERIAL NUMBER

           4

INODE NUMBER

                501817

CORE FILE NAME

/var/opt/VRTSsfmh/core

PROGRAM NAME

vxdclid

LABEL:          CORE_DUMP

IDENTIFIER:     A924A5FC

Date/Time:       Tue Feb  5 06:38:24 USAST 2013

Sequence Number: 14290551

Machine Id:      00C8468E4C00

Node Id:         edmerpr2

Class:           S

Type:            PERM

WPAR:            Global

Resource Name:   SYSPROC

Description

SOFTWARE PROGRAM ABNORMALLY TERMINATED

Probable Causes

SOFTWARE PROGRAM

User Causes

USER GENERATED SIGNAL

        Recommended Actions

        CORRECT THEN RETRY

Failure Causes

SOFTWARE PROGRAM

        Recommended Actions

        RERUN THE APPLICATION PROGRAM

        IF PROBLEM PERSISTS THEN DO THE FOLLOWING

        CONTACT APPROPRIATE SERVICE REPRESENTATIVE

Detail Data

SIGNAL NUMBER

          11

USER'S PROCESS ID:

              34080660

FILE SYSTEM SERIAL NUMBER

           4

INODE NUMBER

                501817

CORE FILE NAME

/var/opt/VRTSsfmh/core

PROGRAM NAME

vxdclid

Good Day,

i need a solution on this topic below:

 VxDRV:vxdmp    Messages reported by vxdmp driver
B258EF25   0217225613 T S VxDRV:vxdmp    Messages reported by vxdmp driver
B258EF25   0217225613 T S VxDRV:vxdmp    Messages reported by vxdmp driver
 

All drives seem to be fine.

veritas:app2:/prod/u03/home/veritas>lsdev -Cc driver
fscsi0   Available 17-T1-01    FC SCSI I/O Controller Protocol Device
fscsi1   Available 18-T1-01    FC SCSI I/O Controller Protocol Device
fscsi2   Available 19-T1-01    FC SCSI I/O Controller Protocol Device
fscsi3   Available 20-T1-01    FC SCSI I/O Controller Protocol Device
iscsi0   Available             iSCSI Protocol Device
sfw0     Available             Storage Framework Module
sfwcomm0 Available 17-T1-01-FF Fibre Channel Storage Framework Comm
sfwcomm1 Available 18-T1-01-FF Fibre Channel Storage Framework Comm
sfwcomm2 Available 19-T1-01-FF Fibre Channel Storage Framework Comm
sfwcomm3 Available 20-T1-01-FF Fibre Channel Storage Framework Comm
 

 All adapters seem fine:

veritas:app2:/prod/u03/home/veritas>lsdev -Cc adapter
ent0 Available       Virtual I/O Ethernet Adapter (l-lan)
ent1 Available       Virtual I/O Ethernet Adapter (l-lan)
fcs0 Available 17-T1 Virtual Fibre Channel Client Adapter
fcs1 Available 18-T1 Virtual Fibre Channel Client Adapter
fcs2 Available 19-T1 Virtual Fibre Channel Client Adapter
fcs3 Available 20-T1 Virtual Fibre Channel Client Adapter
vsa0 Available       LPAR Virtual Serial Adapter
 

Hi,

We upgraded our Clusters from SFHA 5.1 SP2 to SFHA 6.0.1 CP3 a few days ago. Now, after this Upgrade we have an issue when we try to start the VMDg-Ressource after a Server-reboot:
When we stop all the ressources on Node1 and reboot this Node, we aren't able to start the VMDg-Ressource again on that node after the boot until we rescan/refresh the Disks in the Veritas Enterprise Administrator. I also activated the SCSI-3 Support but with no effect.
There's only this Event in the Windows Eventlog: Failed to deport cluster dynamic disk group xxxx_DG.

And this event is stored in the VEA-Log:

ID:
   V-76-58645-10236
Received:
   Montag, 18. Februar 2013 15:38 Uhr MEZ
Severity:
   Error
Source:
   vxvm
Classification:
   dynamic disk and volume class
Brief Description:
   Failed to deport cluster dynamic disk group xxxx_DG.
Details:
   Failed to deport cluster dynamic disk group.
Recommended Action:
   Please try again later.
Other Properties:
   xxxx_DG

The SFHA Nodes are W2008R2 Server VM's on VMware vSphere 5.0 Hosts. Drivers are from the latest VMware Tools.

Does anybody know this issue or has a solution?

Thanks,

Schilli

Environment

OS = SOlaris 10

sfha = 5.1 SP1

2013/02/19 02:20:55 VCS WARNING V-16-2-13139 Thread(2) Canceling thread (3)
2013/02/19 02:20:56 VCS ERROR V-16-2-13011 Thread(4) Resource(Mount): offline procedure did not complete within the expected time.
2013/02/19 02:20:56 VCS ERROR V-16-2-13063 Thread(4) Agent is calling clean for resource(Mount) because offline did not complete within the expected time.
2013/02/19 02:21:55 VCS WARNING V-16-2-13139 Thread(2) Canceling thread (4)
2013/02/19 02:21:57 VCS ERROR V-16-2-13006 Thread(5) Resource(Mount): clean procedure did not complete within the expected time.
2013/02/19 02:23:57 VCS WARNING V-16-2-13139 Thread(2) Canceling thread (5)
2013/02/19 02:23:57 VCS ERROR V-16-10001-5630 Mount:Mount:monitor:The file system on /dev/vx/dsk/XXX-DG/volphx is mounted on /XXXX but not accessible. You may need to manually unmount the file system.
2013/02/19 02:23:57 VCS ERROR V-16-2-13027 Thread(6) Resource(Mount) - monitor procedure did not complete within the expected time.
2013/02/19 02:23:57 VCS ERROR V-16-2-13077 Thread(6) Agent is unable to offline resource(Mount). Administrative intervention may be required.
2013/02/19 02:24:57 VCS WARNING V-16-2-13139 Thread(2) Canceling thread (6)
2013/02/19 02:25:58 VCS WARNING V-16-2-13139 Thread(2) Canceling thread (7)
2013/02/19 02:25:58 VCS ERROR V-16-10001-5630 Mount:Mount:monitor:The file system on /dev/vx/dsk/XXX-DG/volphx is mounted on /XXXX but not accessible. You may need to manually unmount the file system.

Engine LOG

Feb 19 00:03:21 SEC-NODE Had[3789]: [ID 702911 daemon.notice] VCS CRITICAL V-16-1-50086 CPU usage on SEC-NODE is 100%

Feb 19 01:57:31 SEC-NODE AgentFramework[4113]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13011 Thread(4) Resource(APP): offline procedure did not complete within the expected time.

Feb 19 01:57:31 SEC-NODE AgentFramework[4113]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13063 Thread(4) Agent is calling clean for resource(APP) because offline did not complete within the expected time.

Feb 19 01:57:31 SEC-NODE Had[3789]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13063 (SEC-NODE) Agent is calling clean for resource(APP) because offline did not complete within the expected time.

Feb 19 01:57:32 SEC-NODE AgentFramework[4113]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13068 Thread(4) Resource(APP) - clean completed successfully.

Feb 19 02:02:51 SEC-NODE AgentFramework[4113]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13012 Thread(5) Resource(APP): online procedure did not complete within the expected time.

Feb 19 02:02:51 SEC-NODE AgentFramework[4113]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13065 Thread(5) Agent is calling clean for resource(APP) because online did not complete within the expected time.

Feb 19 02:02:51 SEC-NODE Had[3789]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13065 (SEC-NODE) Agent is calling clean for resource(APP) because online did not complete within the expected time.

Feb 19 02:02:52 SEC-NODE AgentFramework[4113]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13068 Thread(5) Resource(APP) - clean completed successfully.

Feb 19 02:02:52 SEC-NODE AgentFramework[4113]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13071 Thread(5) Resource(APP): reached OnlineRetryLimit(0).

Feb 19 02:02:53 SEC-NODE Had[3789]: [ID 702911 daemon.notice] VCS ERROR V-16-1-10303 Resource APP (Owner: Unspecified, Group: SG) is FAULTED (timed out) on sys SEC-NODE

Feb 19 02:03:20 SEC-NODE Had[3789]: [ID 702911 daemon.notice] VCS CRITICAL V-16-1-50086 CPU usage on SEC-NODE is 100%

Feb 19 02:08:23 SEC-NODE AgentFramework[4113]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13012 Thread(6) Resource(APP): online procedure did not complete within the expected time.

Feb 19 02:08:23 SEC-NODE AgentFramework[4113]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13065 Thread(6) Agent is calling clean for resource(APP) because online did not complete within the expected time.

Feb 19 02:08:23 SEC-NODE Had[3789]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13065 (SEC-NODE) Agent is calling clean for resource(APP) because online did not complete within the expected time.

Feb 19 02:08:24 SEC-NODE AgentFramework[4113]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13068 Thread(6) Resource(APP) - clean completed successfully.

Feb 19 02:08:24 SEC-NODE AgentFramework[4113]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13071 Thread(6) Resource(APP): reached OnlineRetryLimit(0).

Feb 19 02:08:25 SEC-NODE Had[3789]: [ID 702911 daemon.notice] VCS ERROR V-16-1-10303 Resource APP (Owner: Unspecified, Group: SG) is FAULTED (timed out) on sys SEC-NODE

Feb 19 02:20:56 SEC-NODE AgentFramework[4120]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13011 Thread(4) Resource(Mount): offline procedure did not complete within the expected time.

Feb 19 02:20:56 SEC-NODE AgentFramework[4120]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13063 Thread(4) Agent is calling clean for resource(Mount) because offline did not complete within the expected time.

Feb 19 02:20:56 SEC-NODE Had[3789]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13063 (SEC-NODE) Agent is calling clean for resource(Mount) because offline did not complete within the expected time.

Feb 19 02:21:57 SEC-NODE Had[3789]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13006 (SEC-NODE) Resource(Mount): clean procedure did not complete within the expected time.

Feb 19 02:21:57 SEC-NODE AgentFramework[4120]: [ID 702911 daemon.notice] VCS ERROR V-16-2-13006 Thread(5) Resource(Mount): clean procedure did not complete within the expected time.

Feb 19 02:23:57 SEC-NODE AgentFramework[4120]: [ID 702911 daemon.notice] VCS ERROR V-16-10001-5630 Mount:Mount:monitor:The file system on /dev/vx/dsk/DISKGROUP/VOLUME is mounted on /MountPoint but not accessible. You may need to manually unmount the file system.

Hi all,

I've tried to configure the Cluster Notifier. Ok, I just needed to reconfigure it with a new IP and it was already setup. How can I test it now? I have tried to simulate a failed login and I can see the result in the engine_A.log:

2013/02/19 11:55:46 VCS WARNING V-16-1-10053 User fd does not exist
2013/02/19 11:55:49 VCS WARNING V-16-1-10053 User ds does not exist

The Notifier is configured to send mails when a Warning occurs. Is there any chance to see if the Notifier sends a mail? I have checked NotifierMngr_A.log which is empty. notifier-out_A.log shows only one single line after I have restarted the notifier:

2013/02/19 11:44:19 VCS WARNING V-16-1-17031 Notifier:activated signal 6

I'm asking because I haven't any access to the mail server it self.

Thanks,

Marek

Starting from Storage Foundation 6.0.1, one of feature that was introduced is Dynamic Reconfiguration tool.

From my point of view, this feature is extremely useful because avoiding any kind of service interruption.

Dynamic Multi-Pathing provides a Dynamic Reconfiguration tool. The Dynamic Reconfiguration tool is an interactive tool to automate dynamic reconfiguration of LUNs or HBAs. Dynamic reconfiguration includes addition, removal or replacement of LUNs, and replacement of certain HBAs, without requiring a reboot. The Dynamic Reconfiguration tool simplifies the process, so that you do not need a complex set of DMP and operating system related commands. System administrators and storage administrators may need to modify the set of LUNs provisioned to a server. You can change the LUN configuration dynamically, without performing a reconfiguration reboot on the host.
The operations are as follows:

• Removing LUNs dynamically from an existing target ID: Veritas Dynamic Multi-Pathing (DMP) provides a Dynamic Reconfiguration tool
  to simplify the removal of LUNs from an existing target ID. Each LUN is unmapped from the host. DMP issues an operating system device scan and cleans up the operating system device tree.

• Adding new LUNs dynamically to a new target ID: the Dynamic Reconfiguration tool simplifies the addition of new LUNs to an
  existing target ID. One or more new LUNs are mapped to the host by way of multiple HBA ports. An operating system device scan is issued for the LUNs to be recognized and added to DMP control.

• Replacing LUNs dynamically from an existing target ID: the Veritas Dynamic Multi-Pathing (DMP) provides a Dynamic Reconfiguration
  tool to simplify the replacement of new LUNs from an existing target ID. Each LUN is unmapped from the host.DMPissues an operating system device scan and cleans up the operating system device tree.

• Dynamic LUN expansion: many modern disk arrays allow existing LUNs to be resized. The Veritas Volume Manager (VxVM) supports dynamic LUN expansion, by providing a facility to update disk headers and other VxVM structures to match a new LUN size. The device must have a SCSI interface that is presented by a smart switch, smart array or RAID controller.

• Changing the characteristics of a LUN from the array side: some arrays provide a way to change the properties of LUNs. For example, the
  EMC Symmetrix array allows write-protected (Read-only), and read-write enabled LUNs. Before changing the properties of a LUN, you must remove the device from Veritas Volume Manager (VxVM) control.

• Replacing a host bus adapter online: Veritas Dynamic Multi-Pathing (DMP) provides a Dynamic Reconfiguration tool to simplify the removal of host bus adapters from an existing system.

For using the Dynamic Reconfiguration tool is sufficient to start the vxdiskadm utility as showed in the pictures below:

From more details, you can consult the following documents for Aix, Linux, HP-UX and Solaris, starting from the version 6.0.1 of Storage Foundation (SF):

• https://sort.symantec.com/public/documents/sfha/6.0.1/aix/productguides/html/sf_admin/
• https://sort.symantec.com/public/documents/sfha/6.0.1/linux/productguides/html/sf_admin/
• https://sort.symantec.com/public/documents/sfha/6.0.1/hp-ux/productguides/html/sf_admin/
• https://sort.symantec.com/public/documents/sfha/6.0.1/solaris/productguides/html/sf_admin/

The Veritas Operations Manager team is launching a new version middle of this year, and we want you (current or new customers) to try out the beta and give us feedback. Check out the VOM and VOM Advanced 6.0 Technology Preview post on what new features you can expect and how to participate in the beta program.

Also, the VOM Tell Your Story Contest for current VOM or VOM Advanced customers has been extended to April 30, 2013. Share your experience about VOM or VOM Advanced to earn extra Symantec Connect points. Here are the contest details.