Failed to Deploy OVF package

Failed to deploy package: File ds:///vmfs/volumes/UUID/_deviceImage-0.iso was not found

While working on setup for a Custom version of our Exchange 2013/2016 Ultimate Bootcamp today, I came across a problem which is not new, but was new to me.

A colleague had saved some VMs to OVA format for deployment in a Master vApp.  As I went to deploy them, I received the following error message:

After doing some research, I came across a couple of articles.  Neither gave me what I needed, but through combining bits and experimenting, I found a solution.

PROBLEM: 

   The VMs had ISOs attached to their CD-ROMs when the were exported to OVA format.
   The OVF file inside the OVA package has information about the attached ISO.

 

SOLUTION:

1. Uncompressed the OVA file using either TAR on Linux/ESXi or 7-Zip on Windows.
2. Located the OVF file which contains the configuration of the VM to deploy.

   
   

   
   

     

3. In the VMname.OVF file, I searched for “vmware.cdrom.iso”
4. I replaced vmware.cdrom.iso with vmware.cdrom.remotepassthrough and saved the file.
   
   
5. Calculated a new SHA1 hash for the updated VMname.OVF file using HashMyFiles (Download here: http://www.nirsoft.net/utils/hash_my_files.html)
   
   
6. Edited the VMname.mf file and replaced the SHA1 hash for the VMname.ovf with the new one & saved the file.
   
      
    
7. Deployed a new VM using the OVF file.
8. Optionally you could repackage the OVF to an OVA using either OVFTool or TAR in a host.
   tar cvf VMname.ova VMname.ovf
   tar uvf VMname.ova *.mf *.vmdk

Another option would be to simply delete the MF file.  vSphere deployment of the OVF then wouldn't do a hash check at all and the install will complete as expected.  Your choice!

Moving the VMware vCenter Server 4.x/5.x/6.0.x SQL database

A simple necessary task that isn’t so simple.  So the scenario plays out like this.  Customer is running vCenter, Update Manager and SRM databases on a SQL server that is running Windows 2003 with SQL 2005 – not extremely uncommon.  With the impending end of Windows 2003 looming we decide to migrate to Windows 2008 R2 and SQL 2008 R2.  

For the most part we followed KB 7960893 from VMware.

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=7960893

Now this KB will get you close, which is nice.  And what does close mean.  It means that we had databases up and running on the SQL server.  It means that we had ODBC connections that were connected and tested OK.  And then….It means that we couldn’t get the service to start – WTF.

So let’s look at the process and the final resolution.  First off it’s a pretty common task and I was doing this for a customer in their DR site, but it is something that everyone will run into at some time. 

Several steps in the process.

1.       Backup the databases on the old SQL server.  Simple enough go into SQL management studio right click the database and back it up.  Stop all of the VMware services on vCenter and SRM first.
2.       Copy the databases over to the new server – Really easy.
3.       Restore the databases to the new SQL server.
4.       Yeah this next step is a little harder.  Permissions.  We had to set the proper permissions.  The customer wanted to use a different service account and instead of having separate accounts for vCenter, UM, and SRM they wanted to use one account for all of them.  Not a huge deal, so whatever, let’s make it happen.  We used a local account in SQL with SQL server authentication.  We then granted dbo permissions to the vCenter, UM, and SRM databases.  You also have to give it dbo for the msdb database.  OK, all of this sounds pretty straightforward so far.
5.       Edit the vcdb.properties file to change the server name – Really easy.
6.       I didn’t update the SQL server agents just yet, because I didn’t care about performance statistics.
7.       Go to ODBC and ODBC32 on the vCenter server and change them to point to the new server and the correct database using the proper username and password.  Then test it.  BOOM, everything worked.  Start the vCenter and UM services up.  And…It failed.  WTF.

Troubleshooting Time

8.       So to do a little troubleshooting I looked in event viewer, got a whole bunch of nothing. 
9.       So what next?  I decided to start the vCenter service using the executable in standalone mode.  Open an administrative command prompt and the run vcdb –s.  Hmmm.  Interesting error.  Login failure and it was related to a DSN, which is telling me SQL login failure.  But the ODBC connections all tested positive.
10.   So I did a registry search and each of the 3 databases all have a section with a DB entry in it.  In that section I looked in there, and it still had the old entry for the name in there.  In the picture it is entry 2.  So I changed it.  DANG IT, the service still wouldn’t start.
11.   Well the line right below the username in the registry is the password hash for the old username, in the picture it is entry 3.  To change that password hash you have to go back to that administrative command prompt and run the vcdb –p command and then enter the new password twice.  And now.  BOOM the service starts.  WOO HOO.

12.   So now let’s start the Update Manager service.  It fails.  WTF.

13.   This was pretty easy.  I ran the Update Manager Utility and change the database in there and it started up fine after that.  Update Manager Utility is mentioned in KB 2011327, and it is located under Program Files (x86), because it is still a 32bit program, also have to update the ODBC32 as well.  So start the service and it worked.

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2011327






14.   OK so now 2 of the 3 databases are connected and running.  Go to the SRM server and start that service up.  DANG IT, no go.  On to troubleshoot that.

15.   Went through several troubleshooting steps, and actually worked with VMware technical support, but since nothing was really set up in SRM we just did a full uninstall/reinstall.  It wasn’t pretty and if there had been any data on there I would have definitely taken the time to get it working, but…I didn’t so this was the easy way out.

So in a nutshell, it wasn’t as easy as I thought it would be.  But it is done, consulting is never quite as easy as it seems, but I still enjoy it.

Unlocking and resetting the VMware vCenter Single Sign-On administrator password (vCenter 5.1, 5.5, 6.0)

I ran across this KB article from VMware to resolve this issue caused by a locked SSO password.  I am going to describe the solution specifically for vCenter 6.0 SSO.  For vCenter 5.1 or 5.1, you can go to the KB article link at the bottom of this page to review the other options.

---

Symptoms

• You are unable to log in to the VMware vSphere Web Client using your single sign-on (SSO) administrator credentials.
• Logging in to the vSphere Web Client using your SSO administrator credentials fails.
• The password is incorrectly entered three times (by default).
• You see the error:
  
  User account is locked. Please contact your administrator. 

---

Cause

The SSO Administrator account is automatically locked after too many failed attempts, default is three.

---

Resolution

For VMware Platform Services Controller 6.0

• Wait for 5 minutes. By default, the account lockout policy is set to unlock after 15 minutes. For more information on account lockout policies for the Platform Services Controller (PSC), see vCenter Server Password Requirements and Lockout Behavior in the vSphere Security Guide.
• Unlock the account using another session that is still logged into the PSC server or using another user account with SSO administrator privileges. 
  
  To unlock an account using another session or using another user account with SSO administrator privileges:
  1. Click Home.
  2. Click Administration.
  3. Click Single Sign-On > Users and Groups.
  4. Click the Users tab.
  5. Right-click the affected user account, such as administrator@vsphere.local, and click Unlock.
• In emergency situations or if the default policies are changed, you can also reset the password to unlock the account. 
  
  To reset the administrator@vsphere.local password on a Windows Platform Services Controller or vCenter Server with Embedded Platform Services Controller:
  1. Log in to the vCenter Server with a domain administrator account. If the Platform Services Controller is installed separate from the vCenter Server, log in to the Platform Services Controller server.
  2. Open an elevated command prompt. For more information, see Opening a command or shell prompt (1003892).
  3. Open the vdcadmintool service tool with this command:c:\> "%VMWARE_CIS_HOME%\vmdird\vdcadmintool.exe"
     This console loads:
     
     ===============================
     Please select:
     0. exit
     1. Test LDAP connectivity
     2. Force start replication cycle
     3. Reset account password
     4. Set log level and mask
     5. Set vmdir state
     ===============================
  4. Press 3 to enter the Reset account password option. 
  5. When prompted for the Account UPN, enter:
     Administrator@vSphere_Domain_Name.local
     
     By default, this is:
     
     Administrator@vSphere.local
     
     A new password is generated.
     Note: if you customized your vSphere Domain name, provide the customized domain name.
  6. Use the generated password to log in to the administrator@vSphere.local account.
  7. After the password is regenerated, log in to the vSphere Web Client and change the password.

To reset the administrator@vsphere.local password on the Platform Services Controller or vCenter Server with Embedded Platform Services Controller Appliance:

1. Log in to the vCenter Server Appliance via SSH.
2. Run this command to enable access the Bash shell:
   
   shell.set --enabled true
3. Type shell and press Enter.
4. Open the vdcadmintool service tool with this command:
   /usr/lib/vmware-vmdir/bin/vdcadmintoolThis console loads:
   ================================
   Please select:
   0. exit
   1. Test LDAP connectivity
   2. Force start replication cycle
   3. Reset account password
   4. Set log level and mask
   5. Set vmdir state
   ================================
5. Press 3 to enter the Reset account password option.
6. When prompted for the Account UPN, enter:
   Administrator@vSphere_Domain_Name.local
   
   By default, this is:
   
   Administrator@vSphere.local
   
   A new password is generated.
   
   Note: if you customized your vSphere Domain name, provide the customized domain name.
7. Use the generated password to log in to the administrator@vSphere.local account.
8. After the password is regenerated, log in to the vSphere Web Client and change the password.

---

KB 2034608: http://kb.vmware.com/kb/2034608

Top 10 vSphere 5.5 Improvements

Attending VMworld this year was very exciting, the release of their NSX product for true virtualization of the network was amongst its biggest announcements. I am looking forward to see this product in action, I believe we will see some very sold security mitigation techniques stem from this deployment, not to mention the ease of network implementation! We will discuss this in more detail in a future blog. Today we are going to look at the top 10 improvements in the upcoming release of vSphere 5.5, here they are:

1. vSphere Web Client Improvements
Why did I choose this for our #1? With the push to the web Client from VMware many of us were dealing with a slow interface that decreased our ability to get work done! I do not know about you but I really do not want to spend even an extra 15 minutes a day at work just because I am waiting! This new interface is fast in comparison and I do not feel like I am waiting, which was not the case in previous versions. This new web client has some additional filtering and navigation features and had the ability to manage the new 64TB VMDK (Yes it is really only 62TB).
2. Single Sign On (SSO) Improvements
The new release of SSO is easier to understand and use, I think you will actually like it now! It still provides a great resource for multiple authentication sources! Surprisingly it provides upgrading from 5.0 to 5.5!
3. vApp HA Additions
 vSphere 5.5 can now monitor applications you have running inside the VM! No need for a 3rd party tool now. I like the idea and in testing it works well, I am not 100% convinced yet but I will try this with clients on non-critical systems until I am confident in the results.
4. vSAN is now available
Software Defined Storage is one of the new hot topics and VMware is attempting to lead the pack, however there are other hardware vendors that have a similar implementation that has worked well. However I like where they are headed, I will need a lot more testing before I am ready to use this in any production environment but there are some details. This feature is enabled on the host and will provide redundant, supported storage across a maximum of 8 hosts. VMware states the primary use cases will be VDI, test/dev, big data, and a disaster recovery target. vSAN functionality will be sold as a standalone offering. The vSAN beta is in public beta and available at vSANbeta.com.
5. Flash Read Cache Solution Added
As expected server-side, host-based caching was a hot topic this year and VMware how their own solution. It requires local SSD on every host that has this feature enabled and support vMotion, HA, DRS and other features of vSphere. The virtual machines and applications are not aware of the Flash Read Cache similar to other implementations of features in vSphere like Transparent Page Sharing. This solution will provide a nice performance boost with read intensive solutions like VDI.
6. vCenter Multi-Hypervisor Manager (MHM)
Surprise, Surprise, vSphere now supports managing a Windows 2012 Hyper-V host! I did not see this one coming, however it does makes sense. With the improvement of the latest version of Hyper-V there are many companies wondering what they should do for the future. With this latest release VMware is at least making it possible for the users to continue working in their well known and loved vCenter. You can even perform a cold migration from Hyper-V to vSphere!
7. vCSA improvements
VMware has stated on many occasions that they are likely to dump the windows version of vCenter, personally I think this will be a huge mistake, but before that can even happen they must prove to the world that the vCenter Server Appliance is going to be a viable option. One that is easy and error free! The latest release allows us to manage 500 hosts and 5000 virtual machines with the embedded database! Yes you read that correctly, the embedded database can handle 500 hosts and 5000 virtual machines! Ready to give it a whirl? No need for a costly sql or oracle server, just download the applicane, run through the basic setup and you are ready. Another key item of note with the vCSA is that there is a new vSphere Update Manager plugin for the vSphere Web Client!
8. vSphere replication improvements
VMware continues to improve its replication solution, the current improvements to the per-VM replication include compatibility with Storage vMotion and Storage DRS along with the ability to have multiple point in time recovery options. You can now have up to 10 appliances per vCenter server which allows for more freedom in your design and lowers the overall cost of vCenter purchases. Site Recovery Manager is updated to support these features as well.
9. vCenter Orchestrator Additions
VMware views Orchestrator as a key component moving forward in the migration to the cloud and they have improved its scalability and availability to allow for that planned migration. It also added features which include a workflow debugger, health check, and auto-alerts
10. vCloud Director New Features
VMware has already announced that vCloud Director 5.5 will be the last release, choosing to migrate to vCloud Automation Center (vCAC) instead. But that did not stop them from improving upon the functionality of this release. It is now sold as part of the vCloud Suite. So what have they done? Added many features related to the vApp lifecycle management such as Open Virtualization Format (OVF) import and export, the option to add or modify virtual hardware that was created from a template, and the ability to clone a VM while preserving its memory state. There are many more but that will have to come in a later blog.