vCSA upgrade from 5.0u1b to 5.1

First try on upgrading to the new 5.1 appliance and a big fail with a filled destination 5.1 vCSA root partition. (( Upgrade that worked here ))

If you have configured remote syslog on your source appliance, this routes to a /new directory on the destination appliance during the upgrade (not /storage/log). Unlucky for us, VMware choose not to have this /new directory with it's own mount point.

My work around is to create a new 20G drive and mount to /new before starting.

Oh, and take a snapshot BEFORE importing/swapping keys between source and destination vCSAs..

I'll update if I come across any other gotchas.


I've pinged VMware's kb with this, hopefully they'll get something up quickly. As for me...to last night's backup I go. Good thing this isn't production. 

vCSA DB2 shenanigans before upgrade

My initial post on upgrading the vCenter appliance to 5.0u1b build 804277 gave some good tips that I found around and figured out myself.
But I'll tell you, dear reader, that the upgrade process kept failing for me. Too much db2 export which I narrowed down to tasks & events.

I had set tasks and events to be kept for 180 days (this might be the default). I bumped this down to 30 days hoping that the rows would be deleted and then my upgrade would work again. This didn't seem to happen. So this post is about clearing your embedded db2 database of history stats, tasks and events before upgrade. Note that it doesn't look like anything in the db2 cli is case sensitive.

I should mention that I am *not* a DBA, that this could hose your installation, it will most definitely hose your stats, events and tasks so this process might not be a good idea in your production environment. The vCSA is a VM: Use snapshots to your advantage here.

• SSH into the appliance as root. Turn off vcenter services:

# service vmware-vpxd stop

• Change your login to db2inst1, get into the db2 cli, connect to the vCenter database:

# su - db2inst1
~> db2
db2 => connect to VCDB

• I couldn't remember if I had bumped the transaction logsize on this appliance, so I did the command anyway: 

db2 => update db configuration FOR VCDB USING logprimary 16 logsecond 112 logfilsiz 8192

• Delete contents of the history tables:

TRUNCATE TABLE vc.vpx_hist_stat1 IMMEDIATE
TRUNCATE TABLE vc.vpx_hist_stat2 IMMEDIATE
TRUNCATE TABLE vc.vpx_hist_stat3 IMMEDIATE
TRUNCATE TABLE vc.vpx_hist_stat4 IMMEDIATE
TRUNCATE TABLE vc.vpx_sample_time1 IMMEDIATE
TRUNCATE TABLE vc.vpx_sample_time2 IMMEDIATE
TRUNCATE TABLE vc.vpx_sample_time3 IMMEDIATE
TRUNCATE TABLE vc.vpx_sample_time4 IMMEDIATE

•  Deleting the contents of the tasks & events is not as straightforward. There are foreign keys associated with these tables that need to be removed before you can delete the rows and then re-create the foreign keys. 
• See how many rows of data: 

select count(*) from vc.vpx_event
select count(*) from vc.vpx_event_arg
select count(*) from vc.vpx_task

• Remove foreign keys: 

alter table vc.VPX_EVENT_ARG drop constraint FK_VPX_EVENT_ARG_REF_EVENT
alter table vc.VPX_EVENT_ARG drop constraint FK_VPX_EVENT_ARG_REF_ENTITY
alter table vc.VPX_ENTITY_LAST_EVENT drop constraint FK_VPX_LAST_EVENT_EVENT
alter table vc.VPX_EVENT drop constraint FK_VPX_CHANGE_TAG
alter table vc.VPX_EVENT drop constraint FK_VPX_EVENT_REF_COMPUTERES
alter table vc.VPX_TASK drop constraint FK_PARENT_TASK_REF
alter table vc.VPX_TASK drop constraint FK_VPX_TASK_CHANGE_TAG
alter table vc.VPX_TASK drop constraint FK_VPX_TASK_REF_ENTITY

• Delete data: 

truncate table vc.VPX_TASK immediate
truncate table vc.VPX_ENTITY_LAST_EVENT immediate
truncate table vc.VPX_EVENT immediate
truncate table vc.VPX_EVENT_ARG immediate

• Re-add foreign keys: 

alter table vc.VPX_EVENT_ARG add constraint FK_VPX_EVENT_ARG_REF_EVENT foreign key(EVENT_ID) references vc.VPX_EVENT (EVENT_ID) on delete cascade
alter table vc.VPX_EVENT_ARG add constraint FK_VPX_EVENT_ARG_REF_ENTITY foreign key (OBJ_TYPE) references vc.VPX_OBJECT_TYPE (ID)
alter table vc.VPX_ENTITY_LAST_EVENT add constraint FK_VPX_LAST_EVENT_EVENT foreign key(LAST_EVENT_ID) references vc.VPX_EVENT (EVENT_ID) on delete cascade
alter table vc.VPX_EVENT add constraint FK_VPX_CHANGE_TAG foreign key(CHANGE_TAG_ID) references vc.VPX_CHANGE_TAG(CHANGE_TAG_ID) on delete set null
alter table vc.VPX_EVENT add constraint FK_VPX_EVENT_REF_COMPUTERES foreign key(COMPUTERESOURCE_TYPE) references vc.VPX_OBJECT_TYPE(ID)
alter table vc.VPX_TASK add constraint FK_PARENT_TASK_REF foreign key(PARENT_TASK_ID) references vc.VPX_TASK(TASK_ID)
alter table vc.VPX_TASK add constraint FK_VPX_TASK_CHANGE_TAG foreign key(CHANGE_TAG_ID) references vc.VPX_CHANGE_TAG(CHANGE_TAG_ID) on delete set null
alter table vc.VPX_TASK add constraint FK_VPX_TASK_REF_ENTITY foreign key(ENTITY_TYPE) references vc.VPX_OBJECT_TYPE(ID)

•  Your tables are now empty:

select count(*) from vc.vpx_event
select count(*) from vc.vpx_event_arg
select count(*) from vc.vpx_task

•  quit from db2 cli, exit from db2inst1 user, restart vcenter services to see if everything powers back up: 

=> quit
~> exit
# service vmware-vpxd start

•  Log into the vSphere Client to check that everything looks good. 
• Try your upgrade again. For me, just truncating the history stats still failed after exporting the data for ~18 hours. Yeah. 18 hours and then I had to revert to snap. Dumping the events/tasks was the only way this was ever going to work. 

Thanks to vfrank, juanma and IBM's db2 documentation.

Sessions

As they are currently available. The website is pulling from aws s3 but I can't easily get to the source links. Hopefully it will go back to the setup that I can script when the videos are available.

Drink Me

~735MB
md5 checksum a07f4590e121798a4cd9273cce3e7dd6

Session number, full title and speakers: the koolaid is so good

VCAP5-DCD Tips

Before taking my VCAP-DCD test at VMworld I got a few tips from Jason Boche who was thankfully in the VMware testing lounge area. It might help you: 

• Go through the test and mark / flag all design questions for review without spending any time on them. Finish the multiple choice and many to many boxes-of-doom questions and save the design questions for last. 
• For the long scenario multiple choice questions, see if you can answer the question from just the last line or last short paragraph without reading the whole thing. 

I'll add to those: 

• Go through the quick demo of how the design questions work. It's not the same as the test but close enough to help you out. 
• For the design questions, check to see what all your available components are before trying to start the project. 
• Hammer out the main components and then move things around to start the design. I actually do this with visio as well..get all the main stencils on the page first. 

I met three people who had failed the test. It's a long test. Time management is crucial. Good luck.

I'll be taking the VCAP5-DCA next, thanks to the VMworld discount code from PearsonVue. 

Die vRAM Die!

vRAM entitlements gone. Now shops that have intentionally purchased smaller memory servers this past year are grumbling. As are the shops that upgraded to Ent+ just to get the higher vRAM entitlement. 

Hopefully it's not too little too late. vRAM made shops wake up and look around at the rest of the competitors...and they can sing a sweet tune.

The Ent+ vCloud Suite upgrade path is like giving your partner flowers after you screwed up: A good idea, but maybe you shouldn't have had a reason to buy the flowers in the first place. 

How about some more trickle down features to Enterprise? I'm still waiting for my vDS. 

Old post on the intro of vRAM

vCSA upgrade tips & recommendations

Upgrading the vCenter Server Appliance looks simple enough. If you run into problems, maybe these tips can help:

• Create the 20G (yes 20G) disk in the kb article. 
• Take a snapshot first (shut down services)
• Really. TAKE A SNAPSHOT FIRST
• If the upgrade does not go smoothly the first time, do not try again with a half-broken install. Revert to snap.
• If your upgrade breaks during the export, check your free inodes (df -i)
• Increase the number of inodes on the /storage/db/export partition (mkfs -t ext2 -i 8192 /dev/sdc1 )
• Problems mounting /dev/sdc1 (doesn't like ext2)? tune2fs -O has_journal /dev/sdc1
• tail -f /opt/vmware/var/log/vami/updatecli.log during the upgrade. This file is created a minute or two after you kick off the upgrade. 
• There will be a long (potentially hours long) pause in the log while the database export is happening. Top will show db2bp kjourneld and db2sync in use. Do not kill any of these processes. 
• You didn't take my advice and create a snapshot? You needed to use a restored backup? You might need to remove the /etc/udev/rules.d/70-persistent-net.rules mac addresses if the appliance says it can't bring up eth0. 

Real competition now

What was that pooled vRAM licensing? Never mind? Good idea.
Now update the goddamn vCenter Appliance and stop making software that requires MS Windows.