Solaris Live Upgrade on SunFire T1000/T2000 servers

The process of installing and configuring Live Upgrade on Solaris is pretty well covered at this stage but, as ever with Solaris, there is always something that is not covered in the many existing documents and blog postings on the subject. So, here is an overview of what I did, what worked, what did not work and how I overcame the bits that did not work.

Setup

I wanted to use Live Upgrade to ease the painful patching process on Solaris (not to upgrade the operating system). I tried this on a SunFire T1000 and a SunFire T2000 both running Solaris 2.10 U3 (06/06). In both cases, the systems contained 2 existing 10GB UFS file systems (/ on c0t0d0s0 and /opt on c0t0d0s3) as well as a swap partition (c0t0d0s1).

What Did Work

Following the Live Upgrade documentation from the Sun website, I narrowed the commands down to the following steps:

Create a new partition to accommodate the Alternative Boot Environment
# format
Created a Live Upgrade configuration:
# lucreate -c solaris_env_1 -n solaris_env_2 -m /:/dev/dsk/c0t0d0s4:ufs
Verified the successful creation using
# lustatus
# lucurr

Downloaded any available patches for this system
# smpatch download
Install the downloaded patches in the alternative boot environment
# smpatch update -b solaris_env_2
Reboot the system in the recommended manner
# init 6

What Did Not Work

No matter how many times I retried to reboot the system (in full accordance with the instructions in the documentation), the system simply refused to boot from the new/alternative boot environment. Each time, the lustatus command showed that both boot environment were correctly configured and that the alternative environment (solaris_env_2) was scheduled to boot at the next reboot. However, it never did boot from the alternative environment.
Apparently, when the patch process completes successfully (or whenever you use the luactivate command), the system should automatically update the boot-device EEPROM environment variable (see eeprom command from Solaris).

Workaround

I found that the only way around this problem was to manually adjust the boot-device environment variable from the system console each time you need to boot an alternative boot environment. This was done (from the system console) as follows:

Shut down the system
# init 0
Determine the appropriate setting for the boot-device variable
ok setenv auto-boot? false
ok reset-all
ok probe-scsi-all
ok show-disks

Using the output from the above commands, determine a new value for the boot-device variable. For example:
ok setenv boot-device /pci@7c0/pci@0/pci@8/scsi@2/disk@0,0:a
Verify that the system still boots from the old/existing environment
ok boot
If so, you should now be able to drop back to the boot prompt and set the boot-device variable to a new value in accordance to the Live Upgrade partition you created earlier (in my case, c0t0d0s4 or disk@0,0:e)
ok setenv boot-device /pci@7c0/pci@0/pci@8/scsi@2/disk@0,0:e
Also, you might like to return auto-boot setting to its normal value before booting the system again
ok setenv auto-boot? true
ok boot

Best of luck!

Upgrading the firmware on a SunFire T2000

In preparation for installing a newer release of Solaris 10 on a SunFire T2000, we first decided to bring the system to the most recent version of firmware. Here is how we got on …

Personally, I have upgraded firmware on many different types of systems in the past and whilst the technique is invariably the same, there are always minor differences that catch you out. This is also true in this case (hence the reason for this article). We found that many of the articles covering this topic made certain assumptions (without realising) and thus, left out certain key information. This article attempts to address those omitted assumptions.

What is the latest version of firmware and where can I get it?

We found links to all versions of firmware for T2000 systems here. The latest version at the time this article was written was 6.4.4. The firmware is distributed in a ZIP file.

What do I do next?

The best source of information on installing the firmware you have just downloaded is contained in a file called Install.info which can be found inside the ZIP file you just downloaded.

What the Install.info does not tell you

Here is a list of things that I did not know before reading the Install.info but found out during …

  1. There are many references on how to use the System Controller Console (SC) to both determine whether you need an upgrade and also to carry out the upgrade. However, they don’t actually tell you how to get to it. I discovered that:
    • You do not need to shut down your system
    • You must type #. at the system console to get to the SC (that’s a hash followed by a dot).
  2. When you first access the SC, you will be asked to provide a password (for future sessions). However, later on when it comes to authenticating yourself, you are also asked for a username but nowhere do they tell you what this is. The default username is admin but this is not well documented.
  3. Once you have establlished your connection to the SC and determined whether or not you actually need a firmware upgrade, it will be time to use the flashupdate command. When I tried this, it kept telling me that it could not establish an FTP connection to the specified server. Once again, I soon discovered that:
    • The flash update process (via FTP) uses a different network port than the one for normal system operation. So I had to move my network connection on the T2000 to the network port labelled NET MGT. The network ports used for normal system operating are actually powered off at this time.
    • You need to assign an IP address to the NET MGT network port. This can be done using the setupsc command from the SC (click here for a separate article on this). Once this is done you then need to reset the SC (resetsc) after which you will require the username and password referred to earlier to regain access to the SC.
  4. Once you finally get to the stage where you have a proper outbound connection from the SC, there is still one last issue. It concerns the name of the file containing the firmware image that you are passing to the flashupdate command from SC (via the -f parameter). I had assumed that this path was relative to the home directory of the FTP user being used but it turns out it was not. I had to specify an absolute path (e.g. /tmp/abc.bin)

Summary

Essentially, the Install.info file is your friend but there were 3 fundamentals that it does not cover:

  1. You need to have a valid network connection to the NET MGT port
  2. Your NET MGT port must have a valid IP address, Mask and Gateway
  3. You must use an absolute path for the image file in the flashupdate command.

Now I fully accept that none of this is rocket science and much of it is indeed documented but if you have not done this before, you will waste time discovering these issues. So hopefully, you will benefit from knowing this and it will save you time some time in the future.

Best of luck!