Error EC950008: Driver mining failed with Norton Ghost or Backup Exec

Error EC950008: Driver mining failed

This happens when trying to create a Symantec Recovery disk with various versions of Backup Exec System Recovery or Norton Ghost (they largely use the same codebase). I've seen this issue all the way up to Ghost 14 with the latest patches.

The issue is most likely due to Daemon Tools (happens even with the latest version to date). It errors out when it encounters it's SPD driver.

The solution is simple: uninstall Daemon Tools, create your System Recovery Disk, reinstall Daemon tools.

Windows 2008 Server Backup

I must say I'm quite pleased with the new Windows 2008 Server backup. A full backup of the system disk (Windows 2008 Enterprise and a few tools, about 15GB uncompressed with 3GB of SWAP. Windows Backup shows the volume as using 14GB) takes 6 minutes. Overall, a great effort, considering I'm doing this over a Firewire 400 connection.



And the backup is done:



A larger backup:

Mondo Backup - GPL Bare Metal Recovery Solution

Here's an interesting alternative to using CloneZilla and the likes for Bare Metal Recovery:

Mondo Rescue is a free (GPL) powerful disaster recovery suite for Linux (i386, x86_64, ia64) and FreeBSD (i386). It's packaged for multiple distributions (RedHat, RHEL, SuSE, SLES, Mandriva, Debian, Gentoo). It's basically the Linux equivalent of the powerful AIX mkcd / mkdvd.

It supports backups to tapes, disks, network and CD/DVD. It also supports multiple filesystems (ext2, ext3, JFS, XFS, ResierFS, VFAT and even NTFS), LVM, software and hardware Raid.

Example of using Mondo Rescue:

Generate a bootable DVD that also backs up /etc and can recover files running mondorestore:

# mondoarchive -OVr -d /dev/dvd -9 -I /etc -gF

Another interesting tool worth checking out is System Imager (automates Linux installs).

Repair MBR in Windows using mbrfix, fixmbr, bootrec, testdisk, dd and gag

Accidentally rewritten your MBR (or even the partition table) and can't find your old Win9x friend, "fdisk /mbr"? Well, here's a couple of ways on how to fix it:

• Fixmbr.exe - Repairs the master boot record of the boot disk. The fixmbr command is only available when you are using the Recovery Console. Example: "fixmbr \Device\HardDisk0"
• Bootrec.exe tool in the Windows Recovery Environment to troubleshoot and repair startup issues in Windows Vista. The /FixMbr option writes a Windows Vista-compatible MBR to the system partition. This option does not overwrite the existing partition table. Use this option when you must resolve MBR corruption issues, or when you have to remove non-standard code from the MBR.
• MbrFix.exe - Free tool that performs several Master Boot Record (MBR) tasks like backup, restore, rewrite (fix) the boot code in the MBR, etc. Supports Windows NT, Windows 2000, Windows XP, Windows Server 2003, Windows Vista and Windows PE. Both 32-bit and 64-bit x64 edition. Don't use for GUID Partition Table (GPT) disks. Example: "MbrFix /drive 0 fixmbr /vista /yes"
• TestDisk is a powerful free data recovery softwar designed to help recover lost partitions and/or make non-booting disks bootable again.
  
• GAG is an open source graphical boot manager which supports multiple operating systems. The floppy or CD can be used to boot Windows (or any other OS), then use recover tools to recover the MBR.

The MBR resides within the first 446 (0x1BE) bytes , the next 64 bytes are the Partition Table, and the last two bytes in the sector are a signature word for the sector and are always 0x55AA.This means that you can use the "dd" tool to backup, restore or modify your MBR "by hand". Example:

dd if=/dev/YOURDISKHERE of=mymbr bs=446 count=1

Will perform a backup copy that can be later restored using dd :-).

What about Linux or other operating systems? Well, you're probably using GRUB or LiLO, so you can simply boot your partition using (any) LiveCD (or GAG) and restore them. As simple as boot, fsck, mount, chroot, grub - you're set. You can also try using SuperGrubBootDisk.

UNIX Deployment Tools - JumpStart, IgniteUX, NIM, KickStart, AutoYaST, FAI

Bare metal recovery and mass deployment tools for UNIX and UNIX-like systems:

On Windows there's RIS, WDS or tools like Ghost, on UNIX platforms we have tools like JumpStart, IgniteUX, NIM, FAI, KickStart, etc. to help with massive deployment of operating systems.

UNIX:

• Sun Solaris - Custom JumpStart and Advanced Installations - The custom JumpStart installation method is a command–line interface that enables you to automatically install or upgrade several systems, based on profiles that you create. The profiles define specific software installation requirements. You can also incorporate shell scripts to include preinstallation and postinstallation tasks. You choose which profile and scripts to use for installation or upgrade. The custom JumpStart installation method installs or upgrades the system, based on the profile and scripts that you select. Also, you can use a sysidcfg file to specify configuration information so that the custom JumpStart installation is completely hands-off.
• Sun Solaris - JumpStart Enterprise Toolkit: provides a framework to simplify and extend the JumpStart functionality provided within the Solaris Operating System.
  
• Sun Solaris Flash Archives (flar) - can be used with JumpStart technology to automate and speed up deployment or disaster recovery.
  
• HP HP-UX Ignite-UX - is an administration toolset that allows: Simultaneous installation of HP-UX on multiple clients, The creation and use of custom installations, The remote recovery of clients, The creation of recovery media.
• IBM AIX mksysb/mkcd/mkdvd: The mksysb command creates a backup of the operating system (that is, the root volume group). You can use this backup to reinstall a system to its original state after it has been corrupted. If you create the backup on tape, the tape is bootable and includes the installation programs needed to install from the backup.
• IBM AIX NIM - Network Installation Management - is an excellent feature of the AIX operating system and is very important for teams or companies that have a need to install or upgrade many RS/6000 machines with the same images at the same time. NIM supports the use of mksysb images. Performing a NIM mksysb installation is faster than performing a NIM rte installation, and with mksysb, you can optionally include other installed software. You can use a mksysb image to install the nodes of a CSM cluster.

Linux:

• RedHat Linux Kickstart provides automation of Linux installation that uses a single kickstart file to install the system on multiple machines.
• SUSE Linux AutoYaST - Automatic Linux Installation and Configuration with YaST2. AutoYaST allows unattended and automated installation. With AutoYaST, administrators can create a consistent baseline configuration for new installations in large or expanding deployments. In addition to AutoYaST, other installation methods include PXE Boot, CD-ROM, NFS, CIFS/SMB, HTTP, FTP, and the Service Location Protocol (SLP), which allows autodetection of install servers. ALICE, SuSEs former auto-installation system was a system built around the auto-installation features that were available with YaST1. In order to be able to use existing ALICE configuration files and resources, a special option is provided in the configuration system will let you convert ALICE configuration files into a control file readable by AutoYaST.
• Debian GNU/Linux FAI - Fully Automatic Installation - is an automated installation tool to install or deploy Debian GNU/Linux and other distributions on a bunch of different hosts or a Cluster. FAI can also be used for configuration management of a running system.
  

BSD:

• Automatic OpenBSD Installation - Jumpstart-style procedure for installing OpenBSD servers
• FreeBSD "JumpStart" Guide - This article details the method used to allow machines to install FreeBSD using the Intel PXE method of booting a machine over a network. Use sysinstall install.cfg for scripting.
• BSD PXEBoot - while not unassisted, BSD systems can easily boot from PXE and install over the network.
  

Tools:

• Cfengine - an adaptive system configuration management engine - is an automated suite of programs for configuring and maintaining Unix-like computers. It has been used on computing arrays of between 1 and 20000 nodes.
  

Ext2 Filesystem for Linux and Solaris

Right now most people are using FAT32 on their external storage media (usually USB memory sticks or external hard drives and such) due to compatibility reasons. FAT32 can be used on anything from Windows and Linux to Solaris and BSD. Problem is, FAT32 has some severe limitations (like the 4GB file size limit). An alternative would be to use ext2.

The ext2 or second extended file system is a file system for the Linux kernel designed as a replacement for the extended file system (ext). Its successor, ext3, provides journaling and is almost completely backward compatible with ext2 (as in, you can mount ext3 as ext2, dropping journaling capabilities). It is implemented via ext2fs in the Linux kernel.

Other implementations:

• FreeBSD: mount_ext2fs(8) -- mount an ext2fs file system
  
• NetBSD: mount_ext2fs(8) - Mount an EXT2FS file system and fsck_ext2fs(8) - EXT2 File System consistency check and interactive repair
  
• OpenBSD: mount_ext2fs(8) - mount an ext2fs file system and fsck_ext2fs(8) - Second Extended File System consistency check and interactive repair
• MacOS X / Darwin: ext2 driver for Mac OS X
  
• Solaris and OpenSolaris: Install FSWpart FSWfsmisc from
  Miscellaneous filesystem support for OpenSolaris on x86 and use "mount -F ext2fs". You can also use the older ext2fs driver for Solaris 7-10.
  
• Windows: Ext2 IFS - Installable File System For Windows. Works on NT4.0/2000/XP/2003 that provides read / write access and integrates as well as a native FAT32 or NTFS filesystem. This will also work in Vista if you install under "XP SP2" compatibility mode.
  
• Windows Vista: Ext2Fsd is an open source linux ext2/ext3 file system driver for Windows systems (NT/2K/XP/VISTA, X86/AMD64).
  

If you practice dual boot (or multi boot) often or use external hard drives for backup or mass storage purposes, consider using the ext2 filesystem for compatibility reasons.

Backup and Recovery Primer - Part I: Backup Strategy

Backup Basics and best practice. Developing a backup strategy.

A proper backup strategy should be part of every disaster recovery plan. Every company should have at least a basic DRP, business continuity plan and employ risk management techniques.

Data backup and recovery falls in the prevention against data loss category, along side surge protection, interruptible power supplies (UPS), fire prevention systems, data security software (IDS, Antivirus Software, etc).

A backup is not a simple thing. You can't just throw your files on some random storage array every now and then and expect things to work out. A proper backup requires planning, a backup strategy, risk assessment and team work.

1) What does a backup strategy help us mitigate against?

• Disk failure. - Face it, disks fail. Often.
• Filesystem corruption or disk corruption, or other events that leave our disk in an inconsistent state. - I've seen this one too many times. You run out of inodes on a filesystem and you also need to run fsck on it, but you cannot. And you cannot mount it either. Or your filesystem decides to break, and corrupts your data. Filesystem corruption is more often in the case of power failures and forced shutdowns.
  
• File deletion and accidents: I've seen things like "# rm -rf $HOME/*" on a UNIX system (where the default root $HOME is /, of course) and various other accidents using pipes or dd. And it's usually easier to restore files from backup than to try to restore them after you've erased them. Also, allowing testing and development on production systems will eventually lead to such accidents taking down your main production database sooner or later...
  
• Stolen or destroyed disks / machines. Laptops are especially vulnerable to this.
• Tampering with the data: Viruses, exploits, hackers may modify and tamper with your data. You may need to perform a roll-back to a previous state of your data.

But, most important, a proper backup strategy will save you and your company money. Data loss and downtime will lead to financial losses due to lost customers, wasted money, image and reputation loss, morale going down among your staff or even legal problems. Not to mention there are things that simply cannot be recreated once lost, like access logs and such.

You must fully understand why you need a backup strategy, what it is you're protecting (don't just think of data, think of your companies reputation, your job security, loss of revenue and such).

2) Who is responsible for the integrity of the data

The data owner. Which, in most cases, is upper management. It's management's responsibility to do a risk assessment, and deploy the proper business continuity and disaster recovery plans. They often decide to delegate permissions and responsibility to such tasks down the chain of command.

3) What do you make backup copies of?

Just your production database? The whole system? Just the critical data tables?
It's your responsibility to asses what data is critical, how long it would take to restore a system in case of a disaster (hope for the best, but expect the worse. Can you cope with a fire? How about and earthquake? How about a disgruntled employee purposely altering or erasing your data?).

4) How often do you backup?

Do you need continuous data protection (CDP)? Can you accept loosing a day's work? How about losing one day of everyone's work? If you have 1000 users, that may as well be 3 years worth of work right there, in a single day. What will that cost you?

5) How do you backup?

Will you be using tapes? Will you purchase a storage array and use it for backup purposes? Will you be using optical media? Are you going to do this across the network? How will the network cope with the load generated by the backup?

What software will you use? How much does it cost? What about the total cost of ownership? Will future versions be supported and be able to access older backups?

Are you storing all the relevant information to restoring your systems? How about permissions or disk volume / partition configuration?

6) Where will you store your backup?

Doing a backup on the same disk, or on the same disk on the machine you're backing up is usually a very bad idea. Or leaving the tape in the tape drive or on top of your machine. Think about it. That tape probably contains vital and confidential information. You don't want it to go up in flames with your production systems, or get stolen along with the confidential business records it contains. You really need to evaluate all possibilities in terms of on-site storage (fire-proof tape safe, tape robots or even a simple storage cabinet) as well as off-site storage.

Also, consider online backup vs. offline backup. Consider something as simple as a rsync-snapshot/rsnapshot/rsync-with snapshots or as complex and powerful as Sun's StorageTek Availability Suite.

7) Monitoring and testing

Don't just dump the fs content to a tape and leave it there. Test and monitor your backup and restore procedures.

Test your backup software. Some companies even go so far as to test a new backup solution for years in parallel to the old solution before committing themselves to using only the new backup method.

Always keep your backup policy up to date. Make sure your plans and strategies reflect real life situations.

8) Restoring data

Backups in themselves have little importance. It's restoring the data that matters. How will you get data off those tapes / cds? How long will it take? Will you have your system up and running to a satisfactory baseline, fast? How about restoring individual files from a certain point in time? Say a user requires an 3 month old email. What will you do then, restore the whole mail database, from 23 tapes by restoring a full backup, the incremental, the differentials, etc? That wouldn't be too much fun, now would it...

It's usually a good idea to take a layered approach to backup. Like in Windows, you have System Restore to revert to a previous state of the operating system, Shadow Copies to restore older versions of files, you have ntbackup to do a system state, registry and filesystem backup, and Windows Complete PC Backup (or tools like Norton Ghost) for bare metal recovery.

Make sure your mail server stores emails for a certain period of time. Have your log servers store logs and rotate / archive them at a specific interval. Here is where products like QFS / SAMfs really shine.

The idea is simple. It's a lot easier to restore and manipulate files from lower backup levels than it is to manipulate bare metal recovery backups or full filesystem backups.

9) Scalability

Data within a company grows at an exponential rate. As data grows, so will your backup needs. You will need to plan ahead, make sure all your backup systems are scalable and can handle growth. If you hit some weird limitation (like with some filesystems for example, that can't grow beyond 1-2 TB and so on), that's pretty much it..

Also, make sure that while your systems are scalable, they don't grow beyond your power to manage them. Keep things simple and easy to understand. Document everything.

OpenBSD RAIDframe mirror - Software RAID

By popular demand, Software RAID mirrors in OpenBSD, using RAIDframe.

For software RAID, OpenBSD has RAIDframe, softraid(4) (but only in 4.2) and ccd(4) (the concatenated disks driver - part of the GENERIC kernel, suppors a mirroring feature and a "striping" effect, but has limitations).

I should warn you that RAIDframe has quite a few problems. It doesn't seem to be properly maintained, code quality is down, it can be quite slow, the configuration process is a PITA and quite low level (as you can probably see), it makes upgrades a difficult and you'd get the same kind of downtime from a backup / restore process (dump/restore). Still, if you're up for it, read the raid(4) manpage, and give it a shot.

The new softraid(4) mirroring is simpler, but it's far from complete or tested as of OpenBSD 4.2.


Step 1. Installing OpenBSD:

The installation is pretty straight forward. You only need to pay attention to the disk labeling part. Create a 512M /, a small swap (128m is fine) and give the rest (a d) to RAID (type RAID in the FS type). Make sure you install compiling tools (comp).

Step 2. Install and boot a RAIDframe kernel

The problem is OpenBSD doesn't ship a RAIDframe enabled kernel by default (in GENERIC or otherwise). And you can't enabled it via config(8) either. So you'll have to build yourself a custom kernel and enable RAIDframe. Read 5.7 - Building a custom kernel of the FAQ for that. Once built, you can use that on all your machines, so it's a good idea to use GENERIC as a template for your RAIDframe kernel. Just uncomment the "pseudo-device raid" and "option RAID_AUTOCONFIG" lines and you're set.

You can use this as an excuse to move to OpenBSD-stable. See http://www.openbsd.org/stable.html - just copy GENERIC to GENERICRAID and edit / config that.

Also remember to turn on SoftUpdates (softdep in /etc/fstab), or it will take forever ;-).

Technically, you could do this on another machine. Once the kernel is built, you can use it to master your own media kits for use in future installations.

Step 3. Configure the mirror disk

You need to clone the disklabel from disk wd0 (or whatever your first disk is) and initialize the second disk (fdisk -i).

# disklabel wd0 > disklabel0
# fdisk -i wd1
# disklabel -R -r wd1 disklabel0

Step 4. Configure the RAID array:

See raidctl(8) and raid(4) for this step. Just setup your /etc/raid0.conf.

An example for a mirror:

# cat /etc/raid0.conf

START array
1 2 0
START disks
/dev/wd0d
/dev/wd1d
START layout
128 1 1 1
START queue
fifo 100

Step 5. Clone the disk

See http://unixsadm.blogspot.com/2007/08/cloning-disk-in-openbsd.html

Step 6. Enable the RAID

# raidctl -C /etc/raid0.conf raid0
# raidctl -I 100 raid0
# raidctl -iv raid0

Step 6. Disklabel and newfs the RAID disk:

# disklabel -E raid0
# newfs /dev/rraid0a

(Newfs created disklabels)

Step 7. Copy the installed OpenBSD on your RAID array:

You should use dump / restore for this process.

# mount /dev/raid0a /mnt
# cd /mnt
# mkdir usr
# mount /dev/raid0d /mnt/home
# dump -0f - / | restore -rf -

(make sure you create mount points and mount all the disk labels you've created).


Step 8. Activate the RAID config:

# raidctl -A root raid0

Remember to edit /etc/fstab to reflect these changes. You should have at least the / line similar to:

/dev/raid0a / ffs rw 1 1

Step 9. Reboot, enjoy, check status:

# raidctl -s raid0

Read the raidctl(8) manpage for details.