Remote code execution via administrative shares

The Sysinternals psexec tool allows you to execute remote code on a Windows box using the Administrative Share (C$). It's part of the pstools package.



You can just run psexec -u Username \\SomeSystem cmd and you get a command prompt on that system, no need to mess with telnet. You can even use it to distribute "batch" files or run something like "gpupdate /force" on remote machines.

Fun with psexec: run the BSOD screensaver on a remote machine :-).

Skype listening on http / https by "default".

Your webserver isn't starting because ports 80 / 443 are in use? Well, it just might be... Skype. For some reason, it listens on ports 80 and 443.

telnet localhost 80
GET / HTTP/1.1
HTTP/1.0 404 Not Found#

Yep, we have a webserver listening allright. Let's find out what process it maps to.

netstat -ban

Active Connections

Proto Local Address Foreign Address State PID
TCP 0.0.0.0:80 0.0.0.0:0 LISTENING 324
[Skype.exe]

TCP 0.0.0.0:443 0.0.0.0:0 LISTENING 324
[Skype.exe]

As sure as hell, there it is. (use lsof / netstat on UNIX systems to match an open port to a process).

Skype listens on these ports by default and acts as a call/message relay, and you can close them by going to Tools - Options - Connection - Unticking "Use Port 80 and 443".

More info on being a Skype "supernode" or relay node is available on the Skype Forums.

Using Sysinternals Autoruns to disable startup programs

The windows tool msconfig doesn't really tell us much about the programs that run at startup in Windows. Let's just compare it to autoruns for a moment.



Knowing how malware can install itself even as a print monitor, it's usually a good idea to rely on autoruns and not msconfig.

Since it can also verify drivers and file signatures, it can hide signed Microsoft programs and show unsigned or fake signature software, usually a sign of malware.

Windows Crash Dump Analysis - Pinpointing Faulty Drivers with Driver Verifier and WinDbg.

I've been having problems with my system lately, random freezes or crashes (BSOD, DRIVER_IRQL_NOT_LESS_OR_EQUAL, etc). Since the system passes memtest86+ and Vista memory tests, and it is updated to the latest and greatest patch level, I'm pretty sure the problems are caused by 3rd party drivers.

I'm going to use a very overlooked tool in Windows, and that is the "Driver Verifier". This tool has been a part of Windows systems ever since Windows 2000, and it's an invaluable tool in debugging faulty drivers.

You'll need to run the tool manually, just start - run - verifier.exe. A wizzard will pop up.



You'll need to create custom settings, Select individual settings from a full list, and pick everything but "Low resource simulation". You can now either pick unsigned drivers (the usual suspects, since signed drivers are usually tested) or select driver names from a list (pick the ones you most suspect... recently installed before crashes appeared and so on). Make sure you don't pick ALL drivers on your machine, that's quite painful.

In my case, I've picked drivers not provided by Microsoft. To be more specific, the Intel Turbo Memory (Robson) Driver, since this is a Santa Rosa platform laptop, and Turbo Memory isn't really know for it's stability or performance boosts. In fact, some laptop providers like HP have said NO to Turbo Memory.

iaStor.sys is the Intel Matrix Storage Manager Driver and ianvstor.sys is the Intel Turbo Memory (Robson) driver. Both install with the Intel Turbo Memory driver.

In case you're more worried about it's actual usefulness than security issues (people reading data from the solid state device) or stability issues, take a look at this article:

Investigating Intel's Turbo Memory: Does it really work?




Once you're done rebooting, make sure you check the results tab:



After this change you'll need to reboot your system, and wait for a crash to occur. Just in case your system will freeze, it's generally a good idea to boot your system in Debugging Mode (F8 at startup) and make sure you have full memory dumps and CrashOnCtrlScroll enabled.

Once you've managed to obtain a crash dump (either by a BSOD or connected a debugger to a hanged system and typing .dump, or even crashing the system yourself with Ctrl - Scroll Lock - Scroll Lock in case of a hang) you'll need to analyze it. For this task, you'll need Windows Debugger (WinDbg) and a Debugging Symbols Server. You can grab them off Microsoft's website. Just make sure they're the right ones for your system.

Once you've got yourself a debugging environment set up, open WinDbg, pick File - Open Crash Dump (Ctrl - D) and open the fresh memory dump (C:\Windows\MEMORY.DMP by default - check the Startup and Recovery Tab to make sure).



As we can see in the image, it did not find any debugging symbols for the iaStor.sys and iaNvStor.sys drivers (the Intel Turbo Memory Drivers) since they are 3rd party drivers.

We're going to type "!analyze -v" to get more details on the error.


Looking at the stack, we can see:

WARNING: Stack unwind information not available. Following frames may be wrong.
88fafd00 80673529 85f2b0e8 8593d194 b92b8f00 iaStor+0x3cf51

That's the Intel TurboMemory driver (well, the Intel Matrix Storage Manager to be more exact).

Since this is a Core 2 system, we have CPU 0 and CPU 1. To run instructions on CPU 1, we need to switch to it, using the ~1 command.

We're going to examine the stack on both CPU's:

0: kd> ~0
0: kd> k
ChildEBP RetAddr
a9563bb0 81ce8651 hal!KeReleaseQueuedSpinLock+0x26
a9563c14 81ce86f6 nt!ExFreePoolWithTag+0xae7
a9563c24 81dec9ae nt!ExFreePool+0xf
a9563c60 81d3949b nt!ObOpenObjectByName+0x47b
a9563d2c 81d39258 nt!CmOpenKey+0x1b1
a9563d50 81c8c92a nt!NtOpenKey+0x39
a9563d50 77ae0f34 nt!KiFastCallEntry+0x12a
WARNING: Frame IP not in any known module. Following frames may be wrong.
0097f0c4 765d5fc9 0x77ae0f34
0097f120 7566548e 0x765d5fc9
0097f214 75664e62 0x7566548e
0097f274 75665581 0x75664e62
0097f2a4 75665f46 0x75665581
0097f450 75665f5f 0x75665f46
0097f474 75664b10 0x75665f5f
0097f4d0 75664a05 0x75664b10
0097f51c 76686d7e 0x75664a05
0097f548 767003a2 0x76686d7e
0097f974 766ff44c 0x767003a2
0097f990 766873cb 0x766ff44c
0097f9cc 76687279 0x766873cb
0: kd> ~1
1: kd> k
ChildEBP RetAddr
88fafb64 828059c1 nt!KeBugCheckEx+0x1e
88fafb94 82805d01 crcdisk!VerifyOrStoreSectorCheckSum+0x111
88fafbc4 8280521a crcdisk!VerifyCheckSum+0xa9
88fafc00 82805570 crcdisk!CompleteXfer+0x16a
88fafc14 81ecec69 crcdisk!CrcScsiReadCompletion+0x20
88fafc4c 81caca3b nt!IovpLocalCompletionRoutine+0xcc
88fafc80 81eceb53 nt!IopfCompleteRequest+0x13d
88fafcf0 8297ef51 nt!IovCompleteRequest+0x11c
WARNING: Stack unwind information not available. Following frames may be wrong.
88fafd00 80673529 iaStor+0x3cf51
88fafd10 8067a8b5 iaNvStor+0x16529
88fafd30 8067b1f1 iaNvStor+0x1d8b5
88fafd4c 80679740 iaNvStor+0x1e1f1
88fafd7c 81e25472 iaNvStor+0x1c740
88fafdc0 81c9141e nt!PspSystemThreadStartup+0x9d
00000000 00000000 nt!KiThreadStartup+0x16

We can use lm kv to list currently loaded drivers and version information. From this list, I've selected the two drivers we're interested in:

8065d000 80699000 iaNvStor (no symbols)
Loaded symbol image file: iaNvStor.sys
Image path: \SystemRoot\system32\DRIVERS\iaNvStor.sys
Image name: iaNvStor.sys
Timestamp: Sun Mar 11 10:11:01 2007 (45F3B995)
CheckSum: 000423B0
ImageSize: 0003C000
Translations: 0000.04b0 0000.04e0 0409.04b0 0409.04e0

82942000 82a00000 iaStor (no symbols)
Loaded symbol image file: iaStor.sys
Image path: \SystemRoot\system32\DRIVERS\iaStor.sys
Image name: iaStor.sys
Timestamp: Mon Feb 12 22:46:47 2007 (45D0D237)
CheckSum: 0004966D
ImageSize: 000BE000
Translations: 0000.04b0 0000.04e0 0409.04b0 0409.04e0

This tells us the driver version and timestamp, to let us know if an update is in order. Usually old drivers are suspect.

Useful commands:

• ~N where N is the CPU number (count starts from 0) changes to that CPU
• !analyze -v gives a detailed heuristic analysis of the problem, and looks for 3rd party drivers that might be responsible for the crash.
• lm kv lists loaded drivers and gives details on them
• !deadlock the deadlock verifier
• !vm prints memory usage. If pool usage is close to pool maximum, then a driver might have a memory leak.
  
• !poolused if pool tagging is on (by default on 2003+) displays kernel memory usage pools by pool tag, and allows you to map the pools back to the drivers. Mapping it for 3rd party drivers require you to grep printable strings in the driver. See "!poolused c".
  
• !thread examine current thread (run on each CPU). If a driver interrupts a running thread, this may not list the cause of the crash.
  
• !process 0 0 list active processes. Look for suspect processes that shouldn't be running, or common 3rd party processes that show up in multiple crashes.

Since all results point at the Turbo Memory drivers, and while working I've got another couple of crashes, with similar results (Probably caused by : iaNvStor.sys ( iaNvStor+3523 )), I've got two choices here: either look for updates or remove the driver, and stop using Turbo Memory. Since most benchmarks say it doesn't provide an actual performance boost, and may provide a mild battery boost, I concluded it's safer to just remove the driver, and stop using Turbo Memory on this machines.

The Windows Experience Index on this machine is 4.7 with Turbo Memory enabled, 4.7 with Turbo Memory caching disabled. No change in performance (Sure, the Experience Index may be no proper benchmark, but it's more than relevant in this case).

Seeing how disabling the caching didn't affect performance, I simply removed the software from my machine. Big mistake. Uninstalling the Turbo Memory software also gave this neat little error when booting Windows:

Please insert the Windows recovery CD.

Windows failed to load because a critical System driver is missing or corrupt.
\WINDOWS\System32\DRIVERS\iaStor.sys

Well, that was fun. At least restoring BOOTSECT managed to fix things.

Final Words:

A vast majority of Windows crashes are caused either by:

Unstable hardware:

• Entry level memory with no ECC or Chipkill, sometimes running and very high speeds it wasn't designed for
• Broken memory: Make sure you run memtest86+ or Vista's Memory Tester (boot the DVD and pick Memory Test).
  
• Overclocked processors and high temperatures

Buggy 3rd party drivers:

• Binary blob drivers that aren't proper tested, verified and updated for all versions of Windows
• Drivers for small time hardware like keyboards with "special" keys or 8 button mice and so on.

My advice is simple: avoid such drivers at all costs. The functionality they add is minor, and the risks and stability problems are not worth it. If your keyboard's "Email" button requires a kernel module, forget about it, the idea is broken by design.

And remember, if you have problems, make sure you obtain a memory dump for analysis, and always perfrom a hardware test before blaming it on software: a full POST, a memtest86+ test, a CPU Prime95 test, monitor system temperatures and disk S.M.A.R.T. data with HD Tune, check your hardware cabling and such, then look at the software. Mainly, at the drivers: remove unneeded 3rd party drivers, software (use Sysinternals autoruns) and hardware, install the latest driver versions, update to the latest Windows patchlevel.

Detecting ARP spoofing with Arpwatch.

ARP spoofing or ARP poisoning is a problem on most network, and can be difficult to mitigate against, short of having switches with ARP security or Dynamic ARP Inspection.

Also, having an attacker (or a clueless user) deploy a DHCP server on your network can be devastating. DHCP Snooping on Cisco Catalyst switches can help against that.

Arpwatch is a free (open source - BSD licensed) tool that monitors ethernet ARP activity using pcap(3) and keeps a database of ethernet/ip address pairings and reports changes via email.

Arpwatch runs on all major UNIX (Solaris, OpenServer, Unixware) and UNIX-like platforms (BSDs, Linux an such).

You can download the current version from:

ftp://ftp.ee.lbl.gov/arpwatch.tar.gz

Google Sketchup

Google SketchUp is a simple, fast and easy to use CAD-like 3D software. It's what they use for the models in Google Earth, so it integrates quite nicely. And best of all, the free version rocks and runs both on Windows and MacOS X.



It also comes with a vast library of objects (bonus packs) from street poles and cars to, well, the kitchen sink...

The software is very good, and it can integrate seamlessly with Graphisoft Archicad and very simple to learn (partly due to the vast collection of video tutorials and available resources).

SketchUp also comes with a Ruby framework for developing scripts, plugins and 3-rd party extensions. Once such extension exports scenes to be rendered to VRay. There is also a Maxwell Render plugin.

X forwarding with ssh -X and XMing

Redirecting X over tcp / udp and using something like "XWin -query machine" is insecure. So are most VNC solutions. You can use X forwarding over ssh:

% ssh -X -C -c blowfish MachineHostname xterm

(Blowfish encryption tends to be a bit faster, and -C enforces compression, though it should be enabled by default. If you have issues with X forwarding, check your /etc/ssh/sshd_config for "X11Forwarding yes" and "ForwardX11 yes").

Sometimes you _have_ to run a graphical interface on a UNIX machine, and do so from your Windows box. Examples include the Oracle Installer and some other Java applications.

While you could install Cygwin to provide you with an XOrg server implementation, Cygwin tends to be quite big. A lighter and faster option would be to use XMing. It's small, fast and just works. Just fire up XWin, and it will listen on 0:0 by default.



Now you can use "ssh -X machineIP xterm" and it will start a remote XTerm right on your machine. Or if you use putty, make sure you redirect X:




Now run your favorite GUI application.

AdRestore - Restore Deleted Active Directory Objects.

AdRestore is a cool tool written by Mark Russinovich that allows you to restore deleted objects from the deleted objects folder. At times, it can be a real life saver. The tool is part of the Sysinternals suite.

For details, read: How to restore deleted user accounts and their group memberships in Active Directory.

Windows Server 2008 x64 Beta 3 - Codename Longhorn

A screenshot preview of the upcoming Microsoft Windows 2008.

The installer.

The option to chose an Enterprise Core install (no GUI) or perform a full blown installation.

Just finished installing SUA/SFU (Services for Unix / Interix).

Interix in action. You can even get OpenSSH packages :-).

Powershell!

Digital Forensic Tools: Imaging, Virtualization, Cryptanalysis, Steganalysis, Data Recovery, Data Carving, Reverse Engineering

"Jrypbzr gb gur bgure fvqr."

Computer Forensics is a science and an art. And to perform it, you need tools to identify, acquisition, preserve and analyze data in a clean, safe, non-destructive manner. Lots of tools. Everything from data acquisition to virtualization and steganalysis.

A list of more or less free tools (mostly open source or freeware, but I have included some relevant commercial products) no digital forensics expert should be without:

Data acquisition, enumeration, imaging and forensics tools: Toolkits and utilities.

1. The Sleuth Kit and Autopsy Browser. Both are open source digital investigation tools (a.k.a digital forensic tools) that run on Unix systems (such as Linux, OS X, FreeBSD, OpenBSD, and Solaris). They can be used to analyze NTFS, FAT, Ext2, Ext3, UFS1, and UFS2 file systems and several volume system types. The Sleuth Kit (TSK) is a collection of command line tools based on The Coroner's Toolkit (TCT). Autopsy is a graphical interface to TSK.
2. The Coroner's Toolkit (TCT) is a forensics toolkit for analysis of UNIX break-ins. It runs on BSD (OpenBSD, FreeBSD, BSD/OS), Solaris/SunOS, Linux and HP-UX.
3. WinHex: Computer Forensics & Data Recovery Software, Hex Editor & Disk Editor.
4. dcfldd is an enhanced version of GNU dd with features useful for forensics and security. GNU ddrescue is a data recovery tool. It copies data from one file or block device trying to rescue data in case of read errors. It's a better alternative to using dd_rescue and dd_rhelp or SpinRite (you can just do a disk refresh with dd: "dd if=/dev/disk of=/dev/disk bs=2m" while the drive isn't mounted - no write operations going on - or something along those lines in order to prevent presently recoverable read errors from progressing into unrecoverable read errors).
5. Sysinternals tools contains programs like streams that help us find data hidden inside alternate streams or strings that grep readable strings from a file. It also has tools like process explorer, procmon, autoruns and rootkit revealer that allow you to dig deep into the Windows operating system to process, disk and data related information.
6. Microsoft Log Parser is a powerful, versatile tool that you can use to extract information from files of almost any format by using Structured Query Language to text-based data such as log files, XML files and CSV files, as well as key data sources on the Windows operating system such as the Event Log, the Registry, the file system, and Active Directory.
7. AccessData Forensic Products: FTK - Forensic Toolkit, Registry Viewier - more neat tools from AccessData. Commercial products.
8. Clonezilla is used to clone many computers simultaneously. It can perform a full disk image or just file backup. It's a backup tool, but it can also perform bit by bit disk imaging.
9. Sysinternals LiveKD is a Live version of Windows Debugger (WinDBG) that allows you to run the Kd and Windbg Microsoft kernel debuggers, which are part of the Debugging Tools for Windows package, locally on a live system. This means that you to easily take a memory dump of a running Windows system (.dump /f YOURUSBDISK:\fullmemorydump.dmp). On UNIX systems you can use dd to take a snapshot of the system memory ("dd if=/dev/kmem of=/path/to/memorydump").
10. Paraben's Device Seizure - Cell Phone and PDA Forensic software. Specialized software for portable device forensics.
11. PDD is a forensic analysis tool for Palm OS platform devices. It is an open source Windows-based tool for Palm OS memory imaging and forensic acquisition. The Palm OS Console Mode is used to acquire memory card information and to create a bit-for-bit image of the selected memory region. No data is modified on the target device and the data retrieval is not detectable by the user of the PDA.
12. CDInfo is an application that will display all ISO descriptors from all attached cd-rom drives (Label, System, Application, VolumeSet, Copyright, Creation Date, Directory Start, Directory Length, extentions, tracks, etc).
13. PMDump is a Windows tool that lets you dump the memory contents (both RAM and swap) of a process to a file without stopping the process.

Virtualization: Once the actual machine is cloned, it's usually put inside a virtual machine (features like snapshots and debugging help quite a bit with the digital forensics process). This is called physical to virtual (P2V) migration.

1. Live View is a Java-based graphical forensics tool that creates a VMware virtual machine out of a raw (dd-style) disk image or physical disk. This allows the forensic examiner to "boot up" the image or disk and gain an interactive, user-level perspective of the environment, all without modifying the underlying image or disk. Because all changes made to the disk are written to a separate file, the examiner can instantly revert all of his or her changes back to the original pristine state of the disk. The end result is that one need not create extra "throw away" copies of the disk or image to create the virtual machine.
2. VMware Converter Starter is a free p2v (physical to virtual) migration tool. VMware Converter quickly converts Microsoft Windows based physical machines and third party image formats to VMware virtual machines. It also converts virtual machines between VMware platforms. Note: for digital forensic images, you should use LiveView, and not the converter.
3. VMware Server allows for free virtualization. You can use it in combination with Live View to virtualize existing environments, and use the snapshots feature to revert back to a previous state of a virtual machine in an instant.
4. QEmu - a much more flexible virtualization program, albeit a bit slower than VMware. Supports emulating IA-32 (x86) PCs, AMD64 PCs, MIPS R4000, Sun's SPARC sun4m, Sun's SPARC sun4u, ARM development boards (Integrator/CP and Versatile/PB), SH4 SHIX board, PowerPC (PReP and Power Macintosh), and ETRAX CRIS architectures. Also, qemu-img can be a valuable tool for converting virtual machine images. Also allows for some really low level debugging features. A modified version of QEmu can even emulate PIX platforms (or Juniper JunOS systems like Olive).
5. VirtualBox is a GPL licensed x86 virtualization platform that runs on Windows, Linux and MacOS hosts, and supports various x86 client machines (Windows, Linux, BSD, Solaris). It's a noteworthy alternative to using VMware, as performance tends to be pretty good.
6. Microsoft VirtualServer / Virtual PC are free virtualization products from Microsoft. They support all major features (snapshots, mounting ISO images and such), and performance tends to be reasonable (to some extent, similar to that of VMware).
7. SIMH is a highly portable, multi-system simulator. It can emulate VAX and PDP-11 platforms. Just in case you need to perform forensics on older minicomputers.
8. Hercules is an open source (QPL licensed) emulator of IBM Mainframe computers (System/370, ESA/390 architectures and even the 64-bit zSeries). Hercules runs under Linux, Windows (98, NT, 2000, and XP), FreeBSD, and Mac OS X (10.3 and later). Hercules will run OS/360, DOS/360, DOS/VS, MVS, VM/370, TSS/370 - all IBM public domain operating system, as well as OS/390, z/OS, VSE/ESA, z/VSE, VM/ESA, and z/VM, and even Linux/390 and Linux (SuSE, RHEL, Debian, CentOS and Slackware) on zSeries.
9. Oracle VM is a server virtualization software based on Xen and Oracle Linux (itself based on RHEL sources) that fully supports both Oracle and non-Oracle applications. It is a free alternative to VMware Virtual Infrastructure (VMware ESX + VirtualCenter). It is certified to run the Linux operating system, Oracle Database, Fusion Middleware, and Application software, thus makes a very good platform for investigating Oracle databases.
10. The Palm OS Emulator is a program based on the Copilot app that emulates the hardware of the various models of Palm-powered handhelds, making it a valuable tool for writing, testing, and debugging applications as well as obtainiwinng evidence from the device.
11. Microsoft Windows CE 5.0 Device Emulator contains the emulator technologies featured in Windows CE 5.0. By using the Device Emulator, you can run emulated-based images created by Windows CE 5.0 without installing Platform Builder, its platform development tool.

Password recovery tools: You may often need to recover keys and passwords.

"This text has been encrypted twice... for double protection!"

1. Ophcrack is a very efficient Windows password cracker based on rainbow tables. It will crack huge tables of LM hashes in under 3 minutes. It also comes in the form of a LiveCD (though in digital forensics cases it's usually best to extract the SAM file containing the password hashes from the disk image and use that. Ophcrack can be a lot more effective if you have more complete rainbow tables.
2. LCP is a free alternative to the now dead L0phtcrack.
3. John the Ripper is a very versatile password cracking tool. It's supported on different architectures and operating systems (UNIX, Windows, OpenVMS, etc) and it's quite fast. Besides several crypt(3) password hash types most commonly found on various Unix flavors, supported out of the box are Kerberos AFS and Windows NT/2000/XP/2003 LM hashes, plus several more with contributed patches.
4. Medusa is a very fast parallel brute force login password cracker.
5. Elcomsoft Password Recovery suite: anything from office, archives, pdf files etc. to email clients. These are commercial products though.
6. CmosPwd decrypts password stored in CMOS used to access BIOS SETUP. Works on a lot of BIOSes (AMI, Award, Phoenix, IBM, etc). It can also be used to backup, restore or erase the BIOS.
7. AccessData Decryption Tools: PRTK - Password Recovery Toolkit, DNA - Distributed Network Attack, PORT - Portable Office Rainbow Tables are some of the best and fastest tools in the business.
8. Offline NT Password and Registry editor - a utility to (re)set the password of any user that has a valid (local) account on your NT system. You do not need to know the old password to set a new one. Features a registry editor. Supports 32 and 64 versions of Vista (and NTFSv5).
9. Elcomsoft Distributed Password Recovery is designed for distributed recovery of forgotten or lost passwords of different documents. Version 2.0 adds support for Windows SYSKEY startup passwords, passwords stored in Domain Cached Credentials, includes updated Adobe Acrobat module, and provides hardware acceleration (now up to 25 times faster!) for NTLM password recovery using GeForce 8 video cards.
10. Dialupass - Dialup Password Recovery - Recovers the passwords of dialup entries (VPN and Internet connections) on Windows systems. NirSoft provides a couple of free password recovery tools for various products such as Instant Messaging applications, cached passwords stored by Internet Browsers, E-mail clients and so on.

Here's a little cool trick for recovering cached passwords (asterisk passwords) stored in your Internet Browser (Firefox, Opera, Internet Explorer or anything with JavaScript).

Steganalysis and stenography: how to detect hidden data using stenography.

1. Stegdetect finds hidden information in JPEG images using such steganography schemes as F5, Invisible Secrets, JPHide, and JSteg. XSteg is a GUI.
2. Stego Suite is a powerful commercial Stenography detection toolkit, consisting of 3 major tools.
3. Stegkit is an Automated Steganalysis Tool.
4. Digital Invisible Ink Toolkit is an open-source cross-platform image steganography suite that includes both steganography and steganalysis implementations.
5. StegSpy will detect steganography and the program used to hide the message.
6. SteGUI is a StegHide GUI.
7. Digital Watermarking allows you to hide copywrite information and such in media (images and such) that's present even after encoding to another format (bmp->jpg), printing, copy/paste, etc. You can use ImageMagick of various Photoshop plugins to do this.
8. Stepic is a Python module and command line tool for hiding arbitrary data within images by slightly modifying the colors. These modifications are generally imperceptible to humans, but are machine detectable.
9. wbStego4 offers steganography in bitmaps, text files, HTML files and PDF files. It is has two very user-friendly interfaces and is ideal for securely transmitting data online or adding copyright information, especially with the copyright information manager.
10. NL Stego is a system for text generation and text-based steganography. It combines Markov Models of several orders to generate random text resembling a given training text (or text corpus). It can also embed secret messages into pseudo-random generated text.
11. Steghide is an Open Source (GPL) steganography program that is able to hide data in various kinds of image- and audio-files. The color- respectivly sample-frequencies are not changed thus making the embedding resistant against first-order statistical tests. Supports compression of embedded data, encryption of embedded data, embedding of a checksum to verify the integrity of the extraced data and has support for JPEG, BMP, WAV and AU files.
12. StegFS is an Open Source (GPL) Steganographic File System for Linux. Not only does it encrypt data, it also hides it such that it cannot be proved to be there.

Filesystem tools: Data Recovery.

1. Undelete Plus is a free deleted file recovery tool that works for all versions of Windows (95-Vista), FAT12/16/32, NTFS and NTFS5 filesystems and can perform recovery on various solid state devices.
2. Avira UnErase Personal - a freely available unerase product.
3. TestDisk is a free (GPL) data recovery software that can fix partition tables, recover deleted partitions and rebuild NTFS boot sectors. It can find lost partitions (anything from BSD disklabels to IBM JFS, it supports pretty much anything).
4. GNU Parted is a program for creating, destroying, resizing, checking, and copying partitions, and the file systems on them. This is useful for creating space for new operating systems, reorganising hard disk usage, copying data between hard disks, and disk imaging. It can also be used to attempt recovery of the partition table similar to TestDisk (rescue START END).
5. Stellar Phoenix has various UNIX and *NIX (SCO OpenServer, Unixware, Sun Solaris, *BSD, HP-UX, MacOS) data recovery tools as well as some Windows Data Recovery tools. They are, however, commercial products.
6. R-Studio is a data recovery software suite that can recover files from FAT(12-32), NTFS, NTFS 5, HFS/HFS+, FFS, UFS/UFS2 (*BSD, Solaris), Ext2/Ext3 (Linux) and so on. This is a commercial product.
7. DiskInternals NTFS Recovery is a fully automatic utility that recovers data from damaged or formatted disks. This is a commercial product.
8. SystemRescueCD is a Linux system on a bootable CD/DVD for repairing your system and your data after a crash. It also aims to provide an easy way to carry out admin tasks on your computer, such as creating and editing the partitions of the hard disk. It contains a lot of system utilities (parted, partimage, fstools) and basic ones (editors, midnight commander, network tools). The kernel of the system supports most important file systems (ext2/ext3, reiserfs, reiser4, xfs, jfs, vfat, ntfs, iso9660), and network ones (SMB/CIFS and NFS).
9. PhotoRec is file data recovery software designed to recover lost files including video, documents and archives from Hard Disks and CDRom and lost pictures (thus, its 'Photo Recovery' name) from digital camera memory. PhotoRec ignores the filesystem and goes after the underlying data, so it will still work even if your media's filesystem has been severely damaged or re-formatted.
10. Datarescue PhotoRescue Advanced is picture and photo data recovery solution made by the creators of IDA Pro. PhotoRescue will undelete, unerase and recover pictures and files lost on corrupted, erased or damaged compact flash (CF) cards, SD Cards, Memory Sticks, SmartMedia and XD cards even when other solutions fail. Once the data is recovered, it guarantees its integrity. It supports recovery of all file types and is optimized for JPG, TIFF, GIF and BMP, as well as most camera RAW formats: CR2, RAW, RAF, CRW, NEF, ORF, MRW, etc and many types of movie files. In some cases, we can even rebuild pictures that have suffered minor corruptions.
11. MiTeC Windows Registry Recovery - crashed machine registry configuration data recovery.

Cryptography tools:Once the data has been collected, disks and media has been imaged, it now needs to be encrypted, hashed and digitally signed in order to be properly stored.

Update: Truecrypt is no longer mainted - see https://www.comparitech.com/blog/information-security/truecrypt-is-discoutinued-try-these-free-alternatives/

1. Truecrypt is a powerful open source encryption software that works on Windows (2000, 2003, XP, Vista) and Linux. It can do on the fly encryption, it can encrypt whole partitions or mass storage devices, it supports stenography (hidden volumes within an encrypted partition) for plausible deniability and supports AES-256, Serpent and Twofish encryption. It can also escrow keys (so you can't access the data without say the two USB sticks with the keys and the passphrase), and supports both password and key authentication. Note: when you're using encryption, you should also use encrypted swap or make sure you zero it out when you're done.
2. Cryptsetup-luks is an interface is based on the original cryptsetup utility and retains full compatibility, but adds extra commands to deal with the Linux Unified Key Setup (LUKS) on-disk format. This format provides additional features such as key management and key strengthening, and remembers encrypted volume configuration across reboots The Linux Unified Key Setup (LUKS) - Under Windows, LUKS encrypted disks can be used with FreeOTFE (a free, open source, "on-the-fly" transparent disk encryption program for PCs and PDAs).
3. FreeBSD GELI - cryptographic GEOM class available as of FreeBSD 6.0. The geli utility is different to gbde; it offers different features and uses a different scheme for doing cryptographic work.
4. NetBSD CGD - cryptographic device driver provides functionality which allows you to use disks or partitions for encrypted storage. After providing the appropriate key, the encrypted partition is accessible using cgd pseudo-devices.
5. OpenBSD vnconfig(8) - provides encrypted svnd's via the "vnconfig -K rounds" options associates an encryption key with the device. All data will be encrypted using the Blowfish cipher before it is written to the disk. The user is asked for both a passphrase and the name of a salt file. OpenBSD also provides encrypted swap by default.
6. OpenPGP - Open Pretty Good Privacy provides data integrity services for messages and data files by using digital signatures, encryption, compression and Radix-64 conversion. In addition, OpenPGP provides key management and certificate services. The GNU Privacy Guard (GnuPG) is the OpenPGP implementation of the GNU project. GnuPG is fully OpenPGP compliant, supports most of the optional features and provides some extra features. GnuPG is used as the standard encryption and signing tool of all GNU/Linux distributions.

Intrusion Detection and vulnerability scanners: Sometimes you need to analyze live systems during the incident. A crime has not yet been committed, or the perpetrator is still active. Intrusion detection, network monitoring and security auditing tools. Tools like AIDE, Tripwire, BART and such would need to be installed and configured before the incident occurs, but in case they are, it's good to know how to extract the right information from the hash database and figure out what files have changed and such.

Sometimes something as simple as a simple vulnerability scan using tools like nmap, Nessus and such may reveal us ways on how a perpetrator might have gained illegal access to a system.

1. AIDE - Advanced Intrusion Detection Environment is an open source intrusion detection system similar to TripWire. It maintains a database of files and hashes (md5,sha1,rmd160,tiger,haval,etc) that are used to check the integrity of a file.
2. Wireshark - Network protocol analyzer for Windows and Unix that allows examination of data from a live network, or from a capture file on disk. Also known as Ethereal (before the name change). Alternatives include tcpdump (most UNIX systems), snoop(1M) (Solaris) and Windows Network Monitor on Windows. Helps detect data theft or covert channels used to leak out information.
3. Tiger is a security tool that can be use both as a security audit and intrusion detection system. It supports multiple UNIX platforms. Unlike other tools, Tiger needs only of POSIX tools and is written entirely in shell language.
4. Lynis is an auditing tool for Unix (specialists). It scans the system configuration and creates an overview of information usable by professional auditors.
5. Rootkit Hunter scans for rootkits, backdoors and local exploits by running various tests. On Windows, you would use Sysinternals "autoruns" and "Rootkit Revealer".
6. Microsoft Baseline Security Analyzer (MBSA) is used to detect common security misconfiguration and missing security updates on your computer systems as well as to ensure the system conforms to a baseline security level.
7. BART (Basic Audit Reporting Tool) and BSM (Basic Security Module) are auditing tools used on Solaris.
8. Bastille is a hardening tool that can maintain logs and audits a forensics expert should know how to parse. Bastille is also used on HP-UX.
9. The Metasploit Framework is a development platform for creating security tools and exploits.

Antivirus and anti-spyware products: Finding malware.The system should be scanned for trojans, keyloggers and other types of malware (make sure you only scan the image, and take no action, just log their existence).

1. Avira AntiVir PersonalEdition Classic is a free Antivirus product that is available on multiple platforms (Windows 2000 / XP / Vista 32 Bit and 64 Bit, Linux / FreeBSD / OpenBSD / Solaris). License is available for home use.
2. Clam AntiVirus is an open source (GPL) anti-virus toolkit for UNIX. This can be used in a commercial environment. ClamWin is a free, open source antivirus software for Microsoft Windows 98/Me/2000/XP/2003/Vista. It provides a graphical user interface to the ClamAV (Clam AntiVirus) engine.
3. AVG Anti-Virus Free Edition - only available for single computer use for home and non commercial use.
4. BitDefender 8 Free Edition is an on-demand virus scanner, which is best used in a system recovery or forensics role.
5. bdc - BitDefender Console Antivirus for FreeBSD.
6. AVG Anti-Spyware Free Edition - free antispyware solution available at no cost to home users and provides a high level of detection capability.
7. Windows Defender - Free anti-spyware tool from Microsoft.
8. Spybot Search and Destroy - Free spyware removal tool.
9. Spyware Terminator - Spyware removal tool. Can integrate ClamAV Antivirus. Free for personal and commercial use.
10. HijackThis is a free utility which quickly scans your Windows computer to find settings that may have been changed by spyware, malware or other unwanted programs. HijackThis creates a report, or log file, with the results of the scan. It generates in depth report of registry and file settings from your computer. HijackThis makes no separation between safe and unsafe settings in its scan results giving you the ability to selectively remove items from your machine. In addition to this scan and remove capability HijackThis comes with several tools useful in manually removing malware from a computer.

Malware analysis, debugging and Reverse Engineering tools:Sometimes you happen to find rootkits, viruses, worms and all kinds of malware or software that requires the use of reverse engineering to figure out. Here's a couple of free to use tools:

1. OllyDbg is a free debugger that emphasizes binary code analysis, which is useful when source code is not available. It traces registers, recognizes procedures, API calls, switches, tables, constants and strings, as well as locates routines from object files and libraries.
2. RR0D is a ring 0 debugger. It offers the possibility to debug any kind of code (kernel/user/rasta land). Its philosophy is to be OS independent. That's why RR0D can today be installed on Linux, *BSD, Wind0ws. It's a free alternative to SoftIce.
3. Hackman Suite is a multi-module all purpose debugging tool. It includes a hex editor, a disassembler, a template editor, a hex calculator and other everyday useful tools to assist programmers and code testers with the most common tasks.
4. IDA Pro 4.9 Freeware (Interactive Disassembler) is a disassembler and debugger with lots of features that is very useful for reverse engineering. It's basically a "lite" version of the powerful IDA Pro.
5. SpyStudio is a powerful application that simplifies the code execution interception operations, also called "hooking". Users can now easily monitor and gain control over processes in their systems, to really know what is happening in the Operating System and it's applications. With SpyStudio you can monitor and intercept API calls at any time, change its parameters, and resume execution.
6. objdump is a program for displaying various information about object files. For instance, it can be used as a disassembler to view executable in assembly form. It is part of the GNU binutils for fine-grained control over executable and other binary data. It runs on multiple platforms.
7. GDB, the GNU Project debugger, allows you to see what is going on `inside' another program while it executes -- or what another program was doing at the moment it crashed.
8. GNU DDD is a graphical front-end for command-line debuggers such as GDB, DBX, WDB, Ladebug, JDB, XDB, the Perl debugger, the bash debugger, or the Python debugger. Besides ``usual'' front-end features such as viewing source texts, DDD has become famous through its interactive graphical data display, where data structures are displayed as graphs.
9. Debugging Tools for Windows are used to debug drivers, applications, and services on systems running Windows NT 4.0, Windows 2000, Windows XP, Windows Server 2003, Windows Vista, and Windows Server 2008 as well as for debugging the operating system itself. Versions of the Debugging Tools for Windows package are available for 32-bit x86, native Intel Itanium, and native x64 platforms. Debugging Tools for Windows includes WinDbg, a powerful debugger with a graphical interface and a console interface, as well as the console-based debuggers NTSD, CDB, and KD and the Windows Debugging Symbols.
10. strace (system call tracer), ltrace (library call tracer), xtrace (eXtended trace utility, similar to strace, ptrace, truss, but with extended functionality and unique features, such as dumping function calls (dynamically or statically linked), dumping call stack and more), truss (trace system calls and signals - Solaris), ktrace (enables kernel process tracing - OpenBSD) and Valgrind (executes a program under emulation, performing analysis according to one of the many plug-in modules as desired. You can write your own plug-in module as desired) are UNIX programs that let you run a program while watching the actions it performs.
11. DTrace is a comprehensive dynamic tracing framework for the Solaris Operating Environment (also ported to MacOS X - XRays and FreeBSD). DTrace provides a powerful infrastructure to permit administrators, developers, and service personnel to concisely answer arbitrary questions about the behavior of the operating system and user programs.

Timeline editors: analyze and evaluate the data obtained from a system and use it to determine what happened.

1. Zeitline is a Java/Swing tool that allows a computer forensic investigator to import events from various sources of a computing system or network and then order and classify them into one or more timelines of events.
2. mac-robber is a digital investigation tool that collects data from allocated files in a mounted file system. This is useful during incident response when analyzing a live system or when analyzing a dead system in a lab. The data can be used by the mactime tool in The Sleuth Kit to make a timeline of file activity. The mac-robber tool is based on the grave-robber tool from TCT and is written in C instead of Perl.

Data carving, analysis: (extracting a collection of data from a larger data set) - a file recovery technique frequently occurs during a digital investigation when the unallocated file system space is analyzed to extract files. The files are "carved" from the unallocated space using file type-specific header and footer values. File system structures are not used during the process.

1. FTimes (File Topography and Integrity Monitoring on an Enterprise Scale) is a system baselining and evidence collection tool. The primary purpose of ftimes is to gather and/or develop topographical information and attributes about specified directories and files in a manner conducive to intrusion and forensic analysis.
2. MiTeC Windows File Analyzer - tool for forensic analysis of Thumbnail Databases, Prefetch files, shortcuts, IExplore Index.DAT files and Recycle Bin contents on a Windows system. Will print a report of analyzed files.
3. Foremost is a console program to recover files based on their headers, footers, and internal data structures. Foremost can work on image files, such as those generated by dd, Safeback, Encase, etc, or directly on a drive. The headers and footers can be specified by a configuration file or you can use command line switches to specify built-in file types. These built-in types look at the data structures of a given file format allowing for a more reliable and faster recovery.
4. Scalpel is a fast file carver that reads a database of header and footer definitions and extracts matching files from a set of image files or raw device files. Scalpel is filesystem-independent and will carve files from FATx, NTFS, ext2/3, or raw partitions. It is useful for both digital forensics investigation and file recovery. Scalpel resulted from a complete rewrite of foremost 0.69, a popular open source file carver, to enhance performance and decrease memory usage.
5. ReviveIT (Revit) - smart data recovery tool (file carving).
6. Magic Rescue looks at "magic bytes" for carving out files from raw data.

Forensic toolkit LiveCDs:

1. Helix is Knoppix based Linux LiveCD containing various forensics tools.
2. Knoppix STD is a Linux based LiveCD containing various digital forensics tools.
3. The MacQuisition Boot Disk is a forensic acquisition tool used to safely and easily image Mac source drives using the source system. MacQuisition provides an intuitive user interface to the traditional command line, providing both beginner and advanced forensic examiners with a valuable tool to:
4. Winternals ERD Commander boots dead systems directly from CD into a Windows-like repair environment. It's now part of the Windows Optimization Pack.
5. BackTrack is the result of merging Auditor and Whax into a single penetration testing Linux LiveCD. It contains an assorted suite of pen-testing, data recovery and digital forensics tools.
6. SMART Linux is a Slackware based Data Forensics and Incident Response LiveCD.
7. Snarl is a FreeBSD based forensics Bootable ISO (includes Autopsy and Sleuth Kit).
8. Penguin SleuthKit Bootable CD - A Linux LiveCD that includes SleuthKit.
9. UBCD4Win - Ultimate Boot CD for Windows - is a bootable recovery CD based on BartPE that contains software used for repairing, restoring, or diagnosing almost computer issues. It's also easy to costomize and add various forensic tools to a Windows XP SP2 LiveCD.
10. The Farmer's Boot CD (FBCD) is a Linux boot CD designed for on-site previewing of systems before acquiring. It contains a number of programs to preview both Windows and Linux systems in a forensically sound manner. A commercial product.

Reference and Documentation, Whitepapers: Material you need to consult and read:

1. Bruce Schneier: His Applied Cryptography and security books as well as his blog posts and articles are a valuable resource for security experts world wide. Beyond Fear there is Bruce Schneier.
2. The Open Source Digital Forensics site is a reference for the use of open source software in digital investigations (a.k.a. digital forensics, computer forensics, incident response). Open source tools may have a legal benefit over closed source tools because they have a documented procedure and allow the investigator to verify that a tool does what it claims.
3. The Forensics Wiki, a Creative Commons-licensed wiki devoted to information about digital forensics.
4. The International Journal of Digital Evidence (IJDE)
5. NIST - National Institute of Standards and Technology - CSRC - Computer Security Division
6. DFRWS (Digital Forensics Research Workshop) is dedicated to the sharing of knowledge and ideas about digital forensics research.
7. Computer Forensics Links & Whitepapers (Forensics.nl)
8. Digital Forensics Links and Resources
9. SANS Institute - Network, Security, Computer, Audit Information Reading Room - Whitepapers on Digital Forensics
10. Forensic Focus - Computer Forensics News, Information and Community
11. The Electronic Evidence Information Center
12. Computer Forensics World - A Community of Computer Forensics Professionals

"Vs lbh pna ernq guvf lbh'er nyzbfg nf fzneg nf cuo."

GPU accelerated password cracking

Modern versions of Windows like Vista no longer use LM (which could be cracked in minutes using rainbow tables and tools like Ophcrack, especially since it grouped passwords in 7 character pieces). NTLM tends to be more difficult to crack.

Now that GPU HPC clustering has been around for a while, it was only a matter of time until someone implemented the concept for usage in password cracking. I've seen specialized FPGA based machines like Copacobana accelerate DES cracking for example. Or custom chip machins like Deep Crack. But there are cheaper and more effective resources available, right on your home desktop!

A simple GeForce 8 card can have 128 stream processing units and they are very suitable for fixed point arithmetics.

Elcomsoft has released a product that uses such a technology: Elcomsoft Distributed Password Recovery 2. And it will soon incorporate the technology into all their products.

What does this mean? It means that cracking a typical 8 character NTLM has can take as little as 4-5 days instead of months. Using a GPU means the process can be 25 times faster than normal! Not to mention you can just use 4 GPU's on your machine to really speed things up :-).

Guess it's time to enforce a more strict password policy. 12 or more characters should be a minimum now.

Want to use GPU clustering for other kind of HPC applications? Grab the NVIDIA CUDA toolkit and an MPI manual, and start coding. Or maybe you're an ATI person? Then take a look at Folding@Home on ATI GPU's page.

Google GMAIL IMAP Support

Google now allows you to enable IMAP support for your GMail account.

Just go to Settings - forwarding and enable IMAP, then configure your MUA to use IMAP.

For those of you that use Outlook 2007 (or some kind of Outlook) you'll need to export your email as as a .pst file, create the IMAP account then import it (or just use it as it is) in order to move from POP3 to IMAP.

I like the fact Google has IMAP now... and even Yahoo mail has unlimited storage and a neat interface (damn thing is quite fast).

Synergy: sharing Keyboard/Video/Mouse across different machines.

Synergy is a neat piece of software that lets you share a single keyboard/video/mouse across your machines. It basically acts like having TwinView or Xinerma, but on different systems. Can you do it across your Sun Solaris machine and your Windows box? Sure!

You can arrange the monitors in patters like square, vertical, horizontal, whatever.

So if you have two or more desktops, and want that Xinerma feel, or just to get rid of two (or more) keyboards on your desk, look at Synergy, it's a pretty amazing piece of software. Beats having to VNC manually into your other Desktop :-).

Tempest - interpreting electromagnetic signals emitted by your monitor

TEMPEST was code name given for to a NSA project regarding technologies used to intercept and decipher the electromagnetic signals that all computers emit. Later, TEMPEST became a backronym: Telecommunications Electronics Material Protected from Emanating Spurious Transmissions.

Today, in military circles, the term has been officially supplanted by Emsec (for Emissions Security); however, the term Tempest is still widely used in the civilian arena.

About two years ago, I've stumbled across this little program in FreeBSD's ports collection:
Tempest For Eliza is a program that allows you to control the electromagnetic signals your monitor emits (by controlling the image displayed on screen) in such a way that you can send AM radio signals with audio (music).


Some demonstrations of Tempest in action:

http://www.youtube.com/watch?v=dWmGNL-ttDY
http://www.youtube.com/watch?v=F-O1QWWmVmg


Although this use seems harmless enough, one can capture the electromagnetic signals emitted by your monitor and reproduce the image on a another machine. This process is know as van Eck phreaking.

A working implementation of that concept is proven in:

http://www.cl.cam.ac.uk/~mgk25/ih98-tempest.pdf

Or, using like 100$ worth of RadioShack components, one can build a fully working receiver and capture data that's good enough to read (may not be effective for modern monitors, but I've seen a few whitepapers on the subject detailing such functional units).

Here is an open source implementation of that concept:
http://eckbox.sourceforge.net/

Guess it's time to bring out the old Faraday cage, hey guys? :-).



And just in case you didn't realize, this is also possible with flat screen monitors, not just CRTs.

So, is this actually used in real life? You bet! Here's an example of a TEMPEST attack on a voting machine (computer). It basically allows people to see who you've voted for. So unless your gear is certified NATO SDIP-27 Level A, you are pretty much vulnerable to such attacks.

Similar attacks can also be made on copper cabling (such as your Ethernet network). When in doubt, use Fibre!

SXCE 75 is out, with Xen support!

Solaris Express Community Edition (SVN_75a) is out, and it comes with Xen!

See http://dlc.sun.com/osol/on/downloads/b75/on-changelog-b75.html for the changelist.
See http://www.opensolaris.org/os/community/on/flag-days for a list of things to come to SXCE :-).

Download SXCE DVD: http://opensolaris.org/os/downloads/sol_ex_dvd/
Download SXCE CD: http://opensolaris.org/os/downloads/sol_ex_cd/

Vodaphone Huawei HSDPA USB Modem - Vista compatible firmware.

Got a Huawei HSDPA modem from Vodaphone and it doesn't work on Vista? You need to perform a firmware upgrade. The risks involved are your own responsibility, of course. If you brick it, don't blame me. Make sure Vodaphone actually supports such an update (they have some details on their site, google!).

Just grab the firmware / software update pack from:
http://www.mobileconnected.co.uk/wp-content/uploads/software/E220_update_Vista.zip

Make sure you read the instructions and read the documentation on Vodaphone's website.


LATER EDIT: There's an official Vodaphone update. Use this!

Multiple file selection (Ctrl-a, shift, ctrl, drag) broken in Vista

Sometimes multiple file selection is broken in Windows Vista. Dragging a box around folders and such, multiple selection with Ctrl or Shift, or even Ctrl-A won't work inside folders (although they may work in the drive view (my computer) or on the Desktop.

For some reason, Explorer saves "single select" to folder views and saved defaults.

I've stumbled across a script that fixes this without too much work:

http://forums.microsoft.com/TechNet/ShowPost.aspx?PostID=981031&SiteID=17
http://www.vistax64.com/vista-file-management/98609-trouble-multiple-file-selection-vista-explorer.html

The script seems to work quite well. I ran it, logged off, logged on and multiple selection worked perfectly :-). Thanks go to Keith Miller.

Backup and Recovery Primer - Part I: Backup Strategy

Backup Basics and best practice. Developing a backup strategy.

A proper backup strategy should be part of every disaster recovery plan. Every company should have at least a basic DRP, business continuity plan and employ risk management techniques.

Data backup and recovery falls in the prevention against data loss category, along side surge protection, interruptible power supplies (UPS), fire prevention systems, data security software (IDS, Antivirus Software, etc).

A backup is not a simple thing. You can't just throw your files on some random storage array every now and then and expect things to work out. A proper backup requires planning, a backup strategy, risk assessment and team work.

1) What does a backup strategy help us mitigate against?

• Disk failure. - Face it, disks fail. Often.
• Filesystem corruption or disk corruption, or other events that leave our disk in an inconsistent state. - I've seen this one too many times. You run out of inodes on a filesystem and you also need to run fsck on it, but you cannot. And you cannot mount it either. Or your filesystem decides to break, and corrupts your data. Filesystem corruption is more often in the case of power failures and forced shutdowns.
  
• File deletion and accidents: I've seen things like "# rm -rf $HOME/*" on a UNIX system (where the default root $HOME is /, of course) and various other accidents using pipes or dd. And it's usually easier to restore files from backup than to try to restore them after you've erased them. Also, allowing testing and development on production systems will eventually lead to such accidents taking down your main production database sooner or later...
  
• Stolen or destroyed disks / machines. Laptops are especially vulnerable to this.
• Tampering with the data: Viruses, exploits, hackers may modify and tamper with your data. You may need to perform a roll-back to a previous state of your data.

But, most important, a proper backup strategy will save you and your company money. Data loss and downtime will lead to financial losses due to lost customers, wasted money, image and reputation loss, morale going down among your staff or even legal problems. Not to mention there are things that simply cannot be recreated once lost, like access logs and such.

You must fully understand why you need a backup strategy, what it is you're protecting (don't just think of data, think of your companies reputation, your job security, loss of revenue and such).

2) Who is responsible for the integrity of the data

The data owner. Which, in most cases, is upper management. It's management's responsibility to do a risk assessment, and deploy the proper business continuity and disaster recovery plans. They often decide to delegate permissions and responsibility to such tasks down the chain of command.

3) What do you make backup copies of?

Just your production database? The whole system? Just the critical data tables?
It's your responsibility to asses what data is critical, how long it would take to restore a system in case of a disaster (hope for the best, but expect the worse. Can you cope with a fire? How about and earthquake? How about a disgruntled employee purposely altering or erasing your data?).

4) How often do you backup?

Do you need continuous data protection (CDP)? Can you accept loosing a day's work? How about losing one day of everyone's work? If you have 1000 users, that may as well be 3 years worth of work right there, in a single day. What will that cost you?

5) How do you backup?

Will you be using tapes? Will you purchase a storage array and use it for backup purposes? Will you be using optical media? Are you going to do this across the network? How will the network cope with the load generated by the backup?

What software will you use? How much does it cost? What about the total cost of ownership? Will future versions be supported and be able to access older backups?

Are you storing all the relevant information to restoring your systems? How about permissions or disk volume / partition configuration?

6) Where will you store your backup?

Doing a backup on the same disk, or on the same disk on the machine you're backing up is usually a very bad idea. Or leaving the tape in the tape drive or on top of your machine. Think about it. That tape probably contains vital and confidential information. You don't want it to go up in flames with your production systems, or get stolen along with the confidential business records it contains. You really need to evaluate all possibilities in terms of on-site storage (fire-proof tape safe, tape robots or even a simple storage cabinet) as well as off-site storage.

Also, consider online backup vs. offline backup. Consider something as simple as a rsync-snapshot/rsnapshot/rsync-with snapshots or as complex and powerful as Sun's StorageTek Availability Suite.

7) Monitoring and testing

Don't just dump the fs content to a tape and leave it there. Test and monitor your backup and restore procedures.

Test your backup software. Some companies even go so far as to test a new backup solution for years in parallel to the old solution before committing themselves to using only the new backup method.

Always keep your backup policy up to date. Make sure your plans and strategies reflect real life situations.

8) Restoring data

Backups in themselves have little importance. It's restoring the data that matters. How will you get data off those tapes / cds? How long will it take? Will you have your system up and running to a satisfactory baseline, fast? How about restoring individual files from a certain point in time? Say a user requires an 3 month old email. What will you do then, restore the whole mail database, from 23 tapes by restoring a full backup, the incremental, the differentials, etc? That wouldn't be too much fun, now would it...

It's usually a good idea to take a layered approach to backup. Like in Windows, you have System Restore to revert to a previous state of the operating system, Shadow Copies to restore older versions of files, you have ntbackup to do a system state, registry and filesystem backup, and Windows Complete PC Backup (or tools like Norton Ghost) for bare metal recovery.

Make sure your mail server stores emails for a certain period of time. Have your log servers store logs and rotate / archive them at a specific interval. Here is where products like QFS / SAMfs really shine.

The idea is simple. It's a lot easier to restore and manipulate files from lower backup levels than it is to manipulate bare metal recovery backups or full filesystem backups.

9) Scalability

Data within a company grows at an exponential rate. As data grows, so will your backup needs. You will need to plan ahead, make sure all your backup systems are scalable and can handle growth. If you hit some weird limitation (like with some filesystems for example, that can't grow beyond 1-2 TB and so on), that's pretty much it..

Also, make sure that while your systems are scalable, they don't grow beyond your power to manage them. Keep things simple and easy to understand. Document everything.