Convert AWK to Perl using a2p

Automatic conversion of AWK scripts to perl.

A little awk script that numbers lines:

ls -la /usr | ./linum.pl
  1 : total 240
  2 : drwxr-xr-x+  12 root root  4096 2010-06-21 06:47 .
  3 : drwxr-xr-x   22 root root  4096 2011-05-11 12:06 ..
  4 : drwxr-xr-x+   2 root root 69632 2011-05-24 14:07 bin
  5 : drwxr-xr-x+   2 root root  4096 2010-05-05 11:33 games
  6 : drwxr-xr-x+   5 root root  4096 2010-06-21 06:47 i586-suse-linux
  7 : drwxr-xr-x+  52 root root  4096 2011-05-13 04:17 include
  8 : drwxr-xr-x+ 156 root root 69632 2011-05-24 14:07 lib
  9 : drwxr-xr-x+  10 root root  4096 2010-06-21 06:47 local
 10 : drwxr-xr-x+   2 root root 12288 2011-05-24 14:07 sbin
 11 : drwxr-xr-x+ 213 root root  4096 2011-05-23 13:19 share

Let's convert this to perl using a2p.

echo '{ printf("%5d : %s\n", NR, $0) }' | a2p > linum.pl
chmod +x linum.pl

Let's try it out:

ls -la /usr | ./linum.pl
  1 : total 240
  2 : drwxr-xr-x+  12 root root  4096 2010-06-21 06:47 .
  3 : drwxr-xr-x   22 root root  4096 2011-05-11 12:06 ..
  4 : drwxr-xr-x+   2 root root 69632 2011-05-24 14:07 bin
  5 : drwxr-xr-x+   2 root root  4096 2010-05-05 11:33 games
  6 : drwxr-xr-x+   5 root root  4096 2010-06-21 06:47 i586-suse-linux
  7 : drwxr-xr-x+  52 root root  4096 2011-05-13 04:17 include
  8 : drwxr-xr-x+ 156 root root 69632 2011-05-24 14:07 lib
  9 : drwxr-xr-x+  10 root root  4096 2010-06-21 06:47 local
 10 : drwxr-xr-x+   2 root root 12288 2011-05-24 14:07 sbin
 11 : drwxr-xr-x+ 213 root root  4096 2011-05-23 13:19 share

The code block looks pretty good:

while (<>) {
chomp;      # strip record separator
printf "%5d : %s\n", $., $_;
}

Unpack functions and BSDProgress - progress bar for your archives.

A colored unpack script using bar:

A little KSH93 function for your .profile. Now you can augomatically unpack most archives while watching the pretty progress bar.

You need bsdprogress (Solaris and Linux port of NetBSD's progress). With a bit work it also works on AIX.

I just re-wrote this for progress, so I didn't check the cpio and such yet.

# Mihai Criveti - Unpack function:
# Progress Unpack - unpack using GNU tar and bsdprogress.
function up {
for archive in $*; do
  print $archive;
  case $archive in
       (*.tar) progress -f $archive gtar xpif - ;;
       (*.tar.gz) progress -f $archive gtar -zpixf - ;;
       (*.tgz) progress -zf $archive gtar -ipxf - ;;
       (*.tar.bz2) progress -f $archive gtar -jipxf - ;;
       (*.tbz) progress -f $archive gtar -jipxf - ;;
       (*.gz) progress -f $archive gunzip -;;
       (*.bz2) progress -f $archive bunzip -;;
       (*.tar.Z) uncompress -c $archive | tar gxvf - ;;
       (*.cpio) cpio -icd < $archive ;;
       (*.Z) uncompress $archive ;;
       (*.rar) unrar x $archive ;;
       (*.zip) unzip $archive ;;
  esac
done
}

Let's say we want to unpack files that end in *bz2 (it will automagically detect .tar.bz2 or tbz2), *gz and a specific tar archive:

cmihai@phobos:/home/cmihai/build$ up g*bz2 c*gz Python\-3.0.1.tar
gnupg-1.4.10.tar.bz2
100% |**********************************************************************|  3330 KB  296.95 KB/s    00:00 ETA
gnupg-2.0.14.tar.bz2
100% |**********************************************************************|  3888 KB  340.20 KB/s    00:00 ETA
ccache-2.4.tar.gz
100% |**********************************************************************| 86363     201.51 KB/s    00:00 ETA
Python-3.0.1.tar
91% |********************************************************               | 43320 KB  787.63 KB/s    05:00 ETA

See: 

http://netbsd.gw.com/cgi-bin/man-cgi?progress++NetBSD-current

http://mac.freshmeat.net/projects/bsdprogress/

It builds fine on AIX (Makefile tweaks), Solaris and Linux. Native on NetBSD.

Other versions:

http://clpbar.sourceforge.net/

http://freshmeat.net/projects/progress

Screenrc hardstatus line

Here's a pretty neat hardstatus line for your .screenrc:

hardstatus alwayslastline "%{-b ck}%?%-Lw%?%{bg}%n*%f %t%?(%u)%?%{wk}%?%+Lw%? %= %{r} %H %{g} %D %d/%m/%Y %0c "

Codepad and Ideone - a new kind of pastebin

Codepad and Ideone are pastebins that let you execute code. Ideone also accepts input:

http://ideone.com/P7WKrT2I

http://ideone.com/o1GGFBV4

http://codepad.org/

This would make an awesome learning tool for schools and such. Make it into some kind of appliance, add a bunch of examples, problems, tests and such and it can teach kids collaborative debugging and the basics of programming.

If I had to guess it's running User Mode Linux (or some similar jail) and ptrace (or some similar systrace or syscall tracer).

I poked a bit with perl "Config" (eg: perl -V) and it looks like they're running Gentoo.. all accounts seem to be on the same instance (nobody). So it's most likely a jail.

osname=linux, osvers=2.6.28-gentoo-r5, archname=i686-linux

uname='linux devel 2.6.28-gentoo-r5 #1 smp sun apr 26 22:00:57 cest 2009 i686 intel(r) core(tm)2 quad cpu q9550 @ 2.83ghz genuineintel gnulinux '

It looks like they did a very good job though, especially with the number of people poking around or running forkbombs :-).

UNIX colors and tools

Just opened a new package of colored chalk so I'll write on my blackboard :-).



Can't really have fun without:

• OpenSSH
  
• GNU screen
• KornShell 93, AWK, 1003.1 Shell & Utilities
  
• Emacs, ed
  
• Mutt
• Midnight Commander (mc)
• Elinks
• Terminus font (sure, it's not a tool, but it sure as hell helps , especially during those 20 hour debugging sessions).

Emacs linum plugin + a 256 color TERMINFO entry + transparent putty can look pretty good:




256 colors _really_ do make quite a difference in Emacs (and vim) btw. Normal dtterm or xterm-color get you the first 8...



All you really need to do to get this working on AIX is to add a new TERMINFO entry (tic). You can easily grab the 256 color xterm with infocmp..

/usr/bin/tput colors
256

VIM looks pretty good too with a 256 color theme also (inkpot):

Export VIM syntax highlighting to HTML

You can export the colorscheme based syntax highlighting in used in your VIM instance to a HTML file.

Just set a colorscheme and enable syntax highligting in VIM"

colorscheme torte
syntax on

Then you can export the same color scheme to a HTML file:

:runtime! syntax/2html.vim

This works for any file type supported by VIM. Just make sure you save the file before using the right extention.

Script for automating hardlink creation

::
:: Linking script
:: Usage: ln "source\REGEXP" destination

@echo off
set sourcedir=%1
set destdir=%2
if "%1" == "" goto usage
echo "Linking" %sourcedir% "\*" to %destdir% "\*"

if not exist %destdir% mkdir %destdir%
:: Create the destination directory if it does not exist

:iterate
shift
if "%1" == "" goto end
for %%i in (%sourcedir%) do fsutil hardlink create %destdir%\%%~nxi %%~fi
goto iterate

:usage
color 4f
echo "Usage: ln.bat source-directory\REGEXP destination-directory
echo "If the destination directory does not exist it will be created."
echo "If a fully qualified path is not used, current directory will be used"
pause
color

:end
set sourcedir=

Scripting VMware Infrastructure with PowerShell

You can script VMware Server and VMware Infrastructure using Microsoft PowerShell CmdLets and the VMware VI Toolkit. It's really great for scheduling and automating tasks, since the scheduler in VMware Infrastructure Client is severely limited



You need to install the .NET Framework (2.0 at least), PowerShell 1.0 and the VMware VI Toolkit. All are freely available.

Get-VICommand to list available cmdlets and read the VI Toolkit Cmdlets Reference for documentation.

SedTris - a Tetris game written in sed.

I've seen irc clients written in sed and I've seen a lot of sed magic in my time, but this is just ridiculous...

A Tetris game written in sed.

http://uuner.doslash.org/forfun/sedtris.sed

AWK tricks - clearing the screen and scrollback buffer.

You can put this in your logout profile to fill up the screen buffer (clear just won't do on some systems with a multiple page console scroll-back buffer).

Filling the screen buffer with new lines:

awk -F 'BEGIN{ for (i = 1; i <= 3000; i++) printf "\n"; exit}'

See also:

http://www.openbsd.org/faq/faq7.html#ConsoleClear

Using AWK to convert UNIX passwords from HP-UX to Solaris

Converting password hashes from HP-UX 11.11 to Solaris is pretty simple if you are using UNIX crypt passwords (if HP-UX isn't a Trusted System. If it is, it will use bigcrypt passwords, > 8 characters, converting them to Solaris UNIX crypt could be problematic).

Here's the gest of it:

On the HP-UX System, we create a test user:

# useradd test
# passwd test
test

Now we convert the passwd file to generate passwd entries for Solaris:

• # awk ' BEGIN { FS = ":" } { print $1":x:" $3 ":" $4 "::/export/home/" $1 ":/usr/bin/sh" }' /etc/passwd
• test:x:107:20::/export/home/test:/usr/bin/sh

And we create the shadow file entries, including the password hash:

• # awk ' BEGIN { FS = ":" } { print $1":"$2"::::::" }' /etc/passwd
• test:lsDWJo7M.iAhY::::::

Just add them using /usr/ucb/vipw to the password file, edit the shadow file for consistency and test. Be sure to backup the files and to test using a few users at first.

• $ su test
• Password:
• $ id
• uid=127(test) gid=120
• $ whoami
• test
• $ echo $HOME
• /export/home/test
• $ echo $SHELL
• /usr/bin/sh

Mix with some shell scripting and mkdir's and you're set :-). Next time, use LDAP :P.

Custom df (diskfree) column output in Solaris using nawk

Let's say you want to combine some features of "df -h" with "df -n" to show filesystem type and some other custom modifications to the output. This is where awk/nakw/gawk/whatever come in handy:

% df -g | nawk '{if (NR % 5 == 1) printf "%-22s", $1 ; if (NR % 5 == 4) printf "%-10s", "fstype " $1 "\n"; if (NR % 5 == 2) printf "%-30s",$1/2/1024/1024 " GB"; if (NR % 5 == 2) printf "%-30s", $4/2/1024/1024 " GB free "}'

/	33.6627 GB	18.4351 GB free	fstype ufs
/devices	0 GB	0 GB free	fstype devfs
/system/contract	0 GB	0 GB free	fstype ctfs
/proc	0 GB	0 GB free	fstype proc
/etc/mnttab	0 GB	0 GB free	fstype mntfs
/etc/svc/volatile	7.88214 GB	7.8813 GB free	fstype tmpfs
/system/object	0 GB	0 GB free	fstype objfs
/lib/libc.so.1	33.6627 GB	18.4351 GB free	fstype ufs
/dev/fd	0 GB	0 GB free	fstype fd
/tmp	7.88142 GB	7.8813 GB free	fstype tmpfs
/var/run	7.88134 GB	7.8813 GB free	fstype tmpfs
/export/home	74.4858 GB	1.87458 GB free	fstype ufs
/storage	108.639 GB	66.9259 GB free	fstype nfs

You can also add a comma (,) to the separators and output > csv (you can open the comma separated values table in Excel or OpenOffice or any other Spreadsheet application) :-).

Convert NRG images to ISO images (nrg2iso) using dd

If you want to convert Nero "nrg" images to ISO format to mount or burn you can either use nrg2iso, or just use dd to skip the 300k header (will work just fine for DAO images):

dd if=image.nrg of=image.iso bs=4096 skip=75

OpenVMS Tips and Tricks - Clear Screen, ls -lha

Want to clear the screen? "TYPE/PAGE NLA0:" will do the trick. Just alias it to your dos like cls or to a UNIX like clear command :-).

$ CLEAR == "TYPE/PAGE NLA0:"
$ CLEAR
$

Might as well alias LS to a DIR/SIZE/DATE/PROTECTION:

$ LS == "DIRECTORY/SIZE/DATE/PROTECTION"
$ LS *.JAVA

Directory SYS$SYSROOT:[SYSMGR]

HELLO.JAVA;1 1 4-MAR-2008 21:40:06.79 (RWED,RWED,RE,)
HW.JAVA;2 1 4-MAR-2008 22:14:36.28 (RWED,RWED,RE,)

Total of 2 files, 2 blocks.

PowerShell, PowerTab, PSCX, Console, Terminus and Vim

Here's something for all you Powershell / WMIC / VB / cmd script monkeys :-).

You can, of course, script in ksh, Perl or Python if that's your thing. (ActiveState Perl / Python).

Powershell + PowerTab + PSCX + Console + Terminus Programmers Fonts + VIM + VIM PowerShell Syntax Highlighting.



Here's another PowerTab Theme and PSCX (PowerShell Community Extensions):



A cyan BackColor to match my prompt:



And of course, a good color scheme and PoSH syntax highlighting in VIM is also nice:

Emacs.NET

"Imagine if someone wanted to write something like Emacs.Net. Actually don't imagine it, it is happening."

"We are looking for developers/testers to build a tool that I will roughly describe as "Emacs.Net"."

^^ duglasp at microsoft dot com

Well, this should be interesting to watch. Hell, notepad could use a rewrite :P.

OpenBSD automated / unattended installation - Yaifo

Don't have remote servers with KVM? Well, this will help:

Yaifo has been updated for OpenBSD 4.2 and is now actively maintained by Mike Erdely. You can download it here:
http://erdelynet.com/category/tech/yaifo/

It's basically a modified installer with ssh support, RAID support and all that. You can basically do remote, scripted, unattended OpenBSD installations, etc.

Uninstalling Blastwave (or any other package group) with a bit of Shell Scripting on Solaris.

Let's say we've installed a bunch of Solaris packages from Blastwave or another source and want to remove them all without going through the boring process of answering all those questions:

$ pkginfo | grep CSW
system CSWcommon common - common files and dirs for CSW packages
application CSWexpat expat - XML Parser Toolkit
system CSWggettext ggettext - GNU gettext
application CSWglib2 glib2 - the low-level core library for GTK+ and GNOME
system CSWiconv libiconv - GNU iconv library
application CSWmc mc - GNU Midnight Commander
system CSWpkgget pkg_get - CSW version of automated package download tool
system CSWslang slang - library primarily to provide an alternative to libcurses
system CSWsunmath libsunmath - Sun maths library

Well, nothing simpler:

# yes | pkgrm $(pkginfo | grep CSW | awk '{ print $2 }' | xargs)

The following package is currently installed:
CSWcommon common - common files and dirs for CSW packages
(i386) 1.4.5

Do you want to remove this package? [y,n,?,q]
## Removing installed package instance
## Verifying package dependencies in global zone

[.....]

Removal of was successful.
# pkginfo | grep CSW
#

Since Blastwave installs by default in /opt/csw, you can probably remove that too if you don't have settings you plan on keeping:

# rm -rf /opt/csw

Enjoy :-).

Killing defunct (zombie) processes on Solaris - preap and orphan processes with init(1M)

A defunct (or zombie) process is one whose exit status has yet to be reaped by its parent. So when a process shows as "defunct" in ps, you need to reap it. Here's how:

preap(1) - force a defunct process to be reaped by its parent

Syntax: /usr/bin/preap PID

So, to get rid of all zombies on our system, all we have to do is script this to real all process marked as defunct:

/usr/bin/preap $(ps -ef | grep defunct | awk '{ print $2 }' | xargs)

So, what's an orphan process then? If the parent of a defunct process happens to exit, it becomes an orphan, and is asigned PID 1 as a parent - see init(1M).

Read more on zombies, defunct processes, orphans, preap and init here.

Debian Etch AMD64 NVIDIA install one-liner

Add contrib and non-free repositories to sources.list:

cmihai@db:~$ cat /etc/apt/sources.list

deb http://ftp.lug.ro/debian/ etch main contrib non-free
deb-src http://ftp.lug.ro/debian/ etch main contrib non-free

deb http://ftp.lug.ro/debian-security etch/updates main contrib non-free
deb-src http://ftp.lug.ro/debian-security etch/updates main contrib n

Install the packages and configure X (nvidia-xconfig).

$ sudo aptitude install nvidia-kernel-2.6-`uname -r | sed 's,.*-,,g'` nvidia-settings nvidia-glx nvidia-xconfig nvidia-glx-ia32 && nvidia-xconfig