S C R I P T I N G & S H E L L . T I P S


Home | perl | perl one liners | shell | sh-libs | tips | Bottom

--Available Perl scripts--

# Send SMTP mail on the CLI or from somme other code / app.

send_SMTP.pl
Related alternatives: sommething more basic?

# A ftp distribution script

Transfer one or more file(s) in a directory or directory hierarchy to a set of different directory's on one or more other servers. You have the choice of "IN script" parameters to CLI options, configuration via config file(s) or finaly completion via a shell. I don't need secure transfers at this time... but they may follow, feel free to implement it yourself and let me know.
fdist.pl man page Last update: 9 May. 2003
Related alternatives: mirror, check_ftp, ange ftp over ssh for emacs.

# Get todays User Friendly comic strip

The script is used to download the Daily Static Commic Strip of the User Friendly site by Illiad, via a proxy that requires authentication. The result of running this script is a gif image in the current directory with the name : "uftoday.gif"
getufcomic.pl
Related alternatives: ufget.pl

# FTP automation with Perl

ftplib.pl
Net::FTP
CPAN ftpstuff ( There is no hyper link available via the CPAN website to this directory! Or none that I found ;)

--Perl one-liners--

# Generic file editing (Tip to remember: eat your daily "perl pie")

# -i=inplace_edit[with_backup] -p=loop -e=code perl -p -i[.backup-extension] -e 's#string_to_replace#string_to_replace_it_with#ig' file(s)

# Add 4 digit linenumbers to a file

perl -p -i -e'$_ = sprintf "%04d %s", $., $_' test

# CPAN shell

perl -MCPAN -eshell

# Replacing tabs with spaces

perl -0011 -pi -e '/\011/&&($_="$'")' filename

# A grep for the DOS CLI

C:\>copy con grep for ( <STDIN> ) {print $_ if $_ =~ /@ARGV/ ;}; ^Z 1 file(s) copied. C:\>dir | perl grep <filter_string> note: you always need to use "perl" or the ".pl" extention to execute perl scripts on Winblows machines!

# This is a lame attempt to wrap long lines to max.70 characters, they don't even filter emty lines!

cat logfile | perl -pe '$_=join("\n", split(/(.{69})/)); s/\n(.+)/$1\//;'

# More at IBM

# Create a directory and all unexisting directorys inbetween

perl -e ' @dirs=split "/","$ARGV[0]"; shift @dirs; foreach $dir ( @dirs ) { $makedir=$makedir."/$dir"; mkdir $makedir,0750 if ( not -e $makedir ); } ' $MAKEFULLPATH "Kinda long to remember but i use it in shell scripts as a function."

# better grepping with perl

cat somefile | perl -ne '/u/ && print'

--Shell script examples--

# Gathering UID and GID using .netrc

Script created for quickley gathering UID and GID from different servers using a temporary .netrc (shame on me) on a AIX box.
idget.sh
Related alternatives:

# RS/6000 system summary

Platform and OS specifications for AIX, using some undocumented options or hard to get data.
platos.sh
Related alternatives:

# Simple ping sweep

$ip=1
$while [ $ip -lt 31 ]
>do
>ping -c 1 10.66.26.$ip >/tmp/ip.log 2>&1 && print $ip
>let ip=$ip+1
>done

# reversed ps - feel free to revise it (used by me on AIX).

#!/usr/bin/ksh                                                                  
function GetPPName {                                                            
PID=$1                                                                          
PRTEXT=$(ps -fp $PID -F "user pid ppid tty pcpu time vsize args"|tail +2 )      
PPID=$(echo $PRTEXT | awk '{print $3}')                                         
if [[ $PID -gt 1 ]]                                                             
  then                                                                          
  print "$PRTEXT"                                                               
  GetPPName $PPID                                                               
elif [[ $PID -eq 1 ]]                                                           
then                                                                            
  print "$PRTEXT"                                                               
  exit                                                                          
else                                                                            
exit                                                                            
fi                                                                              
}
ps -fp  1 -F  "user pid ppid tty pcpu time vsize args" |head -1                 
GetPPName $1 

# Simple telnet and ftp automation, examples at your request:

#!/bin/sh
tpipe() {
sleep 3;echo 'user'
sleep 1;echo 'passwd'
sleep 1;echo 'who'
sleep 1;echo 'exit'
sleep 5
}
tpipe | telnet localhost

ftp ftp://user:passwd@localhost <file_containing_ftp_commands

echo 'cd /tmp' | ftp ftp://user:passwd@localhost

ftp ftp://user:passwd@localhost <<EOF         
>cd /var
>bye
>EOF

ftp -n << EOF
open $SYSTEM
user $USER $PASS
cd /tmp
put $FILE
bye
EOF

Note: Also check out these options for automation: "(auto)expect", ".netrc" or check out "this" article.

# "Shell Notes and Examples" from the Univ. of Lethbridge

# "Examples" from the book "Portable Shell Programming"

"Portable Shell Programming" by Bruce Blinn, Prentice Hall, New Jersey, 1996.

# ASCII convertions: unix2dos2unix2mac2unix - hummm, i like Unix ;)

How do I convert between Unix and Windows text files?
How do I convert between Unix and Mac OS text files?
"tr" Is okay but cannot translate unix2dos becouse you have to replace 1 with 2 characters, for dos2unix you have to replace 2 with 1 character and you can do it like this "tr -s '\r' '\r\n' < infile > outfile".
My Tip: use Perl!

--Shell Library's--

Here are somme library's, they may need somme work - changes / additions are welcome

AIX JFS calculations / convertions
Controlling processes and stuff - i suggest you also take a look at the rc-scripts of any Linux Distribution before you start using this.
Somme things i use(d) sporadicly and are dumped in this sript for later usage

--Shell tips--


Notification: Lots of these tips are specific for Korn shell (ksh)!

# Who has the biggest homedir ?

du -ks /home/* | sort -n
# What is the biggest file ?
find /var -ls -xdev | /usr/bin/sort -nr +6 | head
find /var -mount -ls -xdev | /usr/bin/sort -nr +6 | head

# What are the hidden files in this dir ?

ls -d .??*

# Get this cli script in vi for easy editing in memory using ksh!

[esc] + [v]

# Need to type a LineFeed (^J) in Korn shell?

[ctrl] + [v], [ctrl] + [j]

# Remove top line of a command:

df | tail +2 # kind of a side effect
df | sed '1d'

# Adding numbers in column layouts using awk:

ipcs -m -b | sed '1,3d' | awk '{SIZE+=$7} END {print SIZE}'

# Handy shell loops for the command line:

$ls -s1 | while read ss ff
>do
>print "size $ss - name and type \c"
>file $ff 
>done

$while read LINE
>do
>echo "$LINE" 
>done </tmp/file

$for ff in * # You can also use backtics for somme subshell output.
>do 
>file $ff 
>done

$while `sleep 5` # My favorite, instand, monitor.
>do
>clear
>df
>done

# You switch logs at night and quickly need yesterdays file?

ls -tr |tail -2|head -1

# Like to use find and grep in configs? Notice, there is a difference...

find /etc -type f -exec grep "string" {} \;
find /etc -type f -print | xargs grep "string" # In my favorite, you also get the filname.

# Howto use cat as an editor?

cat << EOF >> /tmp/file.out # Add everything after this line to file.out until the EOF string.
somme crappy text
EOF

# Determining the filesystem types of those partitions?

file -s /dev/hda{,1,2,3,4,5,6,7,8,9,10}

# How do i feed the ex editor in vi a newline?

:%s/sommestr/sommestr^M/g
where ^M is generated by typing Ctrl-V or by Ctrl-V Ctrl-M, and
:set list
will show the EOL (reverse this with :set nolist), you will see $ as end of line via '\n', and ^M via '\r'. The sequence "[ctrl v][enter]" gives you UNIX style end of line, while "[ctrl v][ctrl m]" gives you the carriage return portion of DOS style EOL. A ^M all by itself is a Mac style EOL. So in the above, maybe you want to use [ctrl v][enter] rather than [ctrl v][ctrl m].

# How do i log everything to a file in a script?

Put this in the beginning of the script, it will capture all output and errors and your script will stay readable!
$exec >>/tmp/logfile 2>&1


Notification:
Somme charakters of the code in this page can be displayed wrongly,
do cross link the displayed code with html source to be shure.
And PLEASE notify me if you find somme of these abnormality's!
Last update: 15 Apr. 2003 by W. Van Hooste | Top

Copyright © 2002, 2003 W. Van Hooste.
Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.
Notice: Except for the Geocities banner this page is W3C HTML validated AND perfectly readable using a text based browser!