Wednesday, July 27, 2011

using !$#@*% - pipeline


If you're familiar with UNIX, the pipeline, or pipe, is an integral part of everyday processing. Originally developed by Malcolm McIlroy, the pipeline allows you to redirect the standard output (stdout) of one command to become the standard input (stdin) of the following command in a single chained execution. Using the pipeline isn't limited to one instance per execution. Quite often, the stdout of one command is used as stdin of the following command, and the subsequent stdout is redirected yet again as stdin to another command and so on.
For example, one of the first things most UNIX administrators do on their systems during troubleshooting or daily checks is look at processes running currently on the system. Listing 1 shows such a check.

Listing 1. Example of a daily process check

# ps –ef

     UID     PID    PPID   C    STIME    TTY  TIME CMD
    root       1       0   0   Jul 27      -  0:05 /etc/init
    root   53442  151674   0   Jul 27      -  0:00 /usr/sbin/syslogd
    root   57426       1   0   Jul 27      -  0:00 /usr/lib/errdemon
    root   61510       1   0   Jul 27      - 23:55 /usr/sbin/syncd 60
    root   65634       1   0   Jul 27      -  0:00 /usr/ccs/bin/shlap64
    root   82002  110652   0   Jul 27      -  0:24 /usr/lpp/X11/bin/X -x abx 
       -x dbe -x GLX -D /usr/lib/X11//rgb -T -force :0 -auth /var/dt/A:0-SfIdMa
    root   86102       1   0   Jul 27      -  0:00 /usr/lib/methods/ssa_daemon -l ssa0
    root  106538  151674   0   Jul 27      -  0:01 sendmail: accepting connections
    root  110652       1   0   Jul 27      -  0:00 /usr/dt/bin/dtlogin -daemon
    root  114754  118854   0   Jul 27      - 20:22 dtgreet
    root  118854  110652   0   Jul 27      -  0:00 dtlogin <:0>        -daemon
    root  131088       1   0   Jul 27      -  0:07 /usr/atria/etc/lockmgr 
       -a /var/adm/atria/almd -q 1024 -u 256 -f 256
    root  147584       1   0   Jul 27      -  0:01 /usr/sbin/cron
    root  155816  151674   0   Jul 27      -  0:04 /usr/sbin/portmap
    root  163968  151674   0   Jul 27      -  0:00 /usr/sbin/qdaemon
    root  168018  151674   0   Jul 27      -  0:00 /usr/sbin/inetd
    root  172116  151674   0   Jul 27      -  0:03 /usr/sbin/xntpd
    root  180314  151674   0   Jul 27      -  0:19 /usr/sbin/snmpmibd
    root  184414  151674   0   Jul 27      -  0:21 /usr/sbin/aixmibd
    root  188512  151674   0   Jul 27      -  0:20 /usr/sbin/hostmibd
    root  192608  151674   0   Jul 27      -  7:46 /usr/sbin/muxatmd
    root  196718  151674   0 11:00:27      -  0:00 /usr/sbin/rpc.mountd
    root  200818  151674   0   Jul 27      -  0:00 /usr/sbin/biod 6
    root  213108  151674   0   Jul 27      -  0:00 /usr/sbin/nfsd 3891
    root  221304  245894   0   Jul 27      -  0:05 /bin/nsrexecd
  daemon  225402  151674   0 11:00:27      -  0:00 /usr/sbin/rpc.statd
    root  229498  151674   0 11:00:27      -  0:00 /usr/sbin/rpc.lockd
    root  241794  151674   0   Jul 27      -  0:51 /usr/lib/netsvc/yp/ypbind
    root  245894       1   0   Jul 27      -  0:00 /bin/nsrexecd
    root  253960       1   0   Jul 27      -  0:00 ./mflm_manager
    root  274568  151674   0   Jul 27      -  0:00 /usr/sbin/sshd -D
    root  282766       1   0   Jul 27   lft0  0:00 /usr/sbin/getty /dev/console
    root  290958       1   0   Jul 27      -  0:00 /usr/lpp/diagnostics/bin/diagd
    root  315646  151674   0   Jul 27      -  0:00 /usr/sbin/lpd
    root  319664       1   0   Jul 27      -  0:00 /usr/atria/etc/albd_server
    root  340144  168018   0 12:34:56      -  0:00 rpc.ttdbserver 100083 1
    root  376846  168018   0   Jul 30      -  0:00 rlogind
 cormany  409708  569522   0 19:29:27  pts/1  0:00 -ksh
    root  569522  168018   0 19:29:26      -  0:00 rlogind
 cormany  733188  409708   3 19:30:34  pts/1  0:00 ps -ef
    root  749668  168018   0   Jul 30      -  0:00 rlogind


The listing of the processes currently running on a system can be simple, as shown in Listing 1; however, most production systems run several more processes that make the output of ps much longer. To shorten the list to what you're looking for, redirect the standard output of ps –ef using a pipeline to grep to search for exactly what you want to see. Listing 2 shows the process list from Listing 1 redirected to grep to search for the strings "rpc" and "ksh."

Listing 2. Redirecting the process list to grep

# ps –ef | grep –E "rpc|ksh"

    root  196718  151674   0 11:00:27      -  0:00 /usr/sbin/rpc.mountd
  daemon  225402  151674   0 11:00:27      -  0:00 /usr/sbin/rpc.statd
    root  229498  151674   0 11:00:27      -  0:00 /usr/sbin/rpc.lockd
    root  340144  168018   0 12:34:56      -  0:00 rpc.ttdbserver 100083 1
 cormany  409708  569522   0 19:29:27  pts/1  0:00 -ksh
 cormany  733202  409708   0 19:52:20  pts/1  0:00 grep -E rpc|ksh


Using the pipeline can be much more complicated when you redirect stdout to stdin several times. In the following example, the previous ps and grep example is expanded to pipeline the stdout to another grep to exclude any previous strings found that includes "grep" or "ttdbserver." When the final grep operation has finished, the stdout is redirected again using a pipeline to anawk statement to print any of the processes found with a process identifier (PID) larger than 200,000:
# ps –ef | grep –E "rpc|ksh" | grep -vE "grep|rpc.ttdbserver" | 
   awk -v _MAX_PID=200000 '{if ($2 > _MAX_PID) {print "PID for 
   process",$8,"is greater than", _MAX_PID}}'

PID for process /usr/sbin/rpc.statd is greater than 200000
PID for process /usr/sbin/rpc.lockd is greater than 200000
PID for process -ksh is greater than 200000


Figure 1 provides a graphical representation of the command's stdout redirecting to stdin for the subsequent command.

Figure 1. Pipeline example
Pipeline example

No comments:

Post a Comment