Difference between revisions of "Bash"

Revision as of 04:01, 22 December 2009

Here are some of my BASH tricks

Find files containing MMU in the name, do a line count, and them add them up. The first part finds files, then redirects the output of the line counts to a temp file

[tom@garak gaisler]$ for i in $(find|grep mmu); do wc $i -l; done>~/leon3_mmu_linecount

The output file:
655 ./leon3/mmutlb.vhd
351 ./leon3/mmuconfig.vhd
258 ./leon3/mmutw.vhd
176 ./leon3/mmulru.vhd 
1559 ./leon3/mmu_dcache.vhd
202 ./leon3/libmmu.vhd
245 ./leon3/mmuiface.vhd
609 ./leon3/mmu.vhd
148 ./leon3/mmu_cache.vhd
189 ./leon3/mmutlbcam.vhd
380 ./leon3/mmu_acache.vhd
111 ./leon3/mmulrue.vhd
662 ./leon3/mmu_icache.vhd

The second part cuts the first field of the file, which is an integer containing the line count, and add all of them up

[tom@garak gaisler]$ PASS=0;for i in $(cat ~/leon3_mmu_linecount|cut -d" " -f1); do PASS=$(($PASS+$i)); done; echo $PASS lines total
5545 lines total

--Done--

Remove all synopsys dirs in path, so we can use a different version (need a bit of work, since it doesn't redirect to stdout for some reason)

echo $PATH|sed s/\:/\\n/g|grep -v synopsys|tr '\n' ':'

Find Hacks

Find using wildcards, and multiple names

find -name "*.v" -o -name "*.h"   
This finds all *.v and *.h files. Note that wildcards are enclosed in quotes; or the shell will interpret it!
The -o option is necessary when specifying multiple instances of an option.

Find files, then grep them.

find "*.h" -exec grep "FE_BITS" -Hn {} \; 
This greps every file, and prints the filename and line number of each instance

Sometimes, you want to do something for all files of a particular class. For example, grep *.cpp and *.h files, and see if they contain a specific string or variable. The previous method is to use the find -exec option. This has inherit weaknesses. Using a pipe is very difficult if not impossible with the exec option ie -exec grep "a"|grep "b" will not work, nor will redirects.

What you really want to do in that case, is similar to a bash for loop.

 $for i in `find ~/Download/ -name "*.pdf"`; do echo pdftk \"${i}\" dump_data output ; done

This for loop will work correctly, but only if the filename DON'T contain SPACES. Here's a quick digression of why. The 'for i in' bash construct parses tokens by both the newline, and the space. So when find returns this file:

 /home/tom/Download/lxf/LinuxFormat 2009.pdf

The i variable will become the value "/home/tom/Download/lxf/LinuxFormat" then in the next iteration, it will progress onto "2009.pdf". Clearly not the behaviour we want.

Bash has the read utility, so that is how we will fix the for loop. First, we need to use a while loop, second, it must use the read utility.

 find   ~/Download/ -name "*.pdf"|while read filename; do echo " FILE: $filename *****************" &&pdftk "$filename" dump_data output 2>/dev/null|egrep -i '(InfoKey|InfoValue); done

As you can see in this cumbersome example, we use the while read construct. This will set the filename variable to that of the output of the find utility per line, not per space and per line. Going through all this trouble is quite worth it though, since sometimes you need to pipe and redirect to get what you want. The find -exec option does not allow that.

Another Solution: Create a bash script, and have the find -exec option invoke the script with the filename. Of-course, I am focusing on one-liners here,so my preffered method is the one-liner.

AWK Hacks

--Currently, awk cannot handle files in-place. So redirect to a new_file, then mv the new_file to the old_file.

Print out n'th field, where deliminator is 1 or more spaces, replace $1 with the number field you want e.g. $3 is third field.

cat orig_file|awk '{print $1}'> new_file

Print out only unique lines, first instance only. The uniq command does not always work correctly!

cat orig_file|awk '!x[$0]++' > new_file

BASH Inline Tricks

bash is very powerful. Knowing how to use the 'one-liners' will save significant amounts of time.

Inline Loop

This is a simple for loop, parsing in every line of ls and echoing it.

 for i in `ls`; do echo $i; done

for each `define in fe.h, take second field, and make a $display statement with. Notice the semicolons

 for i in `cat fe.h|grep ^'\`define'|awk '{print $2}' `; do echo "\$display (\"" $i = \%d\"\, $i \)\;; done>fe_print.v