Linux, programming, computers and life

I always knew that ’sort’ command has ‘-k N’ flag which tells to sort based on columns N,…. I just recently find out that this flag is even more versatile, and it can be used as ‘-k N,M’ which will sort based on columns N,…,M. As usuall, it’s in the man page, but i always missed it so far .

sort, cli, shell

Overflow in shell script… while it sounds strange, it actually happened to me . I have a script which does recoding of mp3 files in order to reduce bitrate and space those files take. I tried to be smart and first detect the bitrate in order to avoid redundant recoding. Well, my script actually worked for a long time, but then it refused to recode file with bitrate 256 .
It was my fault of course. In order to determine the bitrate, i used the following ash function:
# function to determine bitrate:
# expects a file as parameter
determine_birate() {
    if [ -z “$1″ ]
    then
        return 0 # invalid
    fi
   
    if [[ “${MP3INFO}” == “” ]]
    then
        return 1 # invalid
    fi

    return `”${MP3INFO}” -p “%r” “$1″`
}
MP3INFO is an application i use in order to read mp3 files meta data. Well, when the bitrate was 256 i got 0 returned, i guess bash has some limit of 1 byte somewhere . My quick fix was (and now it works great):
# function to determine bitrate:
# expects a file as parameter
determine_birate() {
    if [ -z “$1″ ]
    then
        return 0 # invalid
    fi
   
    if [[ “${MP3INFO}” == “” ]]
    then
        return 1 # invalid
    fi

    echo `”${MP3INFO}” -p “%r” “$1″`
}
….
CURRENT_BITRATE=`determine_birate “$file”`

bash, programming, cli, linux, overflow

Let’s assume that you are working in a relatively small organization, 4-5 hundreds people at most. I’m sure you have some kind of intranet and there’s a phone book there. I’m pretty sure you have a page with most used phone numbers pinned on your wall somewhere.
Here’s idea how to search those phones from command line. The solution is based on two scripts. First downloads the names list and formats it as necessary (mine strips all HTML mumbo-jumbo). This one is called periodically by cron. Second is used for names list search.

Here are the scripts:
update_tel_db
#! /bin/sh

cd /your/path/to/database
wget http://your.path.here/file.here >& /dev/null
touch tel.txt ; \rm tel.txt
links -dump file.here | grep @ | grep -v — ——————– | grep -v mailto | grep -vi ‘Click here’ | grep -v javascript | grep -v IMG | grep -v file: | tr ‘|’ ‘ ‘ | sed ’s/^ *//’ | sed ’s/ / /g’ | sed ’s/\[[0-9]*\]//g’ > tel.txt
\rm file.here >& /dev/null
touch tel.txt

tel
#! /bin/bash

echo ‘NAME DPT OFFICE CELL-PHONE HOME E-mail’
grep -i $1 /tour/path/here/tel.txt


Now telephone search becomes tel name on the command line.
Enjoy

I wrote about changing of certain columns in a file, but here let’s solve a simpler task. I’d like to add something, let’s say date, to every line of a file. I’ll emulate a file using `seq 1 10`.

The following is easy enough:
seq 1 10 | awk '{print "echo `date` "$0}' | bash

However, there’s simpler solution:
seq 1 10 | while read ; do echo `date` ${REPLY} ; done

It works because REPLY variable is set to the line of input read by the read builtin command when no arguments are supplied (from bash manpage).

Recently, i had a problem - i needed to run several utilities in parallel in bash script, wait till they finish and then gather the information they produced. It’s sooooooooo easy!
Bash has ‘wait’ command which… (surprisingly) waits for all (or certain) background processes to finish. Knowing that, the rest is easy.
Here’s a bash script which simulates utilities with ’sleep’ command.
nParallelTasks=10
nStartedTasks=0
for i in `seq 1 100`
do
echo "-- $i --"
sleep 5&
nStartedTasks=$(($nStartedTasks + 1))
if [[ "$nStartedTasks" == "$nParallelTasks" ]]
then
echo "waiting"
wait
nStartedTasks=0
fi
done


Technorati Tags: bash, wait, parallel, linux, shell, script

I wanted to call this “advanced awk”, but it’s in fact simple idea with awk and bash integration. I got this idea from wonderful ArchLinux forums.
The problem is as follows. Given a file with several columns, i want to run a command on some columns and keep other columns as is.
For example, given a file with user names and some more information, i need to convert user names to user ids and keep other information unchanged. Example file is:

root a b c
ftp   d e f

The desired result is:

0   a b c
14 d e f

(of course, user id of ‘root’ is 0 and of ‘ftp’- 14)
Well, there’s a way to do this . First we’ll convert the file using awk or sed to ‘echo’ commands and then use bash to interpret those.Take a look at the output of the following command:
cat myFile | awk '{print "echo \"$(id -u "$1") " $2 " " $3"\""}'
And now, only thing left to do is to execute the commands awk created for us - i.e. pipe to bash
The final command is:
cat myFile | awk '{print "echo \"$(id -u "$1") " $2 " " $3"\""}' | bash

Enjoy!

Technorati Tags: bash, awk, cli, shell, script

I recently wanted to record an audio stream. I tried audacity which somehow stopped recording after a while and then found mhwaveedit which did the job. However, i though to myself, how do i do it without GUI?
Gstreamer to the rescue! After examining man gst-launch-0.10 i found the following options.
1. Play a sound:

a: gst-launch-0.10 playbin uri=’file:///mnt/audio/1.mp3′
b: gst-launch-0.10 filesrc location=’1.ogg’ ! decodebin ! alsasink

2. record sound from the sound card

mp3: gst-launch-0.10 alsasrc ! audioconvert ! lame ! filesink location=input.mp3
ogg: gst-launch-0.10 alsasrc ! audioconvert ! vorbisenc ! oggmux ! filesink location=input.ogg

It looks like the library is very versatile but i did not found a good tutorial how to build these command line pipes.

Technorati Tags: audio, gstreamer, record, play, mp3, ogg, linux

More or less two years ago i wrote about atool. I finally managed to do something i wanted to do long ago - to create bash completion for it. In archlinux, create a file named /etc/bash_completion.d/atool, with the following code:

# atool(1) completion by linux4all.blogsome.com
_atool()
{
local cur

COMPREPLY=()
cur=${COMP_WORDS[COMP_CWORD]}

case “$cur” in
-*)
COMPREPLY=( $( compgen -W ‘-l -x -X -a -c- -d -r \
-e -F -D -f -q -v -p -o -E -S –list –extract –extract-to= \
–add –cat –diff –repack –each –format= –subdir –force \
–quiet –verbose –page –null –explain –simulate –config=’ — $cur ) )
;;
*)
_filedir ‘@(rar|RAR|zip|ZIP|tar.gz|tgz|tar.bz|tbz|tar.Z|tZ|tar.lzo|tzo|tar.7z|t7z|tar|jar|JAR|war|lha|lzh|7z|ace|ACE|arj|ARJ|a|arc|ARC|rpm|deb|gz|bz|bz2|Z|lzo|cpio)’
;;
esac

return 0

}
complete -F _atool -o filenames atool

and enjoy custom tab-completion for atool.

Technorati Tags: bash, bash completion, atool, bash tip, tip, linux

One more awk trick . Let’s say you have a huge file. You only want to get all the lines of the file which are:

1. before certain string
2. after certain string

It’s easy to do it using any text editor, but it’ll take time and if the file is really huge you’ll have hard time finding a working editor. awk to the resque!

The solutions:

1. seq 1 30 | awk '/.*8.*/ { nextfile } { print $0 }'
2. seq 1 30 | awk '/.*8.*/ { pr = 1 } { if (pr == 1) print $0 }'
3. seq 1 30 | awk '/.*8.*/ { pr = 1; print $0 } { if (pr == 1) print $0 }'

In this example i use ‘<anything>8<anything>’ as the string which divides the file. The difference between (2) and (3) is whether the string itself is printed or not.

Short explanation: awk is instructed to look for regular expression (which is enclosed in ‘/’) and in (1) stops parsing the file before stopping each line is echoed as is. In (2) and (3) it sets pr variable to be 1 (it’s 0 by default and is short of “print”), and lines are echoed only if pr is 1.

Technorati Tags: awk, truncate file, cli, linux

I tried do do it using awk, but i did not find an easy way to determine the last line in file. So sed did the trick:

cat myFile | sed '1 {d}; $ {d}' > myNewFile

Technorati Tags: sed, awk, linux, cli