Writing loop statement in Unix/Linux shell scripting with examples

What is loop statement in Unix shell scripting

A group of instruction that is executed repeatedly until a termination condition is met is called a loop.

Bash supports for and while loops. Some shell supports until also

How it works

In each and every loop, The variable used in loop condition must be initialized, then execution of the loop begins.
A test (condition) is made at the beginning of each iteration. The body of loop ends with a statement that modifies the value of the test (condition) variable. Loop is executed till the time, the test condition remain valid.

All the loops have a limited life and they come out once the condition is false or true depending on the loop.

A loop may continue forever due to required condition is not met. A loop that executes forever without terminating executes an infinite number of times. For this reason, such loops are called infinite loops

Lets take a look at different loop syntax

For Loop Unix shell

Syntax

The for statement
for command-list1
do
command-list2
done
Let start with one unix shell script for loop example

username=“Jat bob Thor Neil” for uname in $username do echo “Username is $uname” done

1. do and done are reserved words

2. do and done must be preceded by a newline or ; (semicolon).

3 . Small loops can be written on a single line. For example:   for var in list; do commands; done

More unix shell script for loop example

To go through each file in the current directory and mention the filetype of each file #!/bin/sh for i in * do echo $i file $i done

While Loop

Syntax

The while statement
while command-list1
do
command-list2
done

Example

x=20 while [ $x >= 10 ] do echo “$x” echo “\n” sleep 5 x = x – 1 done 20 19 18 17 16 15 14 13 12 11

More examples

i=0 while [ $i -lt 5 ] do echo My id is $i i=`expr $i + 1` done

Until loop

Syntax

The until statement
until command-list1
do
command-list2
done

Example

until test -f $file do sleep 60 done echo “$file now exists”

Nesting of Loops

We can do nesting of loops also

while test condition ; # this is loop1, the outer loop do Statement(s) to be executed if first test condition  is true while test condition  ; # this is loop2, the inner loop do Statement(s) to be executed if test condition 2 is true done done

The break and continue statements in loops
It is often necessary to handle exception conditions within loops. The statements break and continue are used for this.

Break	Continue
It is often necessary to handle exception conditions within loops. The statements break and continue are used for this. The break command terminates the execution of the innermost enclosing loop, causing execution to resume after the nearest done statement. To exit from n levels, use the command: break n This will cause execution to resume after the done n levels up.	The continue command causes execution to resume at the while, until or for statement which begins the loop containing the continue command

Examples for Continue

for filename in `cat filename.txt` do if [ ! -f $filenname ] ; then continue fi echo “Found file $filename” done

Examples for break

for filename in `cat filename.txt` do if [ ! -r $filename ] ; then echo “Cannot read $filename, quitting loop” break fi echo “Found file or directory $filename” done

Learn unix shell scripting: The Basics

We have already written about Unix Shell scripting ,What is its uses, what are various type of shells and what are positional parameters

What is Shell and Shell Scripts

In this post, I will  write about unix shell scripting basics and creating first shell scripts for the beginners and how to learn unix shell scripting

First of all unix shell scripting basics

Scripts must contain complete documentation
Each script must contain internal comments that document
The name of the script
The purpose for the script
The name of the author(s)
The date the script was written
The date(s) the script was modified.
The name of each script must reflect its use
Each script must have a suffix that describes the script, for example “.sh” for Bourne shell scripts, “.ksh” for Korn shell or “.cgi” for Common Gateway Interface scripts.
Lets create the first shell script

#!/bin/ksh # This script displays the date, time, username and # current directory. echo “Current date and time is:” date echo echo “Your username is: `whoami` \\n” echo “Your current directory is: \\c” pwd echo “Your system name is :`hostname`\\n”

Explanation of each line in the scripts is given below

1. #!/bin/ksh as the first line of the script .It is used to tell the script to execute with ksh shell

2. The second two lines beginning with a hash (#) are comments and are not interpreted by the shell.

3.Use comments to document your shell script.

4. The backquotes (`) around the command whoami illustrate the use of command substitution.The command whoami will be executed and output will be printed at that position. Whoami tells the user information

5. The \\n is an option of the echo command that tells the shell to add an extra carriage return at the end of the line.

6.The \\c tells the shell to stay on the same line.

7.  hostname command gives the hostname of the system

Unix/ Linux :: What is shell and Shell Scripts

What is Shell

Whenever you login to a Unix system you are placed in a program called the shell. The shell acts as a command interpreter; it takes each command and passes it to the operating system kernel to be acted upon. It then displays the results of this operation on completion on your screen.

There are different Flavour of shell in Unix

Finding out which shell you are using  Information about which shell you are using is held in the SHELL environment variable. The command echo $SHELL displays the value of this variable.This displays the default shell of the system. You can change the shell though in the script you execute.You can identify which shell you are presently using from the last part of the pathname.

Shell	Description
Bourne shell	This is the original Unix shell written by Steve Bourne of Bell Labs. It is available on all UNIX systems. Below is written  in the beginning of shell scripts to specify the shell to use for the scripts. The default prompt on the Unix for this is $ #!/bin/bsh
C shell	This shell was written at the University of California, Berkeley. It provides a C-like language with which to write shell scripts – hence its name.The default prompt on the Unix for this is % Below is written  in the beginning of shell scripts to specify the shell to use for the scripts #!/bin/csh
Korn shell	This shell was written by David Korn of Bell labs. It is now provided as the standard shell on Unix systems. It provides all the features of the C shell with a shell programming language similar to that of the original Bourne shell. It is the most efficient shell. Consider using this as your standard interactive shell.The default prompt on the Unix for this is $Below  is written in the beginning of shell scripts to specify the shell to use for the scripts #!/bin/ksh

What is shell Scripts:

A shell script is a collection of command which are executed in a order given. There are conditional statement and looping also available like if ,while which helps in finding if a particular value is greater than another value .
To write any comments in the shell scripts ,it has to be written with # preceded
Example
# Auther of the script is

There are some variables which are set internally by the shell and which are available to the user. They are given below in the table.

Name	Description
$1 – $9	these variables are the positional parameters.
$0	the name of the command currently being executed.
$#	the number of positional arguments given to this invocation of the shell.
$?	the exit status of the last command executed is given as a decimal string. When a command completes successfully, it returns the exit status of 0 (zero), otherwise it returns a non-zero exit status.
$$	the process number of this shell – useful for including in filenames, to make them unique.
$!	the process id of the last command run in the background.
$*	a string containing all the arguments to the shell, starting at $1.

Shell scripts and functions are both interpreted. This means they are not compiled.

Commands in Shell

All shells have a number of built-in commands which are executed in the shell’s own process like echo ,cd .

When you enter a command, the shell checks to see, if the command is a shell built-in such as echo or cd ,it is directly interpreted by the shell.
if the command begins with a / shell assumes that the command is the absolute path name of an executable error occurs if the executable is not found.
if not built-in and not a full pathname shell searches the directories in the PATH from left to right for the executable.Current working directory may not be in PATH
If PATH is empty or is not set, only the current working directory is searched for the executable

Unix commands are executable binary files located in directories with the name bin (for binary). Many of the commands that you use are located in the directory /usr/bin.

A typical value for the PATH variable might be:
/bin:/usr/bin:/usr/local/utils/bin:$HOME/bin

Unix Sed Command to Delete Lines in File - Examples

Sed Command to Delete Lines: Sed command can be used to delete or remove specific lines which matches a given pattern or in a particular position in a file. Here we will see how to delete lines using sed command with various examples.



The following file contains a sample data which is used as input file in all the examples:
> cat file
linux
unix
fedora
debian
ubuntu

Sed Command to Delete Lines - Based on Position in File

In the following examples, the sed command removes the lines in file that are in a particular position in a file.

1. Delete first line or header line

The d option in sed command is used to delete a line. The syntax for deleting a line is:
> sed 'Nd' file

Here N indicates Nth line in a file. In the following example, the sed command removes the first line in a file.
> sed '1d' file
unix
fedora
debian
ubuntu

2. Delete last line or footer line or trailer line

The following sed command is used to remove the footer line in a file. The $ indicates the last line of a file.
> sed '$d' file
linux
unix
fedora
debian

3. Delete particular line

This is similar to the first example. The below sed command removes the second line in a file.
> sed '2d' file
linux
fedora
debian
ubuntu

4. Delete range of lines

The sed command can be used to delete a range of lines. The syntax is shown below:
> sed 'm,nd' file

Here m and n are min and max line numbers. The sed command removes the lines from m to n in the file. The following sed command deletes the lines ranging from 2 to 4:
> sed '2,4d' file
linux
ubuntu

5. Delete lines other than the first line or header line

Use the negation (!) operator with d option in sed command. The following sed command removes all the lines except the header line.
> sed '1!d' file
linux

6. Delete lines other than last line or footer line
> sed '$!d' file
ubuntu

7. Delete lines other than the specified range
> sed '2,4!d' file
unix
fedora
debian

Here the sed command removes lines other than 2nd, 3rd and 4th.

8. Delete first and last line

You can specify the list of lines you want to remove in sed command with semicolon as a delimiter.
> sed '1d;$d' file
unix
fedora
debian

9. Delete empty lines or blank lines
> sed '/^$/d' file

The ^$ indicates sed command to delete empty lines. However, this sed do not remove the lines that contain spaces.

Sed Command to Delete Lines - Based on Pattern Match

In the following examples, the sed command deletes the lines in file which match the given pattern.

10. Delete lines that begin with specified character
> sed '/^u/d' file
linux
fedora
debian

^ is to specify the starting of the line. Above sed command removes all the lines that start with character 'u'.

11. Delete lines that end with specified character
> sed '/x$/d' file
fedora
debian
ubuntu

$ is to indicate the end of the line. The above command deletes all the lines that end with character 'x'.

12. Delete lines which are in upper case or capital letters
> sed '/^[A-Z]*$/d' file

13. Delete lines that contain a pattern
> sed '/debian/d' file
linux
unix
fedora
ubuntu

14. Delete lines starting from a pattern till the last line
> sed '/fedora/,$d' file
linux
unix

Here the sed command removes the line that matches the pattern fedora and also deletes all the lines to the end of the file which appear next to this matching line.

15. Delete last line only if it contains the pattern
> sed '${/ubuntu/d;}' file
linux
unix
fedora
debian

Here $ indicates the last line. If you want to delete Nth line only if it contains a pattern, then in place of $ place the line number.

Note: In all the above examples, the sed command prints the contents of the file on the unix or linux terminal by removing the lines. However the sed command does not remove the lines from the source file. To Remove the lines from the source file itself, use the -i option with sed command.
> sed -i '1d' file

If you dont wish to delete the lines from the original source file you can redirect the output of the sed command to another file.
sed '1d' file > newfile

Unix shell script to truncate a large file

I am trying to write a Unix script which will truncate/empty a file which is continuously being written/open by an application when it reaches say 3GB of space. I know that the below command would do it :

cp /dev/null [filename]

But I am going to run this in an production environment automatically as a cron job - just posting here to see if you guys faced any issues while doing something similar to this.

 Answers

Just to add another answer,

: > filename

: is a no-op in bash, so this essentially just opens the file for writing (which of course truncates the file) and then immediately closes it.

EDIT: as shellter commented, you don't actually need a command to go along with the redirection:

$ echo foo > foo.txt
$ cat foo.txt
foo
$ > foo.txt
$ cat foo.txt
$

A simple redirection all by itself will clear the file.

---

That seems reasonable to me. Unix, of course, would let you do this about 50 different ways. For example,

echo -n "" >filename
cat /dev/null >filename

Using SSH and Unix commands

Having some basic knowledge of SSH and Unix commands is very useful. Developed in 1995, SSH (Secure Shell) was created as a secure alternative to Telnet. Telnet is a protocol allowing for command line access to a Unix, Linux or FreeBSD based remote computer. I’ve listed some basic commands to get you familiar with them.

The cd command is used to move to a specific directory.

1 2	Command: cd Format: cd /directory/to/browse

The cp command will copy the file or folder from the source, to the destination.

1 2	Command: cp Format: cp /directory/source /directory/destination

mkdir creates a directory.

1 2	Command: mkdir Format: mkdir /directory_to_create

The rmdir command deletes a directory.

1 2	Command: rmdir Format: rmdir /directory/to/delete

The rm command deletes a file.

1 2	Command: rm -f Format: rm -f filename

The mv command will rename or move a file stated in the first portion to the name or location stated in the second portion.

1 2	Command: mv Format: mv /directory/you/want/to/rename/or/move /new/directory/name/or/location

This command will tar zip the files in the directory specified in the second portion into a tar file specified in the first portion.

1 2	Command: tar cvf Format: tar cvf filename.tar.gz /directory/you/wish/to/archive

The tar -xvf will extract all files from the tarball specified into the directory you are currently in.

1 2	Command: tar -xvf Format: tar -xvf filename.tar.gz

This will create a zip file, with the name specified in the first portion from the file or directory listed in the second portion.

1 2	Command: zip Format: zip filename.zip /file/or/folder/you/want/to/zip

This command will unzip or un pack the named zip file, into the directory you’re currently in.

1 2	Command: unzip Format: unzip filename.zip

The Ls command lists files, and folders within the directory you specify

1 2	Command: ls Format: ls /directory/you/wish/to/list/files/

This will add a forward slash to the directory names within the directory you specify

1 2	Command: ls -f Format: ls -f /directory/you/wish/to/list/files/

This will show “hidden” files in the directory you specify

1 2	Command: ls -a Format: ls -a /directory/you/wish/to/list/files/

This command shows detailed info about each file in the directory you specify.

1 2	Command: ls -l Format: ls -l /directory/you/wish/to/list/files/

Also, a lightweight, freeware application which supports SSH commands for windows is PuTTY if you aren’t on a MAC.