Grep for Windows – findstr example

I love grep command on Linux, it helped to search and filter strings easily, always wonder what is the equivalent tool on Windows, and found this findstr recently.

In this article, I will share some of my favorite “grep” examples on Linux, and how to “port” it to Windows with “findstr” command.

1. Filter a result

1.1 Classic example to filter a listing result.

#Linux
$ ls -ls | grep mkyong

#Windows
c:\> dir | findstr mkyong


Copy

1.2 Add ignore case, and filter the listing result with multiple strings.

#Linux - Need '-E' option and Uses "|" to separate multiple search strings.
$ ls -ls | grep -iE "mkyong|music"

#Windows - Use spaces to separate multiple search strings
c:\> dir | findstr -i "mkyong music"


Copy

2. Search a File

2.1 Search matched string in a file.

#Linux 
$ grep mkyong test.txt

#Windows
c:\> findstr mkyong test.txt


Copy

2.2 Counting the number of matches.

#Linux
$ grep -c mkyong test.txt

#Windows - Piped with find /c command.
c:\> findstr -N "mkyong" test.txt | find /c ":"


Copy

3. Search a list of files

3.1 Search matched string in a list of files.

#Linux
$ grep mkyong -lr /path/folder

#Windows
c:\> findstr /M mkyong c:\folder\*


Copy

* (grep) -l , (findstr) /M = print only name of files containing matches.

4. Help

4.1 The most powerful command ~

#Linux 
$ grep --help 
$ man grep 

#Windows
c:\> findstr -?

Look for a string, word, or sentence in a file with Linux grep command, recursively

Introduction

Sometimes you remember a phrase or a given word or words, you put in a document, but you do not remember the name of the document. How to find it?, well do a search of all your documents, looking for that word or words or sentence, in other words, look for a string or strings.

Now, thanks to computers this is easier now, than it was in our parents' days.

If you are using Linux, you have grep to help on this job.

Look for a document, containing a given string

First the easy case, you know the exact sentence, you are looking for, and you at least remember the folder where the file is.

grep "sentence to look for" /home/user/docs/

Now, let's suppose you do not know if the sentence was in uppercase or in lowercase, so ask grep to ignore case.

grep -i "sentence to look for" /home/user/docs/

Well, let's suppose you have a lot of sub-folders, and you do not remember where your file is.

grep -r "sentence to look for" /home/user/docs/

Let's see an example of the output in my PC

grep -ri "Introduction" /home/user/post/

The output is:

/home/user/post/monthy-newsletter.txt:###Introduction###
/home/user/post/monthy-newsletter.txt:I'm no expert in MySQL, but anyway I have written three introduction-type MySQL posts, something we all need to know to start [how to create databases][1], [How to list those databases][2] and [How to create tables in MySQL][3]
/home/user/post/how-to-debug-bash-shell-scripts.txt:##Introduction##
/home/user/post/how-to-setup-dns-bind-master-slave-linux.txt:##Introduction
/home/user/post/interview-raphael-hertzog.txt:I also have plans for bigger changes concerning Debian, and among them is the introduction of Debian Rolling, a distribution similar to testing but with some design choices to make it more usable at any point in time.
/home/user/post/four-years-with-debian-testing.txt:##Introduction##
Binary file /home/user/post/.find-a-documents-with-given-string.txt.swp matches
/home/user/post/how-to-change-the-priority-of-Linux-processes.txt:##Introduction##

As you can see there are some binary files, also scanned, if we want to avoid that:

grep -riI "introduction" /home/user/post/

The output now is:

/home/user/post/monthy-newsletter.txt:###Introduction###
/home/user/post/monthy-newsletter.txt:I'm no expert in MySQL, but anyway I have written three introduction-type MySQL posts, something we all need to know to start [how to create databases][1], [How to list those databases][2] and [How to create tables in MySQL][3]
/home/user/post/how-to-debug-bash-shell-scripts.txt:##Introduction##
/home/user/post/how-to-setup-dns-bind-master-slave-linux.txt:##Introduction
/home/user/post/interview-raphael-hertzog.txt:I also have plans for bigger changes concerning Debian, and among them is the introduction of Debian Rolling, a distribution similar to testing but with some design choices to make it more usable at any point in time.
/home/user/post/four-years-with-debian-testing.txt:##Introduction##
/home/user/post/how-to-change-the-priority-of-Linux-processes.txt:##Introduction##

Finally, I just want the file names, and not the sentences where the sentence or word appears.

grep -riIl "introduction" /home/user/post/

The output will be:

/home/user/post/monthy-newsletter.txt
/home/user/post/how-to-debug-bash-shell-scripts.txt
/home/user/post/how-to-setup-dns-bind-master-slave-linux.txt
/home/user/post/interview-raphael-hertzog.txt
/home/user/post/four-years-with-debian-testing.txt
/home/user/post/how-to-change-the-priority-of-Linux-processes.txt

Linoux - How To Use awk In Bash Scripting

Print a Text File

awk '{ print }' /etc/passwd
OR
awk '{ print $0 }' /etc/passwd

Print Specific Field

Use : as the input field separator and print first field only i.e. usernames (will print the the first field. all other fields are ignored):
awk -F':' '{ print $1 }' /etc/passwd
Send output to sort command using a shell pipe:
awk -F':' '{ print $1 }' /etc/passwd | sort

Pattern Matching

You can only print line of the file if pattern matched. For e.g. display all lines from Apache log file if HTTP error code is 500 (9th field logs status error code for each http request):
awk '$9 == 500 { print $0}' /var/log/httpd/access.log
The part outside the curly braces is called the “pattern”, and the part inside is the “action”. The comparison operators include the ones from C:

== != < > <= >= ?:

If no pattern is given, then the action applies to all lines. If no action is given, then the entire line is printed. If “print” is used all by itself, the entire line is printed. Thus, the following are equivalent:
awk '$9 == 500 ' /var/log/httpd/access.log
awk '$9 == 500 {print} ' /var/log/httpd/access.log
awk '$9 == 500 {print $0} ' /var/log/httpd/access.log

Print Lines Containing tom, jerry AND vivek

Print pattern possibly on separate lines:
awk '/tom|jerry|vivek/' /etc/passwd

Print 1st Line From File

awk "NR==1{print;exit}" /etc/resolv.conf
awk "NR==$line{print;exit}" /etc/resolv.conf

Simply Arithmetic

You get the sum of all the numbers in a column:
awk '{total += $1} END {print total}' earnings.txt
Shell cannot calculate with floating point numbers, but awk can:
awk 'BEGIN {printf "%.3f\n", 2005.50 / 3}'

Call AWK From Shell Script

A shell script to list all IP addresses that accessing your website. This script use awk for processing log file and verification is done using shell script commands.

#!/bin/bash d=$1 OUT=/tmp/spam.ip.$$ HTTPDLOG="/www/$d/var/log/httpd/access.log" [ $# -eq 0 ] && { echo "Usage: $0 domain-name"; exit 999; } if [ -f $HTTPDLOG ]; then awk '{print}' $HTTPDLOG >$OUT awk '{ print $1}' $OUT | sort -n | uniq -c | sort -n else echo "$HTTPDLOG not found. Make sure domain exists and setup correctly." fi /bin/rm -f $OUT

AWK and Shell Functions

Here is another example. chrootCpSupportFiles() find out the shared libraries required by each program (such as perl / php-cgi) or shared library specified on the command line and copy them to destination. This code calls awk to print selected fields from the ldd output:

chrootCpSupportFiles() { # Set CHROOT directory name local BASE="$1" # JAIL ROOT local pFILE="$2" # copy bin file libs   [ ! -d $BASE ] && mkdir -p $BASE || :   FILES="$(ldd $pFILE | awk '{ print $3 }' |egrep -v ^'\(')" for i in $FILES do dcc="$(dirname $i)" [ ! -d $BASE$dcc ] && mkdir -p $BASE$dcc || : /bin/cp $i $BASE$dcc done   sldl="$(ldd $pFILE | grep 'ld-linux' | awk '{ print $1}')" sldlsubdir="$(dirname $sldl)" if [ ! -f $BASE$sldl ]; then /bin/cp $sldl $BASE$sldlsubdir else : fi }

This function can be called as follows:
chrootCpSupportFiles /lighttpd-jail /usr/local/bin/php-cgi

AWK and Shell Pipes

List your top 10 favorite commands:
history | awk '{print $2}' | sort | uniq -c | sort -rn | head
Sample Output:

   172 ls
    144 cd
     69 vi
     62 grep
     41 dsu
     36 yum
     29 tail
     28 netstat
     21 mysql
     20 cat

Another example to find out domain expiry date:
$ whois cyberciti.com | awk '/Registry Expiry Date:/ { print $4 }'
Sample outputs:

2018-07-31T18:42:58Z

Awk Program File

You can put all awk commands in a file and call the same from a shell script using the following syntax:
awk -f mypgoram.awk input.txt

Awk in Shell Scripts – Passing Shell Variables TO Awk

You can pass shell variables to awk using the -v option:

n1=5 n2=10 echo | awk -v x=$n1 -v y=$n2 -f program.awk

Assign the value n1 to the variable x, before execution of the program begins. Such variable values are available to the BEGIN block of an AWK program:

BEGIN{ans=x+y} {print ans} END{}

AWK command in Linux/Unix

What Operations can AWK do?

• Scanning files line by line
• Splitting each input line into fields
• Comparing input lines and fields to patterns
• Performing specified actions on matching lines

AWK Command Usefulness

• Changing data files
• Producing formatted reports

Programming Concepts for awk command

• Format output lines
• Conditional and loops
• Arithmetic and string operations

AWK Syntax

$ awk options 'selection _criteria {action }' input-file > output-file

To demonstrate more about AWK usage, we are going to use the text file called file.txt

1st column => Item,
2nd column => Model
3rd column => Country
4th column => Cost

Awk Command Examples

Printing specific columns

To print the 2nd and 3rd columns, execute the command below.

$ awk '{print $2 "\t" $3}' file.txt

Output

Printing all lines in a file

If you wish to list all the lines and columns in a file, execute

$ awk ' {print $0}' file.txt

Output

Printing all lines that match a specific pattern

if you want to print lines that match a certain pattern, the syntax is as shown

$ awk '/variable_to_be_matched/ {print $0}' file.txt

For instance, to match all entries with the letter ‘o’, the syntax will be

$ awk '/o/ {print $0}' file.txt

Output

To match all entries with the letter ‘e’

$ awk '/e/ {print $0}' file.txt

Output

Printing columns that match a specific pattern

When AWK locates a pattern match, the command will execute the whole record. You can change the default by issuing an instruction to display only certain fields.

For example:

$ awk '/a/ {print $3 "\t" $4}' file.txt

The above command prints the 3rd and 4th columns where the letter ‘a’ appears in either of the columns

Output

Counting and Printing Matched Pattern

You can use AWK to count and print the number of lines for every pattern match. For example, the command below counts the number of instances a matching pattern appears

$ awk '/a/{++cnt} END {print "Count = ", cnt}' file.txt

Output

Print Lines with More or less than a No. of Characters

AWK has a built-in length function that returns the length of the string. From the command $0 variable stores the entire line and in the absence of a body block, the default action is taken, i.e., the print action. Therefore, in our text file, if a line has more than 18 characters, then the comparison results true, and the line is printed as shown below.

$ awk 'length($0) > 20' file.txt

Output

Saving output of AWK to a different file

If you wish to save the output of your results, use the > redirection operator. For example

$ awk '/a/ {print $3 "\t" $4}' file.txt > Output.txt

You can verify the results using the cat command as shown below

$ cat output.txt

Output

Conclusion

AWK is another simple programming script that you can use to manipulate text in documents or perform specific functions. The shared commands are a few or the many you are yet to know or come across.

Search patterns in files using Linux grep command

Does you job requires you to frequently search for patterns in files through Linux command line? Or, do you feel frustrated when you have to open files in UI editors to search for strings or patterns on Linux? Well, if yes then the Linux grep command is for you. This command can be used to search a pattern in one or more files directly from the command line. 

In this article, we will understand the usage of Linux grep command through practical examples.

 

SYNTAX

Before jumping on to the examples, lets first take a look on how to use the grep command. Here is the basic syntax information of grep command from the man page :

grep [OPTIONS] PATTERN [FILE...]

So we see that the grep command does require PATTERN as a mandatory argument. The OPTION and FILE arguments are non-mandatory. While the OPTION argument tells the grep command to act in a way as specified by the definition of that OPTION, the FILE argument tells the grep command about the files in which the pattern needs to be searched. The ellipsis '...' in the argument FILE indicates that more than one files can be presented in the argument list. 

NOTE : For those who are new to this type of syntax information, any argument specified in square brackets [] are non-mandatory.

 

EXAMPLES

 

1. A basic example

Here is how the grep command can be used in its most basic form.

# grep "Linux" input.txt 
Welcome to Linux.

In the output above, the line in the file input.txt containing the pattern or string "Linux" was displayed as output.

 

2. Pattern matching is case sensitive

The pattern matching done by grep command is case sensitive. For example, if the argument to grep command is "LINUX" (instead of "Linux") then grep will not match the lines containing string "Linux". 

Here is an example :

# grep "LINUX" input.txt 
#

So we see that no output was displayed. If it is desired that grep command should ignore the case sensitiveness then the option -i can be used. Here is the example :

# grep -i "LINUX" input.txt 
Welcome to Linux.

So we see that this time the string "LINUX" matched with the line containing the string "Linux".

 

3. Search in more that one file

If more than one file is supplied in argument list then grep searches for the pattern or string in all the files. 

For example :

# grep "Linux" input.txt output.txt 
input.txt:Welcome to Linux.
output.txt:I hope you enjoyed working on Linux.

As we can see in the output above, the lines containing the string "Linux" along with their respective file names were displayed in the output.

Also, to search in a complete directory, the argument '*' can passed as input.

Here is an example :

# grep "Linux" *
input.txt:Welcome to Linux.
output.txt:I hope you enjoyed working on Linux.
Binary file test_strace matches
test_strace.c:    if(NULL == fopen("Linux","rw"))

So we see that lines in all the files (in current directory) containing the string "Linux" were displayed as output.

 

4. Search recursively using -r option

There exists an option -r through which the grep command can search for pattern (or string) recursively in the sub-directories. 

Here is an example :

# grep -r "Linux" *
input.txt:Welcome to Linux.
new_dir/new.txt:Linux vs Windows
output.txt:I hope you enjoyed working on Linux.
Binary file test_strace matches
test_strace.c:    if(NULL == fopen("Linux","rw"))

So we see that the output contains matching results from files contained in sub-directories.

 

5. Match patterns using regular expressions

The grep command also allows the usage of regular expressions in pattern matching. This provides tremendous power to user using the grep command to search for any possible pattern that can be represented through regular expression. 

Here is an example :

# grep -r ".*Linux" output.txt output1.txt 
output.txt:I hope you enjoyed working on Linux.
output1.txt:Welcome to Linux.
output1.txt:I hope you will have fun with Linux.

As we can see that the grep command above used a regular expression ".*Linux" for pattern matching in files output.txt and ouput1.txt. 

Here is a table of regular expression operators and their effect :

Operator Effect
. Matches any single character.
? The preceding item is optional and will be matched, at most, once.
* The preceding item will be matched zero or more times.
+ The preceding item will be matched one or more times.
{N} The preceding item is matched exactly N times.
{N,} The preceding item is matched N or more times.
{N,M} The preceding item is matched at least N times, but not more than M times.
- represents the range if it's not first or last in a list or the ending point of a range in a list.
^ Matches the empty string at the beginning of a line; also represents the characters not in the range of a list.
$ Matches the empty string at the end of a line.
\b Matches the empty string at the edge of a word.
\B Matches the empty string provided it's not at the edge of a word.
\< Match the empty string at the beginning of word.
\> Match the empty string at the end of word.

Any one or a combination of these operators can be used to form a regular expression that represents the pattern of user's choice.