Showing posts with label Linux cat. Show all posts
Showing posts with label Linux cat. Show all posts

Thursday, 8 August 2019

12 CAT COMMAND EXAMPLES IN LINUX/UNIX

Cat(concatenate) command is an excellent command used to view files. This will conCATenate(Combines two are more files) and display it’s content on the screen. Cat command not only display the file content but it can also do merging files , creating files sending one file to other file etc. Lets learn cat command with with practical examples first and then with some options .
Example 1: Display file content on stranded output (STDOUT)
cat filename.txt
or
cat < filename.txt
Example 2: Send the file content as input to other command. For example count number of lines in a file
cat filename.txt | wc -l
Example 3: Create a file using cat. execute below command start typing the contents on the screen/at terminal once you enter your data, save the file by pressing ctrl+d
cat > filename.txt
asdfasd
a
sdf
sadfsad
fasd
asdf
asdf
(press ctrl+d)
Example 4: Copy one file content to other(over write the file)
cate file1.txt > file2.txt
<, >, | operators etc are called as redirect operators here redirect operators with examples

Example 5:
 Copy one file content to other (don’t overwrite but append it)
cat file1.txt >> file2.txt
Example 6: Copy 2 files in to third file(overwrite if the third file exists)
cat file1.txt file2.txt > file3.txt
or
cat file{1,2}.txt > file3.txt
learn more about {} braces 
Example 7: Copy 2 files in to third file(append the date to third file)
cat file1.txt file2.txt >> file3.txt
Example  8: To copy file abc.txt and write some date using keyboard then redirecting both to a file called xyz.txt.
cat - abc.txt > xyz.txt
cat abc.txt - > xyz.txt
Note: The first command will write keyboard typed date first to file xyz.txt then abc.txt to xyz.txt. Where as the command first abc.txt data is written to xyz.txt then followed with the data entered from keyboard.
Example 9: Display the file content along with number of the line or number the line numbers using cat command
cat -n filename.txt
Example 10: Display file content along with numbers for lines with data
cat -b filename.txt
Example 11: Hide repeated empty lines when displaying content of the file.
cat -s filename.txt
Note: This command will display single empty line for a sequence of multiple empty lines.
Example 12: Display tabs in a file using cat command
cat -t filename.txt
There are many more cat command options which we can get through manual pages.
whatis cat
or
man cat
or
info cat

Friday, 2 August 2019

Linux - Cat Command

Definition and purpose:

Cat command is used to display the content of a file or multiple files.
This command is more helpful in reading the log file or configuration files of the application or system.
Syntax:
Cat file-name1 file-name2

Example:
Cat /etc/passwd
The above command displays the user's details that are available on the machine.
The primary usage of Cat command is for displaying the file content on the terminal window but with Cat command, we can create files and add content also.

Most practical use is for:
  • Checking the Application logs or system logs
  • Concatenate two files or log files information and push to another file.
  • Copying the content from one file to another file.
  • Copying the content from one file to another file along with standard input from the terminal.
  • To empty the file for having new content in it.

Below we have talked about few cat command options and their usage with practical examples.

Example-1:
Creating a new file with cat command:
Syntax:
“cat > filename”

Once the above command is typed and press enter. We see the cursor blinking on the next line, i.e, whatever we may type here, that will be redirected to that file.

Check out the below example,
[rreddy@abclearn abclearn_dir1]$ cat > abc_lab3.txt
Hi
this information saves in new file
3rd line
- must be working
this is 5th line for testing

Note: 
  • We have to press “ctrl+d” to come out of file writing.
  • If the file is not created already then it will be created with the user credentials.
  • If the file is already existing, then its content is overwritten.
  • So using “>” symbol we can redirect the information to given file.

We can use the same cat command to display files content.
To display content of abc_lab3.txt file,
Cat abc_lab3.txt 

We have created a new file. Now, what if we want to add more lines to that existing file.

Example-2:
Add new lines to existing file:
Syntax:
Cat >> filename

Note: 
  • “>” symbol is used to redirect the information into a file.
  • “>>” symbol will add new text at the end of existing file.If the file doesn’t exist already, then it is created.

We can add more information to existing file at the end using “>>” symbol.
[rreddy@abclearnabclearn_dir1]$ cat >>abc_lab3.txt
this is going to be as 6th line
7th one
8th one    
Let’s see the file content and confirm, it is really added.
[rreddy@abclearnabclearn_dir1]$ cat abc_lab3.txt
Hi
this information saves in new file
3rd line
- must be working
this is 5th line for testing
this is going to be as 6th line
7th one
8th one

Example-3:
Copy a file’s content to another file:
Syntax:
Cat source-file-name > target-file-name

As discussed earlier, “>” symbol will redirect the content which should be displayed on terminal window to another file.
So source file content is not removed at all. It is simply redirecting the file content to another file instead of on terminal.
We can use the cat command with “>” symbol to copy the content to another file and ">>" to append at the bottom of another file.
[rreddy@abclearnabclearn_dir1]$ cat abc_lab3.txt > abc_lab4.txt

[rreddy@abclearnabclearn_dir1]$ cat abc_lab4.txt
Hi
this information saves in new file
3rd line
- must be working
this is 5th line for testing this is going to be as
6th line
7th one
8th one

Example-4:
Display more than one file or multiple files output on terminal:
Cat command can concatenate one or more files content and show the output on the terminal.
Syntax:
Cat file-name1 file-name2

Let’s see a practical example for better understanding.
[rreddy@abclearnabclearn_dir1]$ cat abc_lab1.txt abc_lab2.txt abc_lab3.txt
this line should replace the existing one
this is the first line in new file
Hi
this information saves in new file
3rd line
- must be working
this is 5th line for testing
this is going to be as 6th line
7th one
8th one

Example-5:
Redirect multiple files output to another file ( or ) concatenate more than one file output in another file:
Following the explanations are given in above examples, we will make use of “>” symbol for redirecting to file and “>>” symbol for adding the new content at the bottom of the target file.
Syntax:
Cat file-name1 file-name2 > file-name3 

Cat file-name1 file-name2 >> file-name3

[rreddy@abclearnabclearn_dir1]$ cat abc_lab1.txt abc_lab2.txt abc_lab3.txt > abc_lab4.txt

[rreddy@abclearnabclearn_dir1]$ cat abc_lab4.txt
this line should replace the existing one
this is first line in new file
Hi
this information saves in new file
3rd line
- must be working
this is 5th line for testing
this is going to be as 6th line
7th one
8th one

Example-6:
Display the line numbers of a file on terminal:
 we have a couple of options with cat command to display output with line numbering.
 Syntax:
Cat -n file-name

-n will display the line numbers including the empty lines.
Observe the below practical example and see line number 12 & 14. They are empty lines.
[rreddy@abclearnabclearn_dir1]$ cat -n abc_lab4.txt
1   this line should replace the existing one
2   this is first line in new file
3   Hi
4   this information saves in new file
5   3rd line
6   - must be working
7   this is 5th line for testing
8   this is going to be as 6th line
9   7th one
10  8th one
11  I am good
12
13  How are you
14
15  closing this file here

Example-7:
Display line numbers of a file but avoid empty lines:
For avoiding the empty lines from displaying in a file, we have command option is, “-b”
Syntax:
Cat -b file-name

Let’s look at the below cat command example, which completely avoids displaying the empty files.
[rreddy@abclearnabclearn_dir1]$ cat -b abc_lab4.txt
1  this line should replace the existing one
2  this is first line in new file
3  Hi
4  this information saves in new file
5  3rd line
6  - must be working
7  this is 5th line for testing
8  this is going to be as 6th line
9  7th one
10 8th one
11 I am good
12 How are you
13 closing this file here

Example-8:
Display or concatenate 2 files content with terminal one and standard input:
 As we have discussed earlier, cat command comes with lot of options for variety of information.
We can take files information and concatenate it with terminal input.
Syntax :
Cat filename1 – filename2

“-“ option is to support input information from terminal or standard input also.

See the below example and try it out on terminal window,
[rreddy@abclearnabclearn_dir1]$ cat abc_lab1.txt - abc_lab2.txt
this line should replace the existing one----------------------------------->abc_lab1.txt file content
hi-------------------------------------------------------------------------->information typed on terminal
hi
new lines
new lines
sadfa
sadfa
this is first line in new file----------------------------------------------->abc_lab2.txt file content

Example-9:
Emptying the file having the content:
 In such cases we need to empty the files to reduce the utilization of the storage in the machine.
Syntax:
Cat /dev/null > File-name

[rreddy@abclearnabclearn_dir1]$ cat /dev/null > abc_lab2.txt [rreddy@abclearnabclearn_dir1]$ cat abc_lab2.txt

Exploring more cat command options:
For more on command options and their understanding, check out the manual pages or help content given with cat command.
Syntax:
man cat
cat --help
info cat

Thursday, 1 August 2019

Linux: Text Processing: grep, cat, awk, uniq

This page is a basic tutorial on using Linux shell's text processing tools. They are especially useful for processing lines.

Get Lines: grep
grep is the most important command. You should master it.

Show Matching Lines
# show lines containing xyz in myFile
grep 'xyz' myFile
# show lines containing xyz in all files ending in html in current dir top level files
grep 'xyz' *html
Grep for All Files in a Dir
# show matching lines in dir and subdir, file name ending in html
grep -r 'xyz' --include='*html' ~/web
Here's what the options mean:

-r → all subdirectories.
--include='*html' → match file name by a glob pattern (* is a wildcard that matches 0 or more any char.).
grep without regex
Use the option -F. (F means “Fixed string”)

# search ruby source files that contains  .* literally
grep -F '.*' *rb
This is useful when you want to search complicated string in source code, such as *@$.*#+-/\|`.

If your string is really complicated, you can put it in a file, and use the option --file=my_pattern_filename for the search text. Example:

# search js source code in dir and all subdirs. The regex is stored in file named myPattern.txt
grep -r --file=myPattern.txt --include=*js .
Most Useful Grep Options


Options for Pattern String
-F → use fixed string. (no regex)
-P → use Perl's regex syntax. (Perl and Python's regex are basically compatible.)
-i → ignore case.
-v → print lines NOT containing the pattern.
Examples:

# print lines not matching a string, for all files ending in “log”
grep -v 'html HTTP' *log
# print lines containing “png HTTP” or “jpg HTTP”
grep -P 'png HTTP|jpg HTTP' *log
Options for File Selection
*.html = search all files ending in ".html”, in current dir. (files in subdir are ignored)
grep -r --include='*html' pattern dirname = search files for pattern in dirname including subdirs, but only files ending in ".html”.
Output Options
-H = include file name in the result.
-h = do NOT print file name.
-l = print just file name; do NOT print the matched lines.
-L = print just file name that does NOT match.
More Grep Examples
# print lines containing “html HTTP” in a log file, show only the 12th and 7th columns, show only certain lines, then sort, then condense repeation with count, then sort that by the count.

grep 'html HTTP' apache.log | awk '{print $12 , $7}' | grep -i -P "livejournal|blogspot" | sort | uniq -c | sort -n
# print all links in all html files of a dir, except certain links. Output to xx.txt

grep -r --include='*html' -F 'http://' ~/web | grep -v -P 'google.com|twitter.com|reddit.com|wikipedia.org' > xx.txt
text columns, awk, sort, unique, sum column …
show only nth column in a text file
# print the 7th column. (columns are separated by spaces by default.)
cat myFile | awk '{print $7}'
For delimiter other than space, for example tab, use -F option. Example:

# print 12th atd 7th column, Tab is the separator
cat myFile | awk -F\t '{print $12 , $7}'
Alternative solution is to use the cut utility, but it does not accept regex as delimeters. So, if you have column separated by different number of spaces, “cut” cannot do it.

remove duplicate lines
sort myFile -u
or

sort myFile | uniq
To prepend the line with a count of repetition, use sort myFile | uniq -c

sum up 2nd column
awk '{sum += $2} END {print sum}' file_name → sum the 2nd column in a file.

show only first few lines of a huge file
head file_name → show first n lines of a file.

head -n 100 file_name → show first 100 lines of a file.

tail file_name → show the last n lines of a file.

head -n 100 file_name → show last 100 lines of a file.