UNIX and LINUX part 16

The following example matches any line containing the word Apples.
nawk '/Apples/' fruits

Escape Sequences
The following sequences of characters are referred to as escape sequences. Escape sequences specify a notation for characters that are not easily produced from your keyboard and/or cause problems when displayed to your screen. The nawk and echo commands use escape sequences. Although only nawk uses the escape sequences in regular expressions.
Sequence
Description
\b Matches a backspace.
\f Matches a form feed.
\n Matches a new-line.
\r Matches a carriage return.
\t Matches a tab.
\0ddd Matches the ASCII character for the given octal ASCII code.
\c Matches the literal character c. Use \ to generate a \.
Expression Patterns
Expression patterns perform number or string comparisons. Nawk provides eight comparison operators. Six of these operators are relational operators used to perform number or string comparison. The other two are string operators used to perform string comparisons.
String Operators may be used to compare a regular expression pattern to a specific field or the entire input line. The following two operators are supported:
Operator
Function
/RE/ Matches if the current line contains the specified regular expression. Same as the $0 ~ /RE/ command.
expr ~ RE Matches if the string value of expr contains the regular expression RE. For example,
nawk '$1 ~ /[Aa]pples/' fruits

matches lines where field 1 contains "Apples" or "apples."
expr !~ RE Matches if the string value of expr does not contain the regular expression expr. For example,
nawk '$3 !~ /[Aa]pples/' fruits

matches lines where field 1 does not contain the word "Apples" or "apples."
Relational Operators Relational operators compare strings and numbers. The following six operators are used to compare values.
Operator
Function
lv > rv True if lv is greater than rv. For example,
nawk 'NF > 3' fruits

prints each line containing more than three fields.
lv >= rv True if lv is greater than or equal to rv. For example,
nawk ' $1 >= "M" ' fruits

prints each line where the first field begins with a character that has a higher ASCII code than capital M. Such as N, O, P, Q, etc.
lv < rv True if lv is less than rv.
lv <= rv True if lv is less than or equal to rv.
lv == rv True if lv is equal to rv. For instance,
nawk '$1 == "Oranges" ' fruits

prints each line where the first field equals the string "Oranges."
lv != rv True if lv is not equal to rv.
Compound Patterns
A compound pattern is an expression consisting of multiple expression patterns combined with Boolean operators. The Boolean operators are || (OR), && (AND), ! (NOT), and use of parentheses for grouping the operators. If the result of a compound pattern is true, the current input line is matched and the associated action is executed.
Boolean Operators The Boolean operators can be used to combine regular and relational expressions. The following table describes the Boolean operators.
Operator
Function
! Negation. True if expression is NOT true. For example,
nawk '! /Apples/' fruits

prints each line that does not contain the string "Apples."
|| Logical OR. True if either expression is true. For instance,
nawk '$3 > "3,000,000" || $1 == "FRUIT"' fruits

prints the header line and each line with bushel counts of more than "3,000,000." Note 3,000,000 is a string, not a number, because of the commas.
&& Logical AND. True if both expressions are true. For example,
nawk '$3 > "3,000,000" && NF == 5' fruits

prints the header line and each line containing exactly three fields.
( ) Perform comparisons enclosed together(for grouping). For instance,
nawk '( $1 == "Apples" || $1 == "Oranges" ) &&
( NR == 4 )' fruits

prints lines where the first field is either "Apples" or "Oranges" and there are exactly four fields.
Range Patterns
Range patterns contain two patterns separated by a comma (,). A range pattern matches each line from the occurrence of the first pattern to the occurrence of the second pattern. Both patterns may appear on the same line, thus only the one line is matched. If no input line matches the first pattern, no lines are selected. If no input line matches the second pattern, all lines from the first pattern to the end of the file are selected. For example, the following command selects all of the lines from one that contains Apples to one that contains Cherries.
nawk '/Apples/,/Cherries/' fruits

ACTIONS
The action part of an nawk program performs specified actions when a pattern is matched. If no action is specified the default print action is used. The default is the same as specifying { print }, which is equivalent to { print $0 }.
Comments
The nawk command ignores any characters on a line following a # sign. Thus you can document your nawk programs using the number sign.
Constants
Two types of constants are used by nawk, numeric and string. String constants are created by enclosing a sequence of characters in double or single quote marks. For example,
$1 == "FRUIT"

UNIX and LINUX part 15

If an example is given for syntax reasons only, the filename infile is used.
PROGRAMS
The nawk command interprets a set of commands known as the nawk program. The program can be specified as an argument on the shell command line or stored in a file. In either case the program consists of one or more pattern-action statements. The general form of the pattern-action statement is:
pattern { action }

For example,
nawk 'NF < 5 { print $1 }' fruits

prints the first field of a line if less than five fields exist on the line. Fields are space or tab separated words. The built-in variable NF contains the number of fields in the current line.
Nawk reads each line of input automatically. If the pattern matches the input line (record) in part or whole, then the following action is performed. If no pattern is specified, then the action is performed for every input record. The action may consist of multiple programming statements discussed later. The action must be enclosed by '{}' for nawk to function properly. The general format is:
{ action }

For example,
nawk '{ print }' fruits

prints every line of the file infile. Thus you wrote a very simple version of the cat command. This is the default pattern-action statement of nawk.
If no action is specified, each matching input record is displayed on the standard output. The general format is:
pattern

For example,
nawk 'NF == 4' fruits

prints each line consisting of four fields.
WRITING PROGRAMS
It is very common to write small nawk programs in-line; that is, on the command line or even spread over a couple of lines. For example,
nawk 'BEGIN { print "Annual Fruit Report" }
{ print }
END { print "More fruit to Grow" }' fruits

prints the banner line Annual Fruit Report, every line of the fruits file, and a trailer line More fruit to Grow. This line can be typed from your terminal or placed in a shell script.
NOTE:
In-line nawk programs are limited in length to the command line buffer of your system. This buffer varies; some systems are set to 512 characters of information while others are set to 10240 characters. The general rule of thumb for the length of in-line programs is that they should never exceed 2000 characters (approximately one screenful of characters).
The same nawk program written in a file format would be invoked by the command line:
nawk -f fruitprog fruits

The fruitprog file would contain the following lines.
BEGIN { print "Annual Fruit Report" }
{ print }
END { print "More fruit to Grow" }

Notice the single quotes are removed; the nawk command itself is not needed for the input files.
PATTERNS
Patterns are used to select lines from the input. If a pattern matches a string on an input line, the associated action is performed. It is possible to write an nawk program that does not have a pattern, such as
nawk '{ print $1 }' fruits

The default pattern selects all lines from the input.
NOTE:
The BEGIN and END patterns both require an action.
The BEGIN Pattern
The BEGIN pattern does not match any input. Instead, the action associated with the BEGIN pattern is executed before any input is read by nawk. This allows for initialization of variables, the printing of headers, and other coding that needs to be done before the first line of input is read by nawk. The general format is:
nawk 'BEGIN { initialization code; headers; etc. }' infile

The following example prints a header line.
nawk 'BEGIN { print "Fruit Shipped From Primary State" }' fruits

The END pattern
The END pattern does not match any input. Instead, the action associated with the END pattern is executed after all of the input has been read by nawk. The general format is:
nawk 'END { wrap up code }' infile

The following example prints the total number of lines read by nawk.
nawk 'END { print NR }' infile

The built-in variable NR contains the Number of Records read.
Regular Expression Patterns
The following table contains each regular expression and the task that it performs when used inside a pattern. Regular Expressions are often referred to as REs in UNIX terminology. Thus we use the RE notation for uniformity and briefness.
Metacharacters
Metacharacters are the special characters used in regular expression patterns that have special meanings. Metacharacters are often referred to as special or magic characters.
Regular expression patterns must be enclosed in slashes. For example, to match any records (lines) containing the string cpu you would use /cpu/ as the pattern. The string inside the slashes may be any valid combination of the following list.
Special RE
Description
c Matches the character c if c is not a special regular expression character.
\ Escapes the meaning of a metacharacter.
^ Matches the beginning of the line.
$ Matches the end of the line.
. Matches any single character other than the new-line.
[class] A character class. Matches any one character in the class.
[c1-c2] Matches any one of the ASCII characters in the range defined within the brackets.
[^class] Does NOT match any of the ASCII characters listed within the brackets. Ranges may be specified.
| Alternation of regular expressions. Matches either one or the other of the regular expressions provided.
(!) Concatenation operation. Normally the parentheses are omitted. Allows for control of precedence in the interpretation of the regular expressions specified.
RE* Matches zero or more occurrences of the preceding regular expression.
RE+ Matches one or more occurrences of the preceding regular expression.
RE? Matches zero or one occurrence of the preceding regular expression.
// The null RE refers to the last RE defined.