Learning UNIX

What is Unix?

Unix is a time-sharing operating system.

Unix main features are multi-tasking, time-sharing(resource), and multi-users.

In the late 1960's, Ken Thompson and Dennis Ritchie of AT&T's Bell Laboratories developed Unix.

Unix was rewritten in C in the early 1970's.

Unix Components

Unix can be thought of as being comprised of 3 components: the kernel, the shell, and commands.

- the kernel provides basic capability of the operating system such as controlling hardware and devices.

- the shell is a program that interprets user's commands and runs them

- commands are utility program that perform essential tasks such as creating, copying files, etc...

Unix Commands

http://www.linfo.org/man.html

$ man man

options(max.print=1000000)

Find disk space in ubuntu, disk size

$ df -h

sudo apt-get install discus

$ discus

To erase a whole line of typed command

Ctrl-u

To stop the command or program

Ctrl-c

To exit or logout

Ctrl-d

List of users will be displayed

$who

Change your password

$passwd

To show current date

$date

To show month calendar

$cal

Creating Files

an editor (vi) can be used, or the shell redirection (>) can also be used to create file

ex: $echo "In the beginning..." >chapter1

$cat >chapter1 #create a file named "chapter1"

--> enter text here

Ctrl-d #end text insert

$vi chapter1 #use the "vi" editor

Viewing the content of text file

ex: $cat chapter1 #limited amount

$more chapter1 #full

Copying files

ex: $cp chapter1 chapter1.sav #make chapter1.sav copy from chapter1

$cp chapter1 book1 #copy chapter1 to the directory called book1

$cp chapter1 book1/newchpt1 #copy chapter1 to the directory called book1 and name it newchpt1

Deleting (removing) files

ex: $rm chapter1 #delete chapter1

$rm chapter1 chapter2 chapter3 #delete chapter1,2,3

Renaming or moving files (if directory exist, then, move a file to the directory; else, rename it)

ex: $mv draft1 chapter1 #rename draft1 to chapter1

$mv chapter1 book1 #move chapter1 to the directory called book1

Creating directory

ex: $mkdir book1 #create directory book1

$mkdir book1 book2 book3 #create 3 directories (book1,2,3)

Listing the content of directory

ex: $ls #list the content of the current directory

$ls -l #list the content of the current directory (detail)

$ls book1 #list the content of book1 directory

Renaming or moving directory (if directory exist, then, move to a directory to the directory; else, rename it)

ex: $mv book1 final_book #rename book1 directory to final_book

$mv book1 allbooks #move book1 directory under the allbook directory

Deleting (removing) directories

ex: $rmdir book1 #delete book1 directory

$rmdir book1 book2 #delete both book1 and book2 directories

Unix File System

File Names

A file name can be constructed with any characters (except the slash /).

Unix file names are case sensitive.

Extensions

There are no special meanings for file extensions and they are optional.

Extensions are used to add meanings to the filenames.

ex: xx.doc, xx.txt (.doc and .txt have no special meanings that is they need not to be text files)

xx.c (adds meanings - to indicate it's a C language source file)

xx.sh (adds meanings - to indicate it's a shell script file)

File Systems

A file system is a disk or partition of a disk prepared to hold directories and files.

Important System Directories of the File System

/bin contains essential commands and programs

/dev contains special files and device files

/etc contains system and administration files and commands

/home contains users home directories

/usr contains the rest of important files and programs for users

/tmp contains temporary files

/var contains spooling files

/lib contains essential libraries

The Root Directory

There is only one directory called "root", and it's at the top of the file system.

/ is the root directory.

Current and Parent Directories

The current directory is the directory that you're currently in.

"." is the current directory.

The parent directory is the directory above the current directory.

".." is the parent directory.

Home Directories

Each user has a home directory (that belong to her/him).

ex: if you are: jt1

your home directory is: /home/justin/jt1 (in this class)

Moving Around in the File System

The "cd" command is used to move around in the file system.

ex: $cd / #move to the root directory

$cd #move back to home directory

$cd ./book1 or

$cd book1 #go to book1 directory below current directory

$cd .. #go to the parent directory (move up 1 directory)

$cd book1/part1 #go to directory part1 below book1 which is below current directory

$cd ../part2 #go to part2 directory that is at the same level with the current directory

$cd ../.. #go to the grand directory (move up 2 directory)

$cd ~jt9 #go to the user jt9's home directory

Finding Out Where You Are (Your Current Directory)

The "pwd" command is used to print out the name of your current directory.

ex: $pwd #print out the full name(path) of current directory

Absolute Names or Path names

An absolute file (or directory) name is the full name of a file (or directory).

The name of the absolute name is constructed from the root (start with /) through the tree of directories.

ex: /tmp #an absolute or full name of the "tmp" directory which is directly under the root directory

/home/justin/jt1/book1 #full name of the "book1" directory which is under the "/home/justin/jt1" directory

Relative Names

A relative name of a file is the name that does not start with /.

ex: book1/chapter1 #chapter1 of book1 directory which is under current directory

The Shell and File Names

The shell searches for an absolute-named file using the path specified in the name.

The shell searches for a relative-named file using current directory as the reference for starting point.

ex: $cat /home/jt1/chapter1 #(use absolute name)

$cat chapter1 #(use relative name), view the file "chapter1" in current directory

The vi Editor Basics

Starting "vi"

To start "vi" editor, use the "vi" command.

ex: $vi chapter1 #launch vi to edit or create file chapter1

"vi" Modes

vi operate in 2 modes: insert and command.

when in the insert mode: text can be entered

when in the command mode: vi command can be entered

note: vi is in the command mode when first start

Entering Insert Mode

To enter insert mode, "i" or "a" or "o" command can be used.

i enters insert mode to insert text before the cursor

a enters insert mode to insert text after the cursor

o enters insert mode with a new line opened (below the current line) for inserting text

Entering Command Mode

To enter command mode, Escape key can be used.

Inputting Text

Enter the insert mode and type text.

--> hit Escape key when done (to go back to command mode)

Moving Around (cursor movements)

while in the command mode, use:

<- (left arrow key) #to move to the left one character

-> (right arrow key) #to move to the right one character

(up arrow key) #to move up one line

(down arrow key) #to move down one line

w #to move one word forward in a line

b #to move one word backward in a line

G #to move to the end of line

1G # to move to the beginning of file (same as gg)

Scrolling Text

while in the command mode, use:

Ctrl-f #to scroll forward one screen

Ctrl-b #to scroll back one screen

File Movements

while in the command mode, do:

:n or nG #to go to line n

Ctrl-g #to display current file name and line number

Deleting Text

while in the command mode

to delete a character:

position the cursor at the character to be deleted and hit "x"

to delete a word:

position the cursor at the beginning of the word to be deleted and enter "dw"

to delete a line:

move the cursor to the line to be deleted and enter "dd"

Delete block

https://vi.stackexchange.com/questions/1915/how-do-i-delete-a-large-block-of-text-without-counting-the-lines

Go to the starting line of your block, and

type ma 

(mark "a", though you can of course use different letters, and even multiple letters for different marks provided you can keep it straight in your head what each letter is a mark of).

Then go to the last line and enter 

d'a (delete to mark "a") or y'a (yank to mark "a") (a).

Cut and Past

while in the command mode, use: "dd" and "p" command

ex: 5dd #cut 5 lines from the current line at cursor

:15 #move to the line 15

p #past after the line 15

Copy and Past

while in the command mode, use "yy" and "p" commands

ex: 10yy #copy 10 lines from the current line to buffer

:$ #move to the end of file

p #past

Undo Changes

"u" command is used to undo last change #can undo to 1st, 2nd, 3rd... past change

ex: dd #delete a line

u #undo delete

Searching for Text. Ex: /something

"/" command is used to search for text and "n" command is used to repeat the search

ex: /buffer #search for the word "buffer"

n #repeat the previous search

Substitutions

while in the command mode, use "s" command

ex: :1,20s/total/total_amount/g #substitute "total" with "total_amount" from line 1 to line 20

:1,$s/total/total_amount/g #substitute "total" with "total_amount" everywhere in the entire file

Saving and Exiting

while in the command mode, enter:

:q! #to quit without save

:w #to save changes

:wp #to save and quit

Delete all line in the file:

In vi I use

vi delete, delete all text

:%d

where

• : tells vi to go in command mode

• % means all the line

• d : delete

http://www.lagmonster.org/docs/vi.html

ls -ltr

h

Move left

0

Move to the begining of the line

$

Move to the end of the line

1G

Move to the first line of the file

G

Move to the last line of the file

File Permission

Permissions

Both files and directories have permissions.

A user has to have permissions to access or operate on the files (or directories).

For each file, there are 3 groups of ownership permissions: u, g and o.

u=owner of the file

g=primary group (suppose there are several users under(jt1,..jt42) justin.jt42's primary group is "justin".

o=all others

For each group, there are 3 different permissions: r, w and x.

r=read

w=write

x=execute

Checking Permissions of a File

The "ls -l" command is used to check permissions of a file.

ex: $ls -l chapter1 #list permissions of chapter1 file

result: -rwxrwxrwx 1 jt1 justin 120 Sep 10 1998 10:30 chapter1

First character "-" indicates that this is "file" permission.

The first "rwx" permissions are for u(owner).

Second "rwx" permissions are for g(users in the same group).

Third "rwx" permissions are for o(everyone).

Checking Permission of a Directory

The "ls -ld" command is used to check permission of a directory.

ex: $ls -ld book1 #list permission of book1 directory

result: drwxr-xr-x 2 jt1 justin 4096 Oct 12 1998 9:00 book1

First character "d" indicates that this is "directory" permission.

Accordingly for each group of "rwx", "u", "g" and "o" are applied.

"r" means "can look" in the directory

"w" means can create or delete files in this directory

"x" means a user may access any files in the directory if he know filenames but may not search

Setting or Changing Permissions

The "chmod" command is used to set or change permissions.

ex: $chmod u+w chapter1 #add write permission to owner

$chmod go+r chapter1 #add read permission to group and others

$chmod ugo+rwx book1 #add read, write and execute to all on book1 directory

$chmod a+rwx book1 #same as above: "a"="ugo"

$chmod go-w chapter1 #take away write permission from group and others

$chmod o-wx book1 #take away write and execute permission from others

Example:

cd /usr/local/jss/tomcat

sudo chmod ug+rwx *

-R gives the permission recursively under below all files

sudo chmod -R ug+rwx *

########################

• chmod – change permissions

• chown – change ownership.

sudo chown -R jamftomcat:jamftomcat /usr/local/jss/tomcat/webapps/ROOT

sudo chown -R ubuntu:ubuntu /usr/local/jss/tomcat/webapps/

sudo chown -R ubuntu:ubuntu /usr/local/jss/tomcat/webapps/

sudo chown -R jamftomcat:jamftomcat /usr/local/jss/tomcat/webapps/

sudo chmod o+r -R /usr/local/ # give read permission to everyone under the /usr/local/ directory

sudo chmod ug+rwx -R /usr/local/jss/

>> This gives the permission to the files >> not sure why but -R is not working well

sudo chmod a+r -R /usr/local/jss/tomcat/webapps

>> This gives the permission to the directory

sudo chmod o+x /usr/local/jss/tomcat/conf

>> This work better

Add an Existing User Account to a Group (since ubuntu user already belong to ubuntu group, I need to list additional mytomcat group to add)ls

usermod -a -G mytomcat,ubuntu ubuntu

sudo usermod -a -G mytomcat <username>

sudo usermod -a -G mytomcat ubuntu

What is Unix?

Unix is a time-sharing operating system.

Unix main features are multi-tasking, time-sharing(resource), and multi-users.

In the late 1960's, Ken Thompson and Dennis Ritchie of AT&T's Bell Laboratories developed Unix.

Unix was rewritten in C in the early 1970's.

Unix Components

Unix can be thought of as being comprised of 3 components: the kernel, the shell, and commands.

- the kernel provides basic capability of the operating system such as controlling hardware and devices.

- the shell is a program that interprets user's commands and runs them

- commands are utility program that perform essential tasks such as creating, copying files, etc...

Unix Commands

http://www.linfo.org/man.html

$ man man

options(max.print=1000000)

Find disk space in ubuntu, disk size

$ df -h

sudo apt-get install discus

$ discus

To erase a whole line of typed command

Ctrl-u

To stop the command or program

Ctrl-c

To exit or logout

Ctrl-d

List of users will be displayed

$who

Change your password

$passwd

To show current date

$date

To show month calendar

$cal

Creating Files

an editor (vi) can be used, or the shell redirection (>) can also be used to create file

ex: $echo "In the beginning..." >chapter1

$cat >chapter1 #create a file named "chapter1"

--> enter text here

Ctrl-d #end text insert

$vi chapter1 #use the "vi" editor

Viewing the content of text file

ex: $cat chapter1 #limited amount

$more chapter1 #full

Copying files

ex: $cp chapter1 chapter1.sav #make chapter1.sav copy from chapter1

$cp chapter1 book1 #copy chapter1 to the directory called book1

$cp chapter1 book1/newchpt1 #copy chapter1 to the directory called book1 and name it newchpt1

Deleting (removing) files

ex: $rm chapter1 #delete chapter1

$rm chapter1 chapter2 chapter3 #delete chapter1,2,3

Renaming or moving files (if directory exist, then, move a file to the directory; else, rename it)

ex: $mv draft1 chapter1 #rename draft1 to chapter1

$mv chapter1 book1 #move chapter1 to the directory called book1

Creating directory

ex: $mkdir book1 #create directory book1

$mkdir book1 book2 book3 #create 3 directories (book1,2,3)

Listing the content of directory

ex: $ls #list the content of the current directory

$ls -l #list the content of the current directory (detail)

$ls book1 #list the content of book1 directory

Renaming or moving directory (if directory exist, then, move to a directory to the directory; else, rename it)

ex: $mv book1 final_book #rename book1 directory to final_book

$mv book1 allbooks #move book1 directory under the allbook directory

Deleting (removing) directories

ex: $rmdir book1 #delete book1 directory

$rmdir book1 book2 #delete both book1 and book2 directories

Unix File System

File Names

A file name can be constructed with any characters (except the slash /).

Unix file names are case sensitive.

Extensions

There are no special meanings for file extensions and they are optional.

Extensions are used to add meanings to the filenames.

ex: xx.doc, xx.txt (.doc and .txt have no special meanings that is they need not to be text files)

xx.c (adds meanings - to indicate it's a C language source file)

xx.sh (adds meanings - to indicate it's a shell script file)

File Systems

A file system is a disk or partition of a disk prepared to hold directories and files.

Important System Directories of the File System

/bin contains essential commands and programs

/dev contains special files and device files

/etc contains system and administration files and commands

/home contains users home directories

/usr contains the rest of important files and programs for users

/tmp contains temporary files

/var contains spooling files

/lib contains essential libraries

The Root Directory

There is only one directory called "root", and it's at the top of the file system.

/ is the root directory.

Current and Parent Directories

The current directory is the directory that you're currently in.

"." is the current directory.

The parent directory is the directory above the current directory.

".." is the parent directory.

Home Directories

Each user has a home directory (that belong to her/him).

ex: if you are: jt1

your home directory is: /home/justin/jt1 (in this class)

Moving Around in the File System

The "cd" command is used to move around in the file system.

ex: $cd / #move to the root directory

$cd #move back to home directory

$cd ./book1 or

$cd book1 #go to book1 directory below current directory

$cd .. #go to the parent directory (move up 1 directory)

$cd book1/part1 #go to directory part1 below book1 which is below current directory

$cd ../part2 #go to part2 directory that is at the same level with the current directory

$cd ../.. #go to the grand directory (move up 2 directory)

$cd ~jt9 #go to the user jt9's home directory

Finding Out Where You Are (Your Current Directory)

The "pwd" command is used to print out the name of your current directory.

ex: $pwd #print out the full name(path) of current directory

Absolute Names or Path names

An absolute file (or directory) name is the full name of a file (or directory).

The name of the absolute name is constructed from the root (start with /) through the tree of directories.

ex: /tmp #an absolute or full name of the "tmp" directory which is directly under the root directory

/home/justin/jt1/book1 #full name of the "book1" directory which is under the "/home/justin/jt1" directory

Relative Names

A relative name of a file is the name that does not start with /.

ex: book1/chapter1 #chapter1 of book1 directory which is under current directory

The Shell and File Names

The shell searches for an absolute-named file using the path specified in the name.

The shell searches for a relative-named file using current directory as the reference for starting point.

ex: $cat /home/jt1/chapter1 #(use absolute name)

$cat chapter1 #(use relative name), view the file "chapter1" in current directory

The vi Editor Basics

Starting "vi"

To start "vi" editor, use the "vi" command.

ex: $vi chapter1 #launch vi to edit or create file chapter1

"vi" Modes

vi operate in 2 modes: insert and command.

when in the insert mode: text can be entered

when in the command mode: vi command can be entered

note: vi is in the command mode when first start

Entering Insert Mode

To enter insert mode, "i" or "a" or "o" command can be used.

i enters insert mode to insert text before the cursor

a enters insert mode to insert text after the cursor

o enters insert mode with a new line opened (below the current line) for inserting text

Entering Command Mode

To enter command mode, Escape key can be used.

Inputting Text

Enter the insert mode and type text.

--> hit Escape key when done (to go back to command mode)

Moving Around (cursor movements)

while in the command mode, use:

<- (left arrow key) #to move to the left one character

-> (right arrow key) #to move to the right one character

(up arrow key) #to move up one line

(down arrow key) #to move down one line

w #to move one word forward in a line

b #to move one word backward in a line

G #to move to the end of line

1G # to move to the beginning of file (same as gg)

Scrolling Text

while in the command mode, use:

Ctrl-f #to scroll forward one screen

Ctrl-b #to scroll back one screen

File Movements

while in the command mode, do:

:n or nG #to go to line n

Ctrl-g #to display current file name and line number

Deleting Text

while in the command mode

to delete a character:

position the cursor at the character to be deleted and hit "x"

to delete a word:

position the cursor at the beginning of the word to be deleted and enter "dw"

to delete a line:

move the cursor to the line to be deleted and enter "dd"

Delete block

https://vi.stackexchange.com/questions/1915/how-do-i-delete-a-large-block-of-text-without-counting-the-lines

Go to the starting line of your block, and

type ma 

(mark "a", though you can of course use different letters, and even multiple letters for different marks provided you can keep it straight in your head what each letter is a mark of).

Then go to the last line and enter 

d'a (delete to mark "a") or y'a (yank to mark "a") (a).

Cut and Past

while in the command mode, use: "dd" and "p" command

ex: 5dd #cut 5 lines from the current line at cursor

:15 #move to the line 15

p #past after the line 15

Copy and Past

while in the command mode, use "yy" and "p" commands

ex: 10yy #copy 10 lines from the current line to buffer

:$ #move to the end of file

p #past

Undo Changes

"u" command is used to undo last change #can undo to 1st, 2nd, 3rd... past change

ex: dd #delete a line

u #undo delete

Searching for Text. Ex: /something

"/" command is used to search for text and "n" command is used to repeat the search

ex: /buffer #search for the word "buffer"

n #repeat the previous search

Substitutions

while in the command mode, use "s" command

ex: :1,20s/total/total_amount/g #substitute "total" with "total_amount" from line 1 to line 20

:1,$s/total/total_amount/g #substitute "total" with "total_amount" everywhere in the entire file

Saving and Exiting

while in the command mode, enter:

:q! #to quit without save

:w #to save changes

:wp #to save and quit

Delete all line in the file:

In vi I use

vi delete, delete all text

:%d

where

• : tells vi to go in command mode

• % means all the line

• d : delete

http://www.lagmonster.org/docs/vi.html

ls -ltr

h

Move left

0

Move to the begining of the line

$

Move to the end of the line

1G

Move to the first line of the file

G

Move to the last line of the file

File Permission

Permissions

Both files and directories have permissions.

A user has to have permissions to access or operate on the files (or directories).

For each file, there are 3 groups of ownership permissions: u, g and o.

u=owner of the file

g=primary group (suppose there are several users under(jt1,..jt42) justin.jt42's primary group is "justin".

o=all others

For each group, there are 3 different permissions: r, w and x.

r=read

w=write

x=execute

Checking Permissions of a File

The "ls -l" command is used to check permissions of a file.

ex: $ls -l chapter1 #list permissions of chapter1 file

result: -rwxrwxrwx 1 jt1 justin 120 Sep 10 1998 10:30 chapter1

First character "-" indicates that this is "file" permission.

The first "rwx" permissions are for u(owner).

Second "rwx" permissions are for g(users in the same group).

Third "rwx" permissions are for o(everyone).

Checking Permission of a Directory

The "ls -ld" command is used to check permission of a directory.

ex: $ls -ld book1 #list permission of book1 directory

result: drwxr-xr-x 2 jt1 justin 4096 Oct 12 1998 9:00 book1

First character "d" indicates that this is "directory" permission.

Accordingly for each group of "rwx", "u", "g" and "o" are applied.

"r" means "can look" in the directory

"w" means can create or delete files in this directory

"x" means a user may access any files in the directory if he know filenames but may not search

Setting or Changing Permissions

The "chmod" command is used to set or change permissions.

ex: $chmod u+w chapter1 #add write permission to owner

$chmod go+r chapter1 #add read permission to group and others

$chmod ugo+rwx book1 #add read, write and execute to all on book1 directory

$chmod a+rwx book1 #same as above: "a"="ugo"

$chmod go-w chapter1 #take away write permission from group and others

$chmod o-wx book1 #take away write and execute permission from others

Example:

cd /usr/local/jss/tomcat

sudo chmod ug+rwx *

-R gives the permission recursively under below all files

sudo chmod -R ug+rwx *

########################

• chmod – change permissions

• chown – change ownership.

sudo chown -R jamftomcat:jamftomcat /usr/local/jss/tomcat/webapps/ROOT

sudo chown -R ubuntu:ubuntu /usr/local/jss/tomcat/webapps/

sudo chown -R ubuntu:ubuntu /usr/local/jss/tomcat/webapps/

sudo chown -R jamftomcat:jamftomcat /usr/local/jss/tomcat/webapps/

sudo chmod o+r -R /usr/local/ # give read permission to everyone under the /usr/local/ directory

sudo chmod ug+rwx -R /usr/local/jss/

>> This gives the permission to the files >> not sure why but -R is not working well

sudo chmod a+r -R /usr/local/jss/tomcat/webapps

>> This gives the permission to the directory

sudo chmod o+x /usr/local/jss/tomcat/conf

>> This work better

Add an Existing User Account to a Group (since ubuntu user already belong to ubuntu group, I need to list additional mytomcat group to add)ls

usermod -a -G mytomcat,ubuntu ubuntu

sudo usermod -a -G mytomcat <username>

sudo usermod -a -G mytomcat ubuntu

fs 3:23 PM

oh..... I needed to exit the terminal once I do that; then, it works!...

########################

File Types

There are 7 file types: regular, directory, link, device (block, character), pipe and socket.

The "ls-l" or "ls-ld" can be used to find the file type.

ex: $ls -l filename

result: Crwxrwxrwx ... own grp size time filename

If "C" is "-" then file is "regular"

"d" "directory'"

"l" "soft link"

"b" or "c" "block" or "character" device file

"p" "pipe"

"s" "socket"

Links

There are 2 types of links: "hard link" and "soft link".

A hard link is another name to a file.

note: "One Inode for multiple name. File A and B share the same Inode".

A soft link is a file that contains a 'pointer' that points to another file.

note: "File A has Inode/data. File B has Inode/Pointer."

Pipes or Named Pipes (system V)

A pipe is FIFO file (first-in first-out, one way flow file) opened by programs for inter-process communications.

Sockets (BSD)

A socket is special file that is used for inter-process and network communications.

Device Files

A device file is a special file for a device.

Unix treats all peripheral devices as files, all I/Os with a device are done through its device file.

Shell Special Features

Filename Substitution: *, ?, and []

"*" means: substitution of any 'strings' of characters (in the file name)

ex: $ls chapter* #list all files with names begin with 'chapter'

$ls *.txt #list all files with names end with '.txt'

$ls * #list all files

"?" means: substitution of any 'single' character (in the file)

ex: $ls chapter? #list all files with name begin with 'chapter' and

end with any 'single' character like chapter1, chapterA, etc.

note: Thus, chapterABC will be listed by $ls chapter*, but not by $ls chapter?

"[]" mean any 'single' character from group in file name (and range can also be used :[1-9], [a-z], etc...)

ex: $ls hw[569] #list hw5,hw6 and hw9

$ls hw[1-9] #list hw1 to hw9

$ls hw[9-1] #No such file or directory

$ls hw[6-12] #list hw2: ignore '6-1'

$ls hw[1-a] #list hw1 to hw9, and hwA, hwa

$ls hw[1-A] #list hw1 to hw9, and hwA

Turning OFF Special Meanings of *, ?, and []

If the character *, ?, or [] are used as a part of a file name, its special meaning needs to be turned off.

Double quotes (""), or single quotes (''), or escape sequence (\) is used to turn off special meanings of *, ?, and [].

ex: $vi "s*wars" #create or edit the file s*wars (make * a part of the file name)

$vi 's*wars' #same as above

$vi s\*wars #same as above

result: $ls s*wars may list 's*wars' and 'startswars' ;whereas,

$ls "s*wars" will list ONLY 's*wars'

File names with Blanks

Double quotes (""), or single quotes (''), or escape sequence (\) can also be used for file names with blank.

ex: $vi "chapter 1" #create or edit the file 'chapter 1'

$vi 'chapter 1' #same as above

$vi chapter\ 1 #same as above

note: '>chapter\2' creates 'chapter2'. '>chapter\ 2' creates 'chapter 2'.

Redirections: ">", ">>", and "<"

">" is used to redirect output of a commands to a file.

If the file does not exist yet, then, the file will be created. If the file already exists, then, its content will be overwritten.

ex: $echo "Introduce to Unix" >hw1 #write "Introduce to Unix" to a hw1 file

$ls >list

#output of the "ls command", which is the content of current directory, is put in the file named 'list'

">>" is used to append output of a command to an existing file

ex: $cat chapter2>>chapter1 #content of chapter2 is appended to the end of the content of chapter1 file

"<" is used to redirect input from a file instead from the keyboard.

ex: $cat>chapter2<chapter1 #create chapter2 file with content taken from chapter1 file

$mail jt1<letter #send a content written in 'letter' to jt1

Pipe: |

"|" is used to connect output of a program to the input if another program.

ex: $ls /dev | more

#output of "ls" is piped to the 'more' command for paging. (one page output is displayed at a time - enter the Space bar for the next page)

Command Substitutions: ` `

`...` (back quotes) are used to execute a command and place its output where the quotes are at.

ex: $echo "Date is `date`" #the output of 'date' command is echoed out at the terminal together with the phrase 'Date is'

resutl: Date is Sun Oct 15 10:43:58 CDT 2000

$echo "users who currently logged in are: `who`"

"grep" and "find"

The "wc" Command

The "wc" is used to show the numbers of lines, words and characters in a file.

wc [-cwl] file #[option] -c is for character, -w is for word, and -l is for line count

ex: $wc -l chapter1 #show number of lines in chapter1 file

$wc -cw chapter1 #show number of words and characters in chapter1 file

$wc -cwl chapter1 #show number of lines, words, characters (eg. 8 25 147)

note: '$wc -cwl', '$wc -lcw', '$wc -wcl' are the same, and displays exact same order,i.e., (l, w, c)=(8 25 147)

$wc chapter1 #same as "$wc -cwl chapter1"

The "diff" command

The "diff" command is used to show the differences of two files.

ex: $diff file1 file2

result: 12d11 #if you delete line 12 of file1; then, both files match

6,7d5 #if you delete line 6 and 7 of file1; then, both files match

2a3 #at line 2 of file1, if you add a line 3 of file2 to file1; then both files match

5a7,10 #at line 5 of file1, if you add line 7,8,9,10 of file2 to file1; then, both files match

7c7 #if you change line 7 of file1 to line 7 of file2; then, both files match

The "find" Command

The "find" command is used to locate a file or files in the file system

Syntax: find pathname conditions

#pathname is the starting directory where the search begins

. #from the current directory

/ #start from root

/home #from /home

#conditions are conditions of files to be searched for, include options like:

-name filename #name of file to search for

-type f|d|l|b|c|p #file type to search for

-size nbytes #file size

-mtime [+/-]ndays #modified time

ex: $find . -name chapter1 -print #find the file chapter1, from the current directory and below

$find / -name order.c -print #find file order.c, start from root

$find / -name math -type d -print #find the directory math, start from root

$find /home -size0 -print #find all empty files, starting from /home

$find / -mtime -2 -print #find all files from root which were modified within the last 2 days

Regular Expressions

A regular expression is the description of a pattern or a sequence of characters.

Regular expressions are used in pattern searching in vi and grep.

ex: abc #means it a pattern that 'a' is followed by 'b' and then 'c'

pre #is a pattern that matched: expression, precedence, pressure...etc.

The "grep" Command

The "grep" command uses regular expression to search for a pattern in a file.

It reports all lines that have at least one match

ex: $grep Rom chapter1 #search to find where (what lines) in chapter1 where the pattern 'Rom' appears

note: If the file has lines that contain words such as: Rome, Roma, Romeo, CD-Rom...etc, they will be reported.

Special Characters of Regular Expressions

^ :is used to mean 'beginning of line'

$ :is used to mean 'end of line'

. :is used to mean 'a any single character'

[...] :is used for expression in group. Any one of characters in [...]. Ranges used like [a-z] are legal.

\ :used to turn off special meaning of a special character

* :used to mean 'zero' or 'more' occurrences of

ex: $grep "^The" chapter1 #search for all lines that begin with the pattern 'The'

$grep "and$" chapter1 #search for all lines that end with 'and'

$grep "an." chapter1 #search for all lines that has the pattern 'an' and any character follow it

$grep "\$100" #search for all lines that has the pattern '$100'

$grep room[1-9] chapter1 #search to find all occurrences of room1 to room9

$grep bugs* chapter1 #search to find where in chapter1 the word 'bug', 'bugs', or 'bugss'...etc appear

note: bug* displays no 's'(bug) or one or more 's'(bugs, bugss)

Processes

What is a Process?

A process is a single program (instance of the execution/activity) while running in the system.

Processes and Types

There are 3 types of processes:

1. Interactive processes

Interactive processes are processes initiated from and controlled by a terminal session.

2. Batch processes

Batch processes are processes of commands or programs run by the 'batch' or 'at' command.

Batch processes are not attached to any terminal, and are in queues and executed sequentially.

3. Daemons

Daemons are processes of server programs that are usually started at boot time, and run continuously while the system is up.

Process Attributes

A process has associated attributes and the some of them are:

1. PID (process Id) is a unique number assigned to the process by the system and it is used to refer to the process.

2. PPID (parent process Id) is the PID of the parent process. (each process has a parent process that start it)

3. UID (user Id) is a user Id who ran the process. (used to determine the process privileges)

4. Control terminal or login terminal of the process

Managing Processes

A user can list all processes and terminate his running processes.

Listing Processes

The "ps" command can be used to list user and system processes.

ex: $ps #list all processes run by the user

$ps -ef #list all process in the system (for system V Unix)

$ps aux #list all process in the system (for BSD Unix) -- Linux is a BSD

Terminating Processes

The "kill" command can be used to terminate running processes.

ex: $kill -2 645 #interrupt the process with PID=645

$kill -9 8520 #kill (absolute) the process with the PID=8520

Step1: $ps l

Result: FLAGS UID PID PPID PRI ...

100 689 2050 1938 0 ...

40 689 27038 2050 0 ...

Step2: $kill -9 27038 #knowing that PPID is 2050

Running in Background

"&" is used to run a command or program in the background.

(the shell prompt is returned to the user to do other work while the command or program is running)

ex: $who >who_list & #run the 'who' command in the background and redirect the result to 'who_list' file

$monthly_report & #run the program 'monthly_report', that takes a while to finish, in the background

Running at Later Times

A command or program can be run at a later time by using the "at" command.

ex: $at 1830 #mail a 'letter' to jt1 at 6:30PM today

mail jt1 <letter

Ctrl-d

Printing and the 'man' Commands

The "man" Command

The "man" command is used to display information on certain subjects or commands using data from online preference manual.

ex: $man ls #display information on the "ls' command

$man ftp #display information on the "ftp" command

The print Commands

The "lp" (for system V's Unix) or "lpr" (for BSD's Unix, Linux is a BSD Unix) can be used to print.

ex: $lpr chapter1 #print chapter1 using default printer

$lpr -d Laser2 chapter1 #print chapter1 using the printer 'Laser2'

The "lpstat (for system V's Unix) or "lpq" (for BSD's Unix) command can be used to list print jobs.

ex: $lpq #list Id of all print jobs (of the user)

The "cancel" (for system V's Unix) or "lprm" (for BSD's Unix) command can be used to cancel a print job

ex: $lprm 35 #cancel print job 35

Common Commands

File Management

pwd, cd, ls

cat, more, pg, head, tail

cp, rm, ln, mv

mkdir, rmdir

wc, file

chmod, chown, chgrp

Process Management

ps

kill

at, batch, nice, nohup, crontab

Searching

grep, egrep, fgrep

find

Comparison

cmp, diff, diff3, sdiff

dircmp

Printing

lp, lpstat, cancel (sys V)

lpr, lpq, lprm (BSD)

Storage

cpio

tar

compress, uncompress, zcat

pack, unpack, pcat

Communication

write, talk, wall

mail, ftp, rlogin, telnet

Text processing

sort

cut, past, join, fold, uniq

sed, awk

ex, vi

Shell

echo, expr, line, for, read, test, sleep

Programming

cb, cflow, ctags

cc, ld

make

sdb, od

lex, yacc

Miscellaneous

man

passwd, su

date, cal, bc, clear, banner

df, du, printenv, which, who, whoami, id, groups, finger

xargs

Shell Programming I

Shell Program Structure

A shell program (text file) consists of statements that can be Unix commands, shell commands, and comments.

However, the first line of a shell program is usually of the following form:

#!/bin/sh #to indicate scripts should be interpreted by Bourne shell

or

#!/bin/ksh #to indicate scripts should be interpreted by Korn shell

etc...

Creating a Shell Program

The 'vi' editor is usually used to create the program (script) file.

ex: $vi prog1.sh #start vi to create program file 'prog1.sh'

enter program statements

:wq #save and quit

The 'chmod' command is used to make the shell program executable.

ex: $chmod +x prog1.sh #make the file 'prog1.sh' executable

To run or execute the program, just type in the name at the shell prompt to run or execute.

ex: $ ./prog1.sh #type in prog1.sh at the shell prompt to run or execute

A Shell Program Example

The following are the steps to create a shell program that prints out a greeting, current date and time,

users currently logged-in, and content of current directory.

1. Create the program file

$vi start.sh #create a file called 'start.sh' for the shell program

2. Enter statements for the shell program

#!/bin/sh

echo "Hellow"

echo "Today is:"

date

echo "And here are all of the users who are currently logged in:"

who

echo "The content of current directory is:"

ls

3. Save and quit

:wq

4. Make it executable

$chmod +x start.sh #use chmod command to make it executable

5. Execute

$ ./start.sh #type the file name in to run

The 'if' Command

The 'if' command is used for conditional executions.

There are three forms of the 'if' command: if, if-else, and if-elif

Syntax:

if condition

then

commands

fi

or

if condition

then

commands

else

commands

fi

or

if condition

then

commands

elif condition

then

commands

elif condition

then

commands

elif ...

else

commands

fi

The [] Command

The [] command evaluates a condition and returns true or false.

Syntax:

[ condition ] #must have blanks between brackets and the condition

ex:

1. Test files:

[ -f file ] #exists & a regular

[ -x file ] #exists & executable

[ -w file ] #exists & writable

[ -r file ] #exists & readable

[ -d file ] #exists and a directory

2. Test strings:

[ str1=str2 ] #str1 the same as str2

[str1 !=str2 ] #str1 not the same as str2

[ string ] #string is not null

3. Test integer numbers:

[ i1 -eq i2 ] #is number i1 equal to i2

[ i1 -ne i2 ] #is number i1 not equal to i2

[ i1 -lt i2 ] #is number i1 less than i2

[ i1 -gt i2 ] #is number i1 greater than i2

[ i1 -ge i2 ] #is number i1 greater than or equal to i2

[ i1 -le i2 ] #is number i1 less than or equal to i2

'if' and [] together

The [] command is often used in conjunction with the 'if' command

ex:

#check the file permissions on file chapter1

if [ -r chapter1 ]

then

echo "have READ permission on chapter1"

else

echo "do not have READ permission on chapter1"

fi

Shell Variables

Shell variables are used to store values during the shell session.

Define Variables

Syntax:

varname=value #declare and initialize with a value

Note: no blanks between varname and value

ex: count=10 #define the variable 'count' and give a value

Using Variables

A variable is referenced (or used) with the '$' before the variable name.

ex: hour=10 #variable 'count' is defined in this statement

if [ $hour -lt 12 ] #used in this statement

then

echo "Good Morning"

fi

Lab 1:

Write a shell program or script that prints out a greeting depends on the hour of the day.

Delete the 'tmp' file if there is one, and check to see if a friend is logged in.

#!/bin/sh

hour=`date +%H`

if [ $hour -lt 12 ] #Need "Space" after if

then

echo "Good Morning"

elif [ $hour -lt 16 ] #Need "Space" after elif

then

echo "Good Afternoon"

else

echo "Good Night"

fi

#done "prints out a greeting depends on the hour of the day"

if [ -f tmp ] #Need "Space" after if

then

rm tmp

echo "tmp file is deleted"

else

echo "tmp can't be found"

fi

#done "delete the 'tmp' file if there is one"

if [ "`who|grep jt1`" ] #Need "Space" after if

then

echo "jt1 is here"

else

echo "jt1 is not here"

fi

#done "check to see if a friend is logged in"

Shell Programming II

Logical Operators

Logical operators are often used to combine the conditions together in an 'if' statement.

Logical operators are: AND, OR, and NOT

The AND Operator

Syntax:

[ condition1 -a condition2 ]

ex:

#check to see if chapter1 is readable and writable

if [ -r chapter1 -a -w chapter1 ]

then

echo "chapter1 is both readable and writable"

fi

The OR Operator

Syntax:

[ condition1 -o condition2 ]

ex:

#check to see if 'part1' a regular file or a directory

if [ -f part1 -o -d part1 ]

then

echo "part1 is either a regular file or a directory"

fi

The NOT Operator

Syntax:

[ !condition ]

ex:

if [ ! -f part1 ]

then

echo "part1 is not a regular file"

fi

Shell Built-in Variables 0-9

Shell has built-in variables that are used to receive program arguments.

The variable 0, 1, 2, 3, ...., 9 are such variables.

Program arguments are often files that are entered in the command line to run with the shell program.

ex: #run lab2.sh, which compare the 2 arguments,i.e., 'chapter1' and 'chapter2'

$./lab2.sh chapter1 chapter1.bak

#'chapter1' and 'chapter1.bak' are arguments of the shell program 'lab2.sh'

In the shell program 'lab2.sh', the file 'chapter1' and 'chapter1.bak' can be referred to

by using the variable 1 and 2.

#check if 'chapter1' and 'chapter1.bak' exist and readable

if [ -r $1 -a -r $2 ]

then

echo "$1 and $2 exist and readable"

fi

#use the 'cmp' command to check if they are the same

if [ cmp -s $1 $2 ]

then

echo "$1 and $2 are the same"

else

echo "$1 and $2 are not the same"

fi

Lab 2:

Write a shell program that takes two file names and check whether they exist and are

readable. If the files exist and are readable; then, the program determines whether they are

the same. If the they are the same; then, it deletes one of the files and reports which one

was deleted; otherwise, if files are different, then, for each file, the program prints out the

file name together with the size of the file. And finally, if the files do not exist nor are

readable; then, the program indicates so and quits.

#!/bin/sh

if [ -f $1 -a -f $2 ] #Need "Space" after if

then

if [ -r $1 -a -r $2 ]

then

if [ "`cmp $1 $2`" ] #if I get message; then 2 files are different. if I didn't get message; then, 2 files are the same

then

echo

wc -w $1

echo

wc -w $2

else

rm $2

echo "$2 is deleted"

fi

else

echo "Not readable"

fi

else

echo "One or two files won't exist"

fi

Shell Programming III

The "for" loop is used to execute a group of commands for a special number of times.

Syntax:

for

do

command

command

...

done

ex: #a "for" loop that illustrates how it works, it prints out what is put in the variable "file", from the list, for each iteration of the loop.

#!/bin/sh

for file in "$@"

do

echo "" #a blank line for spacing

echo "This time the file is: $file"

echo "and it's content is:"

cat $file

echo "By: at"

date

done

The built-in Variable @

@ is the built-in variable that holds all of the arguments (files) specified in the command for a shell program.

@ is often used together with a 'for' loop.

ex: # 'rename': a shell program that renames any files specified on the command with the extension '.txt' attached.

#!/bin/sh

for file in "$@"

do

mv $file $file.txt #rename it

done

#run 'rename' to rename 3 files

# $./rename chapter1 chapter2 chapter3

#when the command runs, the shell puts all of the program arguments in @

(variable @ holds the file names: chapter1, chapter2, and chapter3)

#the 'for' loop gets each of the name in the list and puts it in the variable 'file' for renaming

The "#" Built-in Variable

The built-in variable # holds the number of arguments for a shell program.

It is often used to check for the correct number of arguments entered in the command.

ex: #if number of argument is less than 1, then the user forgot to enter

any file names for the command, print out message and exit.

#!/bin/sh

if [ $# -lt l ] # or if [$# -eq 0 ]

then

echo "Please enter file names for the command"

exit #exit a shell program

fi

Lab 3:

Write a shell program or script that takes a list offiles and make copies of them (backing up files).

If the user enter a file that doesn't exist, echo the file name indicate the file does not exist.

Also if the user forgot to enter a file or files for the command to back up, print out a message to indicate so.

#!/bin/sh

if [ $# -eq 0 ]

then

echo "Please enter file names for the command"

exit

fi

for file in "$@"

do

if [ -f $file ]

then

cp $file $file.bak

else

echo "$file won't exist"

fi

done

Shell Programming IV

Reading Data: the 'read' Command

The 'read' command can be used to read data from terminals.

ex: read option #read in a value from the terminal and store in the variable 'option'

The 'case' Command

The 'case' command is used for selective executions

ex: #scripts that read in an option selected by an user and execute the associated command for the option

read option

case $option in

1) echo "Current Date and Time is:"

date ;;

2) echo "Current Month is:"

cal ;;

3) echo "Exiting..."

exit ;;

*) echo "Invalid option, please re-enter selection"

esac

The 'while' Loop

The 'while' command is used to execute a group of commands again and again depends upon the status of a condition.

Writing an Infinite Loop

Sometimes, an infinite loop is desired in writing a program.

A program that deals with menu usually employs an infinite loop to keep the menu running until users enter the 'quit' option.

ex: #menu.sh

#program that employs an infinite loop to display a menu

true=1

while [ $true -eq 1 ]

do

echo " Menu"

echo "1. Current Date and Time"

echo "2. Current Month Calendar"

echo "3. Quite"

echo -n "select an option"

#read in and process option

.....

done

#this works the same as above but it is more elegant

while : #this is the infinite while loop statement

do

echo " Menu"

echo "1. Current Date and Time"

echo "2. Current Month Calendar"

echo "3. Quite"

echo -n "select an option"

#read in and process option

.....

done

The 'break' Command

The 'break' command is used to break out of a loop

ex:

while :

do

#print menu

...

read option #read in a value for option

case $option in #process the option using the 'case' command

1) date ;;

2) cal ;;

3) break ;;

esac

done

Lab 4:

Write a shell program or script that print out a menu and ask a user to select an option to program.

If an option is entered, the associated command or commands are executed to perform the desired option.

When it's done, the menu is redisplayed for another selection,

and this process repeats until the user selects the option 'quit' to quit the program.

Also if the user entered an invalid option, the program should print out an error message and redisplay the menu.

#!/bin/sh

while :

do

echo "Menu"

echo "1. Current Date and Time"

echo "2. Current Month Calendar"

echo "3. Users Currently Logged-in"

echo "4. List all files in the current directory"

echo "5. Quite"

echo "Select an Option:"

read option

case $option in

1) echo "Current Date and Time is:"

date ;;

2) echo "Current Month Calendar is:"

cal ;;

3) echo "Users Currently Logged-in"

who ;;

4) echo "Files in the current directory are:"

ls ;;

5) echo "Exiting..."

exit ;;

*) echo "Invalid option. Please re-enter selection"

esac

done

########################

File Types

There are 7 file types: regular, directory, link, device (block, character), pipe and socket.

The "ls-l" or "ls-ld" can be used to find the file type.

ex: $ls -l filename

result: Crwxrwxrwx ... own grp size time filename

If "C" is "-" then file is "regular"

"d" "directory'"

"l" "soft link"

"b" or "c" "block" or "character" device file

"p" "pipe"

"s" "socket"

Links

There are 2 types of links: "hard link" and "soft link".

A hard link is another name to a file.

note: "One Inode for multiple name. File A and B share the same Inode".

A soft link is a file that contains a 'pointer' that points to another file.

note: "File A has Inode/data. File B has Inode/Pointer."

Pipes or Named Pipes (system V)

A pipe is FIFO file (first-in first-out, one way flow file) opened by programs for inter-process communications.

Sockets (BSD)

A socket is special file that is used for inter-process and network communications.

Device Files

A device file is a special file for a device.

Unix treats all peripheral devices as files, all I/Os with a device are done through its device file.

Shell Special Features

Filename Substitution: *, ?, and []

"*" means: substitution of any 'strings' of characters (in the file name)

ex: $ls chapter* #list all files with names begin with 'chapter'

$ls *.txt #list all files with names end with '.txt'

$ls * #list all files

"?" means: substitution of any 'single' character (in the file)

ex: $ls chapter? #list all files with name begin with 'chapter' and

end with any 'single' character like chapter1, chapterA, etc.

note: Thus, chapterABC will be listed by $ls chapter*, but not by $ls chapter?

"[]" mean any 'single' character from group in file name (and range can also be used :[1-9], [a-z], etc...)

ex: $ls hw[569] #list hw5,hw6 and hw9

$ls hw[1-9] #list hw1 to hw9

$ls hw[9-1] #No such file or directory

$ls hw[6-12] #list hw2: ignore '6-1'

$ls hw[1-a] #list hw1 to hw9, and hwA, hwa

$ls hw[1-A] #list hw1 to hw9, and hwA

Turning OFF Special Meanings of *, ?, and []

If the character *, ?, or [] are used as a part of a file name, its special meaning needs to be turned off.

Double quotes (""), or single quotes (''), or escape sequence (\) is used to turn off special meanings of *, ?, and [].

ex: $vi "s*wars" #create or edit the file s*wars (make * a part of the file name)

$vi 's*wars' #same as above

$vi s\*wars #same as above

result: $ls s*wars may list 's*wars' and 'startswars' ;whereas,

$ls "s*wars" will list ONLY 's*wars'

File names with Blanks

Double quotes (""), or single quotes (''), or escape sequence (\) can also be used for file names with blank.

ex: $vi "chapter 1" #create or edit the file 'chapter 1'

$vi 'chapter 1' #same as above

$vi chapter\ 1 #same as above

note: '>chapter\2' creates 'chapter2'. '>chapter\ 2' creates 'chapter 2'.

Redirections: ">", ">>", and "<"

">" is used to redirect output of a commands to a file.

If the file does not exist yet, then, the file will be created. If the file already exists, then, its content will be overwritten.

ex: $echo "Introduce to Unix" >hw1 #write "Introduce to Unix" to a hw1 file

$ls >list

#output of the "ls command", which is the content of current directory, is put in the file named 'list'

">>" is used to append output of a command to an existing file

ex: $cat chapter2>>chapter1 #content of chapter2 is appended to the end of the content of chapter1 file

"<" is used to redirect input from a file instead from the keyboard.

ex: $cat>chapter2<chapter1 #create chapter2 file with content taken from chapter1 file

$mail jt1<letter #send a content written in 'letter' to jt1

Pipe: |

"|" is used to connect output of a program to the input if another program.

ex: $ls /dev | more

#output of "ls" is piped to the 'more' command for paging. (one page output is displayed at a time - enter the Space bar for the next page)

Command Substitutions: ` `

`...` (back quotes) are used to execute a command and place its output where the quotes are at.

ex: $echo "Date is `date`" #the output of 'date' command is echoed out at the terminal together with the phrase 'Date is'

resutl: Date is Sun Oct 15 10:43:58 CDT 2000

$echo "users who currently logged in are: `who`"

"grep" and "find"

The "wc" Command

The "wc" is used to show the numbers of lines, words and characters in a file.

wc [-cwl] file #[option] -c is for character, -w is for word, and -l is for line count

ex: $wc -l chapter1 #show number of lines in chapter1 file

$wc -cw chapter1 #show number of words and characters in chapter1 file

$wc -cwl chapter1 #show number of lines, words, characters (eg. 8 25 147)

note: '$wc -cwl', '$wc -lcw', '$wc -wcl' are the same, and displays exact same order,i.e., (l, w, c)=(8 25 147)

$wc chapter1 #same as "$wc -cwl chapter1"

The "diff" command

The "diff" command is used to show the differences of two files.

ex: $diff file1 file2

result: 12d11 #if you delete line 12 of file1; then, both files match

6,7d5 #if you delete line 6 and 7 of file1; then, both files match

2a3 #at line 2 of file1, if you add a line 3 of file2 to file1; then both files match

5a7,10 #at line 5 of file1, if you add line 7,8,9,10 of file2 to file1; then, both files match

7c7 #if you change line 7 of file1 to line 7 of file2; then, both files match

The "find" Command

The "find" command is used to locate a file or files in the file system

Syntax: find pathname conditions

#pathname is the starting directory where the search begins

. #from the current directory

/ #start from root

/home #from /home

#conditions are conditions of files to be searched for, include options like:

-name filename #name of file to search for

-type f|d|l|b|c|p #file type to search for

-size nbytes #file size

-mtime [+/-]ndays #modified time

ex: $find . -name chapter1 -print #find the file chapter1, from the current directory and below

$find / -name order.c -print #find file order.c, start from root

$find / -name math -type d -print #find the directory math, start from root

$find /home -size0 -print #find all empty files, starting from /home

$find / -mtime -2 -print #find all files from root which were modified within the last 2 days

Regular Expressions

A regular expression is the description of a pattern or a sequence of characters.

Regular expressions are used in pattern searching in vi and grep.

ex: abc #means it a pattern that 'a' is followed by 'b' and then 'c'

pre #is a pattern that matched: expression, precedence, pressure...etc.

The "grep" Command

The "grep" command uses regular expression to search for a pattern in a file.

It reports all lines that have at least one match

ex: $grep Rom chapter1 #search to find where (what lines) in chapter1 where the pattern 'Rom' appears

note: If the file has lines that contain words such as: Rome, Roma, Romeo, CD-Rom...etc, they will be reported.

Special Characters of Regular Expressions

^ :is used to mean 'beginning of line'

$ :is used to mean 'end of line'

. :is used to mean 'a any single character'

[...] :is used for expression in group. Any one of characters in [...]. Ranges used like [a-z] are legal.

\ :used to turn off special meaning of a special character

* :used to mean 'zero' or 'more' occurrences of

ex: $grep "^The" chapter1 #search for all lines that begin with the pattern 'The'

$grep "and$" chapter1 #search for all lines that end with 'and'

$grep "an." chapter1 #search for all lines that has the pattern 'an' and any character follow it

$grep "\$100" #search for all lines that has the pattern '$100'

$grep room[1-9] chapter1 #search to find all occurrences of room1 to room9

$grep bugs* chapter1 #search to find where in chapter1 the word 'bug', 'bugs', or 'bugss'...etc appear

note: bug* displays no 's'(bug) or one or more 's'(bugs, bugss)

Processes

What is a Process?

A process is a single program (instance of the execution/activity) while running in the system.

Processes and Types

There are 3 types of processes:

1. Interactive processes

Interactive processes are processes initiated from and controlled by a terminal session.

2. Batch processes

Batch processes are processes of commands or programs run by the 'batch' or 'at' command.

Batch processes are not attached to any terminal, and are in queues and executed sequentially.

3. Daemons

Daemons are processes of server programs that are usually started at boot time, and run continuously while the system is up.

Process Attributes

A process has associated attributes and the some of them are:

1. PID (process Id) is a unique number assigned to the process by the system and it is used to refer to the process.

2. PPID (parent process Id) is the PID of the parent process. (each process has a parent process that start it)

3. UID (user Id) is a user Id who ran the process. (used to determine the process privileges)

4. Control terminal or login terminal of the process

Managing Processes

A user can list all processes and terminate his running processes.

Listing Processes

The "ps" command can be used to list user and system processes.

ex: $ps #list all processes run by the user

$ps -ef #list all process in the system (for system V Unix)

$ps aux #list all process in the system (for BSD Unix) -- Linux is a BSD

Terminating Processes

The "kill" command can be used to terminate running processes.

ex: $kill -2 645 #interrupt the process with PID=645

$kill -9 8520 #kill (absolute) the process with the PID=8520

Step1: $ps l

Result: FLAGS UID PID PPID PRI ...

100 689 2050 1938 0 ...

40 689 27038 2050 0 ...

Step2: $kill -9 27038 #knowing that PPID is 2050

Running in Background

"&" is used to run a command or program in the background.

(the shell prompt is returned to the user to do other work while the command or program is running)

ex: $who >who_list & #run the 'who' command in the background and redirect the result to 'who_list' file

$monthly_report & #run the program 'monthly_report', that takes a while to finish, in the background

Running at Later Times

A command or program can be run at a later time by using the "at" command.

ex: $at 1830 #mail a 'letter' to jt1 at 6:30PM today

mail jt1 <letter

Ctrl-d

Printing and the 'man' Commands

The "man" Command

The "man" command is used to display information on certain subjects or commands using data from online preference manual.

ex: $man ls #display information on the "ls' command

$man ftp #display information on the "ftp" command

The print Commands

The "lp" (for system V's Unix) or "lpr" (for BSD's Unix, Linux is a BSD Unix) can be used to print.

ex: $lpr chapter1 #print chapter1 using default printer

$lpr -d Laser2 chapter1 #print chapter1 using the printer 'Laser2'

The "lpstat (for system V's Unix) or "lpq" (for BSD's Unix) command can be used to list print jobs.

ex: $lpq #list Id of all print jobs (of the user)

The "cancel" (for system V's Unix) or "lprm" (for BSD's Unix) command can be used to cancel a print job

ex: $lprm 35 #cancel print job 35

Common Commands

File Management

pwd, cd, ls

cat, more, pg, head, tail

cp, rm, ln, mv

mkdir, rmdir

wc, file

chmod, chown, chgrp

Process Management

ps

kill

at, batch, nice, nohup, crontab

Searching

grep, egrep, fgrep

find

Comparison

cmp, diff, diff3, sdiff

dircmp

Printing

lp, lpstat, cancel (sys V)

lpr, lpq, lprm (BSD)

Storage

cpio

tar

compress, uncompress, zcat

pack, unpack, pcat

Communication

write, talk, wall

mail, ftp, rlogin, telnet

Text processing

sort

cut, past, join, fold, uniq

sed, awk

ex, vi

Shell

echo, expr, line, for, read, test, sleep

Programming

cb, cflow, ctags

cc, ld

make

sdb, od

lex, yacc

Miscellaneous

man

passwd, su

date, cal, bc, clear, banner

df, du, printenv, which, who, whoami, id, groups, finger

xargs

Shell Programming I

Shell Program Structure

A shell program (text file) consists of statements that can be Unix commands, shell commands, and comments.

However, the first line of a shell program is usually of the following form:

#!/bin/sh #to indicate scripts should be interpreted by Bourne shell

or

#!/bin/ksh #to indicate scripts should be interpreted by Korn shell

etc...

Creating a Shell Program

The 'vi' editor is usually used to create the program (script) file.

ex: $vi prog1.sh #start vi to create program file 'prog1.sh'

enter program statements

:wq #save and quit

The 'chmod' command is used to make the shell program executable.

ex: $chmod +x prog1.sh #make the file 'prog1.sh' executable

To run or execute the program, just type in the name at the shell prompt to run or execute.

ex: $ ./prog1.sh #type in prog1.sh at the shell prompt to run or execute

A Shell Program Example

The following are the steps to create a shell program that prints out a greeting, current date and time,

users currently logged-in, and content of current directory.

1. Create the program file

$vi start.sh #create a file called 'start.sh' for the shell program

2. Enter statements for the shell program

#!/bin/sh

echo "Hellow"

echo "Today is:"

date

echo "And here are all of the users who are currently logged in:"

who

echo "The content of current directory is:"

ls

3. Save and quit

:wq

4. Make it executable

$chmod +x start.sh #use chmod command to make it executable

5. Execute

$ ./start.sh #type the file name in to run

The 'if' Command

The 'if' command is used for conditional executions.

There are three forms of the 'if' command: if, if-else, and if-elif

Syntax:

if condition

then

commands

fi

or

if condition

then

commands

else

commands

fi

or

if condition

then

commands

elif condition

then

commands

elif condition

then

commands

elif ...

else

commands

fi

The [] Command

The [] command evaluates a condition and returns true or false.

Syntax:

[ condition ] #must have blanks between brackets and the condition

ex:

1. Test files:

[ -f file ] #exists & a regular

[ -x file ] #exists & executable

[ -w file ] #exists & writable

[ -r file ] #exists & readable

[ -d file ] #exists and a directory

2. Test strings:

[ str1=str2 ] #str1 the same as str2

[str1 !=str2 ] #str1 not the same as str2

[ string ] #string is not null

3. Test integer numbers:

[ i1 -eq i2 ] #is number i1 equal to i2

[ i1 -ne i2 ] #is number i1 not equal to i2

[ i1 -lt i2 ] #is number i1 less than i2

[ i1 -gt i2 ] #is number i1 greater than i2

[ i1 -ge i2 ] #is number i1 greater than or equal to i2

[ i1 -le i2 ] #is number i1 less than or equal to i2

'if' and [] together

The [] command is often used in conjunction with the 'if' command

ex:

#check the file permissions on file chapter1

if [ -r chapter1 ]

then

echo "have READ permission on chapter1"

else

echo "do not have READ permission on chapter1"

fi

Shell Variables

Shell variables are used to store values during the shell session.

Define Variables

Syntax:

varname=value #declare and initialize with a value

Note: no blanks between varname and value

ex: count=10 #define the variable 'count' and give a value

Using Variables

A variable is referenced (or used) with the '$' before the variable name.

ex: hour=10 #variable 'count' is defined in this statement

if [ $hour -lt 12 ] #used in this statement

then

echo "Good Morning"

fi

Lab 1:

Write a shell program or script that prints out a greeting depends on the hour of the day.

Delete the 'tmp' file if there is one, and check to see if a friend is logged in.

#!/bin/sh

hour=`date +%H`

if [ $hour -lt 12 ] #Need "Space" after if

then

echo "Good Morning"

elif [ $hour -lt 16 ] #Need "Space" after elif

then

echo "Good Afternoon"

else

echo "Good Night"

fi

#done "prints out a greeting depends on the hour of the day"

if [ -f tmp ] #Need "Space" after if

then

rm tmp

echo "tmp file is deleted"

else

echo "tmp can't be found"

fi

#done "delete the 'tmp' file if there is one"

if [ "`who|grep jt1`" ] #Need "Space" after if

then

echo "jt1 is here"

else

echo "jt1 is not here"

fi

#done "check to see if a friend is logged in"

Shell Programming II

Logical Operators

Logical operators are often used to combine the conditions together in an 'if' statement.

Logical operators are: AND, OR, and NOT

The AND Operator

Syntax:

[ condition1 -a condition2 ]

ex:

#check to see if chapter1 is readable and writable

if [ -r chapter1 -a -w chapter1 ]

then

echo "chapter1 is both readable and writable"

fi

The OR Operator

Syntax:

[ condition1 -o condition2 ]

ex:

#check to see if 'part1' a regular file or a directory

if [ -f part1 -o -d part1 ]

then

echo "part1 is either a regular file or a directory"

fi

The NOT Operator

Syntax:

[ !condition ]

ex:

if [ ! -f part1 ]

then

echo "part1 is not a regular file"

fi

Shell Built-in Variables 0-9

Shell has built-in variables that are used to receive program arguments.

The variable 0, 1, 2, 3, ...., 9 are such variables.

Program arguments are often files that are entered in the command line to run with the shell program.

ex: #run lab2.sh, which compare the 2 arguments,i.e., 'chapter1' and 'chapter2'

$./lab2.sh chapter1 chapter1.bak

#'chapter1' and 'chapter1.bak' are arguments of the shell program 'lab2.sh'

In the shell program 'lab2.sh', the file 'chapter1' and 'chapter1.bak' can be referred to

by using the variable 1 and 2.

#check if 'chapter1' and 'chapter1.bak' exist and readable

if [ -r $1 -a -r $2 ]

then

echo "$1 and $2 exist and readable"

fi

#use the 'cmp' command to check if they are the same

if [ cmp -s $1 $2 ]

then

echo "$1 and $2 are the same"

else

echo "$1 and $2 are not the same"

fi

Lab 2:

Write a shell program that takes two file names and check whether they exist and are

readable. If the files exist and are readable; then, the program determines whether they are

the same. If the they are the same; then, it deletes one of the files and reports which one

was deleted; otherwise, if files are different, then, for each file, the program prints out the

file name together with the size of the file. And finally, if the files do not exist nor are

readable; then, the program indicates so and quits.

#!/bin/sh

if [ -f $1 -a -f $2 ] #Need "Space" after if

then

if [ -r $1 -a -r $2 ]

then

if [ "`cmp $1 $2`" ] #if I get message; then 2 files are different. if I didn't get message; then, 2 files are the same

then

echo

wc -w $1

echo

wc -w $2

else

rm $2

echo "$2 is deleted"

fi

else

echo "Not readable"

fi

else

echo "One or two files won't exist"

fi

Shell Programming III

The "for" loop is used to execute a group of commands for a special number of times.

Syntax:

for

do

command

command

...

done

ex: #a "for" loop that illustrates how it works, it prints out what is put in the variable "file", from the list, for each iteration of the loop.

#!/bin/sh

for file in "$@"

do

echo "" #a blank line for spacing

echo "This time the file is: $file"

echo "and it's content is:"

cat $file

echo "By: at"

date

done

The built-in Variable @

@ is the built-in variable that holds all of the arguments (files) specified in the command for a shell program.

@ is often used together with a 'for' loop.

ex: # 'rename': a shell program that renames any files specified on the command with the extension '.txt' attached.

#!/bin/sh

for file in "$@"

do

mv $file $file.txt #rename it

done

#run 'rename' to rename 3 files

# $./rename chapter1 chapter2 chapter3

#when the command runs, the shell puts all of the program arguments in @

(variable @ holds the file names: chapter1, chapter2, and chapter3)

#the 'for' loop gets each of the name in the list and puts it in the variable 'file' for renaming

The "#" Built-in Variable

The built-in variable # holds the number of arguments for a shell program.

It is often used to check for the correct number of arguments entered in the command.

ex: #if number of argument is less than 1, then the user forgot to enter

any file names for the command, print out message and exit.

#!/bin/sh

if [ $# -lt l ] # or if [$# -eq 0 ]

then

echo "Please enter file names for the command"

exit #exit a shell program

fi

Lab 3:

Write a shell program or script that takes a list offiles and make copies of them (backing up files).

If the user enter a file that doesn't exist, echo the file name indicate the file does not exist.

Also if the user forgot to enter a file or files for the command to back up, print out a message to indicate so.

#!/bin/sh

if [ $# -eq 0 ]

then

echo "Please enter file names for the command"

exit

fi

for file in "$@"

do

if [ -f $file ]

then

cp $file $file.bak

else

echo "$file won't exist"

fi

done

Shell Programming IV

Reading Data: the 'read' Command

The 'read' command can be used to read data from terminals.

ex: read option #read in a value from the terminal and store in the variable 'option'

The 'case' Command

The 'case' command is used for selective executions

ex: #scripts that read in an option selected by an user and execute the associated command for the option

read option

case $option in

1) echo "Current Date and Time is:"

date ;;

2) echo "Current Month is:"

cal ;;

3) echo "Exiting..."

exit ;;

*) echo "Invalid option, please re-enter selection"

esac

The 'while' Loop

The 'while' command is used to execute a group of commands again and again depends upon the status of a condition.

Writing an Infinite Loop

Sometimes, an infinite loop is desired in writing a program.

A program that deals with menu usually employs an infinite loop to keep the menu running until users enter the 'quit' option.

ex: #menu.sh

#program that employs an infinite loop to display a menu

true=1

while [ $true -eq 1 ]

do

echo " Menu"

echo "1. Current Date and Time"

echo "2. Current Month Calendar"

echo "3. Quite"

echo -n "select an option"

#read in and process option

.....

done

#this works the same as above but it is more elegant

while : #this is the infinite while loop statement

do

echo " Menu"

echo "1. Current Date and Time"

echo "2. Current Month Calendar"

echo "3. Quite"

echo -n "select an option"

#read in and process option

.....

done

The 'break' Command

The 'break' command is used to break out of a loop

ex:

while :

do

#print menu

...

read option #read in a value for option

case $option in #process the option using the 'case' command

1) date ;;

2) cal ;;

3) break ;;

esac

done

Lab 4:

Write a shell program or script that print out a menu and ask a user to select an option to program.

If an option is entered, the associated command or commands are executed to perform the desired option.

When it's done, the menu is redisplayed for another selection,

and this process repeats until the user selects the option 'quit' to quit the program.

Also if the user entered an invalid option, the program should print out an error message and redisplay the menu.

#!/bin/sh

while :

do

echo "Menu"

echo "1. Current Date and Time"

echo "2. Current Month Calendar"

echo "3. Users Currently Logged-in"

echo "4. List all files in the current directory"

echo "5. Quite"

echo "Select an Option:"

read option

case $option in

1) echo "Current Date and Time is:"

date ;;

2) echo "Current Month Calendar is:"

cal ;;

3) echo "Users Currently Logged-in"

who ;;

4) echo "Files in the current directory are:"

ls ;;

5) echo "Exiting..."

exit ;;

*) echo "Invalid option. Please re-enter selection"

esac

done