GROUP - C
Table of Contents
S.No.
Topic
Signatur
1.
Introduction to Operating System
2.
Tasks of an Operating System
3.
Classification Of Operating Systems
4.
Memory Management
5.
Virtual Memory
6.
Virtual Memory using Paging
7.
Process Management
8.
Process States
9.
Process Control Block
10.
Scheduling
11.
File System
12.
Deadlocks
13.
File Management
14.
Introduction to UNIX
15.
UNIX Kernal
16.
UNIX Shell
17.
UNIX Utilities
18.
UNIX File Management System
19.
The concept of Directory
20.
System Administration
21.
UNIX File System
22.
Information Node
23.
Data/Storage Blocks
24.
General UNIX Commands
25.
Redirection
26.
Introduction to Shell Programming
27.
Some Shell Programs
Operating System
Introduction to Operating System- An operating system is a program that acts as an intermediate between a "user" and the "computer".
The purpose of an operating system is to provide an environment in which a user can execute programs. The primary goal of an operating system is thus to make the computer system convenient to use.
A secondary goal is to use the computer hardware in an efficient manner.
A computer is a complex system. It is required that a person with minimal technical skills should be able to use computer. This is made possible by the operating system of the computer.
An operating system is a system software. It manages the resources of a computer like-
1 Memory
2 Processor
3 Peripherals i.e. input/output devices
4 Files
An operating system provides an easy to use interface to the user. A user interacts with a computer through its operating system. Even with a limited knowledge of the operating system, it is possible to use a computer. In other words, an operating system makes a computer system easy to use for the user.
Needs of an Operating System:
Every boss needs a secretary and so does a human being when dealing with a computer. The operating system is a set of specialized programs that are used to manage the resources and overall operations of a computer. It acts as an interface between the user and the computer.
Thus, the operating system isolates the hardware from the user, who is a layman. The user communicates with the OS, feeds application programs and input the data that are in a language and format acceptable to the OS, and receives the output results. Therefore, the operating system shields the user from trivial hardware details.
Tasks of an Operating System
An operating system performs three major tasks:
1. Act as an interface between the user and the computer.
2. Manage the resources of the computer efficiently.
1 Memory
2 Processor
3 Files
4 I/O Devices
3. Provide adequate utilities to the user.
1. User Interface:
An interface is a name for a mechanism through which two incompatible systems interact. An operating system provides a user interface. A user interacts with the machine through this interface. The user interface is also called a shell.
The shell is a program that interprets the commands given by the user and initiates appropriate action. The shell could be command or menu driven.
For example, to write a program we use an editor like norton editor.In a menu driven shell, the user is given a list of options like open a file, save a file and so on.
2. Resource Management:
The Memory, Processor, Files and the I/O Devices are considered as the resources of a computer.
An operating system manages the resources of the computer.
1 It interacts with hardware devices.
2 It allocates resources like primary memory and disk space to the users.
3 It keeps track of the processors time.
4 It manages the file system.
For example, in a multi-user system, resources like the CPU, primary memory, hard disk, peripherals like printers, plotters etc. are dynamically shared amongst the users by the operating system.
One important aspect of resource management is the file system. The file system organizes the programs and data as files on the secondary storage devices. It makes the management of programs and data very convenient for the user.
The core part of the operating system which deals with resource management is referred to as its kernel.
Thus, we conclude that the resource management tasks include:
Memory Management:
The function of an operating system that manages memory is called the memory management.
Memory management includes the following tasks:
1 Keep track of the parts of memory which are in use and those that are unused.
2 Allocate memory to a program in execution.
3 Take decisions regarding the amount of memory to be allocated.
4 Free the occupied memory when a task is over and reallocating the same to another task in queue.
5 Ensure that the memory allocated to a task is not allocated to another task.
Processor or Process Management:
Process Management means managing one very essential resource of a computer i.e. the heart of the computer -CPU. Therefore, this functions of operating system deals with-
process
processors
scheduling of processor i.e. CPU
Device Management:
A device management function handles the device for an I/O operation and for error recovery.
File Management:
File management function is used to facilitate easy creation, storage and access of files in order to enable easy sharing of files between programs and their protection against illegal access.
3. Utilities:
An operating system provides a collection of utility programs for the end users and software developers. It includes programs for common tasks like-
Disk Formatting
Displaying the content of a file
Printing
Linking
Loading
Editing
Classification of Operating Systems:
On the basis of "Convenience" and "Efficiency":
It is easier to define operating systems by what they do than by what they are. The primary goal of an operating system is convenience for the user. Operating systems exist because they are supposed to make it easier to compute with them than without them. This view is particularly clear when you look at operating systems for small personal computers.
A secondary goal is efficient operation of the computer system. This goal is particularly important for large, shared multi-user systems. These systems are typically expensive, so it is desirable to make them as efficient as possible. These two goals - convenience and efficiency - are sometimes contradictory.
In the past, efficiency considerations were often more important than convenience. Thus, much of operating-system theory concentrates on optimal use of computing resources. Based on efficiency and convenience, the operating systems have been categorized into various categories. Like-
Batch Processing System:
A batch is a sequence of user jobs. A computer operator forms a batch by arranging user jobs in a sequence and inserting special marker cards to indicate the start and end of the batch. After forming a batch, the operator submits it to the batch processing operating system.
Thus Batch Processing System implements the processing of the jobs in a batch without requiring any intervention of the operator. This is achieved by automating the transition from the execution of one job to that of the next job in the batch.
Batch processing is implemented by locating a component of the BP system, called the batch monitor or supervisor, permanently in one part of the computer's memory. The remaining memory is used to process a user job - the current job in the batch.
The batch monitor is responsible for implementing the various functions initiating the processing of a batch and sets up the processing of the first job of the batch. At the end of the job, it performs job termination of the next batch by the operator.
Multiprogramming System:
To improve utilization of CPU, I/O devices and memory, it is often useful to have more than one process in memory at once. When more than one process are executing, it is called multiprogramming environment.
Multiprogramming is the first instance where the operating system must make decision. Multi-programmed operating systems are therefore fairly sophisticated. All the jobs that enter the system are kept in the job pool. This pool consists of all processes residing on mass storage awaiting allocation of main memory. The operating system picks and begins to execute one of the jobs in the main memory. Eventually, the job may have to wait for some task, such as a tape to be mounted, or an I/O operation complete.
In a multiprogramming system, the operating system simply switches to and executes another job. When that job needs to wait, the CPU is switched to and executes second job. When second job needs to wait, the CPU switched to third job, and so on. Eventually, the first job finishes waiting and gets the CPU back. As long as there is always some job to execute, the CPU will never be idle.
Time Sharing System:
Multi-programmed batched system provides an environment where the various system resources (for example CPU, memory, peripheral devices) are utilized effectively. There are some difficulties with a batch system from the point of view of the user. However, since the user can not interact with the job when it is executing, the user must set up the control cards to handle all possible outcomes.
Time sharing systems were developed to provide interactive use of a computer system at a reasonable cost.
A time sharing operating system uses CPU scheduling and multiprogramming to provide each user with a small portion of a processors time called time slice.
Multiprogramming and time sharing are the central themes of modern operating systems.
Real Time System:
Another form of a special purpose operating system is the real time system.
A real time operating system is used when there are rigid time constraints on the operation of a processor or the flow of data.
The real time operating systems are used often as control device in a dedicated application.
A real time operating system has well defined, fixed time constraints. Processing must be done within the defined constraint, or the system will fail. A hard real time system guarantees that critical tasks complete on time.
A soft real time system is a less restrictive type of real time system, where a critical real time task gets priority over other tasks and retains that priority until it completes.
Classification on the basis of characteristics:
Operating systems can be classified on the basis of their important characteristics. Some of these characteristics are explained below-
Open:
Operating systems which are available on a wide variety of computer hardware from different companies are called open or standard.
Some operating systems like DOS, UNIX, Netware, Windows, Windows 95, Windows NT etc. are not bound to the hardware of a particular company.
Any computer manufacturer can port the operating system to his hardware. For example DOS runs on computer manufactured by various companies like IBM, COMPAQ, Hewlett Packard etc.
Proprietary Operating Systems:
Proprietary Operating Systems are bound to the computers of a particular company.
The proprietary operating systems make it difficult for the users to move from the hardware of one company to the other. Some examples of Proprietary operating systems are listed below-
Company Operating Systems
IBM MVS
DEC VMS
Single User Operating Systems:
An operating system which lets only one user work with a computer at a time is called A Single User Operating System.
These are the simplest operating systems. Most microcomputer operating systems are single user operating systems.
The most widely used single user operating systems are MS-DOS and MS-Windows.
Single User Multitasking Operating Systems:
A Single User Multitasking Operating System lets one user work with a computer at a time.
But, it allows the user to execute multiple programs at the same time.MS-Windows, Windows- 95, Windows-98 and OS/2 are some examples of single user multi tasking operating system.
Multi User Operating Systems:
A multi user operating system lets multiple users work with the computer simultaneously.
In a multi user environment, there is a single computer which is connected to many nodes i.e. a terminal (keyboard and monitor).The operating system lets the users share the primary memory, CPU and other resources of the central computer.
The server has to be very fast and powerful so that it may serve the terminals i.e. multiple user attached to it.
UNIX is the best example of multi user operating system.
Network:
A Network Operating System allows multiple computers to be connected in a network.
In a network environment, each node owns a-
CPU
Keyboard
Monitor
Since these computers are connected in a network, the users can share the resources as well as communicate with each other.
Novell Netware and Windows NT provide the network environment.
Memory Management
Introduction:-
The main purpose of a computer system is to execute programs. These programs together with the data they access must be in main memory (at least partially) during execution.
To improve both utilization of CPU and the speed of its response to its users, the computer, must keep several processes in memory.
The function of operating system which manages the memory of a computer for the processing of programs efficiently is called as memory management.
Who Is A Memory Manager?
Memory of a computer is an important resource that must be carefully managed.
The part of the operating system that manages memory is called the memory manager.
Memory manager performs the following tasks-
1. Keep track of the parts of memory which are in use and those that are unused.
2. Allocate memory to a program in execution.
3. Take decisions regarding the amount of memory to be allocated.
4. Free the occupied memory when a task is over and reallocating the same to another task in queue.
5. Ensure that the memory allocated to a task is not allocated to another task.
The study of memory management can be divide into broad categories.
1 Memory management in Mono-Programming environment.
2 Memory management in Multi-Programming environment.
Memory Management in Multi-Programming Environment:
To improve utilization of CPU, I/O devices and memory, it is often useful to have more than one process in memory at once.
When more than one process are executing, it is called multiprogramming environment.
Different memory managers use different memory management schemes to make sure that processes do not collide at the same memory locations during storage.
Swapping-
Swapping refers to the process of transferring sections of a program from primary memory to secondary memory and vice versa.
Sometimes the entire program is swapped.
The term swapped in is used to refer to a program being transferred from secondary to primary memory.
The term swapped out is used to refer to the reverse program.
It is required when the memory requirements of the process exceeds the primary memory. The program requiring CPU time is swapped in the primary memory and the rest of the processes are swapped out. This process goes on and on till all the processes are processed. Thus all the processes not needing the CPU time are swapped out.
What is Virtual Memory?
As tasks are executed only after they have been loaded into primary memory the size of a program is limited by the amount of primary memory installed in the system.
A part of memory is always occupied by the operating system. Hence memory available for programs is further reduced. This limits number programs that can be executed by the computer.
To increase memory size, the amount of primary memory installed in the system can be increased. Primary memory is fairly expensive. Moreover, computers are designed to work with some maximum amount of memory. There primary memory can be increased only to a certain extent.
The concept of virtual memory supports the primary memory in case of shortage. In this case, the secondary memory acts as primary memory. The processes immediately needing processor are swapped in the primary memory and the processes not currently required are placed in the secondary memory. They are swapped in as and when required.
This transfer of data to and from the primary memory is transparent to the user.
Such kind of technique to extend the primary memory and give a feel of having a large amount of primary is called as virtual memory.
Virtual Memory using Paging-
Most programs do not require all of their instructions to be present in the memory simultaneously. Only a part of the program in the memory is required at times.
Therefore, paged memory technique comes in discussion. In paged memory technique, a program is dividing into equal sized sections or blocks. The size of a block may be from 512 bytes to 4k.Each block is referred to as a page.
Similarly, the primary memory is also divided into blocks. The block size of a program and the primary memory are equal.
The blocks in the primary memory are usually called page frames. Thus pages of a process are loaded into page frames of the primary memory.
In such a system, processes or parts that are not currently required are swapped out. They are stored in the secondary memory. They are swapped in as and when required.
This transfer of data from secondary to primary and vice versa is accomplished by operating system and the process is completely transparent to the user.
Whenever a program is swapped in, memory is allocated to it. Whenever a program is swapped out, memory previously allocated to it is freed.
All operating systems do not support virtual memory.
Paged Memory:
Most programs do not require all of their instruction to be present in memory at any given point of time.
Two programs divided into pages of fixed and same size
Primary memory divided into page frames having the size same as the page size of the programs
The concept of paged memory subscribes to this philosophy. In a paged memory, a program is divided into equal sized blocks. The size of a block may be from 512 bytes to 4k.Each block is referred to as a page.
The primary memory is also divided into blocks. The block size of a program and the primary memory are equal. The blocks in the primary memory are usually called page frames. Thus pages of a process are loaded into page frames of the primary memory.
To execute a program, the entire program is not loaded into memory. Only a few pages are loaded. The page frames into which these pages are loaded may not be contiguous.
To keep track of which page of which process is loaded into which page frame, a page table is used.
Page Table:
A page table also stores other information regarding the pages of a process. It also keeps track of which page frames are free in the memory.
Page Fault
Fields in the PMT (page map table) entry of page pi whether the page currently exists in memory. If pi does not exist in memory when referenced, ATU (address translation unit) raises a missing page interrupt, also called a page fault.
Whenever a process refers to an instruction or data that lies outside the currently executing page, it triggers a page fault. Whenever occurs, the operating system consults the page table to see if the desired instruction or data item is available in a page frame. If it is, control is handed over the desired page. If it is not, the desired page is swapped in and control is handed over to it.
Demand Paging:
Demand paging is a variation of paged memory. In demand paging, a page is brought into main memory only when it is required.
Anticipation Paging:
In anticipation paging, a page is brought in memory anticipating that it will be required in near future. As a programs execution is largely sequential, the next page is brought in memory while the currently page is being executed.
Page Replacement Policy:
When paged memory is used, pages are continually swapped in and swapped out.
The performance of the computer system can improved, if the operating system follows a logical policy for swapping pages in and out. This policy is known as a page replacement policy.
When a page is brought in main memory, an old page has to be swapped out. The page which should be swapped out is determined by page replacement policy. Different operating systems use different policies for replacing pages.
The following are the commonly used page replacement policies.
First in first out (FIFO):
The oldest (or first) page is replaced by the new page.
Least recently used (LRU):
The page which was not used for the maximum amount of time is replaced by the new page. In this policy, the operating system maintains a count for each page. The count is incremented at regular intervals. Whenever, the page is referenced, the count is set to zero.
Page replacement becomes necessary when a page fault occurs and no free page blocks exist in memory. An important issue in page replacement is deciding which page to replace. If a replaced page is referenced in immediate future, it would cause another page fault.
Segmented Memory:
One disadvantage of using paged memory is that pages do not define logical boundaries of a program. For example, a loop or procedure may not be of the same size as A PAGE.
Consider an example where a program contains a loop which can fit only two pages. In the worst possible case, we will assume that during execution, only one page can be in memory.
So it execute the loop 1000 times, each page has to be swapped in and swapped out 1000 times. A total of 1000 swap operations are required. This will drastically decrease the performance of the system as each page transfer involves accessing the secondary memory.
One solution to the above problem is to divide the program into segments.
Segmentation is a memory management scheme where the program is divide into parts and stored in various memory segments.
A logical address space is a collection of segments. Each segment has a name and a length. The address specifies both the segment name and the offset within the segment. The user therefore specifies each address by two quantities: a segment name and an offset.
Features of segments:
1 Segments may have different lengths.
2 Segment lengths may change during execution.
These segments mark natural boundaries of the program. In segmented memory, a segment table is used to keep track of where a segment can be found.
Process Management
Introduction-
Operating system functions fall into broad categories. It becomes essential to understand each of its function very clearly and deeply. Process management plays an important role.
Process Management means managing one very essential resource of a computer i.e. the heart of the computer - CPU. Therefore, this functions of operating system deals with-
1 Process
2 Processors
3 Scheduling of processor i.e. CPU
Concept of a Process:
A process is a program in execution.
A process is a tuple having fields:
(process id, code, data, register values, pc value)
Where,
process id is unique in the system,
code is the program code,
data is the data used during its execution i.e. the data space and the values of the data,
register values are the values in the machine registers,
and
pc value is the address in the program counter.
Job and process are the terms which are almost used interchangeably.
Process States
A process is a program or job in execution. During execution, the process changes its states. The states of a process are defined by its current action.A process can have 4 states:
1 New: The process is being created.
2 Active: Instructions are being executed.
3 Waiting: The process is waiting for some event to occur.
4 Terminated: The process has finished execution.
Various states of a process
Commonly used terms-
1. Throughput:
It is the amount of work accomplished in a given time interval (e.g. 20 job per hour)
It is calculated as follows-
The number of jobs completed (n)
Throughput =
Total time taken to complete the job(t)
2. Turnaround Time:
It is the delay between the job submission and job completion.
3. Response Time:
The time taken by the computer to the processing request is called response time.
Process Control Block (PCB)-
Each process in the OS is represented by a data block called Process Control Block. A PCB contains following information about a process:
1 Process State
2 Program counter: It indicates the address of next instruction to be executed.
3 CPU register: Stores system related information.
4 Memory limits
5 List of open files
Process Switch
To control the execution of processes, the kernel must be able to suspend the execution of the process running on the CPU and resume the execution of some other process previously suspended. This activity goes variously by the names process switch, task switch, or context switch.
Performing the Process Switch:
A process switch may occur at just one well-defined point: the schedule( ) function.
Essentially, every process switch consists of two steps:
1. Switching the Page Global Directory to install a new address space.
2. Switching the Kernel Mode stack and the hardware context, which provides all the information needed by the kernel to execute the new process, including the CPU registers.
SCHEDULING
This concept originated when there were many processes requesting for the one and only CPU and its time and every process was urgent.
Scheduling is the activity of determining which service request should be handled next by a server.
Scheduling is a major function of an operating used in process should be allotted or given the CPU time and which process should wait in the queue for its turn.
In job scheduling, a priority is assigned to the jobs and new jobs are called for processing at appropriate times.
Criteria for Scheduling:
In making decisions about the scheduling of processor work, a number of criteria can be taken into account by the operating system. The precise effect of these factors becomes more evident shortly when we describe specific scheduling algorithms. The criteria to be considered include the following:
1 Priority assigned to job. This is discussed in more detail below.
2 Class of job; i.e. batch or on-line or real-time. On -line users require a tolerable response time, while real-time systems often demand instant service.
3 Resource requirements; e.g. expected run-time, memory required, etc.
4 I/O or CPU bound; i.e. whether job uses predominantly I/O time or processor time. This criterion is often of consequence because of the need to balance the use of processor and the I/O system. If the processor is absorbed in CPU-intensive work, it is unlikely that the I/O devices are being serviced frequently enough to sustain maximum throughput.
5 Resources used to date; e.g. the amount of processor time already consumed.
6 Waiting time to date; i.e. the amount of time spent waiting for service so far.
Scheduling falls into two broad categories -
1 Non-Preemptive Scheduling
2 Preemptive Scheduling
Non-preemptive Scheduling:
In Non-Preemptive Scheduling, a scheduled job is always completed before another scheduling decision is made. The scheduling techniques which use non-preemptive scheduling are:
1. First come first served (FCFS) scheduling
2. Shortest job Next (SJN) scheduling
3. Deadline scheduling
First Come First Served Scheduling:
This is the simplest scheduling technique which is managed by FIFO (First in First out).That is, the process, which requests the CPU first, is allocated the CPU first.
Queue:
A queue is maintained to store processes. It is also called a ready queue in which all the processes, that want CPU time, are entered. The CPU executes the jobs in the ready queue one by one.
But turnaround time for the very first job in the batch is the best and for the very last job, is worst.(Turnaround time is the delay between job submission and job completion).
Let’s consider an example,
Let there be, three jobs, job1, job2 and job3 in a queue
Now, the second job requires only one second and the third job requires only 2 seconds for execution. But, they have to wait for 45 seconds so that the first job may leave the CPU for them.
This technique fails in such cases. A rescheduling of jobs should be done in order to give fair a chance to each job. The next scheduling technique solves this problem.
Shortest Job Next (SJN) Scheduling:
In SJN Scheduling, whenever a new job is to be admitted, the shortest of the arrived jobs is selected and given the CPU time.
Throughput remains the same as in FCFS scheduling but the waiting time improves. SJN associates with each job the length of its next CPU burst.
CPU burst is the CPU time required by a job to execute its continuous executable part e.g. if there are three executable statements and two I/O operations then again executable parts. Then the job will have two CPU bursts one for executable part before I/O and the next for executable part after I/O. When the CPU is available, it is allocated to the job with smallest next CPU burst. If two jobs have the same next CPU burst, then FCFS is used. For example, if we have the following set of jobs:
Using SJN , the scheduling order of these jobs would be JOB 4; JOB 3; JOB 6; JOB 1; JOB 2; JOB 5
It is difficult to implement in ptctice since the next CPU burst time may not always be known.
In deadline scheduling , the job with the earliest deadline is selected for scheduling.Deadline of a job is the time limit within which a job must be over.If a job overshoots its deadline.It is said to be deadline over run.Deadline is calculated as
Z= CT-D
Where,
Z= deadline iover run
CT= job completion time
D= deadline for a job
Preemptive Scheduling;
As compared to non preemptive scheduling methods , preemptive scheduling methods gives a fair chance to each process in queue to share the CPU time.A decision can be made to release the processor for a new process , even when an old process is using the processor.
Therfore , preemptive scheduling may force a job in execution to release the processor, so that execution of some other job can be undertaken , in order to improve the throughput considerably.
There are various techniques which use preemptive scheduling:
i) Round Robin Scheduling
ii) Response Ratio Scheduling
Round Robin Scheduling:
"All the processes are equally important and no process is a special process." This point is kept in mind by the operating system before starting with round robin scheduling.
In order to give same treatment and equal chance to all the processes in queue , Round robin scheduling is aimed at giving all programs equal opportunity to make progress.This is implemented by ensuring that no program gets a second chance to execute unless all other programs have had at least one opportunity.
A small unit of time, called a time quantum or time slice, is defined.the ready queue (queue of programs waiting for CPU time) is treated as a circular queue.The program in the ready queue are processed time slice, one by one.
When a program time comes, it can utilize the processor for the duration of time slice.However, if it requires I/O before t elapses, it is passed over to I/O handler and CPU moves over to the next program and executes it.After the time t is over, it puts the program in the end of the queue and moves over to the next program.
If the program which required I/O is now through queue.This way, CPU processes all the programs in the ready queue one by one until their execution is over.Once a program's execution is over, its output is processed and it is removed from the ready queue.
Actual progress made by each program depends upon its capacity to utilize the processor.It implies that turnaround time of a job also depend upon its nature (CPU bound, I/O bound etc.).Thus, a short job may not get better turnaround time than a long job.
The drawbacks of RR scheduling, gave birth to a new technique known as Least completed next (LCN).
Least completed next (LCN):
This strategy favors the job which has consumed the least processor time amongst all active jobs in the system.However, sill it does not ensure good turnaround service to shorter jobs.
Shortest - Time - To - Go (STTG):
Shortest - Time - To- Go is a policy where a job would be scheduled when its remaining processing requirements are smaller than those of all other jobs in the system.The drawback of this policy is that it favors any jobs which is near to completion, irrespective of the processing time already consumed by it.
Response Ratio Scheduling:
To overcome the drawbacks of both LCN and STTG scheduling, concepts of Response Ratio Scheduling technique became popular.
A response ratio is calculated as-
Multi Processor Scheduling:
Our major discussion in processor management has focussed on the single processor system.Most of these scheduling aspects apply equally well in multi processor systems as only the processor is to be chosen for a job and then processor processes the job using one of the techniques.
Processing of more than one job simultaneously by multiple processors is called multiprocessing.
Multi-level Feedback Queues (MFQ):
The policies described above are relatively limited in their ability to cope with wide variability in the behaviour of the processes in a system.Ideally, a scheduling scheme should give priority to short jobs and favour I/O-bound jobs, while otherwise being 'fair' to all processes.The MFQ scheme is an attempt to provide a more adaptive policy which will treat processes on the basis of their past behaviour.
This figure shows a typical set up for a MFQ system.It consists of a number of separate queues of entries which represent active processes.Each queue represents a different priority, with the top queue being highest priority and lower queues successively lower priorities.Within each queue, the queued processes are trated in a FIFO fashion, with a time quantum being applied to limit the amount of processor time given to each process.Processes in a lower level queue are not allocated the processor unless all the queues above that queue are empty.A new process enters the system at the end of the top queue and will eventually work its way to the front and be dispatched.If it uses up its time quantum, it moves to the end of the queue in the next lower level, with the exception of the lowest queue where a round robin scheme applies (i.e. it simply moves to the end of that queue).If it relinquishes the processor due to a wait condition, the process leaves the queueing system.As a process uses more and more CPU time, it will migrate down the levels, thus obtaining a reducing level of access.
This arrangement militates to some extent against long processes and starvation is a possibility.Various modifications exist to the basic scheme which attempt to meet some of its problems.One option is to use increasing size of time quantum on lower levels, so that when a lower queue process does get a chance it holds the processor for a longer period.Also, some systems promote processes to a higher level queue when they have spent a certain amount of time in a queue without being serviced.
File System
In computing, a file system (often also written as filesystem) is a method for storing and organizing computer files and the data they contain to make it easy to find and access them. File systems may use a data storage device such as a hard disk or CD-ROM and involve maintaining the physical location of the files, they might provide access to data on a file server by acting as clients for a network protocol (e.g., NFS, SMB, or 9P clients), or they may be virtual and exist only as an access method for virtual data (e.g., procfs). It is distinguished from a directory service and registry.
More formally, a file system is a special-purpose database for the storage, organization, manipulation, and retrieval of data.
File names:
Whether the file system has an underlying storage device or not, file systems typically have directories which associate file names with files, usually by connecting the file name to an index in a file allocation table of some sort, such as the FAT in a DOS file system, or an inode in a Unix-like file system. Directory structures may be flat, or allow hierarchies where directories may contain subdirectories. In some file systems, file names are structured, with special syntax for filename extensions and version numbers. In others, file names are simple strings, and per-file metadata is stored elsewhere.
File systems under Unix-like operating systems:
Unix-like operating systems create a virtual file system, which makes all the files on all the devices appear to exist in a single hierarchy. This means, in those systems, there is one root directory, and every file existing on the system is located under it somewhere. Furthermore, the root directory does not have to be in any physical place. It might not be on your first hard drive - it might not even be on your computer. Unix-like systems can use a network shared resource as its root directory.
Unix-like systems assign a device name to each device, but this is not how the files on that device are accessed. Instead, to gain access to files on another device, you must first inform the operating system where in the directory tree you would like those files to appear. This process is called mounting a file system. For example, to access the files on a CD-ROM, one must tell the operating system "Take the file system from this CD-ROM and make it appear under such-and-such directory". The directory given to the operating system is called the mount point - it might, for example, be /media. The /media directory exists on many Unix systems (as specified in the Filesystem Hierarchy Standard) and is intended specifically for use as a mount point for removable media such as CDs, DVDs and like floppy disks. It may be empty, or it may contain subdirectories for mounting individual devices. Generally, only the administrator (i.e. root user) may authorize the mounting of file systems.
Unix-like operating systems often include software and tools that assist in the mounting process and provide it new functionality. Some of these strategies have been coined "auto-mounting" as a reflection of their purpose.
1. In many situations, file systems other than the root need to be available as soon as the operating system has booted. All Unix-like systems therefore provide a facility for mounting file systems at boot time. System administrators define these file systems in the configuration file fstab or vfstab in Solaris Operating Environment, which also indicates options and mount points.
2. In some situations, there is no need to mount certain file systems at boot time, although their use may be desired thereafter. There are some utilities for Unix-like systems that allow the mounting of predefined file systems upon demand.
3. Removable media have become very common with microcomputer platforms. They allow programs and data to be transferred between machines without a physical connection. Common examples include USB flash drives, CD-ROMs, and DVDs. Utilities have therefore been developed to detect the presence and availability of a medium and then mount that medium without any user intervention.
4. Progressive Unix-like systems have also introduced a concept called supermounting; see, for example, the Linux supermount-ng project. For example, a floppy disk that has been supermounted can be physically removed from the system. Under normal circumstances, the disk should have been synchronized and then unmounted before its removal. Provided synchronization has occurred, a different disk can be inserted into the drive. The system automatically notices that the disk has changed and updates the mount point contents to reflect the new medium. Similar functionality is found on standard Windows machines.
5. A similar innovation preferred by some users is the use of autofs, a system that, like supermounting, eliminates the need for manual mounting commands. The difference from supermount, other than compatibility in an apparent greater range of applications such as access to file systems on network servers, is that devices are mounted transparently when requests to their file systems are made, as would be appropriate for file systems on network servers, rather than relying on events such as the insertion of media, as would be appropriate for removable media.
Deadlocks
A deadlock is said to arise when a set of processes enter into indefinite wait conditions.
Consider two processes P1 and P2, let us say both the processes are running on different processors.P1 can not complete its execution unless it reads 'STUDENT' file and P2 can not complete unless it reads 'RESULT'.Now files 'STUDENT' and 'RESULT' can not be used by other processes unless the process which opened them closes them.Thus P1 is waiting for P2 to get over and P2 is waiting for P1 get over.This is a deadlock situation.
In the case of deadlock, the processor chooses one of the jobs and cancels it.Then the cancelled job is started all over again.The cancelling of a job is called Rollback.So in a deadlock, one job is rolled back in order to break the deadlock.
File Management
File management function is used to faciliate easy creation, storage and access of files in order to enable easy sharing of files between programs and their protection against illegal access.
The major issues of file management are:
1. It provides file naming freedom to the users and permits controlled sharing of files (protection).
2. It provides long and short term storage of files.
3. It provides security against loss of information due to system failure (security).
File system also facilitate grouping of files in form of directories or folders, for the purpose of their access and protection.
The file system also involves the file recovery module if required (in case of system failure or any other error)
File Operations-
A file is an abstract data type.To define a file properly, we need to consider the operations that can be performed on files.The operating system provides system call to create, write, read, reposition, delete and truncate files.
1 Creating a file: Two steps are necessary to create a file.First, space in the file system must be found for the file.Second an entry records the name of the file and the location in the file.
2 Writing a file: To write a file, we make a system call specifying both the name of the file and the information to be written to the file.
3 Reading a file: To read from a file, we use a system call that specifies the name of the file and where the next block of the file should be put.Again, the directory is searched for the associated directory entry and the system needs to keep a read pointer to the location in the file where the next read is to take place.Once read has taken place, the read pointer is updated.
4 Reposition within a file: The directory is searched for the appropriate entry and the current file position is set to a given value.Repositioning within a file does not need to involve any actual I/O.This file operation is also known as a file seek.
5 Deleting a file: To delete a file, we search the directory for the named file.Having found the associated directory entry, we release all file space.
6 Truncating a file: There are occasions when the user wants the attributes of a file to remain the same , but wants to erase the contents of the file.Rather than forcing the user to delete the file and then recreate it, this function allows all attributes to remain unchanged but reset to zero.
These six basic file operations certainly comprise the minimal set of required file operations. Other common operations include appending, renaming, copy etc.
Introduction to UNIX
Introduction:-
UNIX is a multiuser, multi tasking operating system developed at Bell laboratories.It is emerging as a worldwide standard in operating systems.
you can consider UNIX as an operating system kernal providing many functions-
1 A File System
2 Memory Management
3 Device Management
4 Process Management
From the user point of view, UNIX can also be thought of as
1 A User Interface(Shell)
2 A set of Utility Programs
3 An Emerging Industry Standard
4 A complete Programming Environment
Structure of UNIX:-
The major components are as follows
1. UNIX KERNAL
2. SHELL
3. UNIX UTILITIES
4. USER PROGRAM/APPLICATIONS
1. UNIX Kernal:
Kernal is the core or heart of the UNIX system.Kernal interacts directly with the hardware and thus creates the UNIX environment on a Computer System.There are numerous services & functions provided by Kernal:
1 System Accounting
2 Input/Output Services
3 Memory Management
4 File Management
5 Date and Time services
6 Processors Management
UNIX System Call Interface
System call are the interface between UNIX kernal and the utilities/applications programs.
Firstly, the UNIX utilities and application programs invoke the appropriate system call depending upon the service required from the kernal.
Secondaly, kernal provide the requested service and returns the control back to the requesting utility/Applications programs.
2. UNIX Shell:
Shell is the component of UNIX System which directly interacts with the user.So, it can be thought of as the interface between user and the UNIX system.
Features of Shell:
I. A command Interpreter:
Shell takes the commands from the user, interprets them and causes them to be carried out one by one.Thus shell is basically a command interpreter.
II. A complete Programming Language:
Shell provides a complete programming language.It supports all the basic language constructs.Hence it can be used for
1 Writing new command
2 Using existing set of commands
3 Language constructs provided by the shell
In UNIX system hierarchy shell and application program occupy same level.
3. UNIX Utilities:
Unix Utilities are a collection of more than 200 programs that come with almost every UNIX System.
These Utilities are nothing but UNIX command, which are functionally similar to the DOS Commands provided by DOS.These utilities are executed through the shell.UNIX provides utilities for almost every important application area.They can be clssified into different groups, depending upon their functions.Some important utility groups are as follows:
1 General purpose utilities for daily use
2 Directory and file Management utilities
3 Text processing utilities
4 Office Support Utilities
5 Help Utilities for on-line
6 Graphics utilities
7 Games
8 Communication utilities
9 Networking utilities
10 System administration utilities
11 System Development Utilities
4. Application/User Programs:
Many UNIX-based Applications can be purchased off-the shelf.These applications are either developed by the user having a knowledge of programming or by a software company.
These can be run like UNIX utilities.These are in addition to the utilities supplied with the UNIX.
Some of the applications software available for the UNIX based systems are as follows:
I. Graphics
II. Computer Aided Design (CAD)
III. Communications
IV. Database Management
V. Word Processing
VI. Accounting
VII. Language Compiler/ Assemblers
UNIX File Management System
What is a File?
The fundamental structure that UNIX uses to store information is the file.Each and every type of information is stored in some sort of file, irrespective of its type.
A UNIX file can be compared to a file folder in desk drawer.In an organisation or business environment you may have many file folders for storing different kind of information.Similarly in UNIX you may have several kind of files for storing different information.
A file can store letter, documents, accounts, accounts information, payroll data or high level language programs.In short, UNIX files are the structures used to store virtually every kind of information required for the preparation of a typical UNIX system.
Valid Filenames:
Every UNIX file has a name associated with it just like a file folder and it is generally referred to as filename.
1 A filename is written as one word
2 Any character otherthan / is legal
3 Afilename can be at the most, 14 characters long
4 Uppercase and lowercase letters in a filename are distinct
Although you can use almost any character in a filename including metacharacters, but you should use only visible characters and should avoid characters that might be having special meanings.
For example a- (hyphen) sign is used for specifying a command option.So if you have a file whose name starts with - minus you may face problem while using the filename in some commands.Besides, the minus sign as a first character , there are other characters with special meaning.To avoid any difficult situation, you should use only
1 Alphabet (a-z, A-Z)
2 Numbers (0-9)
3 Period (.)
4 Underscore ( _ )
The period and underscore are conventially used to divide filenames into parts as in payslip.jan.
You should also remember the case distinctions.
For example
computer,
Computer,
COMPUTER are three different names.
Some legal filenames
counter
Counter
COUNTER
first.c
jaipur.rp.lane1
cd-ny..15
The concept of a directory:
UNIX keeps track of files internally by giving each file a unique identification number.Thus a customer file may be stored in file number 12, a file containing payroll data may be stored in file number 30 and so on.On a typical UNIX system, it is possible to have thousand of files existing at a time.
Identifying files by numbers can be very difficult and time consuming.UNIX identifies the files by numbers, called i-node numbers, but they are used only within the UNIX System.File system allows the user to assign a filename to each file but this method will become very tedious for a multi user which can have 1000 of files.
For example, you can not be certain when assigning a name to a new file, that tha name selected has not been used previously by some other user.
UNIX provides a simple way of organizing files by grouping them.
A group of related files is called a directory.A directory performs the same function as a file drawer in a filing cabinet, arranging together related files in a separate drawer where they can be found easily.
With directories, the user has complete flexibility in grouping files in a meaningful way.
For example,: A college may maintain files of information on students, lecturers, suppliers of various items, result, laboratory orders, fee collection etc. for each of its department with each directory containing only files related to a particular department.
How to name a directory?
1 UNIX directory names may contain upto 14 characters.
2 Directory names are chosen such that they indicate the kinds of files in the directory.
Internally, a directory is also a file.It contain a list of filename and their corresponding i-node numbers. The directory performs exactly the same function as a common telephone directory for storing telephone numbers.
For example, the files for the computer department might be called results, orders, students and lecturers. The directory containing these files might then be called computer.deptt.
System Administration:
System administration functions are operations need to be performed periodically so as to ensure the smooth administration of the network environment.
Operations of a multi-user system are quite complex.These operations are referred as the Important System Administration issues.
1 Adding and deleting users and groups
2 Managing the file system
3 Maintaining Security
4 Bringing up the UNIX system
5 Shutting down the UNIX system
6 Bracking up File System
7 System Accounting
8 Print Spooling
Some of these facilities are available to all the users on the UNIX system.But certain administration functions on a UNIX system require access to files and commands that are normally denied to all users.Adding a new name to the list of authorized user names is an example of one of these functions.
System Administrator:
Administrative functions are usually performed by a system administrator, a "special" user charged with maintenance and general operation of the computer system.
UNIX provides an exception of its file security scheme to this single user of the system, called the super-user or system administrator.
Duties of a system administrator
1 By convention, the super-user is given the user name "root".As the name implies, the super-user owns the root directory.
2 The super-user also owns the directories and files that contain the UNIX operating system kernal, the utilities, the devices etc.
3 With this ownership, the super user can authorize new users on the system and perform similar administrative functions.
4 All UNIX permission and protection mechanisms are bypassed for the super-user.The user name root is, therefore, always password protected on a production UNIX system and the password is supplied on a "need to know" basis.Protection of the super-user password is the most critical link in preserving a UNIX system's security and integrity.
All the system administrstion functions can be classified into the following 2 categories:
1) Building a new system
2) Maintaining the UNIX system
Knowledge of file system is must for maintaining a UNIX System.Knowledge of start up and shut down operations is also very important.
UNIX File System:
File system is one of the most powerful and attrative feature of UNIX.File system manages all the data stored on the computer's main storage devices.
Features of the File System
1 The file system imposes hierarachical structure for organizing the stored information in a very natural way.
2 It is File System which maintains all types of files whether they are ordinary, directory or the special files.
A file system can correspond to a physical device like magnetic disk or floppy disk.If a disk is large, it can be thought of as consisting several logical disks.Each logical disk occupy a portion of physical disk.We have a file system corresponding to each logical disk.
But every UNIX File System has the same internal structure, whether it corresponds to a physical disk or logical disk.The operating system views a file system as an arrangement of addressable blocks of disk space.Traditional UNIX file system consists of four parts, in sequential blocks of disk partition.
The parts are:
1 Boot block or block 0
2 Block 1 or super block
3 A variable number of blocks comprising the i-node list
4 A variable number of blocks for data storage
BLOCK 0: THE BOOT BLOCK:
Block 0, although considered to be part of the file system, is actually not used by it.It is reserved for storing Bootstra program.
BLOCK 1: THE SUPER BLOCK:
Much of the information about the file system is stored in the super block, including such things as:
1 File system size in blocks
2 File system name
3 Number of blocks reserved for i-nodes
4 Number of free i-nodes
5 Total number of free (data) blocks
6 Start of the chain of free blocks
7 Time of the alst up-date
8 Time of the last back-up
I-Node or Information Node:
The term i-nodes stand for information node.
There is one i-node for every directory and file in the file system.If an i-node is marked allocated then it mustcontain description of a file or directory somewhere in the file system.
The i-node contains all the information about a file except for its name, which is kept in a directory.An i-node is 64 bytes long, so there are 8 i-nodes in a physical block (512 bytes).The number of i-nodes is specified at the time the file system is created.
DATA / STORAGE BLOCKS:
The remainder of the space allocated to the file system is taken up by storage blocks, also called data blocks.
Data blocks contain
1 The actual data stored in the files and directories
2 Some of the data blocks are used as indirect blocks
For a regular file, the storage blocks contain the contents of the file.For a directory, the storage blocks contain 16 bytes entries, each entry represents a file or subdirectory.It consists of 2 bytes for i-node and 14 bytes for the filename or subdirectory name.
FREE BLOCKS:
Blocks not currently being used as i-nodes, as indirect address blocks, or as a storage blocks are chained together in a linked list.Each block in the list carries the next block in the chain.These blocks are called as free blocks.
UNIX GENERAL COMMANDS
Introduction:
These commands are similar to the command you have done in DOS.They perform all function like listing a directory, copy a file, deleting a file and so on.
Parameters of UNIX Commands
-x -Multicolumn display
-f -exe file*
-r -Sort all the files in reverse order
-l -For long listing
-d -Listing of directory
-t -Sorts files by time
-u -Sort by access time
1. Date: This command returns the systems date and time.
$date
mon nov 1 12:40:30 edt
To print only month no.:
$ date + % m
1
To print only month name:
$ date + % h
nov
2. who: This command gives the information about the users which are currently logged in
$who
3. who am i: This command returns the information about the user currently logged on the terminal from which the command is given.
$who am i
4. man: Complete details of any command can be obtained using man command
eg:
$man date
This command will give the details of the date command.
5. Banner (Blownup Message): This command takes string (sequence of characters) as an argument and print the characters as banners.
for eg:
$banner xi
# # # # # # # # # # #
# # #
# # #
# # #
# # #
# # #
# # #
# # #
# # # # # # # # # # #
6. mail: The UNIX system provides a postal system for communicating with other users, so some day when you login, you can see the following message:
you have mail
before the first prompt
To read the mail, just type:
Your mail will be printed one message at a time, most recent first.After displaying each message, mail waits for you to say what to do with it, the basic responses are
d
For deletion of mail message return or enter-which does not delete the message (so it will be there the next time you read your mail)
q
quit from the mail command
s
to save the mail message with a filename
sending mail
You can send mail to any user whether he is logged on or not suppose you want to send mail to a person with login name "shally"
to do this type as follows:
$mail shally
Dear mam shally
I want to have an urgent meeting with you.
your sincerely
Nipurnika
$
As you type the mail command followed by the user name and press return, it will wait for you to type the message.Immediately after this you can start typing your message.When your message is complete type ctrl+d which will signal the end of message by telling the mail command that there is no more input.
7. write: Using this command you can send message to any user on the system, but that user should be currently logged on.
eg:
$write shally
If the user with login shally is currently logged on, following message is displayed on her terminal
message from sonu tty0
If you are working on terminal tty0 with login name sonu
If user shally wants to communicate with you she should respond by typing:
$write sonu
After this a two way communication path is established.Now whatever you type will be displayed on user shally's terminal.
You can exit from write command by typing ctrl+d or pressing del key.
Insulation from other users(mesg):
$ mesg y (yes)- This will allow communication.
$ mesg n (no)- This will not allow communication.
Addressing all users who are currently logged(wall):
This command allows user to send a message to all othher users who are currently logged in.
$ write wall
8. echo: This command takes strings (sequence of characters) as argument.Execution of this command results in display of strings on the terminal.
For eg:
1. $ echo my account id on rediff
my account id on rediff
$
2. $echo*- * is used for files.It will list all the file name in the current directory.
3. $ echo '\t xyz\n abc'
4. $ echo'\007'- For the beep sound
5. $ x=37
$ echo x- It will display only x.
$ echo $x- To display the value of x=37
6. $ msg=you have a mail
$ echo $msg
Output-you
7. $ msg=you\have\a\mail
$ echo $msg
O/P-you have a mail
8. $ msg='you have a mail'
$ echo $msg
O/P-you have a mail
9. $ echo ${variable name}
Example-i)fname=dept.lst
$ echo ${fname}
$ echo ${fname}x
dept.lst x
ii)$ c=add.c
$ echo ${c}pp
add.cpp
Head
For displaying the beginning of a file the head command is used.
$ head dept.lst-It will display first 10 lines of the file.
$ head -3 dept.lst- It will display first 3 lines of the file.
$ head -25 dept.lst- It will display first 25 lines of the file.
Tail
To display tail of the file.
$ tail dept.lst-It will display last 10 lines of the file.
$ tail dept.lst- It will display last 3 lines of the file.
Concept of a Path:
A path is a way to reach the detination.
Absolute Path Name:
Path name that traces a complete path down from root directory to the directory containing the file is called the absolute path name.
For eg:
/ student / icg / bsc_ii / shally
Relative Path Name:
Identifying each file by its full path name can be very cumbersome, especially when directory structure contains a number of level.A pahtname that starts from the current working directoryis called as relative pathname.UNIX solves the problem of specifying full pahtname by allowing the user to use shorthand notation, known as partially qualified path name or relative path name to identify a file whose path name includes the current working directory.This path name simply omits the initial slash and the directory names upto and including the current working directory.
Let the current working directory be shally, to reach another subdirectory muskan in icg directory, the relative path would be::
eg: ../ ../muskan
Printing Working Directory:
The pwd (Print Working Directory) command displays the fully qualified path name of the directory in which you are working currently.For example, let's assume thatthe current working directory is / student/ icg/ boom_ii
$pwd
/ student/ icg/ boom_ii
Change a Directory:
The cd (change directory) command moves the user into a different working directory.For example, if files for Banglore office are to be used frequently then make the Banglore as the current working directory by giving command
$ cd / sales / south / banglore
Listing file names in a directory:
The ls (list command) lists the file in a directory.If you do not speccify any directory with ls command, it will display the names of the files in the current working directory, for eg:
$ls
Long Listing
$ ls -x
Making new directory:
The mkdir (make directory) utility creates a new directory.For eg, the command:
$mkdir /sales/north/kota
can be used to make a directory for a new sales office in kota.
Removing Directory:
The rmdir (remove directory) command removes directories that are no longer needed.For example, to remove the directory kota created above, give the following command:
$rm dir /sales/north/kota
Note: rmdir command removes a directory only if it is empty.
Creating a File:
To create a text file the following command is used-
$ cat > filename
Copying file:
The cp (copy) command makes a duplicate copy of a file.For example, the command:
$cp orders oldorders
Copies the contents of the orders file to a new file named oldorders
Renaming/moving Files:
The mv(move) command changes the name of file.Both the old and new names should be given with the command.
For example, to change the name of a file from account to account.feb,give the command:
$mv account account.feb
Linking filenames:
A file may be known by several different names.The ln (link) command is used to give an additional name to a file.Both the current name and the new name should be specified with the command.
$ln sale sale.jan
Removing a file:
The rm (remove) command removes unwanted file names from the directory.
for eg: To remove a file named "prediction" from the directory, give the command:
$rm prediction
$ rm*
To delete all files from current directory.
Interactive option- (i)
$ rm -i result prediction
result-?Y/N
prediction-?Y/N
$ rm -i*
Displaying a file:
The cat (concatenate files) command is often used to see the contents of a file the screen.Cat is a general purpose utility having many uses.
$cat result
This command will display the contents of the file 'result'.
To display contents of more than one file-
$ cat result prediction
Counting lines, words and characters in a file:
The wc (word count) command counts lines, words and characters in a file.
For eg:
$wc orders
20 120 900 orders
switches:
-l to display only number of lines
-c to display only number of characters
-w to display only number of words
tee
The tee command is used to send output of suitable UNIX command into a file as well as on screen.
$ who tee
Paginating File:
The pr command prepares a file for printing by adding header, footer or page set up.
$ pr dept.lst
top margin-5 lines
bottom margin-5 lines
left margin-0
right margin-0
In top margin 1 header line is reserved.
Header contents may be- date, time, file name, page no.
$ pr-l 72 dept.lst
$ pr-w 132 dept.lst-132 characters in a line.
$ pr-h dept.lst
$ pr-h "IMS" dept.lst= h stands for header.
$ pr-n-h dept.lst=n stands for line number.
$ pr-o20 dept.lst=o stands for offsite (address) (Left Margin).
$ pr -m chap1 chap2= mstants for merge.
$ pr + 10 dept.lst= starts printing from page no. 10.
$ pr - d chap1= double space printing.
$ pr -l 72 -w 80 -n -d -m chap1 chap2
$ pr -t chap1= eleminates header, footer, margins totally.
Cut:
Slitting if i vertically the cut command is used.The cut command can be used to extract specific columns from the file.Suppose NAME (the second field) and DESIGNATION (third field).NAME starts from column no. 6 goes upto column 22.The DESIGNATION from 24 to 32.Use c (column option).
$ cut -c 6-22,24-32 emp.lst
$ cut -c 5,6-22,24-emp.lst
-d= stands for delimiter.
Paste:
The paste command pastes more than one file vertically.
Oredering a File:
The sort command is used to ordering a file.
$ sort emp.lst
It sorts the entire lines.
$ sort -r+1 emp.lst
Reverse order, 2 column primary.
Splitting a File:
The split command breakes up its input into several equi-line segments.
$ split chap1
It has no standard output but a group of files like xaa xab xac......by default it creates 1000 lines segment for each file.
$ split -72 chap1
To normalize a splitted file
$ cat x ?? > new1
or
$ cat xaa xab xac > new1
To remove these files
$ rm xaa xab xac
or
$ rm x??
Line Numbering(nl)
$ nl emp.lst
News
It is normally invoked by end user to read message that is sent by system administrator.
$ cat/ user/ news/ dinner
The director invites you on to a dinner at Laxmi Palace Hotel.
^d
$ news
Telly Type (tty)
To know your terminal the tty command is used.TTy number begins from 01.
$ tty
/dev/tty25
General purpose utilities
Halted the output the more command is used.
$ more ch01
It displays the contents of the file ch01 on the screen one page at a time.
File Type:
The file command displays the type of file of UNIX.There are three types of file- ordinary, directory, device file.
$ file emp.list (file name)
emp.lst : ascii.txt
$ file*
To show the types of all existing files.
Other file types:
:english.txt
:directory
:empty
:data
:executable
:c program file
:device file
Od - Octal Dump:
It displays information byte wise.Displaying the contents of a file in octal form by using od command.
Syntax-
$ od filename
Example-
$ od emp.lst
152157141015
Comparing two files with cmp command:
The cmp command compares the contents of two files byte by byte and mismatches are displayed on the screen.
Example-
$ cmp xyz abc
xyz abc
(a) difference char 4 line 1
The -l (line) gives detailed list of thebyte number and differing bytes in octal form.
xyz abc
(a) byte no.-04, difference-255, bytes-64
Line Printing (lp):
To print a document (file) on line printer the lp command is used.The lp command also displays the request ID on the user terminal.
Syntax-
$ lp filename
Example-
$ lp emp.lst
$ lp-t 'Emp Database' emp.lst
The -t (title) command is used to print a title on the first page.
$ lp -t 'I /n (next line) database' emp.lst
Changing File Access Permissions:
The chmod (change mode) command is used to change access permissions associated with a file.Changing access permissions can be done in two ways:
1 By using symbols descriptions i.e. absolutely specify whether a permission is to be granted or to be denied.
2 Or by using octal values.
Using Symbolic Description:
Following symbols are used with chmod command:
u - to specify user (owner)
g - to specify group
o - to specify other
r - to specify read access
w - to specify write access
x - to specify execute access
+ - to specify grant a permission
- - to specify denial a permission
= - to absolutely specify all the permissions
Some Commands and Their Effects:
1. chmod u+w sale
User (owner) is allowed write permissions for file sale.
2. chmod gc-w sale
Group users and other users are denied write permission for all sale.
3. chmod u=rxw go=rx dir
User (owner) is allowed all read, write and execute permission for file dir, while group members and others are allowed only read and execute permissions.
4. $ chmod a+mcm1
To provide all the rights to all.
5. $ chmod a-mcm1
To remove the rights.
chmod command is used to set three permissions (read, write, execution) for all the three categories of user (user, group, owner) to a file.
Syntax-
$ chmod
Category-1. U -Stand for user
2. G -Group
3. O -Owner
Operation-(+) -Assign
(-) -Remove
(=) -Assign absolute permission
Attribute-R -Read
W -Write
X -Execute
$ chmod u+w mcm1
P- To add two values:
$ a=5
$ b=6
$ c='expr $a+$b'
$ echo $c
To see the output of c.
Using Octal Values:
In the octal mode, each access permiission read, write and execute is aassigned an octal values as follows:
1 Read is assigned the value 4
2 Write is assigned the value 2
3 Execute is assigned the value 1
The total of these values varying 0 to 7 represents all possible combination erad, write and execute permissions for a given category of user.Some combinations of the permissions in symbolic form and their equivalent octal numbers are specified below:
Permission Equivalent Octal Numbers
rwxrwxrwx 777
rw-rw-rw- 666
r-r-r- 444
rwxrw-rw- 755
rw-r-r- 644
rwx--- 700
-x-x-x 111
So, to change access permissions associated with a file simply specify the octal equivalent of new set of permissions with the chmod command.
Some of the uses of the chmod command with octal number for changing permissions are given below.
Some Commands and Their Effects:
1. chmod 777 orders
All the users on the system are allowed all type of access.
2. chmod 666 orders
All the users on the system are allowed read and write access but denied execute access.
3. chmod 600 letter
User (owner) is allowed read and write access while all group users and other users are denied any access.
REDIRECTION, PIPES AND FILTERS
Introduction to Redirection:
Input and output operations are done using keyboard and monitor.Generally, a UNIX command or a UNIX program takes input from standard input file i.e. keyboard and displays its output on the standard file i.e. screen or visual display unit (V.D.U.).
But UNIX allows a user to change the standard input and standard output files associated with a program without making any changes in the program.Thus you can make any file other than keyboard as the standard input file and similarly any file other than monitor (screen) as the standard output file.
Thus input/output redirection is a facility through which one can change standard input/output files.It is very powerful shell efature.
Input Redirection(<):
UNIX can direct the shell to take standard input for We can change the primary string as well as the secondary string.
Example-
$ ps1 = "c:>"
$ ps1= "MCM 1"
Create executable file:
To make executable file the following command is used-
x=executable
$ chmod u+x find.sh
$ find.sh
Output goes or stored in a.out file.The result of executable file always stores in a file a.out instead of appearing on screen.
$ cat a.out = To see the result.
Filters in detail:
Sort:
Often the contents of a file must be sorted into some order before useful processing can proceed.The sort command performs this function.It takes lines from one or more file containing the lines in sorted order.With sort, each line from an input file is treated as a series of one another or more fields, separated from one another by spaces.
The sort command accepts a number of options to control the order of sorting.
Without any option it sorts the lines in the input according to ASCII order. Some commonly used options with the sort command are given below:
Some commands with their actions:
sort-r
sorts in descending order.
sort-n
sorts in numeric order.
sort-nr
sorts in reverse numeric order.
sort-f
sorts without differentiating uppercase and lowercase letters.
sort+n
sorts starting from n+1 st field
Use of sort command are given below:
Examples:
$ ls sort -f
Sorts filenames in alphabetic order.
$ls -l sort +3n
Sorts the users by time of login, oldest first.
Pg
This is a very sophisticated file viewing utility.This command pg supports page by page viewing and file searching capability.The pg command can be used to view one or more files.
The pg command displays the content of text files, whose names are given as argument.For eg:
$ pg temp1 temp2 temp3
Will display the contents of files temp1, temp2 and temp3.
Infact, you can display the output of any command through pg if it is very long.So that you are able to see the output one page at a time.
For example, if a directory listing is very long then you can view it page by page by giving this command.
$ ls -l pg
cat
The cat is the simplest of all the filters.It does not do any transformation on its data.It simply copies its standard input to the standard output.Cat can perform some useful operations on files, like:
1 Reading files
2 Creating new files
3 Cpying existing files
You can read and display the contents of a file named temp with any of the following command:
$ cat temp
$ cat < temp
You can create a new file by using cat command as follows:
$ cat > newfile
CTRL+D
$
You can copy the contents of a file to some file.For eg. to copy the contents of file temp1 into file temp2, give the command:
$cat
wc
The command wc counts the lines, words and characters in its input and displays these counts on the standard output, if output of the command is not redirected to a file.
Examples:
$ who wc-l
Counts the users who are currently logged in.
$ ls wc-l
Counts numbers of files in the current directory.
The grep filter and file searching:
The grep (global regular expression print) utility is the standard file searching and selection utility on UNIX.
The user specifies a pattern to guide grep's search through a file.This examines the input file, line by line, checking each one to see if it contains the pattern.
When a match is found, the line is copied to the standard output file.If a line does not contain the pattern, the line is not copied to the output file.When grep completes its search the output file contains the input file's lines that contain the pattern.For example:
$ grep jaipur orders
This command will display all the lines in file orders which contain the pattern "jaipur" in them.
1. grep's first argument is a pattern.
2. Arguments after first argument specify the files to be searched for the pattern.If no file is specified with the grep then it will take input i.e. keyboard.
Switches with their effects:
-N
Displays line numbers alongwith the lines selected by grep command.
-v
It reverses the normal functioning of grep command.It displays the lines which do not match with the pattern instead of displaying matching lines.
-e
Gives only the number of lines matching with the pattern.
-i
Ignores the letter case.
Pattern specified with the grep can be simply a string or it an also be a regular expression.
Some examples illustrating use of regular expression for specifying the search pattern are given below:
Some examples with their output:
$grep sha.ly temp
Displays all the lines from the file temp that contain shally, shaily, shaely etc.
$grep t.e temp
Displays all the lines from the file temp that contain a "t", followed by any single character, followed by an "e".For example the, tee, the, tae etc.
$grep sto*p temp
Displays all the lines from the file temp that contain stop, stoop, stopper etc.
grep abc.*temp
Displays all the lines from the file temp that contain any string beginning with abc.
grep t[aou]rn temp
Displays all the lines from the file temp that contain string tarn, torn, turn etc.
grep appendix-[A-Z] temp
Displays all the lines from the file temp having appendix-A, appendix-B, appendix-Z etc.
grep ^the temp
Displays all the lines from the file temp containing the in the beginning.
grep est$ temp
Dis plays all the lines from the file temp containing est in the end.
Stream editor : sed
sed is non interactive editor that can be used to edit large files.
The sed is an automative version of the UNIX editor ed.sed can edit large files because it does not read the entire file into a buffer.
The sed reads lines one by one, from an input file and applies aset of editing commands to the lines and send the edited lines to the standard output.
Features of sed editor
1 sed is a filter because it does not change the original file but reads the input file and creates a new file consisting of edited lines which is sent to standard output by default.
2 The editing commands used with sed include a pattern matching capability like the one used with grep.
3 The sed editing commands often more flexibility in adding, deleting, modifying and searching lines of the input file as it is processed.
4 The sed utility can copy files from the input file into a buffer and copy them to new positions in the output file.
5 This utility can also merge lines from other files into the output file.
6 The sed editing commands can be specified on the command line or stored in a file.
The syntax of sed command is:
$ sed 'list of editing commands' filenames
awk
awk is a pattern scanning and processing language.
Features of awk
1 It is most popular and the most powerful filter available in the UNIX environment.
2 It is particularly helpful because it supports field oriented processing.
3 It also has capabilities for matching on regular expressions.
4 It has many built-in functions and supports many conventional arithmetic expression.
5 Its statements and expressions resemble withthat of c-language.
6 It supports variables and arrays, has many built in variable and a wide range of operators.
7 awk does not need any declaration of variables.
INTRODUCTION TO SHELL PROGRAMME
What is an editor?
It is a program using which you can process textual information. With the help of an editor you can perform the following tasks.
· You can create a new file for storing information.
· You can see the contents of an existing file on the screen.
· You can modify the information stored in an existing file.
Editors available on UNIX System:
A number of text editors are available on different UNIX system for writing text and saving it for future use. Most popular of them are:
1) ed-A line editor
A line editor is one which edit line by line. Working with line editor is very tedious as while working with a line editor, we can type or edit only one line at a time.
2) vi – A screen editor
A page editor is one which edits page by page. While in case of screen editor a page of text is displayed on the screen and can be manipulated as a whole. You have the facility to move anywhere on the screen and insert or edit the text.
3) ex – A line editor associated with vi
vi Screen Editor
vi (visual) works in two distinct modes of operation :
1. COMMAND MODE
2. INPUT MODE
COMMAND MODE:
This is the mode you are in when you first enter vi.
In command mode , most of the characters including lower and upper case characters , control characters , punctuation characters and digits have special meanings and are associated with vi commands.
Features of Command Mode
· Almost all vi commands are entered from the command mode.
· Most of the vi commands are not themselves displayed on the screen. Only their result are displayed.
· Most of the vi commands do not need pressing return for termination. So unless specified , do not press return after a vi command.
· You can switch to command mode from input mode , just pressing Esc key. If you are unsure which mode you are in , press Esc repeatedly until the terminal beeps. You will then definitely be in command mode.
Note : You cannot enter text from the command mode.
INPUT MODE:
Input mode is used to type text into buffer. To enter text you will have to switch to input mode by giving appropriate vi command – appending text, inserting text or changing / replacing text commands.
Features of Input Mode
· You can enter text into a file only from input mode. Because in command mode , most of the characters including lower and upper case characters , control characters , punctuation characters and digits have special meanings and are associated with vi commands.
· To enter input mode , type one of the appending text , inserting text or changing / replacing text commands. Then you can start entering the text as character will now have their usual meaning.
· While input mode , anything you type will not be taking as command, but will be inserted directly into the buffer.
· You can return to command mode by pressing ESC. Key.
Entering vi
To invoke the vi editor , give the command vi with the filename.
e.g.
$ vi abc
Exiting vi – A variety of ways to exit vi !
Commands and their actions
zz
Writes Buffer contents to the file specified at the time of invoking vi , then exits vi and returns to shell prompt $.
:wq
RETURN (Write and Quit) same as zz, except that it writes buffer contents even if there is no change after the last saving operation.
:q
RETURN (Quit) same as zz except that it does not writes buffer contents.
:q!
RETURN same as: q except that it always quits without checking for changes since last saving operation. So you should be careful; while using this command as it throws away all changes to the buffer since your last Write / Save operation.
:w
RETURN (Write) Writes Buffer to the specified file you can quit the editor by giving : q command immediately after this operation.
proceeds to execute the next command in the shell file . For eg:
$ cat hello
echo “what is you name?”
read name
echo “welcome , Mr. $name”
$
The Expr Statement:
Using expr you can do arithmetic operations on the shell variables and numbers.
Syntax:
expr expression
expr evaluates the expression given as argument. Arguments of expr must be separated by blank space from another and special meaning of “*” asterisk must be suppressed by the back slash “\” character.
Using “expr” you can do the following arithmetic operation :
· addition ( +)
· subtraction ( - )
· division ( / )
· multiplication ( \* )
· mod (remainder) ( % )
Some of the usage of expr statement are as follows :
$ expr 50 + 40
90
$ expr 3 \* 9
27
$ expr 15 % 4
$
$ x=8
$ x=’expr $x + 2’
$ echo $x
10
$
Conditional Test:
Some times you may require that certain commands should be executed
depending upon the results of other commands.
For example , you may want to print an error message when the commands cannot do the required job. Shell provides you the facility to test conditions.
Successful execution of a UNIX command returns a zero value while failure of a command execution returns a non zero value.
Shell provides a statement “test” for checking validity of an expression. This is used by conditional statement for testing expression.
“test” statement can be used for :
· testing file existence
· testing file access permission
· string comparision
· numerical comparision
Syntax:
test <>
Statement “test” returns a zero value if the expression is true otherwise it returns a non zero value. Expression associated with test may be any one of the following :
Expression evaluates to true if
-f filename
named file exists and is an ordinary file
-r filename
named file exists and is readable by the user
-w filename
named file exists and is writable
-x filename
named file exists and is executable
-d filename
named file exists and its size is greater than 0
s1 .. s2
strings s1 and s2 are not equal
n1 –gt n2
n1 is numerically greater than n2
n1 –ge n2
n1 is numerically greater than or equal n2
n1 –ne n2
n1 is not equal to n2
n1 –lt n2
n1 is less than n2
n1 –le n2
n1 is less than or equal to n2
n1 –eq n2
n1 is equal to n2
Expression may also be built by combining above expressions within the test statement. While combining the expression normal rules of boolean logic are followed. Following logical operators are used for combining the expressions :
· ! for “negation”
· -a for “ logical and”
· -o for “logical or”
So using these you can write complex expression as follows :
Expression and Explanation:
! expr
will be true if expr is false.
expr1 –a expr2
will be true if and only if expr1 and expr2 are true.
expr1 –o expr2
will be true if and only if expr1 or expr2 are true.
Branching Constructs:
The shell provides a branching construct similar to other programming language. Branching can lead to execution of selective statements. The if command is used for conditional branching operations in a program.
The if .. then …if command
Syntax:
Form 1:
if condition
then
sequence of commands
fi
Form 2:
if condition
then
sequence of commands
else
sequence of commands
fi
Form 3:
if condition1
then
sequence of commands
elif condition2
then
sequence of commands
elif condition3
then
sequence of commands
else
sequence of commands
fi
fi
fi
Looping:
Looping is the most powerful feature of programming. It can execute a task a fixed number of times. The iteration can be performed n ( where n is a variable ) number of times depending on the terminating condition.
The While Do .. Done Command:
The execution of the while command depends on the outcome of a condition. The statements placed with the while are executed if the condition returns “true” and the execution terminates, the moment condition returns false. Thus the execution is performed till the condition is true.
Syntax
while <>
do
statement 1
statement 2
statement n
done
Example:
1. while (x -le 10)
do
echo "$x"
$x=$x+1
done
2. Odd no series (1 to 21):-
$ x=1
while [x -le 21]
do
echo "$x"
$x=$x+2
done
3. while [true]
do
ps-f
sleep 5
done
For Loop:
1. for x in 12345
do
echo "$x"
done
Output: 1
2
3
4
5
2. for x in ls-l who who am i ps-f date
do
echo "$x"
done
3. for var in $PATH $HOME $MAIL
do
echo "$var"
done
4. $ ch= "unit01 unit02 unit03"
$ for var in $ch
>do
>wc $var
>done
wc=word count
5. for file in *.c
>do
>cc $file
>done
cc=compiler
6. for file in *.exe
>do
>file
>cat a.out
>done
For execute all executable files.
If then else condition-
It takes two way decision depending upon the fulfilment of the certain condition.
Syntax:
if (condition)
then
statement(&)
else
statement(&)
fi
Some Shell Programs:
Example 1: A shell program to read two numbers and print their sum.
Solution:
echo “ enter x”
read x
echo “ enter y”
read y
z = ‘expr $x + $y’
echo $z
Example 2: A shell program to accept age and check whether the person is eligible to cast vote or not.
Solution:
echo “ enter age”
read age
if test [$age –ge 18]
then
echo “eligible”
else
echo “not eligible”
fi
Example 3: A shell program to print the largest of two numbers.
Solution:
echo “ enter a”
read a
echo “ enter b”
read b
if test [$a –gt $b]
then
echo $a
else
echo $b
fi
Example 4: A shell program to print 1st 10 natural numbers.
Solution:
count =1
while $count –le 10
do
echo $count
count= ‘expr $count + 1’
done
Example 5: A shell program to print the square of 1st 10 natural numbers.
Solution:
count=1
while $count –le 10
do
c= ‘expr $count \* $count’
echo $c
count= ‘expr $count + 1’
done
Example 6: A shell program to print any name.
Solution:
$ cat > xyz.sh
echo "enter your name"
read name
echo $name
^d
Example7: Write a program in UNIX to search a pattern from a file, the file name and pattern is given by the user.
Solution:
$ cat > find.sh
echo "this program finds a pattern from a file"
echo
echo "please enter file name"
read file name
echo "please enter pattern"
read pattern
grep "$ pattern" $fname
^d
Note: grep= search
To search a data from a file
grep "$ value" $ fname
Pattern= Record
echo= printf statement
read= scanf statement
Example8:
echo "press enter key to continue"
tput clear (clrscr( ))
echo "/enter the pattern and fname"
read pattern fname
grep "$ p name" $ fname
Example9: Write a program to find largest number among two numbers.
Solution:
$ x=5
$ y=10
$ if [$x gt $y]
>then
>echo "the largest value is $x"
>else
>echo "the largest value is $y"
>fi
$ ^d
Note-
-lt - less than
-gt - greater than
-eq - equal to
-le - less than equal to
-ge - greater than equal to
-ne - not equal to
Test exact status-
-a whether file is available or not.True.
-r True if file exist and is readable.
-w True if file exist and is writable.
-x True if file exist and is exexcutable.
-s True if file exist and the size is zero.
-f True if file exist and a regular file (file is UNIX file or not).
Example10:
$ cat > file test.sh
echo "enter the name of the file"
read fname
if [! -f $ fname]
then file does not exist
elif [! -r $ fname]
then
echo "file is not readable"
elif [-w $ fname]
then
echo "file is not writable"
else
echo "file is both readable and writable"
fi
!- bang use to check for not pupose.
The string test can be used the avaibility of the string and we can also compare the string.
Test Exact status
-n stg True if string is not null string.
-z stg True if string is null string.
s1=s2 True if string s1 equals to string s2.
s1!=s2 True if string s1 and s2 not equals to.
stg True if string stg is assigned and not null.
Example11:
$ cat > emp5.sh
echo "enter the string to be searched"
read stg
if [-z "$stg"]
then
echo "you have not entered the string"
exit1
fi
echo "enter the file name to be used"
read fname
if [! -n "$fname"]
then
echo "you have not entered the file name"
exit2
else
echo "you want to search for $stg from $fname?"
read answer
if ["$ answer = y"]
then
grep "$stg" "$fname" ¦ ¦ echo "pattern not found"
else
exit3
fi
fi
Example12:
cat > Menu.sh
cat << END
MENU
1. List of files
2. Processes of users
3. Today's date
4. Users of system
5. Quit to UNIX
END
echo "enter your option"
read ch
echo
case "$ ch" in
1) ls-l;;
2) ps-f;;
3) date;;
4) who;;
5) exit;;
esac
Example13:
cat > arithmatic.sh
cat << END
MENU
1. Addition
2. Subtrction
3. Multiplication
4. Division
5. Exit
END
echo "enter your choice"
read ch
echo "enter the value of a"
read a
echo "enter the value of b"
read b
case "$ ch" in
1) c=a+b
echo "$c"
2) c= a-b
echo "$c"
3) c= a*b
echo "$c"
4) c= a//b
echo "$c"
5) Exit
esac
