New Document
Fundamental performance
  • You can't have too much RAM
  • The computer is never fast enough
  • There is always one more bottleneck
  • Every 2 years your computer is obsolete
  • Every new software upgrade will cost performance
  • Every new O/S upgrade will cost performance
  • Performance is money
  • Money can't performance
Fundamental laws of performance tuning
  • Be a scientist FIRST: Measure SECOND: Hypothesize THIRD: Test/Verify/Fiddle with things GOTO FIRST
  • Change only one thing at a time
  • Note what you changed
  • Save measurement data
  • TANSTAAFL: There Ainít No Such Thing As A Free Lunch
  • Don't look for a panacea
The uncertainty principle
  • Quantum mechanics discover that it is impossible to simultaneously measure both a particle's locationand its speed
  • Heisenberg's uncertainty principle indicates that the act of measuring a system affects it - thereby throwing the measurement off
  • The same applies for performance tuning! Most system monitoring programs are resource pigs! Some fancy X-windows based monitoring tools can completely crunch a system! You must be able to factor (roughly) the effect of measuring into your measurements
Hardware and O/S interaction
  • The key to understanding system performance is to understand how your software uses the underlying hardware
  • Most performance problems are a result of one of: Unbalanced systems Incorrect algorithms System bugs Resource starvation
Hardware and O/S i unbalanced systems
  • System capabilities must be on par to take advantage of eachother: If you have a super hot CPU connected via a 14.4K PPP link your network will be sloweven if you upgrade your CPU If you have a super hot on a system with slow disk drives it will spend most of its cycles waiting for the disk to spin If you have an incredibly fast disk connected to a slow controller it cannot transfer data faster than the controller
  • Balancing system depends largely on what you want to do with it
Hardware and O/S interaction: incorrect algorithms
  • Sometimes system capabilities are ignored or even defeated by using the wrong algorithms either in the kernel or in applications: One example is System V filesystem which (unwittingly) acted to defeat disk track caches UFS (aka: fast file system) disk geometry logic makes no sense on SCSI disks but is incredibly important on CDROMs
  • What was a smart way to do things years ago may be a bad idea today because of changing technology
  • If you are on or near the cutting edge you willget cut
Hardware and O/S interaction: system bugs
  • Bugs can manifest as performance problems Network problems may cause NFS errors and retransmits which will appearto be a disk or server performance problem Multiprocessor lock contention or lost locks may appearto be slow disks or unpredictable hangs Memory leaks may cause slow system degradation [A favorite example is an X-Window server that slowly leaks memory and grows to 20-30MB before crashing the system]
  • Tracking vendor bug reports and keeping the O/S upgraded regularly can be a cheap and easy form of performance tuning!
Hardware and O/S interaction: resource starvation
  • Resource starvation situations are the hardest to debug: Often the system tries desperately to reallocate resources to get work done This means that a shortage of RAM may appearto the user as slow disk I/O Shortage of network buffers caused by shortage of RAM may appearas a slow network!
  • Most system performance problems are the result of resource starvation
CPU performance measures: key concepts: interactive response
  • When your mind-bogglingly fast RISC system feels slow to you, it probably is!
  • Use the time command to get a rough idea how long some basic things normally take Usually the same tasks should take approximately the same amount of time each time, all things being equal
  • Scientific studies* show that computer user frustration rises sharply with delays in response More than 1/2 second delay produces visible stress Users begin repeatedly pressing [3 times, on average] Donít worry: if youíre the systems admin they willlet you know
CPU performance measures: key concepts: load average
  • Load average prints the average number of runnableprocesses during: The last minute The last 5 minutes The last 15 minutes
  • A runnable process is one that is ready to do work but cannot because the system is busy
  • Load average is a very coarse approximation but usually when it is high you have a problem
  • Some systems compute load average wrong and get very strange results
CPU performance measures: key concepts: CPU idle/system/user
  • Many system diagnosis tools report CPU usage in terms of percent of time spent in:
  •  Idle - The CPU is twiddling its little silicon thumbs
     User - The CPU is doing computation on behalf of an
    application
     System - The CPU is busy doing some kind of internal 
    bookkeeping
    
  • High values of system time usually indicate a performance problem:
  •  I/O waits
     Paging
     System calls
     Network service routines
    

The purpose of this tutorial is to introduce the performance analyst to some of the free tools available to monitor and manage performance on UNIX systems, and to provide a guideline on how to diagnose and fix performance problems in Unix environment.

UNIX has following major resource types that need to be monitored and tuned:

  • CPU

  • Memory

  • Disk space

  • Communications lines

  • I/O Time

  • Network Time

  • Applications programs

Peformance Components:

There are following major five component where total system time goes:

ComponentDescription
User state CPUThe actual amount of time the CPU spends running the users program in the user state. It includes time spent executing library calls, but does not include time spent in the kernel on its behalf.
System state CPU This is the amount of time the CPU spends in the system state on behalf of this program. All I/O routines require kernel services. The programmer can affect this value by the use of blocking for I/O transfers.
I/O Time and Network Time These are the amount of time spent moving data and servicing I/O requests
Virtual Memory Performance This includes context switching and swapping.
Application ProgramTime spent running other programs - when the system is not servicing this application because another application currently has the CPU.
Peformance Tools:

Unix provides following important tools to measure and fine tune Unix system performance:

CommandDescription
nice/renice Run a program with modified scheduling priority
netstat Print network connections, routing tables, interface statistics, masquerade connections, and multicast memberships
timeTime a simple command or give resource usage
uptimeSystem Load Average
psReport a snapshot of the current processes.
vmstatReport virtual memory statistics
gprofDisplay call graph profile data
profProcess Profiling
topDisplay system tasks
Previous                                                                                                                                                       Next

Back to Top