Tuesday, November 18, 2008

SQL Server 2008 - Resource Governor Questions

SQL Server 2008 - Resource Governor Questions

A couple of common questions have surfaced related to the SQL Server 2008 Resource Governor feature. The SQL Server 2008 BOL has a lot of good information so start don't forget to review that as well.
Metadata and Runtime Data
A set of DMVs exist for resource governor.  The 'dm_' indicates the current 'in-use' (running values and statistics).  These DMVs are helpful to view the various pool and group activities and supplement the performance counters well.
sys.dm_resource* Running values and statistics
sys.resource* Metadata values
Classification
Classification takes place during login the sessions maintains the classification for its lifetime.  The classifier is a UDF similar to a login trigger returns a group name.
ALTER RESOURCE GOVERNOR DISABLE
Resource governor is dynamic and as such a disable is not absolute. Assume you have GroupA and sessions are assigned to GroupA (see sys.dm_exec_sessions group_id column).  You then issue the disable command.
SQL Server will:
  1. Return the "default" group and pool to default values.  For example the memory and cpu values will return to 0:100 and the query memory grant percentage to 25%.
  2. The classifier is ignored and all new sessions are assigned to the "default" group
  3. Existing sessions will remain assigned to GroupA until disconnected.
Internal Group
Activities such as the lazy writer, ghost record cleanup and others are assigned to the internal group.   This is not a group that can be used by classifier.  If "internal" or an invalid group is returned from the classifier the "default" group is used instead.
There are limited locations where the session is temporarily permitted to the internal group.   One area is when producing a trace event.   As an administrator you do not want to enable tracing on the server and commands to start failing memory allocation.   If you had a pool set to 0:10 for memory and the production of the trace event exceeded the 10% max goal you don't want it to fail.  It makes it really hard to support a server when support functions are limited.   Instead SQL Server temporarily promotes the session to the "internal" group to produce the event and then returns the session to the assigned group.  
Importance is not priority
The importance setting should not be looked at as a thread or process priority setting.  For example a pool using 0:10 CPU with a group of high importance may not get resources ahead of a pool using 0:100 CPU and a normal importance.   The resource governor is designed to avoid live and deadlock scenarios while attempting to maintain a high level of concurrency.   There are times when other decisions will outweigh the importance decision.
Why when I disable resource governor does dm_exec_sessions and error messages indicate the "default" group assignment?
Disabling resource governor resets the "default" group and pool values to the defaults 0:100, 25% memory grant limit and such.   All new sessions are assigned to the default pool as the classifier is disabled.

This behavior makes the SQL Server 2008 work similar to SQL Server 2005 but there are some subtle differences.  You are really running with the "internal" and "default" pools and groups.   Aspects of memory and scheduling can be applied to these groups.  For example the default memory grant percent in SQL Server 2005 was 20.  When resource governor is disabled the "default" group setting is "25%"
Active query is taking too much CPU how can I throttle it?
Individual queries can't be throttled but the groups and pools can be altered dynamically.  It is important to remember that once a session is established it is bound to the group until disconnected. 
Assume you have PoolA with a 50:100 min:max CPU configuration.  You can alter the pool to 25:50 and issue a reconfigure.  This is a change that can be dynamically applied so all groups associated with pool take on the 25:50 CPU behavior.
There are several configuration options that can be dynamically applied.  Others are applied but may not take effect until the next batch due to the nature of the setting.  An example of this would be group max requests.  This is applied when the request is started.
If PoolMarketing is max CPU percent is set to 10% and PoolExecutive to 100% why does marketing exceed 10%?
This example assumes a single query executing from each pool.
CPU percentage is determined at a per scheduler level.   The max CPU percent is only a guideline to use when scheduler resources are under contention.  So there are several possibilities here.
  1. This a a multiple processor computer and each session is using a separate CPU.  Since there is no contention the CPU usage would not be limited for either query.
  2. I will tell you that this was a single CPU computer.  Notice that the Marketing pool query start and used almost 100% CPU until the Executive query starts.  This is where the SQL Server scheduler encounters contention and the balancing begins.  It still exceeds 10% and this can be expected.  Because Executive is not requiring 90% CPU Marketing is allowed to use the extra bandwidth.   Perhaps Executive is waiting on I/O or other resource and does not need the CPU resource at a 90% level.
 image
My classifier has a bug and is leading to connection timeouts.  How can I correct it?
The classifier should run quickly.  However, we have all made coding mistakes that cause infinite loops or other bad behavior.   This will cause all new login attempts to timeout.   So I would you login to disable the classifier and correct it?
  1. DAC logins do not execute the classifier.
  2. Starting SQL Server in single user move (-m command line parameter) does not execute the classifier.

------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
CUSTOMER SUPPORT QUICK START  (spring 2007 as is)
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
Here is an overview I provided to the support team in the spring of 2007.
The resource governor is designed to provide a fair level of scheduling, apply memory, concurrency and wait constraints.   The design involves pools, groups and session classification assignments.  The pool and group have separate controls that are combined to provide the governance activities.    Pools can have more than one group assignment but a group can only be assigned to one pool.

Pools, Groups

Pool
Limited to 20 (18 user defined and 2 built-in [internal and default])   - pool and group names are case sensitive
The variance goal of RG is 5%.  100% / 5% is 20 or the logical max break down based on granularity of the feature

Pool Controls
                Pool controls provide the key min and max limits for CPU and Memory.

                                Min/Max CPU%   (Note: Applied per scheduler not as percent of all schedulers)
                                Min/Max Memory%

Warning: Pools that establish min and max memory values should be done with caution.   As soon as the pool is available at runtime the memory boundaries are applied.   This occurs even if a group is not yet bound to the pool.   It is a bad practice to create pools with memory limitation that are seldom used as the SQL Server memory manager has to account for the pool and use the limits even when sessions are not active.



Group
                A group can be associated with a single Pool but a Pool can have many group associations.

The code does not limit the groups but we should recommend to customers they keep a reasonable set of groups or maintenance and troubleshooting could become complex. 

                                Group Controls
                                                Group controls allow limitations that are generally applied to an individual worker request.  

                                                Importance  (Low, Normal, High   1:3:9)

                                                Max Concurrent Requests   (Note:  If transaction active request won’t honor this limit to avoid deadlocking.)

                                                Request Max CPU Time Seconds
                                                Request Max Memory Grant Percent
                                                Request Max Memory Grant Timeout Seconds
                                                Request MAX DOP

Special Pools and Groups: internal and default

The internal and default pools and groups are created during installation of the master database and are special, system groups.  The internal pool and group is assigned to system tasks such as checkpoint, lazy writer and others.  The default group is used whenever the classification is disabled (UDF not installed), invalid group name returned or any error in the classifier UDF is encountered.

The default pool and group can be ALTERED but the internal group parameters are fixed values. 
 

Classification UDF

Classification is controlled by a server wide UDF, stored in master) that is run just like a login trigger.   Once the authentication portion of the login has completed the ‘PreConnect’ activity takes place.   This PreConnect state can be traced with the new PreConnect trace events as well and the ‘PreConnect’ state shown in sys.dm_exec_sessions and sys.dm_exec_requests.    The classifier is not added to sys.dm_exec_query_stats but the individual statements are.  Using the sql_handle in sys.dm_exec_requests the classifier can be identified.

The rules for the classifier are not limited to but include the following:

·         UDF must return a group name of type NVARCHAR(128) == sysname
·         Group names returned are case sensitive comparisons  (this may use server setting in the future)
·         Returning an invalid group results in assignment to the ‘default’ group
·         UDF can’t assign to the ‘internal’ group.  Returning internal results in assignment to ‘default’ group
·         UDF can’t call XPROCs, Linked Server or other such T-SQL such as waitfor delay.   
·         UDF is part of the login process so it should execute quickly or logins can timeout
·         Session is bound to the group for the life of the session
·         If UDF fails (ex_raise) or returns invalid group name the session is assigned the default group.
·         If no classification UDF is active/present the session is assigned the default group.
·         System tasks (checkpoint, lazy writer, …) are assigned to internal and don’t execute the classifier
·         DAC does not execute the classification UDF allowing the DBA to access the server if the UDF is misbehaving
·         Starting SQL Server with –m –f avoids execution of the classifier UDF allowing resource governor meta data changes
·         Classifier allows access to common tables, DMVs and intrinsic to obtain information such as application name for group assignment decisions

Classifier Invocation

The classifier is a UDF that is fired right after the login triggers and before the T-SQL debugger is enabled during login.  The classifier runs under the ‘interal’ group as classification can’t take place until after the classifier returns the proper group.
 
Importance (HIGH, MEDIUM (normal - default), LOW)

A group property that should be clearly understood is the importance.  Importance does not imply a priority scheme such as one might be used to with thread and process priorities in the Windows schedulers.  

Importance is just used as a simple weighting among active workers for the pool.   When a worker is added to the runnable list (Resume) the importance is use as a factor for position in the list against other workers in the same pool on the same scheduler.   The importance does not carry across multiple schedulers nor does it carry across multiple pools on the same scheduler.   It only applies to active workers of groups assigned to the same pool.

Session Boundary

The classification only takes place during the initial creation of the session.  Version 1 is not shipping with a SET statement allowing a session to change their group explicitly.   This means that we don’t have to conver all the possible scenarios about behavior when a SET is used in a proc versus a batch or transaction scope activities.   It also means that sp_resetconnection does not re-execute the classifier.  The classification is done in the login function only (same place as login triggers) and the classification is bound to the session for life of the session.

There are limited internal locations that the group may be altered.   An example of this is producing a trace event.   The production of the trace event is wrapped with an auto scoped class that binds the working to the internal group until the event is produced.   This prevents unwanted conditions such as out of memory when a memory limited pool is producing an event.   When running in C2 audit mode is an event can’t be produced the SQL Server is shutdown.   The SQL Server will prevent such issues by establishing a temporary assignment to the internal pool when required.  

Metadata – Master.mdf

The metadata is stored in the master database.  When I first read about the feature I was thinking about a hosting scenario and that the hosted databases could have different groups and pools and that put me in the mindset that the database had the metadata.   This is not the case.  Master holds the metadata for resource governor, including the classification UDF.   They don’t have to be but the pools, groups and classifier would need be scripted and established on alternate servers you want to restore or attach databases on and maintain the resource governor settings.    If the database is restored or attached on a server without the classifier, pools or groups the default behavior applies.

This is critical to CSS for attempting reproductions.   It will be important that when cloning a database you also get the pools, groups and classifier.   For example, the pool may have a memory limit the changes the plan selection.   If the memory limitation is not applied a different plan could be selected.   Outside of the clone a DTA tuning exercise may need the proper settings to perform the what-if analysis.

This is also important to customers because they will need to properly maintain the resource governor metadata on test servers to be able properly establish testing environments.

Alternate Workloads

The resource governor does not force the min and max setting for all pool or group properties.   The scheduling decisions are only made when there are 2 or more active users on the same scheduler.   If scheduler bandwidth is available the worker can exceed the max configured value because doing so does not violate the scheduling for another pool.

Memory limits are upheld because it would be extremely difficult to exceed the max value and be able to quickly return to the max limit when another pool needed to use it.

The group provided a max concurrent user limit.   This can be exceeded by something like an open transaction.   For example, if a worker is returning to run a batch and already has an open transaction the worker may be allowed to execute and exceed the concurrent user limit.   If the server did not allow this we can create unwanted blocking and deadlock situations.   Allowing the user to continue is the best solution.

With all this variations possible to properly test alternate pools, groups and workloads should be applied to testing environments.   You can’t simply configure a pool to support 0:50 CPU and test for proper query duration.   Without alternate load the pool will get 100% of the CPU.   You would have to create an alternate pool set to 0:50 and have something like ‘while(1=1) declare @iID int’ running on each scheduler to make sure the 50% target is maintained and the testing can validate the true response parameters.

Variance

Variance among this is common.  The CPU% values should never be set to anything less than a 5% boundary.    Resource governor does not provide strict, percentage guarantees but instead we should use the terms like ‘in aggregate’ or ‘in average’ the values are upheld.   The scheduling goes into the possible CPU variances and explains how over time, with the proper loads the designated values should average near the designated settings.

DMVs

There are several for resource governor.  The following are the meta data DMVs showing the currently established meta data.   These can be different from the running values until the proper reconfigure has taken place.


select * from sys.resource_governor_resource_pools
select * from sys.resource_governor_workload_groups
select * from sys.resource_governor_configuration

The following dm_* DMVs expose the running values

select * from sys.dm_resource_governor_resource_pools
select * from sys.dm_resource_governor_workload_groups

 
Examples

The following example creates a pool for 0:25 CPU, 0:100 Memory then creates and assigns a group with low importance to the pool.   All of the create commands have matching alter commands. 

Create a Pool

CREATE RESOURCE POOL Pool1
with
(
      MAX_CPU_PERCENT = 25
)

Create A Group

CREATE WORKLOAD GROUP Group1
with
(
      IMPORTANCE = Low
)
using Pool1

Create Classifier

create function RDORR_CLASSIFICATION ()
RETURNS sysname
with SCHEMABINDING                  --          Schema binding is required
as
begin
return N'Group1'              --          All new sessions are assigned to Group1
end


Enable the Classifier

ALTER RESOURCE GOVERNOR WITH (CLASSIFIER_FUNCTION = dbo.RDORR_CLASSIFICATION)
ALTER RESOURCE GOVERNOR RECONFIGURE


Disable the Classifier

ALTER RESOURCE GOVERNOR WITH (CLASSIFIER_FUNCTION = NULL)
ALTER RESOURCE GOVERNOR RECONFIGURE

------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
CPU Scheduling  (Fair Scheduling)  (spring 2007 as is)
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
Here is an overview I provided to the support team in the spring of 2007Some of this has changed and is patented but the overall idea stands and is helpful.


The resource governor design includes scheduling.  I want to make it very clear that the resource governor does NOT attempt to limit actual CPU.   The design is based on balancing activity on a per scheduler bases based on the pool settings and active workers and when contention is not a factor all CPU resources are made available to the worker.   This design makes it critical that customers test pool parameters using alternate workloads when in QA testing environments.
The scheduling portion of the RG improvement is SOS based and happens on a per scheduler basis.   SOS does not look across schedulers when determining what the local runnable list should look like.   The runnable list has changed from simple linked list like SQL Server 2005 were you add to tail or head to a percentile design.
Under limited explanation the percentile design is essentially an array of runnable list used like a time window.   When a worker is ‘Resumed’ (added to the runnable list tail) the proper percentile bucket is selected.  

Say the current runnable list dequeue position is at the percentile location of 2.   The time window would move forward as 2, 3 … and 1 would be the end of the time window.   Workers enqueued at location 2 (current time) would be processed.   Once the workers at location 2 are processed the current dequeue location moves ahead to position 3 and so forth. 
           C
[0][1][2][3][4][5][6]
When a worker is resumed the pool information and importance is used to determine the location in the time window that is appropriate for the enqueue.   Let’s take an example of 2 workers.   Worker A is assigned to pool 1 which is configured for 0:25 and Worker B is assigned to pool 2 which is configured for 0:75.    Both workers are active at the current (C) location =2.   It helps to think of the activity in terms of quantums and not CPU usage.   In this case we want Worker A to get 1 quantum and Worker B to get 3 quantums out of the next 4 quantums to uphold the pool goal targets.
In a simplified discussion what will happen is Worker B will get placed in the runnable queue in bucket 3 and Worker A will get placed in bucket 6.   The dequeue will advance to bucket 3 and scheduler worker B.   Worker B yields (quantum exceeded) and determines the next location to enqueue.   It should enqueue in location 4, repeat this for 5 and then 6.   When the dequeue activity reaches position 6 both worker A and worker B get scheduling quantums and the activity repeats.
Remember this is all per scheduler based using the current active tasks to determine the pools allotted scheduling resources.   As active tasks come and go from the scheduler the percentile enqueue locations are adjusted, attempting to provide fair scheduling quantums to the active workers based on pool settings.
There are conditions that the worker is just added to the head of the current buckets runnable list.  This has been the case since SQL Server 7.0 and the conditions remain similar.   An example might be that a worker was waiting on a I/O request that just finished.   Since it was not using any scheduling quantums it is added to the beginning of the current runnable list for the current quantum and the next yield will apply the RG activity.
NO SLEEP:  The scheduler never adds logic to maintain the max CPU by adding sleeps.
CSS is likely to get inquiries that I set my CPU max to 50% and SQL Server is still using 100% CPU.   This is by design.  If the CPU has bandwidth the scheduler just works off the runnable queues and allows the workers to execute.  As long as the quantum sharing takes place SQL Server still uses the available hardware.   This is again why testing of the various settings is importing with alternate T-SQL workloads using CPU bandwidth.   As long as there is nothing in the current dequeue location the runnable list will progress to the next slot and so forth until it finds a runnable user or the scheduler becomes idle.   So a pool with a cap of 20% will queue at the proper percentile buckets but if no other activity is present the worker gets to use the scheduler quantums. 
Preemptive Activities
The RG CPU implementation is only responsible for CPU time acquired from the SQL_OS schedulers.  It does not attempt to control any preemptive (XPROC, Linked Server, sp_OA,…) CPU usage.   The one exception to this statement is SQLCLR.  The SQLCLR activities are supported by SOS_Tasks and as such some of the CLR activity can be controlled by the SQL_OS schedulers.   As a reminder not all CLR activity maps to a SOS_Task so some aspects of SQLCLR may not be controlled by the RG.
Importance
Groups and thus SOS_Tasks can be assigned importance levels (Low, Normal or High.)   This is distinctly different from thread priorities as changing thread priorities can cause unwanted side effects such as:
·         Irresolvable spinlock and other synchronization object starvation
·         Alteration of I/O priority
·         Memory allocation operations
The importance indicates to a single scheduler the relative priority of the request. 
·         Importance does not span multiple schedulers and SOS_Tasks can’t be migrated to different SOS_Schedulers.  
·         The task remains bound to the scheduler.
·         Tasks with internally boosted priority always go to the head of the runnable list.  For example: An aborted task is placed at the head of the runnable list.
Ratio
The current importance ratio is LOW:NORMAL:HIGH - 1:3:9. 
Current Tasks
The CPU activity for RG only applies to runnable tasks.   Tasks that are in a wait state don’t affect current calculations.
The CPU ratios are recalculated as needed when a user is resumed on the scheduler. 
Note: If no other tasks are present on the CPU the full bandwidth of the CPU can be used by the task.
Use Available
Whenever CPU is available it is assigned to one or more of the runnable tasks to use.  The Max CPU limit only becomes a hard limit when exceeding it would cause another pool to drop below its Max setting.   SQLOS will always attempt to maximize the CPU resource usage. 
This is an important point because you won’t be able to develop a set of pools and groups in SQL Server as a way to manage CPU on the machine in a way to give CPU resources to other applications.
SQL 2005 Compatibility
The RG change slightly changes the scheduling from what SQL Server 2005 and previous SQL_OS or UMS schedulers provided.   Even when all users are running under the DEFAULT POOL and DEFAULT GROUP the fairness is being applied.   In previous versions of SQL Server you went to the end of the runnable list and it was more of a round-robin approach.   The RG design places the worker in the proper percentile bucket so overall the fairness of schedule resources may slightly change the scheduling behavior.
Reference :- http://blogs.msdn.com/psssql/archive/2008/01/10/sql-server-2008-resource-governor-questions.aspx

Friday, November 7, 2008

How to use FTP from the command line.

How to use FTP from the command line.

FTP (File Transfer Protocol) allows you to transfer files between your PC and other Internet systems (hosts). You can transfer files, work with local or remote directories, rename and display files, and execute system commands. Before you start, you must know how to log on to the remote system and have a userid and password on that system. 

Note: Some systems allow anonymous ftp access. To use anonymous ftp, use anonymous as your userid andyour e-mail address as the logon password.

Logging Onto and Off of a Remote System

To begin using Microsoft's FTP client, Open a command prompt and switch to the destination directory (where you want the download file).

To start an FTP session, enter: ftp host_name 
where hostname is the name or IP address of the remote system.

You will then be asked to enter your userid and password.

Once you have successfully logged onto a remote system, you will be able to use ftp commands to view a listing of files on the remote system and transfer files between the two systems.

Example: Download i386.exe (Windows NT 3.5 Resource Kit) from ftp://ftp.microsoft.com/bussys/winnt/winnt-public/reskit/nt35/i386 to C:\Temp\Download

  1. Open a command prompt. Enter CD C:\Temp\Download (assuming that directory exists). 
    Enter: ftp ftp.microsoft.com 

    You should now see a prompt similar to this: 
    Connected to ftp.microsoft.com. 
    220 Microsoft FTP Service 
    User (ftp.microsoft.com:(none)):
  2. For the userid, Enter: anonymous 

    You should see a prompt similar to this: 
    331 Anonymous access allowed, send identity (e-mail name) as password. 
    Password:
  3. Enter: userid@domain.com as the password at the "Password:" prompt. 
    Note: Any e-mail address in a userid@domain.com format should work. You will not be able to see the password as you type it.
  4. To download i386.exe from the bussys/winnt/winnt-public/reskit/nt35/i386 directory, Enter: get bussys/winnt/winnt-public/reskit/nt35/i386/i386.exe

    Note: You could have also used ls to view the directory and file names, cd bussys/winnt/winnt-public/reskit/nt35/i386 to switch directories, and get i386.exe to download the file from within that directory.

  5. To end the FTP session, Enter: quit or bye.
Note: Once you have extracted the resource kit, you will have to expand individual files 
example: expand choice.ex_ choice.exe

FTP Commands

For a list of FTP commands, at the "ftp>" prompt, Enter: help

When using ftp from the command prompt, the following list of supported commands will be displayed: 
Note: Hover your mouse over a command to see what the output of "help *" is for that command.

!   
delete
literal
prompt
send
?   
debug
ls  
put 
status
append
dir
mdelete
pwd 
trace
ascii
disconnect
mdir
quit
type
bell
get 
mget
quote
user
binary
glob
mkdir
recv
verbose
bye 
hash
mls 
remotehelp
 
cd  
help
mput
rename
 
close
lcd 
open
rmdir
 

The question mark (?) command is equivalent to the help command. Typing help or ? followed by the name of a command will display a brief description of the command's purpose.

The exclamation point (!) can be used to shell to the system (command) prompt. Type Exit to return to the FTP session. You can also issue a subset of system commands to perform as you shell out, e.g., ! dir %windir% | more. When the commands in the shell have completed, you will be returned to the FTP session.

The pwd command will list the current directory on the remote machine. To change directories on the remote machine, use the cd command. To create a new directory on the remote machine, use the mkdir command followed by the name you would like to assign to the new directory. The lcd command can be used to change directories on the local (PC) machine.

To display a listing of files on the remote system, enter: ls or dir.

To download a file (copy a file from the remote system to your PC), you can use the command get or recvfollowed by the name of the file you would like to download. Optionally, you can follow the filename with a second filename which will be assigned to the file when it is downloaded to your PC. To download multiple files, you can use the mget command followed by a descriptor for the files you would like to download (e.g.: *.f for all files ending in ".f" or *.* for all files). You will be prompted to indicate whether you would like to download each file in turn. To turn off this prompting, enter the prompt command prior to entering the mget command; you will receive the message "Interactive mode OFF" indicating that prompting has been deactivated.

By default, files are downloaded and uploaded in ASCII file transfer mode. To download or upload files using Binary format mode, enter the command Binary at the "ftp>" prompt prior to downloading or uploading the file(s). To return to ASCII file transfer mode, enter the ASCII command.

To upload a file (copy a file from your PC to the remote system), you can use the command put or send followed by the name of the file you would like to upload. Optionally, you can follow the filename with a second filename which will be assigned to the file when it is uploaded to the remote system. The mput command can be used to upload multiple files.

You can use the close or disconnect command to drop the current ftp connection without exiting from the command enironment and then use the open command to connect to a new host.

Referenced by :- http://www.tburke.net/info/misc/cmdline-ftp.htm

What is FTP?

What is FTP?

The FTP (File Transfer Protocol) utility program is commonly used for copying files to and from other computers. These computers may be at the same site or at different sites thousands of miles apart. FTP is a general protocol that works on UNIX systems as well as a variety of other (non-UNIX) systems. 

For the purposes of this Web page, the local machine refers to the machine you are initially logged into, the one on which you type the ftp command. Theremote machine is the other one, the one that is the argument of the ftp command. 

A user interface for the standard File Transfer Protocol for ARPANET, FTP acts as an interpreter on the remote machine. The user may type a number of UNIX-like commands under this interpreter to perform desired actions on the remote machine. 

Most operating systems and communication programs now include some form of an FTP utility program, but the commands differ slightly between them. The following explanations and alphabetical list of commands refers to the common FTP utility program as provided on a UNIX machine. Check the documentation for your own machine to determine the comparable commands. 

Most computers today include a windows-based type FTP program that is more PC-oriented and does not require full knowledge of these commands.
You can also perform FTP through a browser. For example, bring up Internet Explorer and type in

            ftp://yourLoginName@IPaddress

instead of a normal web page URL. 

The FTP site of the Computer Science department at CSU requires the user to use sftp, the secure version of FTP. Just type sftp instead of ftp, when you are using FTP in a terminal window. 


Getting Started

To connect your local machine to the remote machine, type

            ftp machinename

where machinename is the full machine name of the remote machine, e.g., purcell.cs.colostate.edu. If the name of the machine is unknown, you may type

            ftp machinennumber

where machinennumber is the net address of the remote machine, e.g., 129.82.45.181. In either case, this command is similar to logging onto the remote machine. If the remote machine has been reached successfully, FTP responds by asking for a loginname and password.

When you enter your own loginname and password for the remote machine, it returns the prompt

            ftp>

and permits you access to your own home directory on the remote machine. You should be able to move around in your own directory and to copy files to and from your local machine using the FTP interface commands given on the following page.


Anonymous FTP

At times you may wish to copy files from a remote machine on which you do not have a loginname. This can be done using anonymous FTP.

When the remote machine asks for your loginname, you should type in the word anonymous. Instead of a password, you should enter your own electronic mail address. This allows the remote site to keep records of the anonymous FTP requests.

Once you have been logged in, you are in the anonymous directory for the remote machine. This usually contains a number of public files and directories. Again you should be able to move around in these directories. However, you are only able to copy the files from the remote machine to your own local machine; you are not able to write on the remote machine or to delete any files there.


Common FTP Commands

?to request help or information about the FTP commands
asciito set the mode of file transfer to ASCII 
(this is the default and transmits seven bits per character)
binaryto set the mode of file transfer to binary 
(the binary mode transmits all eight bits per byte and thus provides less chance of a transmission error and must be used to transmit files other than ASCII files)
byeto exit the FTP environment (same as quit)
cdto change directory on the remote machine
closeto terminate a connection with another computer
 close brubeckcloses the current FTP connection with brubeck
  but still leaves you within the FTP environment.
deleteto delete (remove) a file in the current remote directory (same as rm in UNIX)
getto copy one file from the remote machine to the local machine
 get ABC DEFcopies file ABC in the current remote directory to (or on top of) a file named DEF in your current local directory.
 get ABCcopies file ABC in the current remote directory to (or on top of) a file with the same name, ABC, in your current local directory.
helpto request a list of all available FTP commands
lcdto change directory on your local machine (same as UNIX cd)
lsto list the names of the files in the current remote directory
mkdirto make a new directory within the current remote directory
mgetto copy multiple files from the remote machine to the local machine; 
  you are prompted for a y/n answer before transferring each file
 mget *copies all the files in the current remote directory to your current local directory, using the same filenames. Notice the use of the wild card character, *.
mputto copy multiple files from the local machine to the remote machine; 
  you are prompted for a y/n answer before transferring each file
opento open a connection with another computer
 open brubeckopens a new FTP connection with brubeck
  you must enter a username and password for a brubeck account 
      (unless it is to be an anonymous connection).
putto copy one file from the local machine to the remote machine
pwdto find out the pathname of the current directory on the remote machine
quitto exit the FTP environment (same as bye)
rmdirto to remove (delete) a directory in the current remote directory


Further Information

Many other interface commands are available. Also FTP can be run with different options. Please refer to your manual or the UNIX man page on ftp for more information.

See also the SILS FTP Tutorial


Example Sessions

Examples of two FTP sessions are given on the next two pages. These show the type of interaction you may expect when using the ftp utility.


Example of Anonymous FTP Session

%   ftp cs.colorado.edu   Connected to cs.colorado.edu.   220 bruno FTP server (SunOS 4.1) ready.   Name (cs.colorado.edu:yourlogin): anonymous   331 Guest login ok, send ident as password.   Password:   230-This server is courtesy of Sun Microsystems, Inc.   230-   230-The data on this FTP server can be searched and accessed via WAIS, using   230-our Essence semantic indexing system.  Users can pick up a copy of the   230-WAIS ".src" file for accessing this service by anonymous FTP from   230-ftp.cs.colorado.edu, in pub/cs/distribs/essence/aftp-cs-colorado-edu.src   230-This file also describes where to get the prototype source code and a   230-paper about this system.   230-   230-   230 Guest login ok, access restrictions apply.   ftp> cd /pub/HPSC   250 CWD command successful.   ftp> ls   200 PORT command successful.   150 ASCII data connection for /bin/ls (128.138.242.10,3133) (0 bytes).   ElementsofAVS.ps.Z      . . . execsumm_tr.ps.Z   viShortRef.ps.Z   226 ASCII Transfer complete.   418 bytes received in 0.043 seconds (9.5 Kbytes/s)   ftp> get README   200 PORT command successful.   150 ASCII data connection for README (128.138.242.10,3134) (2881 bytes).   226 ASCII Transfer complete.   local: README remote: README   2939 bytes received in 0.066 seconds (43 Kbytes/s)   ftp> bye   221 Goodbye.   %   ls      . . . README        . . . 

An FTP session to obtain the HPSC README file from the cs.colorado.edu anonymous ftp directory using a loginname of anonymous and apassword of one's own electronic mail address.


Example of Regular FTP Session

%   ftp nordsieck.cs.colorado.edu   Connected to nordsieck.cs.colorado.edu.   220 nordsieck FTP server (Version 5.53 Tue Aug 25 10:46:12 MDT 1992) ready.   Name (nordsieck.cs.colorado.edu:yourlogin): yourlogin   331 Password required for yourlogin.   Password:   230 User yourlogin logged in.   ftp> cd HPSC/exercises   250 CWD command successful.   ftp> ls   200 PORT command successful.   550 No files found.   ftp> put tmul.out   200 PORT command successful.   150 Opening ASCII mode data connection for tmul.out.   226 Transfer complete.   local: tmul.out remote: tmul.out   1882 bytes sent in 0.0095 seconds (1.9e+02 Kbytes/s)   ftp> ls   200 PORT command successful.   150 Opening ASCII mode data connection for file list.   tmul.out   226 Transfer complete.   9 bytes received in 0.0021 seconds (4.3 Kbytes/s)   ftp> mput *   mput Makefile? y   200 PORT command successful.   150 Opening ASCII mode data connection for Makefile.   226 Transfer complete.   local: Makefile remote: Makefile   1020 bytes sent in 0.0062 seconds (1.6e+02 Kbytes/s)   mput tmul.out? n   ftp> quit   221 Goodbye.   %   ls      . . . Makefile     tmul.out        . . . 

An FTP session to copy files from a remote machine back to nordsieck.cs.colorado.edu using one's own login and password.

Referenced by :- http://www.cs.colostate.edu/helpdocs/ftp.html