Introductory Guide to the Unix Host

Revision Record


Preface

Throughout this manual, commands are specified in courier bold and what you need to type is specified in bold italics. Where a userid is used in examples, our user had the userid, abcdefg.


Contents


1 Introduction

It is expected that users will access MC via X-windows, such as xterm on a workstation or using X software, such as eXceed on a PC or exodus on a Mac. For those of you without these facilities, you should use telnet or rlogin wherever possible. Solaris on the CS6400 system at MC is supplemented by two important additional software components. These are a batch jobs service, controlled by the Network Queuing System (NQS) software, originally developed by NASA to provide a mechanism to control the running of background processes, and remote job submission facilities, and file backup and archive software provided by the Legato Networker Backup and Archive suite. There are further modifications from Cray which provide extra performance and reliability.

1.1 The Operating System and Its Shells

Solaris, like any other UNIX system, uses shells to interpret the commands you type, and pass them on to the operating system kernel. When you are registered on CS6400, you are provided with a login shell; this is normally the only one you will need to use, and it is started up automatically each time you login to the CS6400. As well as interpreting commands, your login shell maintains your user environment (for example, it contains details of: the type of terminal you are using/emulating; which editor is to be used for creating files; which editor is to be used when typing commands; the identity of the PC/workstation for screen displays; etc). Manchester Computing gives new users the Korn shell (also written as ksh) as their default login shell, but there are various others available, such as the Bourne shell (sh), the C shell (csh), and the Bash (Bourne Again) shell (bash). All the examples in this guide are for the Korn shell, although many of the features are common to all the shells. The Korn shell prompt (when the system is waiting for you to type in another command) is a dollar sign, $. You can also write executable programs consisting of shell commands, known as shell scripts; these are similar to DOS batch files. UNIX is case-sensitive (that is, lower case and capitals are interpreted differently). It is important to remember this when logging in to the CS6400, typing commands, or naming files.

If you are familiar with UNIX, and use a different shell to the MC default, we provide the chsh (change shell) command so that you can alter the default shell to any of the supported shells. For example, to change your default shell from the Korn shell to the Bash shell, you would type:


	chsh (Rtn)

You will then see the following dialogue, and should enter bash when prompted for the new shell:

DBD Host: cs6400.mcc.ac.uk
Changing login shell for abcdefg.
Old shell: /usr/bin/ksh
New shell: bash
Send packet.
Packet Received.
$

This change is permanent unless you issue the chsh command again.

1.2 Your Terminal or PC/Workstation

You can use a variety of terminal types to connect to the system, and the way in which you connect depends on your location. The ideal way is to use X-windows access over TCP/IP. Failing that, you should use a TCP/IP connection to telnet in to the CS6400, either from a workstation or a PC, but there is an X.29 coloured book route for those sites without a JIPS (JANET IP Service) connection. In the event that you do not have access to any of the types of terminal UNIX supports, it is also possible to use a dumb terminal. This is unlikely however, as most terminals or PC software will emulate a VT100, which is one of the supported terminal types. For more details of supported terminal types, please see Chapter 2, Section 2.1.

1.3 Filestore

Filestore on the CS6400 is organized such that a user can create a tree-like structure of directories and subdirectories in which to store files. This system is very similar to that used by DOS, except that path names to directories are denoted by a / not a \. Your main working directory on UNIX is known as your home directory. This can also be referred to as $HOME. Each user's home directory containing their files is itself a subdirectory of the entire filestore available on the system. The top level system directory is known as the root directory, or /.

You can let other users have access to your files through a system of access permissions. There are three levels of access permission in UNIX: user, group, and other (sometimes referred to as world access), and three types of access, read, write, and execute. Permissions can be set both for files and for directories. By default, your own files are accessible only to yourself (if created on the CS6400; if transferred from other machines, the world has read and write permissions - see Note in Section 3.8.2) when you are logged in, but you may give access permission to other people in your group, or make your files available to everyone on the system. The group you are assigned to will have been set up by your site representative when you registered. It is possible to be associated with more than one group.

1.4 Security

The passwd command enables you to change your login password. The chmod command allows you to alter read, write, and execute permissions on files and directories, at the user, group and other levels (see Section 3.8.2 on how to use chmod).

1.5 Scripts

You will want to know how to create simple shell scripts to make interactive tasks, such as running programs, more efficient. You will also need to use scripts if you want to run programs in the background, or using the NQS batch system (see Chapters 6 and 7).

1.6 Becoming a User

You need to be a datasets user in order to use the CS6400 (these may be datasets held on the system, such as the Census, or may be large datasets of your own). Contact your MC Representative for a Registration Form and complete the appropriate section(s). Your representative will complete the remainder of the form which is sent to the Centre for processing.

On registration, you will be given a unique user name, which will identify you and your department/university/site for administrative and accounting purposes. You will also be given a password. The system will prompt you to change the password the first time you login. (See Chapter 2, Section 2.6 for further details).

Normally, you will be registered within 48 hours of your Registration Form being received at Manchester Computing.

1.7 Documentation

Besides this guide, a number of other manuals and CSS Notes have been produced by Manchester Computing. These describe in more detail various aspects of the CS6400 service and the use of particular datasets and software packages. A user note, CSS 001, contains a list of all current CSS Notes and manuals which are relevant to the CS6400.

1.8 Control of Resources

The main resources which are allocated or controlled are filestore (disc space), and processor time - other resources may include memory usage, I/O, etc. These resources are controlled at the user level, although each site has an overall allocation of resources. To check how much filestore you have available and how much of it you have used, type the command

	quota -v

This will display both your allocation or quota, and your current usage. The display will look something like this:

Disk quotas for abcdefg (uid 1148):
Filesystem      usage  quota  limit timeleft files quota limit   timeleft
/home/cs6400b    5371  20000  22000   	     96

The figure listed under the usage heading is the amount of space you have used up in Kilobytes, in this case 5371. The allocation or quota is 20000 Kilobytes or 20 Megabytes, and the hard limit is 22000 Kilobytes or 22 Megabytes (the allocation + 10%). The "timeleft" field indicates how many days are left before action is taken when you exceed your filestore hard limit.

The command


	diskq

shows the information in a more concise format:

abcdefg : allocated 20 Mb, used 5 Mb ( 25.00%)

Each department/university/site has a share of filestore. The computer representative tells MC staff how much filestore to allocate each user at the time of registration, so it is important to talk to your representative if you require a large amount of filestore. The quota you have been given is a soft limit which may be exceeded temporarily, up to a hard limit which cannot be exceeded. If you exceed the soft limit, you will get a warning message when you login. If after a week you are still exceeding the soft limit, you will be prevented from creating more files.

Usage of CPU time is monitored and controlled by a package similar in function to RACS on the old VM/CMS scalar service at MCC. All usage is collected and monitored, and usage reports for each site are available on the CS6400 gopher system. The racsinfo command shows current usage for your site; the pkguse command shows details of the CPU time used by your last 10 jobs. There is currently a limit of 15 minutes CPU time for interactive jobs; jobs requiring more than 15 minutes of CPU should be submitted to the NQS batch system (see Chapter 7). The interactive limit is enforced by 'governor' software, which sends an email message to your CS6400 mailbox (see Section 8.1 for details of how to use the mail system) if it has killed off an interactive process which used more than 15 minutes of CPU time.

2 Getting Started

This chapter tells you how to begin a terminal session (login) and how to end it (logout). It also describes how to use the terminal, and how to get help. The general format of a UNIX command is also explained.

2.1 Types of Terminal

UNIX can operate with many terminal types, but it is necessary to 'inform' the system which kind of terminal is in use. At Manchester Computing, we use the qterm program to do this automatically. It asks your terminal to identify itself. Many, but not all, terminals will respond with a string that can be used to select the correct value to set in an environment variable called TERM. You may see the message "Terminal recognized as vt100 (ANSI/VT100 Clone)", or something similar, when you log in - this is a message from qterm.

Sometimes the terminal will not respond to the request and sometimes it will respond with a message that is not recognized by qterm. If there is no response you will have to manually set the TERM variable to a valid value - "vt100" will work most of the time with simple full screen displays, that is:


	export TERM=vt100 	# for users of sh, ksh, or bash
	setenv TERM vt100 	# for users of csh

export means 'pass (export) the value of one or more shell variables, giving global meaning to the variables (which are local by default)'. In this example, the shell variable is TERM and its value is vt100. The C shell command, setenv, has the same effect as export.

You can place this command in your .profile (sh/ksh/bash) or .login (csh) files and have it set each time you login. For more information about the .profile file, see Section 2.4 below.

It is recommended that you use X-windows, for those of you having access to an X-terminal or X-windows emulation software on a PC (such as eXceed). For those of you without access to an X-terminal or X-windows, it is recommended that you use software emulating a VT100, preferably connecting via TCP/IP telnet, although coloured book software enables those coming in via a JANET PAD to use a terminal/PC, again preferably emulating a VT100. The following terminal types are amongst those supported:


	xterm	X-terminal or X-windows software (e.g. eXceed)
	VT100	DEC VT100 or compatible terminal
	Z29	Zenith Z29
	ADM31	ADM 31
	VT52	VT52
	Telnet	TCP/IP VT100 terminal emulators for the IBM PC or     
		compatible microcomputer
	Kermit	VT100 or compatible emulation from an IBM PC or     
		compatible microcomputer

When you log in to the CS6400 with a functioning terminal, you can check whether your particular terminal type is supported by using the command:

	infocmp   -I   <terminalname>

to search through the terminfo database.

There are many TCP/IP telnet public domain utilities, such as CUTCP and NCSA telnet. Most other TCP/IP software, e.g. PC-NFS, LAN WorkPlace, also support VT100 emulation.

In this guide, you are expected to press RETURN (denoted by (Rtn) in this manual) after each command when you want the system to process it. In other documentation, you may see the terms ENTER or RETURN used. They mean the same thing.

On all terminals the key marked RETURN (or the large key with the symbol if there is no RETURN key) performs this function. Some terminals have both a RETURN and an ENTER key which may do different things.

We will start by showing you how to access MC and login to the CS6400 and then describe more of the basic features.

2.2 Access to Manchester Computing

If you have a Network connection (JANET or JIPS/Ethernet), there are three possible methods of access:

(a) If you have an X-terminal, or a PC running X-windows software such as eXceed, you can use X-windows to start up an xterm session on the CS6400. You will probably need to specify the full path /usr/openwin/bin/xterm and the display number of your PC/workstation. It is also possible to access the CS6400 using XDMCP.

(b) If you have an IBM PC (or compatible) or other micro or mainframe system supporting the TCP/IP telnet protocol, you can call the Internet addresses:


	cs6400.mcc.ac.uk
or 	midas.ac.uk
or	130.88.203.18

(c) If you have an IBM PC (or compatible) running a VT100 terminal emulator or any other system providing VT100 emulation, for example, a mainframe or workstation, which is connected to a PAD, you can access Manchester Computing as follows:

At the prompt, PAD>, type:

 
	call net.ja.telnet (numeric address: 000040010900)

This will connect you to the JNT's X29 to Telnet converter. At the prompt:

	Telnet to:

enter

	midas.ac.uk	or 	cs6400.mcc.ac.uk

If you are unable to access the CS6400 from a VT100 or other supported terminal type, dumb terminal connection is still possible, although screen editors such as vi and joe cannot be used. The ed line mode editor can be used in such a case.

2.3 Logging In

You will see the following welcome message when you have contacted the CS6400.

	UNIX(r) System V Release 4.0 (cs6400)
	login:

This is your prompt to enter your userid. The following dialogue shows the login procedure. Note: UNIX is case-sensitive, and all userids must be entered in lower case and all system commands. Your initial password must be entered exactly as given on the registration notification form. What you type is shown in bold, italic, lower case.

login: abcdefg
Password:@@@@@@@

Last login: Mon Jan 28 11:04:01 from gerbel.camford.ac.uk
****  MIDAS  Manchester Information Datasets and Associated Services  ****
       Cray CS6400  -   University of Manchester  -   Solaris 2.5
                 For assistance type "helpme" or "gopher"

news: sw.news gauss emacs quotas passwords kermit joe_editor gopher email 
nohup ftp compilers spss.news windowsize
You have new mail.

$

Note that your password is not echoed as you type. As stated in Section 1.6, if you are logging in for the very first time, the password given on the registration notification form is set to expire immediately. You must therefore change your password to something different (see Section 2.6 for further details). In addition to the above, a "message of the day" may also be displayed. This is used to inform users of scheduled down-time, etc.

If you are accessing MIDAS in X-windows mode using XDMCP, you will be presented with a 'login window'. You should enter your MIDAS userid and password here, remembering that UNIX is case sensitive.

2.3.1 System news

As illustrated in Section 2.3, when you login, you may see a message similar to:

news: sw.news gauss emacs quotas passwords kermit joe_editor gopher email nohup ftp compilers spss.news windowsize

This means that there are messages to read on the news system. You should always read the system news when you login, as it may contain essential messages about system availability, new software, etc.

The command


	news | pg
or  	news | more

will give you the news one 'page' or screenful, respectively, at a time, after which you will get the message

	:
or 	--more--(11%)

To see the next page, press the space bar for more or the Return key for pg. When you have reached the end of the messages, more will exit automatically.

Pressing RETURN from within pg when you see


	(EOF):

will exit back to the $ prompt, which is what you will see each time the Korn shell is awaiting your next command.

If you want to re-read old news messages, use


	news -a | more

which will give you everything that is available.

It is also possible to read the news for a specific item. For example, if you simply want to read the news files for sas and spss you could type:


	$ news sas | pg
	$ news spss

Everything which is posted as a news item, is also available via the gopher information server (see Section 2.3.3 below).

2.3.2 The World Wide Web information server

The World Wide Web (also known as WWW, W3 or just abbreviated to the Web) is a global, interconnected network of computers providing information in many different formats, including hypertext. The MIDAS Web server holds a lot of information and documentation on the service, as well as useful links to other sites of interest. By following a link, you may be accessing a computer on the other side of the world without knowing it; the process is transparent to the user. Any software which is used to access the WWW is known as a browser. Like gopher, the MIDAS Web pages can be accessed directly by starting up a browser when logged in to the CS6400, or it may be accessed from a browser running on a machine at your local site, or on your PC if you have the correct software installed.

There are three different browsers installed on the CS6400; Netscape, Mosaic, and Lynx. The first two can only be accessed if you are using the X-windows system to connect to the CS6400, as they display graphical images, whereas Lynx is a text-based browser and can be used by anyone. To use Netscape (also known as Mozilla), type


	netscape &

To start up Mosaic, type

	mosaic &

To access Lynx, type

	lynx

You will automatically enter the Web at the MIDAS home page.

You can follow links by clicking on them with your left-hand mouse button if using Netscape or Mosaic, or by using the cursor keys to highlight the link and pressing ENTER/RETURN if using Lynx.

If you are browsing the Web from a remote machine, typing in the URL (Uniform Resource Locator, or Web address)


	  http://midas.ac.uk

will get you to the MIDAS home page.

2.3.3 The gopher information server

Gopher is a networked information system that may be used to access information about the CS6400 machine from both login sessions on the CS6400 and other, remote, systems. If you are logged on to the CS6400, you can access gopher with the command

	gopher

A menu will appear from which you can select the areas you wish to see.

Type


	man gopher 

for more details.

If your site supports gopher, you may be able to access the CS6400 gopher by typing:


	gopher   cs6400.mcc.ac.uk		or 	gopher   midas.ac.uk

We intend to transfer all the information on the gopher to the WWW during the next few months.

2.4 The Use of Your .profile File

Each Korn shell user is supplied with the default system .profile file. This is a shell script which executes each time you login, and contains links or paths to the system software and various applications packages. (Users who have a different shell as their login shell may have a different file, for example, for csh, there are two files called .login and .cshrc which perform similar functions). When you have used the CS6400 for some time, you may find that you wish to make certain changes or additions to your .profile.

To edit your .profile file, you will need to use one of the UNIX editors, for example joe or vi (see Chapter 4, Sections 4.1 and 4.2, respectively).

Here is a typical .profile:


#
#       This is the CS6400 default shell "profile" command file.
#  MCC_Version: 1.0 : Do NOT remove this line
#
#       Comment lines begin with a "#".
#
#       See the shell manual page (man sh) and the CS6400 User
#       Guide for information concerning this file and its possible
#       contents.
#

#       This file is for you to change and add to.  As you become more
#       accustomed to using the CS6400, you may remove these comments, making
#       it a little faster at logon time for the profile to execute.
#

MAIL=/usr/mail/${LOGNAME:?}

# The search path for command execution is set in /etc/profile

# set default file protection masking bits for new files.
# "066" means "no read/write permission for other users"
umask 066

# Store current user logname in variable "LOGNAME"
LOGNAME=`logname`; export LOGNAME

# directories to be searched by the cd command
CDPATH=:$HOME; export CDPATH

# terminaldependent stuff
# VISUAL is the name of your fullscreen editor
# EDITOR is some editor, not necessary fullscreen
VISUAL=/usr/bin/vi 	; export VISUAL
EDITOR=/usr/bin/ed     	; export EDITOR

# Set your terminal keys to the usual defaults

stty erase '^H' kill '^U' intr '^C' echoe tostop
# Change to the following if you want <delete> to
# delete characters rather than using <backspace>
#stty erase '^?' kill '^U' intr '^C' echoe tostop

# For X-windows uncomment the next line and add your terminal's IP address
# DISPLAY=[terminal_IP_address]:0; export DISPLAY

# Aliases
alias bye='exit'
alias f='finger | more'
alias h='history | more'
alias rd='rmdir'
alias md='mkdir'

Notes

(a) Your .profile should be edited with care, as it is possible to mess up the file in such a way that you will not be able to log in again!

(b) Copies of the original .profile files may be found in the directory /etc/mccskel; these may be copied to replace a badly mangled file.

2.4.1 Backspace/delete key

In the example .profile file shown above, you will notice that there is a comment line which mentions default settings for terminal keys. Usually, you will want your backspace key (the grey key located immediately above Return/Enter on a PC keyboard) to delete characters which you have entered by mistake. Although the settings in the default CS6400 .profile file supplied by Manchester Computing should work in most cases, they will not be correct for all terminal types. If you find that your backspace key does not delete but echoes strange characters, you will need to edit your .profile file, and uncomment the line starting #stty erase '^?' by removing the #. Make sure that you also comment out the line starting stty erase '^H' by inserting a # at the beginning of the line. When you next login, your backspace key should work as you would expect.

2.4.2 Environment variables

Some environment variables are set up in the .profile; their names are in upper case, for example: PATH, VISUAL, CDPATH, TERM, DISPLAY. These variables contain values which are referred to each time certain commands or processes are executed. The PATH variable is very important, as it contains the path or 'route' the shell will search for executable files or commands. It has already been set up for you, so that all the system commands you need will be found, but you may want to add extra paths by using the command:


	export PATH=$PATH:/new pathname

(the / is not required if the path name begins with $HOME), for example:

	export PATH=$PATH:$HOME/progs

Putting a $ in front of the variable name gives the value of that variable. In this particular example, the .profile has got the value of the variable PATH from the path contained in the standard system profile, and added directions to the user's directory $HOME/progs, so that any executable files in progs will be found.

PATH includes an entry to a directory, bin, in your own home directory. To make things easier, we suggest that you create this directory and install all of your own scripts or other executable programs in it, as files in bin will automatically be found and you will not have to add other directories to your PATH.

To see what values the shell environment variables are currently set to, type:


	set

To display the value of just one variable, type echo $[variablename]; for example:

	echo $VISUAL

echo sends the value of the variable to the standard output (the terminal screen).

Many applications, such as saspac, sas, spss, etc, can be run interactively in an X-window environment. If you access MIDAS via an X terminal or from a PC using, say, eXceed, you must ensure that the environment variables TERM and DISPLAY are correctly set. TERM must be set to xterm, and DISPLAY must be set to the IP address of your terminal (this is so that the system knows where to send displayed output of panels, etc.). Note: if you are using XDMCP, this is done automatically.

To set the TERM environment variable, use:


	export  TERM=xterm	(Korn, bash, sh users)
or 	setenv TERM xterm  	(csh users)

To set the DISPLAY environment variable, use:

	export  DISPLAY=nnn.mm.xxx.yy:0.0 	(Korn, bash, sh users)
or 	setenv  DISPLAY  nnn.mm.xxx.yy:0.0 	(csh users)

where nnn.mm.xxx.yy is the IP address of your terminal.

Both the TERM and DISPLAY environment variables can be set permanently in your .profile (Korn, bash, sh users) or .login (csh users).

2.4.3 Aliases

You may also wish to set up a system of aliases. An alias allows you to abbreviate or change the name of a command to something else. This can be automatically done in your .profile; for example:


	alias bye='exit'

will mean that in future, you will be able to type bye to leave UNIX rather than type exit.

To get rid of an alias temporarily, without deleting it from your .profile, type:


	unalias <aliasname>

for example:

	unalias bye

2.5 Logging Out

To leave UNIX, type


	exit
or 	bye 	(if it is set as an alias for exit in your .profile file)

If you have started up another shell after logging in, you will need to type exit to get out of the sub-shell before typing exit again to actually leave the system. If you get the message:

	You have running jobs

this means that you have left a process or job running in the background. To exit anyway and kill off the background job(s), type exit again. Use nohup to run the job if you don't wish it to terminate when you logout (see Section 6.3.1).

2.6 Changing Your Password

When you are registered to use the CS6400, you are given a login password. The first time you login you will be asked to change the password, thus:

	Choose a new password.
	Old password:

Enter your initial password again. This is just a security measure to check that you are the real owner of the userid, and not just someone who happens to be walking past your terminal! You will then be asked for the new password you have chosen:

	New password:

Since the system does not echo the password as you are typing it, you are asked to type it again for verification. The system checks that the two new passwords are identical. If they do not match, the process is abandoned, your original password is retained, and you can then try again.

If at any time you want to change your password, type:

 
	passwd

A valid password must have at least 6 characters, of which two or more must be alphabetic and one must be a number. It must not be the same as your userid, or an anagram thereof; and the new password must differ from the old one by at least 3 characters. If you choose an invalid password, UNIX will reject it. New passwords will be set to expire after a period of 98 days; when your password is getting rather old, you will get a warning message at login time informing you of how long it will be before it expires. Please note that UNIX is case-sensitive, so you must remember not only your password, but whether it was lower or upper case, or even a mixture! Although it is possible to use a non-alphanumeric character in a valid password, problems may be caused by using spaces, tabs, forward slash (/), backward slash (\), or commas, so we recommend that you do not use these.

2.6.1 Notes about passwords

Choose your passwords with care. Badly chosen passwords are the single most common cause of 'hacking' into University computer systems. When you choose a password, please keep in mind the following advice ...

DON'Ts

DOs If you find it difficult to remember your password, write it down and either carry it with you or place it in an envelope, seal it shut, and store in a safe place (a lockable drawer or at home). If you lose the written copy of your password change it immediately.

2.7 Forgetting Your Password

It is possible that you may forget your login password. As all passwords are checked for validity at login, if you type in the wrong password, the following message will appear on the screen:


	Login incorrect
	login:

At this point you will have to enter both your userid and password again. If you incorrectly type your userid, you will also get the same message. This is because the system does not check the userid until after you have already entered the password. Check carefully that both are correct and in the correct case (lower case for the userid, and whichever case - or a mixture - you originally set your password to).

After five attempts at entering the correct password, the system will disconnect your terminal from the network. If you get into difficulties, contact your MC representative and ask him/her to change your password to something new.

When asking for a password to be changed, please suggest a new one (which you will not forget). Once it has been reset to this new password, please change it immediately after logging on to the system, as two or three people will know the temporary one.

2.8 Command Format

While logged in to the CS6400 you will use various commands. These come from different sources, and may be: Most of the commands you will use, however, are UNIX commands, and the convention for the format of UNIX commands is defined and fairly rigorously observed. UNIX commands always look like this:

	command   [options]   [arguments]   [files] 

where:

	command		a word understood by the system that tells it to take some action.

	option 		generally, a letter known to the system which modifies the way in which the system 
			is to perform the command (although sometimes it may be a number).  It is 
			separated from the rest of the line by a space and usually preceded by a hyphen '-'.  
			If more than one option is to be specified, the letters may follow each other after 
			the '-'.

	argument	the information the command needs for it to perform the action; for example, it 
			might be a filename.  Not all commands require arguments.

	files		the files on which to use the command.

In the following example, the command name is ls (to list your files); the option is -a (list all your files in the current directory, including hidden or 'dot' files), and there are no arguments.

	 ls -a

To list every file in every directory, make sure you are in your 'home' directory and type:

	ls  -R

As stated, more than one option can be specified (you do not need to include the '-' each time); for example:

	ls -a -l
and 	ls -al

which will list all files (a), in long form (l), have the same effect.

2.9 Recalling Commands

If you have vi set as your 'visual' editor (this is done by setting up the environment variable VISUAL in your .profile file (Korn shell) or .login (Csh users)), the sequence <Esc> followed by k brings back the previously-entered command to the command line so you can edit or reissue it. You can edit this command using the vi editor sequences (see Section 4.2). If you keep pressing k, you get back, one by one, all the commands the system has stored in its command buffer. This is very useful if there are several complicated commands which you wish to reissue (with or without editing them).

It is also possible to set your visual editor to be emacs; in this case, the sequence Ctrl-P recalls previously-entered commands, and the usual emacs commands (Ctrl-B to move backwards one character, Ctrl-F forward one character, etc.) will work.

If you are using the bash shell, commands can be recalled using the up-arrow key.

There is also something called a history list. The system stores a list of the last commands you typed (the number it 'remembers' is set up by the system). To see your 'history list', use the command


	history

The history list might look something like this:

	3 	finger
	4 	set | more
	5 	echo $VISUAL
	6 	echo $HISTORY
	7 	echo $HISTSIZE
	8 	history
	9 	r -1	[Number 'one' not letter 'l' (ell)]
	10 	mna history
	11 	man history
	12 	history
	13 	ls
	14 	ls -a
	15 	f
	16	vi .job
	17 	more .job
	18 	history > myhist

You can 'replay' any of the commands in your history list by referring to it by the history number and using the r command; for example:

	r 11

will replay the command at number 11 on the history list, which in the above example is man history.

To replay the last command you typed, you can use the command


	r -1

which in this case will be history > myhist.

You can also use pattern matching; for example:


	r v

will find the last command which starts with a 'v', in the above example, vi .job.

2.10 How To Get Help

If you type the command helpme, you will receive limited help information. To obtain more detailed help and information , you should go to the MIDAS WWW Home Page. The Home Page looks like the following:

Simply click on the topic button for the type of information you require. As stated earlier, we are in the process of transferring information from the MIDAS gopher to the WWW. If you click on a topic which has not yet been transferred, the link will automatically take you to the gopher information.

If you wish to access information directly from the gopher information server, just type:


	gopher

You will be presented with a screen similar to the following:

Using the arrow keys, move the pointer to the required topic and press the RETURN key.

2.11 The man Pages

As well as the WWW and gopher information server on the CS6400, more detailed information on command syntax is given in the on-line man (manual) pages. The format of the man command is:

	man  [option]  [command-name]

This shows you all the information available on that topic, page by page. You may notice that at the end of a man page on a particular topic, it lists other man pages which may be of interest. For example, the man page for stat refers to the man pages stat(2) and environ(5). The number in brackets refers to the section. To get to the appropriate section of the manual, use the option -s followed by the section number; for example:


	man -s 2 stat

Unfortunately, the man pages are not written in a very user friendly way if you are unfamiliar with UNIX.

With all the various compilers and packages which have been installed, there are a number of manual pages with the same name, but referring to different items! By default, man will only display the first matching page it finds. If you are not sure which manual page you require, you can see them all by typing:


	man -a <name>

The -a option (all), causes all the manual pages with the name <name> to be displayed one after the other. (This is a rare occurrence - you should not need to use this option often.)

If you are not sure of the precise command name, but have an idea of the area for which you want information, you can use the apropos command; for example:


	apropos files

will give you a long list of commands relating to files. To see the list one page at a time, use more or pg to read it, that is:

	apropos files | more

Manual pages are searched according to the value of the env variable, MANPATH. This may be examined by typing:

	set  |  more

If you can change the order of the paths (by including MANPATH in your .profile/.login file), it will alter the order in which manual pages are displayed.

2.12 "Pipe" and Redirection Characters

In the previous examples, we have used the | character. The | character is called 'pipe'. Pipes are used to connect commands together, by redirecting the standard output from one command to the standard input of the second. For example:

	set | more

means 'send the output from set to the input required by more' - which will display the output one screenful at a time.

The < character can be used to redirect standard input to be taken from a file rather than from the terminal keyboard. For example, to sort the file animals and display the sorted list on the screen, you would type:


	sort   <  animals

The > character can be used to redirect standard output from the terminal screen into a file. For example, to sort the file animals and save the sorted list in the file animals.sorted, you would type:

	sort   <  animals   >   animals.sorted

2.13 Getting Out of Trouble

This section tells you what to do if you have problems. When you use UNIX, most of your problems will be caused by being unfamiliar with the commands. If you are new to UNIX, it is a good idea to keep a copy of this document close by. A summary of the more commonly used commands is given in Appendix A.

Sometimes, your session may 'hang up' mysteriously, and will not respond to any key pressed. It may be that you have started a process running which you wish to terminate. The interrupt sequence is set to Ctrl-C (you can change this in your .profile if you wish); you will then get back to the normal Korn shell prompt $.

If you press Ctrl-D by accident, you will log yourself off! Do not worry, your session will have terminated properly and you can just log back in again. This sequence can be changed, if required, and is described in the man page for ksh and csh.

Pressing Ctrl-Z by accident may stop jobs running. See Section 6.4 for details of how to restart stopped jobs.

If a mass of output is being displayed on the screen, pressing Ctrl-S will stop terminal output. To restart it, press Ctrl-Q.

3 File Handling and Directories

This chapter describes how to name your files and gives details about the commands you need to maintain your filestore.

3.1 Directories

UNIX systems have the potential for "infinite" levels of subdirectories. The top level of files on the system is called the root directory and is referred to simply as /; for example:

	ls /

will show you the files and directories in the root directory.

Other directories which you will come across are /bin, /etc, /work, and /tmp. /bin is where the system stores a lot of executable files. /tmp and /work are where scratch files can be written by anybody, though they will not be saved permanently; /tmp is cleared every 24 hours unless the files are in use. Some software, for example, sas, automatically writes scratch files to /work in order to prevent your own disc quota from being exceeded.

3.1.1 Your home directory

Your home directory is the top-level directory of your personal filestore; by default (this can be changed, if required, in your .profile/.login) this is the directory in which you will automatically be placed when you login to the CS6400. It is stored in the variable $HOME. Your home directory comprises:

	/home/cs6400x/userid

where:

	/home		is one level below the root directory.

	/cs6400x	this is the filestore system on which your files will reside.  x is one of the letters 'a' 
			to 'h'. This is purely for administrative purposes, as it would take a very long time 
			to back-up files if they all resided on a single filestore system (but see Section 3.2.1 
			below).

	/userid		is your cs6400 userid.

From your home directory, you can create, edit, and delete files; create subdirectories; issue jobs; etc.

3.1.2 Creating new directories

To create a new directory, you use the command:


	mkdir new directory name

For example:

	mkdir   jobfiles

will create the directory, jobfiles, under your home directory.

3.1.3 Changing directories

To go down a level, that is, to a subdirectory, use the change directory command, cd; for example, to change to your subdirectory called saspac, you would type:

	cd saspac

saspac then becomes your current working directory, in which you can then create, edit, or delete files. The current directory is referred to as .; thus typing:

	cd .

will change to your current directory (that is, it will do nothing).

To go up one level (that is, to the parent directory), type


	cd .. (like DOS, ". ." is the shorthand notation for the parent directory)

To go straight back to your home directory, type:

	cd

If, for example, you wish to create another subdirectory within a subdirectory, you simply change directory to the one under which the new subdirectory is to be created, and create a new one. For example, our user has created the subdirectory datafiles and wishes to create one called sasdata. He types the following commands:

	cd   datafiles
	mkdir  sasdata

3.2 Directory Path Names

A path name is a list of directories which tell you how to get from one place in the file system to another. There are two ways of specifying directory path names:

3.2.1 Absolute path names

Absolute path names comprise the full path (starting with /) to a given subdirectory. For example, our user has a subdirectory called datafiles, and within that subdirectory, another one called sasdata. The absolute path name of subdirectory sasdata would be:

	/home/cs6400a/abcdefg/datafiles/sasdata

If you have access permission to a file(s) belonging to a colleague, and you use absolute path names, you will need to know on which file system your colleague's files reside. It is recommended that, where possible, you use relative path names, as this alleviates the 'need to know' on which file system files reside (see Section 3.2.2); because absolute path names may not stay the same if, for example, system disc space is reorganized, it is better to use relative path names where possible.

3.2.2 Relative path names

Relative pathnames comprise a 'short-hand' version of the user's home directory. This can be specified in two ways:

The ~ character is a useful abbreviation, which the system interprets as 'the home directory belonging to' the specified userid. If no userid is specified, it relates to the issuing userid.

Once again, it is strongly recommended that you use relative path names instead of absolute, because the path name will always be correct even if the filestore system is reorganized.

3.3 What is My Current Directory?

To check which directory you are in, type either:

	pwd
or 	echo $PWD

This stands for print working directory, and will give you an absolute rather than a relative pathname. An absolute pathname starts with a slash (that is, from the root directory). For example, issuing the pwd command when in the subdirectory datafiles belonging to our user, abcdefg, will give:

	/home/cs6400a/abcdefg/datafiles

3.4 Listing Files in a Directory

To look at files in directories belonging to other users, or a higher level system directory, you need to specify the correct path; for example:

	ls saspac

will list the contents of the directory saspac, one level down from the current working directory.

	ls ~hijklmn/saspac

will list files in the directory saspac belonging to the user hijklmn, provided that you have permission to access his/her particular directory. If you do not have permission, you will get the error:

	[full-directory-path-name]:  Permission denied

3.5 File Naming Rules

File names comprise a string of characters, numerics, and symbols. Valid characters are a-z, A-Z, 09, or any non-alphanumeric character such as $ # @ + - : _ . (dot or fullstop). It is not a good idea, however, to start a file name with a special character as it may make it impossible to use normal directory commands (for example, rm and cp). One particularly nasty example is using a '-' (minus sign); the shell thinks it is an option, not a file name!! The maximum number of characters, etc., which may be specified in a file name is 255. Typical file names might be:

	runsir
	greaterlondon.listing
	gauss.run1

Sometimes it is essential that a file name has a suffix appended to it; a suffix comprises a dot or full stop, followed by a specific string. For example, a SASPAC command file must have a suffix .cmd, and SPSS command files should have a suffix of .sps (especially if you use the spssmcc command - see Section 5.4.3).

Below is a list of the types of file which have a mandatory suffix:


 Contents of file  	Mandatory suffix

 SASPAC command file  	.cmd
 FORTRAN 77 program  	.f or .for
 Pascal program   	.p or .pas
 C program   		.c
 C++ program   		.c or .cc or .C or .cxx

There are also special hidden files called 'dot' files, which as you might expect, start with a dot. They may only be seen by using the -a option to ls (see below). They can be edited in the normal way. They are usually scripts or executables (for example, your .profile), although most user-created scripts will not be dot files.

3.6 The ls Command

ls (list) provides information about your files. To use ls without any options, type:

	ls

The names of your files will then be displayed, for example:

	Mail   animals   bin   saspac   sasuser

Some of the more useful options to ls are:

	-a	shows all files in the current directory, including hidden ones ('dot' files)
	-l	shows files in the current directory in a longer form, including access modes
	-C	shows the listing in column form (default)
	-F	adds the suffix * to the filename if executable, / if a directory

The ls command is usually entered without arguments. ls can take a wildcard. The UNIX wildcard character for a string is asterisk, *. So to see all your files which start with an 'a', type

	ls a*

To see all files which have the extension .sps, type

	ls *.sps
 
You can list the contents of a directory other than the current one by typing

	ls   directory name

3.7 Examining the Contents of Your Files

A good way to look at a file without changing it is to use the more command; type:

	more   filename

This displays the file on the screen in pages. After each page, it will pause, giving a prompt and an indication of how much of the file has been displayed; for example:

	--More--(11%)

which means you have seen 11% of the file, and it is waiting for you to press either the space bar or the RETURN key. Pressing the space bar will give you another page; pressing the RETURN key will give you the next line. At the end of the file, more will automatically finish and redisplay your default shell prompt ($ for Korn; % for C shell). If you wish to quit from more before you have finished reading everything, press q (Rtn).

Another way to look at a file is to use the pg command. Type:


	pg   filename

This will also display the file a screenful at a time, but the prompt after each page will look like this:

	:

when it is expecting you to press the RETURN key. At the end of the file, pg will display

	(EOF):

which stands for end-of-file. It waits for you to press RETURN before exiting to your default shell. As with more, pressing q (Rtn) will allow you to quit before the end of the file.

To list the contents of a file on the screen you can also use the cat command:


	cat   filename

which does not pause between pages. If you want the listing to pause after each screenful, you can type either:

	cat   filename  |  more

or 	cat   filename  |  pg 

3.8 File and Group Access Permissions

3.8.1 Assigning files to another group

When you are registered on the CS6400, your userid is assigned to a group for accounting and filestore management purposes. To see which groups you are a member of, type the command:

	groups

For our user, the system might then display the following:

	ab abchp ab

The first group denotes the default group associated with their files - that is, every time they create a file, it will belong to the group ab. This group is also the group against which their computer usage will be accounted.

If you and colleagues are working on a project, you will have been given a filestore group name so that you can access each other's files. In the example above, abcdefg is associated with the filestore group abchp. Any files they create which are to be accessible to other members of their group need to be assigned to the group abchp. This is done using the chgrp command, which takes the following format:


	chgrp  groupid   filename

where:

	groupid		is the name of the group to which the file(s) is/are to be assigned.  You must be a 
			member of that group.
 
	filename	is the name of the file(s) to be assigned to groupid.  You must own the file(s) 
			specified.

Suppose our user has created the file, chpproj.data1, which they wish other users associated with their project to be able to access (and who are also members of the group abchp), they type:

	chgrp abchp  chpproj.data1

Once a file or files have been assigned to the correct filestore group, you can give read, and/or write, and/or execute permission to other users.

Note: The chgrp command also applies to directories as well as files. In order to give others in your group access to files within any sub-directory on your userid, you will need to ensure that all the relevant directories below the home directory are owned by the correct group, and have a minimum of execute permission (see Section 3.8.2 for details) set for the group, as well as checking that the relevant permissions are set correctly for the file(s) contained within those directories. You will also need to change the permissions for your home directory.

3.8.2 Granting file access permissions

File access permissions can be changed at three levels of access, read or copy (r), write (w) - which includes edit and delete - and execute (x), and for three classes of user, the owner (u), the group (g), and all others (o).

The command which does this is chmod (which stands for change mode), and it can be used in two different ways, to achieve the same result; these are:

(a) Symbolic Method

This method may be easier to remember if you are unfamiliar with UNIX. It uses the system of symbols (or letters) to denote types of access, that is, r, w, and x.

When you get a long listing of your files using ls with the -l option, the file permissions are shown in the symbolic way:


total 10
drwx------   2 abcdefg  ab           512 Jan 19 15:43 Mail
-rw-------   1 abcdefg  ab            64 Feb 19 10:46 animals
drwxr-xr-x   2 abcdefg  ab           512 Jan 21 15:25 bin
-rwxr-----   2 abcdefg  ab         11256 Feb 14 16:21 chpdat
drwx--x--x   3 abcdefg  ab          1024 Jan 19 15:31 saspac
drwx--x--x   2 abcdefg  ab           512 Dec 22 16:04 sasuser

The 'd' in the first column means that the file is a directory; the following nine characters indicate access permissions for the file (read, write, execute) and are shown in the order user, group, other. For example, the bin directory has read, write, and execute permissions for the user, but only read and execute permissions for the group and others. (The hyphens denote lack of permission). The abcdefg denotes the owner, and the ab the group under which the files were created. The next figure along represents the number of bytes of disc space the file is taking up. After this comes the date and time when the file was last modified, followed by the filename itself.

To give a permission to a file, the syntax is:

To remove permission, a minus is substituted for the plus sign, for example, to remove execute permission for other users, you would type:

	chmod   o-x   filename

(b) Octal Method

This method uses octal numbers to denote both access permission and class of user. The types of access, r, w, and x are individually assigned a number of 'points'. These points are added together for each class of user, giving a three digit code (each digit representing user, group, and others) thus:


	777
or  	711

Read access counts as 4 points, write as 2 and execute as 1. Therefore a file having permissions of 777 means that user, group, and others can all read, write and execute it. 711 means that all classes of user can execute it, but only the owner can read or write to it.

To give yourself r, w, and x permissions, people in your group read and execute, and others no permissions at all, you would type:


	chmod 750 filename

This system also works for directories. However, execute permission in the case of a directory means that a user can change directory to it and execute any executable files it contains (subject to those files having the necessary permissions), but not list the files it contains. To do this, read permission must be granted.

Note: If you have transferred files to the CS6400 using ftp, all these files will have read and write permissions for all other users; that is, files transferred from other systems may not have the permissions you would expect! It is therefore recommended that you issue the chmod command to reset these permissions.

3.9 Printing Your Files

Hardcopy listings of files can be obtained using printers on 'networked' machines on JANET, using either the lp or lpr commands; the latter is now recommended as being more reliable and is used in the following examples.

3.9.1 Using networked printers

To get hardcopy listings of files on a networked printer (that is, one at your site/department) which is attached to a UNIX machine, use the command:

		lpr -P printername -S servername filename

for example: 		lpr  -P  lp  -S   aardvark.lonuni.ac.uk   friday.data

This may also work for machines running operating systems other than UNIX, if they are using the correct (Berkeley, or BSD) print protocol (contact your MC site representative if you are not sure). It may also be necessary for the computing service at your site to arrange for remote print requests from the CS6400 to be accepted by the local machine.

You can include two environment variables, PRINTER and LPD_SERVER, in your .profile file, to avoid having to type in the full printer and server addresses each time, thus:


	export PRINTER=printername
	export LPD_SERVER=servername

Next time you log in, and in the future, the command

	lpr filename 

will then work.

For further information and details of the options to lpr, type:


	man  lpr

The lpq command shows the status of a printer, for example, how many jobs are in the queue. The format is similar to that of lpr, that is:

	lpq -P printername -S servername

If you have set up the environment variables PRINTER and LPD_SERVER, the command lpq without any arguments will show the status of your default printer.

3.9.2 Cancelling a print job

A print job can be cancelled using the lprm command; this can be done in several different ways. Each time you use lpr to print a file, that job is given an identifier, or 'job number'. You can see all the jobs you have submitted with their job numbers by using the lpq command. To cancel a print job using its job number, the syntax is:

	lprm  job number

For example, if the output from lpq looks something like this (it will vary slightly according to the system at the remote end, but should always have a 'Job' column):

Queue 	Dev 	Status 	Job 	Files  	 User  	 %	Size
------- ---- 	------- ---- 	-------  ------- --- 	----
lp  	lp 	active 	328 	.profile abcdefg 33 	5016 bytes

So, the command

	lprm  328

will remove that job. Typing lprm on its own with no arguments, will remove the currently active job provided that it is owned by you.

3.10 Sharing Files

If you want another user to be able to read or copy one of your files, you have to set the appropriate access permission on the file using chmod (see Section 3.8.2). Note that all the parent directories, including your top level directory, need to have execute permission granted in order for them to look at the file.

As stated, your home directory is usually just referred to as $HOME, but when accessing someone else's home directory, you will need to specify the path name, for example:


	~hijklmn

Directories underneath hijklmn's home directory, for example, saspac, can be referred to in the same way, for example:

	~hijklmn/saspac

To copy a file from this directory, for example, test1.cmd, you use the cp command as follows:

	cp   ~hijklmn/saspac/test1.cmd   test1.cmd
or 	cp   ~hijklmn/saspac/test1.cmd   .

This will put a copy of hijklmn's file test1.cmd into your current working directory (denoted by .).

3.11 Erasing Files and Directories

3.11.1 Erasing files

The command to erase files is rm (remove). The format of the command is:

	rm [option] filename(s)

For example, to erase the file friday.data from your current working directory, type:

	rm   friday.data

This command, like ls, can take a wildcard. Thus, to delete all the files in your current working directory which start with friday., type:

	rm   friday.* 

To delete all files starting with fr, type:

	rm   fr*

To delete all files with the suffix .data, type:

	rm   *.data

Similarly, to erase all the files in your current working directory (except hidden files), type:

	rm   *

(this should obviously be used with extreme caution)! It is strongly recommended that you use the -i option with rm as this asks you to confirm that each file is to be deleted. For example:

	rm   -i   fr*

You could edit your .profile file to set this up as an alias called say, del, to force this as a default by adding the line:

	alias del="rm  -i"

3.11.2 Deleting directories

As with DOS, directories can be deleted using the rmdir command, provided that they do not contain any files; for example:

	rmdir  fred 

Alternatively, you can recursively delete a directory and all the files and subdirectories it contains by typing:

	rm   -r  directoryname

Be very careful when using this command! It is strongly recommended that you read the section on Backup and File Recovery (Section 3.12 below) before attempting to delete a directory recursively.

3.12 Help! I've Erased the Wrong File

If you inadvertently erase the wrong file do not despair; all may not be lost. Provided that the erased file was backed up overnight at some point, a copy of it will be available. If, however, the file was created the same day as you erased it, you will not be able to restore it, as there will be no back-up copy available.

Following the move to Solaris 2.5, we have installed a new Backup and Recover system called, Legato Networker (Solstice) Backup. This system allows users to retrieve erased files themselves, without having to involve members of staff at Manchester Computing.

There are two interfaces for Solstice Backup:

3.12.1 X-windows access

To invoke the GUI interface to Solstice Backup, type:

	nwrecover   &

This presents you with a window entitled 'Solstice Backup Recover'. By default, the 'directory tree' displayed will be your MIDAS home directory. What follows is a very brief description of the action you take.

(a) Browse the index of files for the required file name(s) - this is like using MS Windows 'File Manager' where you 'double click' on folders (directories) displayed on the left-hand side of the screen, to view files displayed on the right-hand side of the screen.

(b) Mark the required directory/file(s) for recovery by clicking the box to the left of the directory/file name(s).

(c) Click on the Start button to start the recovery.

For full details of the nwrecover command, see CSS 204 - 'Legato Networker (Solstice) Backup' (Second Edition), or type:


	man  nwrecover

3.12.2 Line mode access

To restore a copy of an erased file, use the command:

	recover [options]  path_name
or 	recover

The latter command puts you into an interactive session, which displays the backed-up files as though they were in your home directory. The ls, cd, pwd, etc., commands are available for you to move around the file system. By doing this, you move to the directory which contained the file(s) you accidentally deleted and then list the 'backup' files by typing ls. Wildcards, etc., can be used in the normal way. Once you have located the file(s) you wish to recover, type:

	add   fileid(s)

to add a file to the list of those to restore. You can now move to another directory and add other files to the recover list as required. To view the contents of the recover list, simply type:

	list

Once you are satisfied that all the files you wish to recover are in the list, type:

	recover

For a full description of the recover command, including a description of the options available, see CSS 204 - 'Legato Networker (Solstice) Backup' (Second Edition), or type:

	man   recover

3.12.3 Further Information

A fuller description of Legato Networker (Solstice) Backup is contained in the note, CSS 204 - 'Legato Networker Solstice Backup: New Backup and Recover Interface' (Second Edition), copies of which may be obtained from the Information Point at Manchester Computing. This document provides worked examples of how to recover files in X-Windows GUI and Line Mode access.

In order to prevent accidentally erasing important files (for example, data files), the safest option is to change the owner's permissions on the relevant file using the chmod command.

Removing the write permission by typing


	chmod   u-w   very-important.data

will mean that very-important.data can no longer be edited or deleted (without confirmation) by the owner, until write access permission is re-enabled using chmod. The file can, of course, still be read, or executed if it is a script.

3.13 Copying Files

Use the cp command to copy your files, as follows:

	cp  infilename   outfilename

where infilename is the complete file name of your original file, and outfilename is the complete file name of the copy you are creating. If outfilename already exists, its contents will be overwritten without warning.

You can copy a file to another directory using the format


	cp   infilename   directoryname/outfilename
or  	cp   infilename   directory

if you wish to keep the filename the same in both directories.

Type:


	man  cp

for more information.

3.14 Renaming Files

To rename files use the mv or move command, which takes the following format:

	mv  oldfilename   newfilename

To move the file to another directory, type

	mv  oldfilename   directoryname/newfilename
or 	mv  oldfilename   directoryname

to keep the same name in the new directory. Once again, if newfilename already exists, it is overwritten without warning.

3.15 Comparing Two Files

To compare the contents of two files, use the command

	cmp file1  file2

cmp will only give any output if there are differences, with a summary of where these differences are located in the files. Alternatively, the diff command will give a more detailed output showing exactly what the differences are. See the man pages for cmp and diff for more details.

4 Creating and Editing Files

There are numerous editors available for UNIX systems. Unfortunately, those supplied as part of the standard configuration are not necessarily the most powerful or the easiest to use; they are, however, likely to be available almost universally across UNIX systems.

The editors described below are called vi and joe. It is recommended that (if you are new to UNIX), you use joe to create and edit files, and vi for command line editing. You may, of course, use vi to create new files and to edit existing files if you prefer; you can move backward and forward freely, using commands and cursor keys to modify the text in any order.

joe and vi work with a copy of your file in memory while you are editing. No changes are made to the original file until you save your edit. You can choose whether to save your edit to the original file, or to a new file or you can choose to scrap the edit completely. A fact card for joe is provided in Appendix B of this guide, and a brief description is given in Section 4.1. A fact card for vi (HPS 201) is also available from the MC Information Point and a brief description is given in Section 4.2.

Another editor on the CS6400 is gnu emacs. Please note, however, that this editor has little support at Manchester Computing.

4.1 The joe Editor

joe is the text editor which is recommended by Manchester Computing for file creation and editing. It is easy to use and is available on all the UNIX computers operated by MC.

All commands to joe involve pressing the Control key (generally marked Ctrl on most keyboards) and a letter simultaneously; for example, to move down a screenful of text, you would press Ctrl and V together. It does not matter whether the V is in upper case or lower case; for clarity in this document, however, Ctrl will be indicated by "^" (caret) and all letters will be in upper case. Thus Ctrl V will be represented as "^V". Some commands require a second letter to be pressed. In these cases, the Control key does not need to be held down when entering the second letter.

joe has a comprehensive Help system, and you may choose which Help screen you need. The message, Ctrl-K H for help, is always displayed in the top right-hand corner of the screen when you invoke joe.

4.1.1 Invoking joe

To create or edit a file, you need to invoke joe using the following command:

	joe   filename

If the specified file does not already exist a new file will be created. If the file already exists, a copy of the file will be put in the edit buffer.

Supposing you type:


	joe  new.f

The screen will change to look like this:

The top and bottom lines are highlighted. If you are new to joe, it is useful to invoke the Help screen, as it can act as an aide memoire for the various commands. Pressing ^KH will give you the following display:

You can now enter your text. Each time you wish to go onto a new line, simply press the RETURN key.

4.1.2 Terminating the edit

The usual method of saving the information is to type

	^KX

This writes out your work to the file you named at the start of the edit, and quits. If it already exists, the file will be overwritten.

The following are also valid


^KE	quit the current edit and edit another file (you will be asked if you wish to save 
	any changes you have made to the current file, and then prompted for the new 
	name)
^KD  	write out the current file to a different file (you are prompted to supply the new 
	name).  A back-up copy of the original file is made; this comprises the file 
	name followed by a "~".
^C  	quit without saving changes
^KQ	is a more refined way of exiting.

4.1.3 Editing an existing file

You call up joe to edit an existing file in the same way as if you were creating a new one. For example, to edit new.f, type

	joe new.f

as before.

The screen will now look something like this:

You can now use the following techniques to add, modify, and delete text.

(a) Moving Around

Here is a summary of how to move through the file:


	      cursor left one character
       	      cursor right one character
	^Z  	move left one word
  	^X  	move right one word
  	^A  	move to the start of the current line
  	^E  	move to the end of the line
              cursor up one line
       	      cursor down a line

  	^U  	move to the previous screen
  	^V	move to the next screen (think of 'V' as a downward-pointing arrow)
  	^KU  	move to the top of the file
  	^KV  	move to the bottom of the file
  	^KL  	move to a specific line number (you are prompted for the line number).
  	^KF  	locate the line containing a given string.  You will be prompted to enter 
		the required string.  Options for the search are:
      		
		B, I, and R can be combined, if required.

(b) Changing Strings

To change strings use the following command:


	^KF

The following prompt will appear at the bottom of the screen:

	Find (^C to abort): 

Enter the string to be changed and press RETURN. The following prompt will appear at the bottom of the screen:

	(I)gnore case (R)eplace (B)ackwards NNN (^C to abort): 

Type R. The next prompt will be:

	Replace with (^C to abort):

Enter the new string and press RETURN. joe will highlight the string to be changed, and display the following at the bottom of the screen:

	Replace (Y)es (N)o (R)est (^C to abort)?

Type Y (if you want to replace this occurrence of the string) or R (if you want all occurrences of the specified string to be replaced with the new one). For example, to substitute "new tricks" for all occurrences of "old dogs", you would enter the following sequence:

	^KF
	Find (^C to abort): old dogs
 	(I)gnore case (R)eplace (B)ackwards NNN (^C to abort): r
 	Replace with (^C to abort): new tricks
 	Replace (Y)es (N)o (R)est (^C to abort)? r

(c) Moving text (cut and paste)

To move or copy text within a file you define the start and end of a block using:


	^KB	define start of block (then move the cursor and ...)
  	^KK	define end of block

Move the cursor to the point where the block of text is to be moved/copied and enter:

	^KM	move the block to the new position
  	^KC	copy the block at the new position

(d) Other Commands

 	^D		delete the current character (at the cursor position)
  	Del key 	delete the previous character (delete left)
or 	Backspace

	^W		delete next word
  	^O		delete previous word

	^J		delete to end of line
  	^Y		delete a whole line

	^KR		insert text from another file

There are many other features to joe; these are given in greater detail in Appendix B of this manual.

4.2 The vi Editor

The purpose of this section is to show you a simple way to create a new file or edit an existing file. It also introduces how the vi editor works. Once you are familiar with the basics, there are shortcuts to simplify editing tasks. You create new or edit existing files by the following procedure:

4.2.1 Invoking vi

To create or edit a file, you need to invoke vi using the command:

	vi  filename

If the specified file does not already exist a new file will be created. If the file already exists, a copy of the file will be put in the edit buffer.

Supposing you type:


	vi  new.f  

The screen will change to look like this:

A tilde (~) displays on each line to indicate the file is empty, and the words "new file" appear at the bottom of the screen. The cursor will be at the beginning of the file, waiting for the first command.

If you invoke the editor with an existing file name, the first screen of text in the file will be shown. The number of lines and characters in the file and the file name are printed at the bottom of the screen.

When you invoke vi, you are in command mode. You must get into input mode before you can enter text.

4.2.2 Using vi

Having started vi as described in Section 4.2.1, press

	i

to get into input mode. You will not see any changes to the display, but you may now type in a line of text. Press RETURN and continue on the next line, and so on, as illustrated below.

To come out of input mode, press <ESC>. (It doesn't do any harm if you press <ESC> several times: the terminal may beep, but the file is not changed).

4.2.3 Terminating the edit

The usual method of saving the information is to type

	:wq

in command mode. The colon ":" is significant and mandatory; it is also essential that there is no space between :wq. This writes out your work to the file you named at the start of the edit, and quits. If it already exists, the file will be overwritten.

The following are also valid


	:wq filename	write out to named file and quit
 	:x   		write out to file and quit (like :wq)
 	:x  filename	write out to named file and quit
 	:q		quit if you haven't made any changes
 	:q!		quit without saving changes (this will always get you out of vi)
 	:w		write out work in progress
 	:w  filename	write out work in progress to named file

4.2.4 Editing an existing file

You call up vi to edit an existing file in the same way as if you were creating a new one. For example, to edit new.f, type

	vi new.f

as before. The screen will now look something like this:

You can now use the following techniques to add, modify and delete text.

(a) Moving Around

Here is a summary of how to move through the file, while you are in command mode:


	h or          cursor left one character
  	l or          cursor right one character
  	k or          cursor up one line
  	j or          cursor down a line

  	^f  		scroll forward (down) a full window
  	^d  		scroll forward (down) half  a window
  	^b  		scroll backwards (up) a full window
  	^u		scroll backwards (up) half a window

	G  		go to the bottom of the file  must be capital G
  	1G  		go to the top of the file  must be capital G

	n             move forward n lines
  	n             move back n lines
  	nG  		go to line n

(b) Locating Strings

To find the next occurrence of a string you can type (in command mode - that is, press <Esc> first):


	/string

and press RETURN.

To find the previous occurrence, type:


	?string

If you need to search for a string which contains a slash or other special character, precede it with a backslash (\). Thus, to find "//", you would have to specify /\/\/ !!

To repeat the last search, press


	n  	to search in the same direction (the letter n, not a number)
  	N  	to search in the reverse direction

and the RETURN key.

(c) Changing Strings

To change strings use the following command:


	:n1,n2s/string1/string2/g

where:

	n1		Specifies the number of first line to be subject to the changes.

  	n2  		Specifies the number of the last line to be subject to the change.  Use 
			a dollar sign ($) to denote end-of-file.

  	string1  	Is the existing text which is to be changed.

  	string2  	Is the text which is to replace the existing text.

  	s  		Means 'substitute'

  	g  		Means 'global'

For example, to substitute 'new tricks' for all occurrences of 'old dogs', you might type

	:1,$s/old dogs/new tricks/g

(d) Other Commands

All of the commands in this section require you to be in command mode. Remember to press <Esc> first!


	^		move the cursor to the beginning of line
 	$		move the cursor to the end of the current line

  	x		delete a character
 	X		delete 1 character backwards

  	dd		delete a line
 	cc		replace a line

	yy		yank (that is, remove) the current line
 	p (little p) 	put any yanked text after the current line
  	P (big P) 	put any yanked text before the current line

After giving any of the following commands, vi switches to input mode:

 	I		insert text before cursor (input mode)
  	a		insert text after cursor (append mode)

 	I		insert text at beginning of line
  	A		insert text at end of line

 	o		insert new line after current line (open mode)
  	O		insert new line before the current line

  	r  		enter overstrike mode (replace mode - single character only)
 	R  		enter overstrike mode (replace mode - more than zero characters)

Having entered text after one of the above commands, remember to press <Esc> again to return to command mode.

4.2.5 Recovering from interrupted sessions

If there is a problem with your session whilst you are in vi, and for some reason you are forced out of the editor without being able to save work in progress, you may try restarting vi specifying the -r recovery switch:

	vi -r filename

where filename is the name of the file you were editing. With luck, this will take you back to where you were, more or less, when you were interrupted.

5 Access to Packages

A variety of packages and tools are available on the CS6400 to access and manipulate datasets. Below are listed the commands needed to access some of the more popular packages (more information is available on the WWW or gopher server, see Section 2.3.2). They can all be accessed in interactive mode, in background mode, or submitted as batch jobs.

5.1 SASPAC

The 1991 and 1981 Census of Population for the UK, and the Northern Ireland 1991 Census data are invoked using the command saspac, which takes the form:

	saspac [filename]

if filename is specified, saspac will execute the SASPAC commands in a named file, which must have the suffix .cmd. If filename is omitted, the interactive version of SASPAC will be invoked. This bears close resemblance to the PC version of the package, providing a menu interface to the usual SASPAC commands, and is very easy to use.

5.1.1 An example job

The following job, stored in the file glndrun1.cmd, reads in data from a system file and creates a formatted output file:

	input system file name=glndiclh
  	* 100% local base statistics
  	* (ward level) for Inner London
  	input framework file name=tlbs132
  	print tables l50
  	* l50 = Country of birth: household heads and residents
  	output print file name=mylist
  	end
  	finish

This can be run by typing:

	saspac   glndrun1

5.1.2 Further information

Further information about running SASPAC is provided in CSS 603 - 'A Guide to the SASPAC Interactive Menu System', CSS 604 - 'A Guide to Accessing the Census 1991 (SAS/LBS) and 1981 (SAS) Data using SASPAC', and CSS 635 - Guide to the Northern Ireland 1991 Census (SAS)', copies of which are available from the MC Information Point, or contact your site representative or Liaison Officer.

5.2 SAS

To run SAS, use the command sas, which has the following syntax:

	sas [filename]

where:

	filename 	is the name of the file containing the SAS commands.  This file must have the 
			suffix .sas.  If filename is omitted, the interactive (display manager) interface 
			version of SAS will be invoked.

The log file is written to a file with the same name as the input file, but with a suffix of .log. Similarly, the listing file is written to a file with the same name as the input file, but with an suffix of .lst.

To use the interactive X-windows interface to SAS, you must:

(1) access the CS6400 using X-windows (for example, from an X-terminal or using eXceed)

(2) if necessary, have your terminal type in the TERM environment variable set to xterm, using the command


	export TERM=xterm (Korn, bash, sh users)
  or 	setenv TERM xterm (csh users)

either in your .profile or .login, respectively, or at the terminal before entering SAS (this may be done automatically)

(3) have your IP address stored in the DISPLAY environment variable, using the command


	export DISPLAY=AAA.BB.CCC.DD:0.0 (Korn, bash, users)
  or 	setenv DISPLAY  AAA.BB.CCC.DD:0.0 (csh users)

in your .profile or .login, respectively, or at the terminal before entering SAS, where AAA.BB.CCC.DD is your IP address (again, this may be set automatically, depending on how you access MIDAS using X)

To start up SAS (assuming you have logged on with a properly configured terminal), use the -fsdevice option:


	sas   -fsdevice   x11
 or 	xsas

5.2.1 An example job

The following job, stored in simple.sas, analyses data in the SAS dataset MYLIB.FAMILY:

	TITLE 'Simple SAS program file';
 	* Set width of log and listing files for screen output;
 	OPTIONS LINESIZE=78;
 	* Specify libref for the permanent SAS formats;
 	LIBNAME LIBRARY 'famdir';
 	* Specify libref for the permanent SAS dataset;
 	LIBNAME MYLIB 'famdir';
 
 	* One-way table;
 	PROC FREQ DATA=MYLIB.FAMILY;
  	TABLES MARSTAT SEX;
 	RUN;

 	* One-way table;
 	PROC FREQ DATA=MYLIB.FAMILY;
  	TABLES MARSTAT*SEX;
 	RUN;

	* Sort data by sex into new data set;
 	PROC SORT DATA=MYLIB.FAMILY OUT=FAMBYSEX;
  	BY SEX;
 	RUN;

	* Univariate statistics;
 	PROC UNIVARIATE DATA=FAMBYSEX PLOT;
  	VAR HEIGHT;
  	BY SEX;
 	RUN;

To run this, simply type:

	sas   simple

5.2.2 Further information

Further information about running SAS is provided in CSS 601 - SAS, copies of which are available from the MC Information Point, or contact your site representative or Liaison Officer.

5.3 SIR

To run SIR, use the command sir-dbms, which has the following basic syntax:

	sir-dbms db=[filename] /in=[filename] /out=[filename]

where:

 	db=filename is the name of the file containing the SIR database.

 	in=filename is the name of the file containing the SIR commands.

 	out=filename is the name of the file containing the SIR output.  If this parameter is omitted, 
	the output is automatically written to the standard output file, stdout.

There are about sixty other possible command-line parameters! For further details, refer to the SIR manuals.

For command files in a different directory, use backslashes and quotes; for example:


  	sir-dbms in=\'~abcdxyz/temp/mypalsjob.sir\'

Interactive SIR with the menu interface is obtained with:

  	sir-dbms

but the menu interface will only work if you are on a VT100-type terminal. Since most software is capable of emulating a VT100, it should only be necessary to set the TERM environment variable, that is, you will need to type:

	export TERM=vt100 (Korn, bash users)

 or 	setenv TERM vt100 (csh users)

before using the SIR menu interface. Before proceeding, you need to set the terminal type as above if your terminal is set to xterm.

To get interactive SIR with a prompt (as in SIR version 2) type:


  	sir-dbms ia

For databases in other directories, use the p parameter, for example:

  	sir-dbms  p=\'~abcdefg/temp\'

All the other SIR modules are also available in addition to dbms, for example:

  	sir-sql

5.3.1 An example job

The following job, stored in the SIR command file, mymstat1.sir, retrieves marital status, stored in the SIR database, mysirdb, and produces a frequency table. The output will automatically be sent to stdout:

    	retrieval
    	. process cases
    	.   process rec person
    	.     get vars marstat
    	.     perform procs
    	.   end rec
    	. end cases
    	frequencies integer=marstat (1,7)
    	end retrieval  

This job can be run by typing:

  	sir-dbms   db=mysirdb   /in=mymstat1.sir

5.3.2 Further information

Further information about using SIR with the Government Survey Data is provided in CSS 606 - General Household Survey, CSS 608 - Labour Force Survey, and CSS 609 - Family Expenditure Survey, copies of which are available from the MC Information Point, or contact your site representative or Liaison Officer.

5.4 SPSS

SPSS can be invoked in four different modes:

5.4.1 Full interactive Manager mode

For the Full Interactive Manager mode, use

  	spss +m

By default, the Manager version of SPSS only assumes 512 Kbytes of workspace. If SPSS tells you that you have run out of memory, you can request extra on the UNIX command line using the -s switch. For example:

  	spss  +m  -s32m

requests 32 Mbytes of workspace.

This interface is similar to that of SPSS PC+, that is, it has optional menus for creating command files. To display help with key combinations, press


  	Esc-1 then select manageR help

(that is, press the Escape and numeric 1 keys together). To toggle the menu system, press

  	Esc-m

To switch between the input window (at the bottom of the screen) and the output window (at the top of the screen), press

  	Esc-s

To exit from interactive Manager mode, press

  	Esc-0 (zero), and select Exit.

5.4.2 Interactive Motif mode

To use the interactive Motif interface, you must be using X-windows, and have set the TERM and DISPLAY environment variables correctly (see Section 2.4.2 for more details). If you attempt to run the Motif interface without having set TERM and DISPLAY, you will be defaulted to the interactive Manager interface.

To run the Motif interface on an X-terminal, enter


  	spss   &

This command may also be used to run the Motif interface from a PC using eXceed, but the SPSS windows will extend beyond the screen boundaries.

You may prefer the command line


  	spss   -name vgaSPSS   &

or its MC abbreviation

  	spssvga   &

This draws SPSS windows which fit within the small screen boundaries, but the text characters are rather small!

By default, the Motif version of SPSS only assumes 1512 Kbytes of workspace. If SPSS tells you that you have run out of memory, you can request extra on the UNIX command line using the -s switch, for example


  	spss   -s32m   &
 or 	spssvga   -s32m   &

start off Motif sessions with 32 Mbytes of workspace.

If you wish to start the Motif interface with an SPSS system file in the Data Editor window, then use either


  	spss   data fred.sys   &  (if the extension is not .sav)
  	spssvga   data fred.sys   &

 or 	spss   fred.sav   &
	spssvga   fred.sav   &

The Motif interface assumes you have a monitor with more than 16 colours. It runs adequately on a 16-colour VGA monitor, but some things will not be visible, particularly the high-resolution graphics. The graphics can still be printed, however.

5.4.3 Non-interactive mode

To run SPSS in non-interactive mode, the spss command takes the following form:

  	spss -m filename

This will execute the SPSS commands in a named file, which should (for the sake of clarity and portability) have extension .sps. Output by default appears on the screen, but may be redirected with > filename, for example:

  	spss   -m    myjob.sps   >   myjob.lst

takes the input from myjob.sps and writes the output to myjob.lst. Error and other messages may also be redirected from the standard error file by using 2> filename; for example:

  	spss   -m    myjob.sps   >   myjob.lst   2>   myjob.err

If you always want your standard output and any error messages to be sent automatically to files beginning with the name of your input file (minus the extension) followed by .lst and .err, respectively, Manchester Computing has written a script called spssmcc which satisfies this procedure. The command takes the form:

  	spssmcc filename

where:

 	filename 	is the name of the file containing your SPSS commands.  This file must have 
			an extension of .sps.  Standard output will be written to filename.lst and 
			any error or other messages will be written to filename.err.

By default, the noninteractive version of SPSS called up via the spss command only assumes 1512 Kbytes of workspace; the spssmcc command asks for 10 Mbytes. If SPSS tells you that you have run out of memory, you can request extra on the UNIX command line using the -s switch. For example:

  	spss  -s32m  -m   myjob.sps   > myjob.lst   2> myjob.err
 or 	spssmcc   -s32m   myjob 

start off jobs with 32 Mbytes of workspace.

5.4.4 Prompt mode

To run in prompt mode, enter

       	spss  -m

To leave prompt mode, enter

       	finish.

5.4.5 An example job

The following job, stored in mynewsys.sps, reads in an SPSS export file, computes a new variable, and writes out the resulting dataset as a system file. The standard output is redirected to mynewsys.lis and any error messages are redirected to mynewsys.err.

 	import file='mydata.spsspff'.
 	compute agemths = 12 * ageyears.
 	save outfile='mydata.sys'.
 	finish.

This job can be run, by typing either:

  	spss   <   mynewsys.sps   >   mynewsys.lis   2> mynewsys.err
 or 	spssmcc   mynewsys

5.5 GLIM

To run GLIM, use the command glim, which has the following syntax:


  	glim [pip=input_file_name] [pop=output_file_name]

where:

 	input_file_name 	is the name of your file which contains the GLIM commands.  This file 
				should have the suffix .glim for clarity and portability.  If 
				input_file_name is not specified, the interactive version will be 
				invoked with prompting.

 	output_file_name 	is the name of the results file to be written to (or the screen if no output 
				file is given).

5.5.1 An example job

The following job reads in two variables from a raw data file, and fits a simple regression model:

 	$units 50$
 	$data x y$
 	$dinput 'myfile.data'$
 	$yvar y$
 	$fit x$
 	$stop$

These instructions are stored in the file myglim1.glim and can be run by typing:

  	glim  myglim1.glim

The output will be written to the screen.

5.6 NAG

The NAG library is available on the Sun CS6400. If your FORTRAN code makes use of any NAG routine, you must include the NAG library in your f77 command, for example:

	  f77   myprog.f   -lnag

6 Creating and Running Shell Scripts

6.1 Introduction

Shell scripts are the UNIX equivalent of a command file on DOS. They allow you to specify several commands in a single file and then run them. When you execute commands, such as saspac, sas, spss, etc., you are actually running a shell script.

The execution of a shell script is carried out in a sub-shell. It is important to note that the default sub-shell invoked for the execution of scripts is not automatically the same as the shell being used by you. The most commonly used shell for writing shell scripts is the Bourne Shell (sh). (Korn shell - the default shell on the CS6400 - is a backward-compatible extension of the Bourne shell). If you use the C shell, the differences between the C shell and the Bourne shell are sufficient that scripts written for one shell rarely execute correctly with the other.

As a rule, the first line of every shell script should indicate which shell will be used to interpret it. For example, to cause the script to be executed using the Korn shell, the first line should contain:


  	#!/bin/ksh

and for C shell it should contain:

  	#!/bin/csh
	

6.2 Creating Scripts

Invoke your preferred editor and give the script a meaningful name. The following examples vary in complexity.

6.2.1 A very simple script

This Korn shell script, hello, prints out a message on the screen.

   	#!/bin/ksh
  	print "Hello world"

6.2.2 More complex scripts

(a) This script, wottime, checks the hour of the day (found by executing the UNIX date command and fiddling with the output), and then prints an appropriate message.

  	#!/bin/ksh
 	DATE=`date`   		# store the date in shell variable DATE
      					# e.g.     Wed Mar 16 11:45:40 GMT 1996
 	HOUR=${DATE%%:*}   	# chop off everything after the first colon (:)
                                # e.g.     Wed Mar 16 11
 	HOUR=${HOUR##* }	# chop off everything up to the last space
      				# e.g.                11
 	if [ $HOUR -lt 18 ]  	# if it's earlier than 18.00 hours...
 	then			# print a suitable message
     		print "Time for a cup of tea."
 	else
     		print "Better make that a gin and tonic."
 	fi

(b) This Korn shell script, ruthere, checks whether a named user is currently logged on. It is invoked with one parameter, thus:

  	ruthere  abcdxyz

where 'abcdxyz' is the userid we are interested in.
 
 	#!/bin/ksh
 	USER=`who | grep "$1"`		# use grep to look through list of logged on users
 	if [ "$USER" != "" ] 		# and see if it has found the one we want
 	then
     		print "$1 is logged on"
 	else
     		print "$1 is not logged on"
 	fi

(c) This Korn shell script runs SAS, but first checks whether the file you name actually exists. It is invoked with one parameter, thus:

  	runsas  filename

where 'filename' is the name of the file containing the SAS commands.

 	#!/bin/ksh
 	if [ "$#" -lt 1 ]   				# check for command-line arguments
 	then			      			# if none ...
     		print "You have not supplied a file name."
     		print
     		print "Usage:      runsas FILENAME"
     		exit 1
 	fi
 	FN="${1%.sas}.sas"   				# put in a trailing '.sas' if necessary
 	if [ -f "$FN" ]    				# see if the file exists ...
 	then
     		print "Executing SAS ..."		# if it does, run SAS
     		sas "$1"
     		print "Finished."
     		exit 0
 	else      					# if not, tell the user
     		print "File '$FN' does not exist."
     		exit 1
 	fi

6.3 Running Scripts

Before you can execute a script, you must make it executable using the chmod command. For example, before our user can execute the script stored in the file, runsas (created in Section 6.2.2, Example (c)), he types:

  	chmod  u+x  runsas

(Note: if the u option is omitted, execute permission is given to user, group, and other). Once a script is an 'executable', you simply type the name of the script, followed by any parameters if required. For example:

  	runsas   mysasjob1

6.3.1 Running scripts in background

To run a command or script in the background, you can type the name of the command or script (for example, myrun) followed by an &:

  	myrun &

You can then carry on with other tasks while the job runs. In order to prevent the job from terminating when you logoff the CS6400, you should type

  	nohup myrun > myrun.out &

nohup stands for no hangup. Note that you need to redirect output, which normally goes to the screen, to a file; in this example the ouptut is redirected to myrun.out.

6.4 Accidentally Stopping Background Jobs

Sometimes, if you press Ctrl-Z by accident, you may have stopped a job from running. The command

  	jobs

will show you if you have any stopped jobs. The output might look like the following:

 	[1]  - Stopped 
 	[2]  + Running

To start a job running again in the background, type

  	bg %jobnumber

where 'jobnumber' is the number shown in square brackets.

To start a job running again in the foreground, type


  	fg %jobnumber

To see all the processes you are currently running, use the command ps. The output looks like this:

     	PID 	TTY        TIME COMD
   	11484 	pts/6      0:00 ps
    	1356 	pts/6      0:02 ksh

The process id (PID) is important. You can kill (stop) a process by typing:

  	kill  PID

Be careful with this, as killing your login shell process (in this case, ksh, PID 1356) will log you out!

7 Submitting Batch Jobs to the NQS system

You will need to make use of the batch job system on the CS6400 if you intend to run jobs which require more than 15 minutes of CPU time or more than 32 Mb of memory. Batch job management facilities on the CS6400 are provided by the NQS (Network Queuing System) software. There are many different versions of NQS available from a range of sources. Whilst they can all interconnect with one another, different levels of functionality are provided.

Manchester Computing has installed on the CS6400 the same version of NQS as on the campus workstations and the Fujitsu VPX. This version provides us with the added functionality we require in our environment; in particular, it enables us to store and control the system resources according to your allocation, as on previous systems.

This version of NQS enables us to maintain a single, central job input or 'pipe' queue which is ordered according to your rate of working. From the pipe queue, your job is allocated automatically to the correct execution queue according to the time and memory parameters you have selected. For this reason, we recommend that you do not ask for more time and/or memory than you think the job is likely to require, as it may have to wait for a slot in one of the 'bigger' queues. The command pkguse allows you to see how much CPU time your last 20 package runs used up.

7.1 Submitting a Job

To submit a job to NQS, you will first need to create a shell script containing the commands to be executed, as you would have typed them on the command line (see Chapter 6 for more information about shell scripts). This script can then be sent to NQS using the command qsub. The format of the qsub command is:

  	qsub [options] [script-name]

Generally, the only options you need to specify are the time and memory required. For example, to submit a shell script to run a SAS job such as the example in section 6.2.2, giving the job 45 minutes of CPU time and 64 Mb of memory, you would type:

  	qsub   -lt 45:00 -lm 64mb   runsas

If you just type qsub script-name without any options, the job will run with the default amount of time and memory, which is currently 30 minutes of CPU and 32 Mb of memory. Jobs using these parameters will be eligible to run in the special 'express' or short job queue.

For other options, please see the on-line man pages (man qsub) or refer to the NQS user note CSS 202.

7.2 Specifying Options within an NQS Script

It is also possible to specify NQS options within the shell script, in a special comment line at the top of the script, for example:

 	# An ordinary comment
 	# QSUB -lt 2:00:00
 	# this gives the job 2 hours of CPU time

This means that you would no longer have to specify the time using the -lt option when submitting the job to NQS, but if you did, then the command line option would take precedence over the comment line in the script.

Your output files are returned automatically to the directory you submitted the job from; by default, they are called script_name.onnnn and script_name.ennnn where the .onnnn file is the standard output file, and the .ennnn file contains any error messages.

7.3 Monitoring Jobs

To check up on the progress of your jobs through the NQS system, use the command qstat. If you type qstat by itself, you will just see your own jobs. To see all the jobs in the system, type

  	qstat   -a

The output will look something like this:

$ qstat -a
=========================================================================
NQS Version: 2.4  BATCH REQUESTS on Midas:cs6400
=========================================================================
REQUEST     NAME    OWNER    ALLOCGP QUEUE   PR  CPU      MEM   STATE
339.cs6400  x94run  nopqrst  nop      cpq    28  1:00:00  32Mb  WAITING
551.cs6400  runjob  abcdefg  abc      cpq    58  2:00:00 128Mb  WAITING
443.cs6400  testsas abclsjl  abcls    EXP    32    30:00  32Mb  RUNNING
493.cs6400  submit  hijklmn  hijk     xbq    49  1:00:00 256Mb  RUNNING
690.cs6400  link    abcdefg  abc      abq    58  6:00:00  64Mb  RUNNING
$

In this example, there are 2 jobs in the input or pipe queue, cpq, with three currently executing.

7.4 Cancelling a Job

If you accidentally submit a job and wish to cancel it, you can do so using the qdel command. The format of the command is:

  	qdel [options] request_id

where request_id is the number given to your job by the NQS system, listed in the REQUEST column in the example above. You do not need to specify the '.cs6400' part, just the number, that is:

  	qdel 551

If the job is already running, you will need to use the -k option, thus:

  	qdel -k 690

7.5 Running Applications under NQS

Some applications such as GAUSS, SPSS and STATA can be used in either interactive or non-interactive mode; the command used in your NQS shell script must be the form for non-interactive use, for example:

  	gauss -B file_name

	spss -m file_name (or spssmcc file_name)

	stata -b do file_name

7.5.1 Running SAS jobs

Manchester Computing has set up a special script which makes it easier to submit SAS jobs to NQS. Instead of having to create a separate script file containing the commands to run your job, all you need to do is to specify a time parameter on the command line, and the job will automatically be submitted to NQS as a batch job.

The syntax of the command is:


 	 sas -lt time  jobname.sas

To submit a job called myjob.sas with a CPU time limit of 45 minutes, you would type:

  	sas   -lt 45:00   myjob.sas

7.6 Further Information

Full details about NQS and the options used on MIDAS are given in CSS 202 - 'Using NQS, Network Queuing System' (Second Edition), copies of which are available from the MC Information Point or via your local representative or Liaison Officer.

8 Communicating with Other Users

8.1 Electronic Mail

Manchester Computing recommends that users receive mail on the machine they use most regularly. If this is the MIDAS datasets machine, we recommend that they use the elm user interface. Mail can be sent both to users on the CS6400 and other machines.

Your mail address on the CS6400 is userid@cs6400.mcc.ac.uk.

It is recommended that the email address you publish is independent of the machine where you currently read mail. Most sites allow users to specify names of the form:


  	F.Bloggs@camford.ac.uk

to be published which will redirect mail to the actual mail address; that is:

  	userid@cs6400.mcc.ac.uk

Your cs6400 mailbox will be stored in the file:

  	/usr/mail/userid

which holds all the unread email for that userid.

To start elm, just type:


  	elm

The first time you use elm, it will display the following message:

This version of elm requires the use of a .elm directory in your home
directory to store your elmrc and alias files. Shall I create the
directory .elm for you and set it up (y/n/q)?

Type y in response to set it up. When this has been done it will give you a full screen menu with several options:

Typing m will give:


  	Command: Mail
	Send the message to:

Type in the email address of the person to whom you want to send mail, for example:

  	hijklmn   			(to send mail to user hijklmn on the CS6400)

 or  	a.n.other@auniversity.ac.uk  	(to send mail to user name a.n.other at a remote site 
					called auniversity.ac.uk)

Note that addresses must be given in Internet (reverse NRS or 'world') notation. If our user had mailed hijklmn, the following message would be displayed:

	Command: Mail   		To: hijklmn
	Subject of message:

Type in a subject name; the following message will then be displayed:

  	Copies to:

Press RETURN if you do not want to send a copy to anyone else, otherwise type in their email address. elm will then invoke the default editor.

To set up the editor of your choice, type o from the main menu.

You will get into the Options Menu. The Options Menu looks like this:

Use E and V to change the editor to the one you prefer, then type > to save, followed by i to return to the index.

When you have composed a mail message and saved the file using the editor you prefer, you will see:


Please choose one of the following options by parenthesized letter:
             e)dit message, edit h)eaders, s)end it, or f)orget it.

If the message is ok, type s to send. You will be returned to the main elm menu. It will display:

  	Command:
			Mail sent!

To read your electronic mail, type elm to get into the main menu. If you have messages, the screen will look like this:

Just press RETURN to read the message which is currently highlighted. When you have finished reading it, press i to get back to the main menu. When you exit mail, it will ask you if you want to move the message to your 'received' mail folder.

Once you have become familiar with elm, you can also set your user level to Intermediate, then to Expert by using the Options Menu (type o from the main menu). You will then have the ability to set up aliases, for example, if you mail someone called fred.bloggs@londuni.ac.uk quite frequently, you could set up an alias called fred, so that to send him mail in future all you would have to type is elm fred. You can also set up mailing lists in this way. When you are familiar with the options, which are stored in the file elmrc in your .elm directory, you can edit this file to change them.

A useful email address to remember is:


  	info@midas.ac.uk

Mail asking for help with any problems you are having with using the CS6400 can always be sent to this address. If you do send email, please give as much detail as you can about the particular problem you are experiencing, and include your own email address and day-time telephone number in the message if possible.

8.2 File Transfer

8.2.1 The TCP/IP ftp command

If the site you wish to transfer files to is on JANET or the Internet, you can use the interactive ftp command.

Just typing ftp will give you the prompt


  	ftp>

To connect to the remote site, type:

  	open  site-name

(this can either be the Internet name of the site, or the IP address)

You can also issue the site name as an argument, that is:


  	ftp  site-name

In either case, the system will respond with

	Connected to [site-name]

followed by other messages, dependent on the remote system's ftp; then

  	Name (sitename:cs6400_userid): abcdefg
	Password:  (your password is not echoed)

(where abcdefg is your userid on the remote system). This will be followed by the message:

  	User abcdefg logged in.
	ftp>

You can send a file to the remote system by typing:

  	put   filename

or fetch a file by typing:

  	get   filename

You can also use the cd and ls commands to change the remote directory and list remote files. ASCII (text) file transfer is the default; to send binary files, type

  	bin

before you issue the put or get command. Use the command

  	quit 

when you have finished. For more information, see the manual page for ftp or type ? for help within ftp.

8.2.2 Blue Book FTP (NIFTP)

The CS6400 does not support Coloured Book protocols, consequently file transfer between the CS6400 and 'Blue Book' FTP hosts must be performed through a protocol conversion gateway, uk.ac.ft-relay. (N.B. this relay is scheduled to be removed on 1 August 1997.)

(a) Starting file transfers from the remote machine to the CS6400

To transfer a file to/from the CS6400 you must supply the following parameters to your local Blue Book interface: the Internet (DNS) name of the CS6400, your CS6400 username and password and the CS6400 filename. You must specify the NRS name of the gateway, uk.ac.ft-relay, as the remote host name. The gateway will forward the parameters as received and therefore they must be sent in the appropriate case where necessary.

Two alternative forms are provided (they should not be mixed).


 	Blue Book parameter      Form A

	Remote site              UK.AC.FT-RELAY
	Remote username          cs6400_username@cs6400.mcc.ac.uk
	Remote password          cs6400_password
	Remote filename          cs6400_file name

	Blue Book parameter      Form B

	Remote site              UK.AC.FT-RELAY
	Remote user              cs6400_username
	Remote password          cs6400_password
	Remote filename          cs6400.mcc.ac.uk::CS6400_filename

For example, to send the file, test file, from a CMS service to the CS6400 using the standard CMS Blue Book FTP, you would type:

ftp send test file as test.file to <userid>@cs6400.mcc.ac.uk <password> at uk.ac.ft-relay

To fetch the file, test.file from the CS6400 to the CMS machine, type:

ftp fetch test.file as test file from <userid>@cs6400.mcc.ac.uk <password> at uk.ac.ft-relay

(it must be on a single line).

(b) Starting file transfers from the CS6400

To transfer a file to/from a Blue Book host you must first use the ftp command to call the gateway, that is, type:


   	ftp   ft-relay.ac.uk

You will then be prompted for a username and this must be given in the form username@remotehost where 'remotehost' is the NRS name of your remote host and username is your remote userid. You will also be prompted for your remote password (this must be the disk read password in the case of CMS). Having supplied your password you may then perform file transfers using the get and put commands in the way described in Section 8.2.1. It is not possible to use mget and mput, because the Blue Book protocol does not support directory listings.

For example, to send/fetch a file from a machine UK.AC.LONUNI.HOST1:


  	ftp   ft-relay.ac.uk
Connected to FT-RELAY.AC.UK
220 FT-RELAY FTP server (Version 3.2) at Mon Feb 24 14:25:01
Name (FT-RELAY.AC.UK:roger): userid@uk.ac.lonuni.host1
331 Give password for user USERID on host uk.ac.lonuni.host1
Password: msecret
230-User anonymous, Host:uk.ac.lonuni.host1, Network:NIFTP
Login information will be checked at file access.
230 End reply
ftp>

Then having logged on to the gateway, get a file from LONUNI.HOST1.

  	ftp> get mytest.dat mytest
   	200 PORT command accepted.
	125 transfer started.(mytest.dat)
	226 Transfer completed.
	local: mytest remote mytest.dat
	2560 bytes received in 00:00:03:02 seconds
	ftp>

To send a file to LONUNI.HOST1:

  	ftp> put mytest mytest.dat
   	200 PORT command accepted.
	125 transfer started.(mytest)
	226 Transfer completed.
	local: mytest remote mytest.dat
	2560 bytes received in 00:00:03:02 seconds
	ftp>

Finally, most sites have or are planing to provide, Internet style networking facilities. If these are available to you please make use of them, as Internet ftp will be both much faster and more reliable than Coloured Book NIFTP protocols for file transfer to/from the CS6400.

8.3 Checking Who is Logged In

Use the finger command to see who is logged in to the CS6400.

The output displayed is similar to this:


Login      Name             TTY     Idle    When     Where
root     0000-Admin(0000)   pts/0   1:15  Fri 17:16  cs6400ssp.mcc.ac.uk
root     0000-Admin(0000)   pts/3   3:55  Mon 09:49  cs6400ssp.mcc.ac.uk
seagull  Mr P O'Toole       pts/2         Mon 19:22  158.143.103.161
zephyrs  IM Windrush        pts/2         Mon 07:31 
pelican  W. Byrd            pts/1   2:45  Mon 07:57
venice   P.D. Glass         pts/5    79d  Mon 09:16
liverpl  Penny Lane         pts/7    79d  Mon 09:30
artery   Richard Ticker     pts/1   2:45  Mon 16:25  adonis.auniv.ac.uk
aldous   BRV Newworld       pts/11   79d  Mon 10:16
lampoon  Michael Taker      pts/12   79d  Mon 10:33
abcdefg  Fred Bloggs        pts/15  3:11  Mon 13:02  harvest.camford.ac.uk
liverpl  Penny Lane         pts/13   79d  Mon 12:08
shopper  ISA Trolley        pts/6   5:50  Mon 12:35
shopper  ISA Trolley        pts/10  5:50  Mon 12:53
liverpl  Penny Lane         pts/14   79d  Mon 13:46
lampoon  Michael Taker      pts/9         Mon 16:45

Alternatively, you can use the who command. The output is similar to the above except that the second and fourth fields are not displayed, and there are no field headers.

8.4 Sending Messages to Another Logged In User

To talk to another user, for example, hijklmn, type the command:

  	write   hijklmn

Everything you type will then be echoed on their screen, until you press Ctrl-D to terminate the connection. They will get a banner like this:

Message from abcdefg on cs6400 (pts/15 [ Mon Jan 24 13:29:36 ] ...

before seeing any messages from you.

If they are logged on from more than one location for example, pts/12 and pts/9, you can type:


  	write  hijklmn  pts/12

to make sure that your message goes to terminal pts/12.

To have a two-way discussion with user hijklmn, type:


  	talk  hijklmn

Your screen will divide itself into two parts. In the top half, you will see the message

 	[Waiting for your party to respond]

They will receive the message

	Message from Talk_Daemon@cs6400 at 19:37 ...
	talk: connection requested by abcdefg@cs6400.mcc.ac.uk.
	talk: respond with: talk abcdefg@cs6400.mcc.ac.uk

If they wish to respond, you will get the message

  	[Connection established]

at the top of your screen.

You can then both type messages interactively. What you type will appear in the top half of your screen, with their messages appearing in the bottom half.

If either of you presses Ctrl-C (or whatever the interrupt sequence has been set to in your .profile), the connection will finish, you will see the message


  	[Connection closing]

at the top of the screen, and you will return to the $ prompt.

9 Using Tapes

Access to user tapes on the CS6400, following the removal of the old tape management system (which used the tape command), is now performed using the standard UNIX interface found on most UNIX systems. This means that, in order to mount a tape, you need to contact Manchester Computing and tell us which tape you wish to access (see below for how to register a tape with us). Once the operator has loaded the tape, you will be informed on which 'drive' it is mounted. The tape can then be accessed using the command:

  	/dev/rmt/xbn

where x is the drive number.

9.1 Important Note

Once mounted, tape access is insecure and there is no checking. This means that any logged-on user can access the tape, reposition it, off-line it, etc. You should therefore exercise extreme caution when using tapes. You are strongly recommended to take the following steps when using tapes: For users who intend to access tapes, a document describing this system in more detail (CSS 201 - Using Tapes (Third Edition)) has been published. This document is available from the MC Information Point or your site representative/Liaison Officer, and you are strongly recommended to obtain a copy.

At present, the following types of tape can be handled: Exabyte, QIC, ½" 9 track open-reel, and IBM 3480 cartridges. There is no user access to DAT tapes, but these can be processed in conjunction with MC staff help. Send an email to info@midas.ac.uk or ring the MIDAS Helpline (0161-275 6109) if you have a DAT tape to process. Tape formats which can be read are: tar, tapes written using the UNIX dd, cpio and cp commands, IBM SL and NL tapes in FB and VB format, CMS DUMP format.

9.2 Registering Tapes

All tapes need to be registered with the Tape Librarian; if sending a tape to MC, please remember to include your name, userid on the CS6400 and postal address. Once the tape has been registered, you will be sent a receipt which includes the identifier, henceforth referred to as the volser (volume serial number). You are strongly recommended to make a copy of your tape before you send it to us, especially if it contains important and/or irreplaceable data.

9.3 Mounting a Tape

As stated in Section 9.1 above, if you wish to use a tape, you must first contact Manchester Computing on either 0161-275 6109 (during normal working hours) or 0161-275 6090 (between 07:30 and 09:00, and 17:00 to 22:30, Monday to Friday). The operators will attach your tape to a drive and inform you of its number. To check that it is okay, you can use the command:

  	mt   -f /dev/rmt/xbn   status

where x is the number of the tape drive. If you receive the following response, the tape is ready for use:

	HP 88780 ½-inch tape drive:
           sense key(0x0)= No Additional Sense   residual= 0   retires= 0
           file no= 0   block no= 0

9.4 Dismounting a Tape

When you have finished with a tape, it is essential that you do not leave it on the drive (remember, tapes are not secure). To dismount the tape, type:

  	mt  -f /dev/rmt/xbn offl

9.5 Positioning a Tape

To position the tape, use the mt command (see man mt for more details). The general form of the command is:

  	mt  -f /dev/tape/volser  command [count]

where command is one of the tape positioning commands, including fsf (skip forward [count] tape marks), bsf (skip backwards [count] tape marks), rew (rewind the tape). You will need to rewind the tape yourself when necessary.

9.6 The tar Command

The tar (tape archive) UNIX utility is used for creating and manipulating archives of data, which are usually but not necessarily on tape. The general format of the command is:

 	tar <function> [device] filename(s)

where function may be

   	c 	create a tar archive file
   	r 	write files at the end of an existing tar archive
   	t 	list the contents of the tar archive
   	u 	update the archive
   	x 	extract files from the archive

In addition, several function modifiers can be used, for example:

   	f 		file - causes tar to process the archive on the device specified, which is 
			used for accessing files on tape on the CS6400
   	v 		verbose - will give information about what tar is doing 

and  	filename(s) 	is a list of files (you can use wildcards) to be processed.

To check the contents of a tape using tar, for a tape mounted on /dev/rmt/3bn, you would use the following command:

  	tar   tvf   /dev/rmt/3bn

Your output should look something like this:

-rw------- abcdefg/ab  2714  Mar  3 16:20 1994 ansi.list
-rw------- abcdefg/ab  8132  Feb 22 10:47 1994 ansir.list
-rw------- abcdefg/ab   448  Mar  3 16:48 1994 survey.list
-rw------- abcdefg/ab 12018  Mar  3 16:50 1994 surveyp.list
-rw------- abcdefg/ab  1556  Feb 21 14:11 1994 t2.list
-rw------- abcdefg/ab  1556  Mar  3 12:11 1994 tcopy.list

To read the first tar file off the tape, the command:

  	tar   xvf   /dev/rmt/3bn

will extract all the files in the archive.

To read a tar file which is the third file on the tape, you would first need to position the tape at the correct file:


  	mt  -f   /dev/rmt/3bn   asf   2
	tar  xvf   /dev/rmt/3bn   *.f

This would skip past the first two files on the tape, then extract all the requisite .f files.

To write a tar tape for transfer to another machine, we recommend using the gnu utilities version of tar, gtar, which resides in the file /packages/liaison/bin/gtar; this is less specific than the default version, and it is probably wiser to always use the gnu version. If the tape is mounted on device /dev/rmt/3bn and has been correctly positioned, then:


  	gtar   cvf   /dev/rmt/3bn   progs

will create a tar archive file containing the contents of the subdirectory progs. The command works recursively, so all subdirectories below progs will also be archived.

9.7 IBM Standard Label (SL) Tapes

To read all the files off an IBM SL open-reel tape, you should use the following procedure.

(a) Contact the Centre and request that your tape is mounted as described in Section 9.3.

(b) Check that the tape is there (see Section 9.3).

(c) Use ansir to read all the files off the tape (in the following example, from device 0) into the current directory:


 	 ansir   -c   /dev/rmt/0bn

(d) Dismount the tape explicitly, using:

 	 mt   -f   /dev/rmt/0bn   offl

The ansir utility has been provided to read IBM labelled (SL) tapes. The program is in the directory /packages/tapes/bin and the man pages are in /packages/tapes/man. In order to use it, these names should be added to your PATH and MANPATH environment variables by editing your .profile file (see Section 2.4). The command:

  	ansir   -pnc   /dev/rmt/0bn   >   tape.list

will provide a list of the tape's contents in a file tape.list.

10 Archiving Files

Users can now archive their files directly to the new Storagetek 9710 Digital Linear Tape (DLT) based robotic library using the Legato Networker interface. More detailed information about the Legato Networker software is given in CSS 203 - 'Archiving Data on the CS6400 using Legato Networker' (Third Edition).

10.1 Two Different Interfaces to Legato Networker

As stated above, the interface to the new DLT system is provided by Legato Networker. There are two different user interfaces to the archive, depending on whether you are accessing MIDAS via X-windows software (for example, by running eXceed on a PC or Exodus on a Mac) or whether you access MIDAS in 'line mode', usually done by using Telnet.

10.2 This is a Permanent Archive

Before describing the user interfaces, it should be pointed out that the archive facility is provided for long term storage of important data, and not as a means of temporarily exceeding your filestore quota (there are other remedies for this - contact MC through the normal channels for solutions). It should be noted that, at present, once a file has been archived on the DLT system, there is no way of deleting it and recovering the space; it is there "forever". Please bear this in mind when archiving data.

10.3 Registration to use the Archive

In this initial phase of the transition, it will be necessary for those users wishing to use the archive to register with MC first. In order to do this, contact the centre via the MIDAS help line (tel: 0161-275 6109 or email: info@midas.ac.uk).

10.4 Using Legato Networker - GUI Interface

For X-windows users, there is a user-friendly GUI interface, accessed by typing:

  	nwarchive &

You are then presented with a filestore 'tree', based on your current directory (that is, the one you were in when you issued the nwarchive command). You then select the files (and/or directories) you wish to archive by 'checking' the little boxes to the left-hand side of the file names. An important point to note here is that individual files are not archived separately, but that all the files and directories marked are archived together as a "save-set". It is not possible, subsequently, to de-archive single files, that is, the whole save-set must be de-archived. Please bear this in mind when archiving files and directories. (Of course a save-set may comprise of a single file, but users are recommended to archive files together in batches, for the sake of efficiency. If the single file is very large, say hundreds of Megabytes, then it may be archived singly.)

Once you have marked the requisite files and directories, you start the process by clicking on the Start button. A second window now appears; it is here that you specify the options to be in effect during the archive session. First, you must enter a description field with text that is meaningful to you, and that will enable you to identify your files both now and several years hence! It is strongly recommended that you start your description with your userid; for example:


 	abcdefg : This is a set of files relating to Bury FC's promotion year 1995/6 ...

The reason for the above is that it is the Annotation field that is used when searching for your files on de-archiving (see below). Including your userid and a meaningful description will thus make your life easier in the future.

Having entered a userid and meaningful description, you now need to select the name of the Archive and the Archive Clone "pools" to be used. Select Archive for the archive pool and Archive Clone for the Clone pool. You should also switch OFF the Compression box, as this is not required (in fact, it actually takes up more space if you select Compression)!

In order to create two archive copies, which we recommend, rather than just one, you should now select the Clone button. It is also wise to check the Verification button, which ensures that the contents written to tape are checked (that is, verified) for errors.

If you wish the file to be deleted from your 'on-line' filestore on successful completion of the archive transfer, then you should select the 'Grooming' button. If you do not select this, you will need to deal with the on-line file yourself later.

After doing all of the above, click on the Archive button to start the transfer. A progress log of what is happening is displayed on the screen. When the archival has been completed, you will be given a (numerical) save-set identifier. Record this number and the files that it refers to for future reference (when you wish to retrieve your files). Remember also that it is not possible to retrieve single files, unless they have been archived as individual save-sets.

As a guide to how long the process takes, we recently archived 375 Mbytes of files, selecting both Cloning and Verification (so two verified copies were made). The process to successful completion took 25 minutes. Therefore please be patient! Unlike the UniTree system, where archived files were copied to an intermediate disc cache, here the files are copied directly to (DLT) tape.

10.5 Using Legato Networker - Line Mode Interface

For the minority of users who use 'line mode', rather than the recommended X-windows access, the line commands are as follows:

 	 nsrarchive -b Archive -C "Archive Clone" -V file_name(s)

To specify the Archive backup pool, we use the -b Archive option. To instruct the system to make a Clone copy in the Archive Clone pool, we use -C "Archive Clone" option (the "" are needed because of the space) and to have the system verify the transfer we use -V. As with all UNIX commands, case is sensitive here, so please make sure you follow the above closely.

After issuing the above command, you are prompted for the Annotation text to go with the Save_set. You are recommended to start the annotation text with your userid, as this will facilitate recovery of the archive files later. End the data entry by pressing Enter followed by <Ctrl> D. Please wait. Do not press Enter again until prompted. As with the GUI method of archiving, this process may take several minutes, or tens of minutes, to complete, as the files are copied directly to tape, and not via a (faster) intermediate disc cache.

For more details on the available options, type:


  	man  nsrarchive

10.6 File Retrieval

10.6.1 X-window (GUI) interface

The GUI interface for file retrieval is started by typing:

  	nwretrieve  &

If you click on the Query button, you are presented with a list of all the archived save-sets (that is, everybody's, not just your own). However, you only have access to your own, of course. You can now search the Annotation field for a given string in order to identify your save-sets. This is why we recommend that you prefix your save-sets with your userid You do this by typing a value in the Selection field (your userid, say) and then clicking on Query again.

One thing to note however, is that, in typical UNIX fashion, the search is case sensitive, therefore if you included your userid in upper-case, you will need to search for it in upper-case. You can of course always specify the save-set id, which will make life much simpler, which is why you are recommended to make a record of it.

To select a save-set for retrieval, highlight it in the archive list and click on the Start button. The Retrieve Status window is then displayed, showing the progress of the restoration. By default, files are retrieved to their original locations. You can use the 'Relocate To' switch to restore the file to a different directory. If you select 'Overwrite Existing Files', any on-line files which have the same filename as an archive file will be overwritten by the de-archived copy. The default is to prompt you if any conflict occurs. When you are ready to start, click on the Start button.

10.6.2 Line mode interface

For 'line mode' users, the command is:

 	 nsrretrieve -s  backup.mcc.ac.uk  -S  <save_set_id>

where <save_set_id> is the unique numerical value you were given when the file(s) was/were archived. When the save_set is restored, the file names are listed on the screen and your prompt is returned.

At present, you also need to specify the name of the archive server (backup.mcc.ac.uk) using the -s option. For further details, type:


  	man   nsrretrieve

10.7 Further Information

Full details on how to archive files and retrieve them are provided in the CSS note, CSS 203 - Archiving Data on the CS6400 using Legato Networker' (Third Edition), copies of which are available from the MC Information Point (0161-275 6042; email: info@midas.ac.uk) or via your local MIDAS representative.

Appendix A - Summary of Useful UNIX Commands


alias 	Abbreviate or change the name of a command
apropos	Show commands related to a particular topic
cat  	Display the contents of a file on the screen
cd  	Change your working directory
chmod 	Change access permission modes for a file or directory
cmp  	Compare the contents of two files
cp  	Copy an existing file to a new file
csh  	A shell with a C-like syntax
date  	Show the system date and time
diff  	Find the difference(s) between two files
echo  	Echo input to the standard output
ed  	Edit or create a file using the ed line editor
elm  	Use the elm electronic mailer to send and read email
exit  	Logout of the system
finger 	Show who is logged in to the system
ftp  	Transfer files to/from a remote system
gopher 	Use the gopher on-line help and information server
grep  	Search a file for a pattern
groups 	Display the user groups to which you belong
history Show a list of the last few commands you typed
jobs  	Show your background processes
joe  	Edit or create a file using the joe screen editor
kill  	Terminate a process
ksh  	The Korn (default) shell
lpr  	Print a file on a lineprinter
lpq  	Display jobs in the print queue
lprm  	Remove a job from the print queue
ls  	List the contents of a directory
man  	Look at the on-line manual pages
mkdir 	Create a new directory
more  	Display the contents of a file a page at a time
mv  	Move or rename a file
news  	Display the system login news
nohup 	Continue with background process after logout
passwd 	Change your login password
pg  	Display the contents of a file a page at a time
ps  	Show the status of your processes
pwd  	Show the current working directory
qsub  	Submit a job to the NQS batch system
quota 	Show your filestore allocation and current usage
recover Restore a copy of an erased file
rm  	Remove (delete) file(s)
rmdir 	Delete an empty directory 
spell 	Check a file for spelling mistakes
talk  	Talk to another user
telnet 	Login to a remote system
unalias Remove an existing alias
vi  	Edit or create a file using the vi screen editor
write 	Send a single message to another user

Appendix B - Joe Editor Fact Card

Unix - The joe editor


Joe is the text editor supported by the Computer Centre for use on Unix systems. It is easy to use for most text creation and alteration tasks - the input of numeric data, the typing of program text, or the preparation of files for submission to formatting and typesetting systems such as Latex. It is available on all Unix computers operated by the Centre, and its operation is the same on all styles of terminal, PC and workstation, whatever the keyboard layout.

Commands to joe all involve the use of the Control key. This is usually marked Ctrl. It is used in conjunction with other keys. For example, the command to delete a word is Control W i.e. hold down the Ctrl key while you press W. The W can be upper or lower case. Such letters will be shown here as upper case for clarity, but normally you will press them without the Shift key, although it will not matter if Caps Lock is on.

Some commands require two characters. For example, the command to obtain help on the screen is Control K H. As long as you hold down Ctrl for the first of the characters, it does not matter whether it is down for the second.

Some commands ask questions. For example if you give the command to find some existing text, joe will ask for the text to find. If during such a dialogue you wish to abandon the operation, give Ctrl C as the response.

Help is available on the screen, and there are several different screens of Help information to choose from. The basic one is already displayed when you enter joe. Giving the Control KH command removes it, and another Control KH obtains the menu of Help screens. In the Help information, the use of the Ctrl key is indicated by ^ e.g. Control K H is shown as ^KH. This convention is also used in this document from this point on.


If your screen gets into a mess

If you receive a broadcast message, joe may put some text on your screen while the cursor is in the wrong place, and will not be aware of the message text so will never delete it. To restore the correct picture, give the ^R command.

If your screen freezes it may just be the system going slow, but if the cursor will not move after a few seconds you must have accidentally pressed ^S. This is not a joe command, but can have the effect of locking the screen. The antidote is to press ^Q.

Creating a new file

Give the joe command with a new file name, e.g.

	joe  data1
The joe screen will appear, with basic Help information at the top, and the words New File at the top of the text area. These words will disappear when you start to type.

Type normally. Use the Backspace key to delete if necessary, and the arrow keys to move around. For large scale changes, see the next section Editing existing text.

Note that joe will start new lines as necessary. If you are typing running text there is no need to use Return except at the ends of paragraphs. If the default line length of 77 characters is unsuitable, you can change the right margin with ^TR or you can switch off word-wrap with ^TW and then you will always have to press Return (marked Enter on some keyboards) to start new lines.

When you have finished give the ^KX command to save your work and leave joe.

Editing an existing file

Give the name of the existing file in the joe command. The first screenful of text will then appear, with the cursor sitting on the first character. You can move through the text and insert or delete text, or move blocks of text around.

Moving the cursor


Left		One character   	            Right
 		One word  		^Z   ^X
 		Start/end of line 	^A   ^E
Up 		One line		            Down
 		One screenful 		^U   ^V
 		Start/end of text 	^KU  ^KV
By line number  			^KL
By context - see Search and replace

Deleting


Back 		One character 	Backspace ^D  
Forward
 		Start/end word 		^O   ^W
 		To end of line  	     ^J
Whole line   				^Y
Block 		Define block by  	^KB at start
    					^KK at end
 		Delete with  		^KY

Inserting

Move to required position and type new text. If the last line is left too long, use ^KJ to reformat paragraph.

Text from another file can be inserted at any point. Move the cursor to where you want the text and give the ^KR command. All text in the file will be inserted; delete any you do not want.

Moving text (cut and paste)

To move a block of text, or to reproduce a block in a new position, mark the block with ^KB at the beginning and ^KK at the end. Move to the new position and use ^KM to move the block there or ^KC to copy the block there.

The newly positioned block will still be highlighted, so you can move or copy it again, or delete it if you wish. To remove the highlighting give another ^KK command. To cancel blocking if you change your mind and do not move it, give another ^KB and a ^KK.

Search and replace

To find a particular group of characters, use the ^KF command. Joe will ask for the characters to be found. Type them as they appear in the text. Press Return. Joe will then ask you to type in a group of characters to describe how to do the search. If you just want to search forwards for the characters, press Return. If you want to search Backwards press B. To ignore case, i.e. to find occurrences of your characters in upper or lower case, press I. To replace occurrences that are found, press R. Use B I and R in combination if necessary. If you have used R, Joe will ask for the characters to be used as a replacement.

Joe will position the cursor after the next occurrence of the characters you have asked it to find, in the direction you asked it to search, or will report that no occurrences were found. If you specified replacement, joe will ask you to confirm that it is to replace this occurrence. You can tell joe to go ahead, or you can cancel the replacement, or you can tell joe to replace this and all subsequent occurrences without asking again, if you wish.

The characters you specify as the group to be searched for will not be found if they spread over a line ending. If you want to find occurrences of characters specifically at the ends or beginnings of lines, or at the ends or beginnings of words, include in the group:


 	\^  	to indicate the beginning of a line - note 
		this is not a Control character, you actually 
		type \ then ^
 	\& 	for end of line
 	\< 	for beginning of word
 	\> 	for end of word

For example, to find all words that end in ise search for ise\>

You can also use the normal Unix wild card characters and regular expressions in searches if you put a \ in front of them. For example, to find all three letter words that begin with letter a, use a\?\?

If you need to search for \ itself, specify it as \\

Repeating a search - joe remembers what you last searched for, and will repeat the search in the same direction if you give the ^L command.


Editing strategy
Remember that changing something on the screen does not change it in the stored disc file. This does not happen until you save the altered version.

Saving occurs automatically when you finish with ^KX but in a long editing session it is wise to save frequently, while you are working, with the command ^KD. In this way you will not have to repeat much work if equipment malfunctions and prevents you from finishing the job.

When you are editing an existing file, the first time you save with ^KD, or at the end if you have not used ^KD, joe will save the original version of the file under a new name. This is the old name with ~ at the end. For example, after you have edited the file called data1 the new version will be in data1 and the previous version will be in data1~. The next time you edit data1, the new version will go into data1, the previous version will be saved in data1~ and the original will be lost.
You therefore always have the most recent version, and can go back to the last previous version. If you need to keep older versions, take copies before using joe, or save new versions in files with different names by specifying a new name to the ^KD command (^KX does not give you the opportunity to specify a name, unless you entered joe without a file name to create a new file, which is not a recommended practice).

If you realise after performing modifications that you do not want to save them, do not use ^KX to leave joe, but use ^C as a command to abort the session.

Undoing and redoing

Joe keeps a log of the changes you make, so it is possible to undo them. If you press ^_ (Ctrl and underline) joe will reinstate the last piece of text you deleted or delete the last piece of text you typed or replace a block you moved, i.e. it will undo the last operation you performed on a piece of text. A piece of text is defined as consecutive characters; successive backspaces deleting several characters would count as one operation, but a cursor move between deletions makes two separate pieces of text. If you use ^_ again before making another alteration, the previous alteration (the one before the one you have just undone) will be undone, and so on, back to the text in its original state.

If you undo an alteration and realise that was a mistake, you can reinstate the alteration by immediately pressing ^^ (Control and ^). You can reverse a whole series of ^_ commands with a series of ^^ commands.

More information

When you are using joe, you have access to several screens of information through the Help command ^KH, not just the 'Basic' screen that is initially displayed. There are screens on 'Advanced' commands, searching, and other topics. The Help information within Joe gives a far more exhaustive list of commands and special character sequences than is possible here, although it provides little explanation. A lengthy explanatory text is available in the man page for joe. To see this, give the command

	man joe

at the normal Unix prompt (not within joe).
Word processing functions
Joe is not a word processor. You cannot choose fonts, underline, embolden, spell check etc, but you can lay out text in a tidy way. Joe has commands for setting left and right margins, for indenting blocks, for centring lines, and for setting tabs. Many of these features are particularly useful for laying out program text (see also the feature for finding matching brackets).

Joe does not reformat lines lengthened or shortened by alterations as a word processor would. You must remember to reformat paragraphs with ^KJ.

Remember that text saved by joe will have newline codes at the end of every line, whether you pressed Return or not. You may have to arrange to get rid of these if you ever import the file into a genuine word processor.