This document describes how users can move files to the permanent archive and retrieve them using the Legato Networker software. Depending on how users access Unix Host, there are two interfaces from which to choose; there is the user-friendly GUI interface for X-windows users, and a command-line interface for those using Telnet connections.
Until 31 October 1997 some users will have access to files which were archived under the old UniTree system, so Section 3 of this document describes how to recover them.
During 1996, new Backup and Archive facilities were implemented at MC. The new facilities cover the Unix Host service and comprise a Storagetek 9710 DLT (Digital Linear Tape) based robotic library, having an initial capacity at MC of 5 Terabytes (uncompressed). This is expandable up to 11 Terabytes (uncompressed). The software interface to this system is the Legato Networker suite, which will also be used for filestore backup (see user note CSS 204).
Legato Networker has replaced the UniTree system for the archiving of data. Since 31/10/96, the old UniTree archive has not been accessible by using the unitree command. All the UniTree files archived before 21/10/96 are available, however, via the Legato Networker Backup interface, described in Section 3 of this note. These old archive files will be available for a period of 1 year (until 31/10/97). This should give you adequate time to retrieve your files from the Backup and, where appropriate, re-archive them in the new Legato Archive system described below. In the interest of saving space, you are asked to re-archive only those files that you require, and not just re-archive everything en masse.
As stated above, the interface to the new DLT system is provided by Legato Networker. There are two different user interfaces to the archive, depending on whether you are accessing Unix Host via X-windows software (for example, by running eXceed on a PC or Exodus on a Mac) or whether you access Unix Host in 'line mode', usually done by using Telnet.
Before describing the user interfaces, it should be pointed out that the archive facility is provided for long term storage of important data, and not as a means of temporarily exceeding your filestore quota (there are other remedies for this - contact MC through the normal channels for solutions). It should be noted that, at present, once a file has been archived on the DLT system, there is no way of deleting it and recovering the space; it is there "forever". Please bear this in mind when archiving data.
In this initial phase of the transition, it will be necessary for those users wishing to use the archive to register with MC first. In order to do this, contact the centre via the Unix Host help line .
2. Legato Networker Interfaces
There are two different user interfaces to the archive, depending on whether you are accessing Unix Host via X-windows software (for example, by running eXceed on a PC or Exodus on a Mac) or whether you access Unix Host in 'line mode', usually done by using Telnet.
2.1 X-Windows Access - GUI Interface
If you have X-Windows you will have access to the 'Graphical User Interface' (GUI) program which looks and behaves like any MS-Windows program. You can use the mouse and pointer to explore and select directories, files and menu items and click on buttons to perform actions.
For X-windows users, there is a user-friendly GUI interface, accessed by typing :
nwarchive &
(Typing the '&' will run the program in the background). You are then presented with a filestore 'tree', based on your current directory (that is, the one you were in when you issued the nwarchive command).
You then select the files (and/or directories) you wish to archive by 'checking' the little boxes to the left hand side of the file names.
An important point to note here is that individual files are not archived separately, but that all the files and directories marked are archived together as a "Saveset". It is not possible subsequently to de-archive single files, but the whole Saveset must be de-archived. Please bear this in mind when archiving files and directories. (Of course a Saveset may comprise of a single file, but users are recommended to archive files together in batches, for the sake of efficiency. If the single file is very large, say hundreds of Megabytes, then it may be archived singly).
Once you have marked the requisite files and directories, you start the process by clicking on the Start button. A second window now appears; it is here that you specify the options to be in effect during the archive session. First, you must enter a description field with text that is meaningful to you, and that will enable you to identify your files both now and several years hence! It is strongly recommended that you start your description with your userid as in this example:
The reason for the above is that it is the Annotation field that is used when searching for your files on de-archiving (see Section 2.1.2 below).
Including your userid and a meaningful description will thus make your life easier in the future.
Having entered a userid and meaningful description, you now need to select the name of the Archive and the Archive Clone "pools" to be used. From the drop down menus select Archive for the archive pool and Archive Clone for the Clone pool. You should also switch OFF the Compression box, as this is not required (in fact it actually takes up more space if you select Compression)!
In order to create two archive copies, which we recommend, rather than just one, you should now select the Clone button. (The clone copy will be saved onto a second tape). It is also wise to check the Verification button, which ensures that the contents written to tape are checked (that is, verified) for errors.
If you wish the file to be deleted from your 'on-line' filestore on successful completion of the archive transfer, then you should select the 'Grooming' button. If you do not select this, then you will need to deal with the on-line file yourself later. If any problems are detected during the archive run, however, grooming does not take place.
After doing all of the above, click on the Archive button to start the transfer. A progress log of what is happening is displayed on the screen. When the archival has been completed, you will be given a (numerical) Saveset identifier (ssid). Record this number and the files that it refers to for future reference (when you wish to retrieve your files). Remember also that it is not possible to retrieve single files, unless they have been archived as individual Savesets.
As a guide to how long the process takes, I recently archived 375 Megabytes of files, selecting both Cloning and Verification (so two verified copies were made). The process to successful completion took 25 minutes. Therefore please be patient! Unlike the UniTree system, where archived files were copied to an intermediate disk cache, here the files are copied directly to (DLT) tape. The time taken depends to some degree on the activity taking place on the robotic system. The robot has 4 tape decks and the cloning process needs 2 decks, so you may need to wait for a second one to become free after the initial archive phase.
The GUI interface for file retrieval is started by typing :
nwretrieve &
If you click on the Query button, you are presented with a list of all the archived Savesets (that is everybody's , not just your own). You only have access to your own, however, of course. You can now search the Annotation field for a given string in order to identify your Savesets. This is why you are recommended to always include your userid in the description. You do this by typing a value in the Selection field (your userid say) and then clicking on Query again.
One thing to note, however, is that the search is case sensitive, therefore if you included your userid in upper case, you will need to search for it in upper case. You can of course always specify the Saveset id (ssid ), which will make life much simpler, which is why you are recommended to make a record of it.
To see additional details, highlight the Saveset you are interested in and click on the Details button:
To select a Saveset for retrieval, highlight it in the archive list and click on the Start button.
The Retrieve Status window is then displayed.
By default, files are retrieved to their original locations. You can type a new path into the 'Relocate to:' box before the process begins if you wish to restore the file to a different directory.
If you select 'Overwrite Existing Files' then any files on-line that have the same filename as an archive file will be overwritten by the archive copy. The default is to prompt you if any conflict occurs.
When you are ready to start, click the OK button. The Status window will show the progress of the restoration. It will take some time.
As you can see, to restore this one (small) file took approximately 3 minutes, but the same file took 15 minutes on another occasion as the server was busy when the restoration was first attempted. The Status window keeps you fully informed.
When you are told that the retrieval has ended, click on the Cancel button. Beware - If you click on Ok, the process will begin all over again!
If you access Unix Host in 'line mode' (for example, via telnet), you will have to type UNIX commands at the prompt. This means you have to know, or copy accurately, the appropriate commands. Many Unix Host users access their accounts in line mode, and will therefore have to archive and restore data in this way. Some of the longer strings of commands have been put into UNIX command files to make life easier for you.
For the majority of users who use 'line mode' access, the line command is as follows :
nsrarchive -b Archive -C "Archive Clone" -V [file_name(s)]
To specify the Archive backup pool, we use the -b Archive option. To instruct the system to make a Clone copy in the Archive Clone pool, we use -C "Archive Clone" option (the "" are needed because of the space between the words Archive and Clone). To have the system verify the transfer we use -V. As with all UNIX commands, case is sensitive here, so please make sure you follow the above closely.
Beware! If no filenames are specified, the full current directory and all sub-directories will be archived!
After issuing the above command, you are prompted for the Annotation text to go with the Saveset. You are recommended to start the annotation text with your userid, as this will facilitate recovery of the archive files later.
For example you might type:
zzalstm -Course directory containing Unix course exercise files
End the data entry by typing <Enter> followed by Ctrl-D (that is, press the <Control> and D keys together). Do not press <Enter> again!
What you will see on your screen will look something like this:
As with the GUI method of archiving, this process may take several minutes, or tens of minutes, to complete, as the files are copied directly to tape, and not via a (faster) intermediate disc cache. This is the result of the commands above:
For more details on the available options, type:
man nsrarchive
Once the files have been archived successfully, find out what the ssid (the numerical Saveset id) is by typing:
lsarch[identifier] (more about this command in section 2.2.3)
where [identifier] is either your userid or any other descriptive word you have used in the annotation. If you have used the procedure outlined above, there will of course be two references to your archived material - the original and the clone archives.
Here I have listed Savesets in the archive that have the word "Course" in them. You can see that the ssid is the same for both the original archive and clone archive copies of the Saveset: 30762.
Note the ssid down for future use.
For 'line mode' users, the command is :-
nsrretrieve -s backup.mcc.ac.uk -S [Saveset id]
where [Saveset id]is the unique numerical value you were given when the file(s) was archived (more about this below). When the Saveset is restored, the file names are listed on the screen and your prompt is returned.
The files are restored by default to their original locations. If files of the same name exist you will be prompted to either rename or overwrite the files, as in the following example:
At present, you also need to specify the name of the archive server ( backup.mcc.ac.uk) using the -s option.
If you wish to write the files recovered into a different directory, you must also type
-d[destination] after the nsrretrieve command, as follows:
nsrretrieve -d [destination] -s backup.mcc.ac.uk -S [ssid]
For example, to restore the Course archive into a subdirectory I created named zzalstm off the temporary directory /tmp, I typed:
nsrretrieve -d /tmp/zzalstm -s backup.mcc.ac.uk -S 30762
as in the following example:
As I created the directory /tmp/zzalstm myself beforehand, I am able to list the restored files as in my own home directory. If, however, I had allowed the system to create it for me, the directory would only have execute permission and I would be denied permission to list or read the files! I could, however, copy the files from there to another directory (for example to my home directory) using the command:
cp /tmp/zzalstm/Course/* $HOME
Note: The /tmp and /work directories are only temporary directories, cleared out frequently, so any files you put there should be moved to another location as soon as possible.
For further details, type:
man nsrretrieve
2.2.3 Finding the ssid (Saveset id)
To restore your archived files using Line Mode access (eg via telnet ) you need to know the numerical Saveset id (ssid). A routine has been written to help you to do this. The command is
lsarch [-a]or
lsarch [identifier]
lsarch -a gives a list of all Savesets that have been archived on the system. You only need to use this if you cannot remember what you typed in the annotation for your files.
If you know that you added your username or a description of the contents of the archive, then use the command:
lsarch [identifier]
where [identifier] is either your userid or a descriptive word; for example:
lsarch zzalsas
will find archive sets with "zzalsas" in the annotation, and
lsarch UNIDO
will find archive sets with "UNIDO" in the annotation. (Note: this will show all archive files which contain the word "UNIDO" - not just those that belong to you.)
You will be presented with a list of all the archived files that contain the identifier. For example, a search for the identifier 'zzalsas' produced the following result:
The command includes pg capabilities, so if you are presented with a long list, you can move backwards and forwards through it in pages by using -n or +n at the colon prompt (where n is the number of pages you want to move backwards or forwards). You could also search for a word in the list - for example search for the word 'Census' by typing /Census at the colon prompt.
3. Recovering Old UniTree Archived Files
The UniTree archive system is no longer in operation. Files which were stored under the UniTree archiving system have now been stored in a special UniTree directory on the Legato Backup System (that is, not in the new permanent Legato Archive System described in this document) as many of these files may be redundant. Users with UniTree files who wish to recover them should use the following procedures. These procedures are fully documented in the User Note 'Legato Networker (Solstice) Backup' CSS 204 (2nd edition). A brief description follows:
3.1 Recovering UniTree Data (GUI Interface)
Launch the Legato Networker GUI interface by typing
nwrecover &
at the prompt. After a few seconds the Networker Recover interface will appear.
From the Change menu select
Browse Time
The following panel is displayed:
In the dialogue box, change the browse time to 1st November 1996, as illustrated. Then click Ok to return to the main Networker Recover interface.
In the Selection box, change the path to /unitree/userid (where userid is your own user name, for example, abcdefg).
You will be presented with a list of files in that directory. You can browse the files and sub-directories. Select the files you wish to recover by clicking in the box to the left of each filename. Selected files will be marked with a tick.
So that the files are recovered to a different location, from the Options Menu select relocate and type in the name and path of your home directory (for example /home/cs6400/abcdefg or wherever you want the files to be moved to).
Finally, click on the Start button. It may take some time for your files to be recovered to the new location.
3.2 Recovering UniTree Data (Line Mode)
If you are not able to access the GUI interface, you can also recover old UniTree files using a command line interface. This is achieved by taking the following steps:
1. At the prompt type:
recover
The prompt will become recover>
Type help for a list of commands you can use. The display from help is shown in the next figure:
2. You must change the time you are browsing by typing:
changetime1 November 1996
as this is the date at which the UniTree files were backed up (see panel above).
You may see an error message similar to the following:
cannot browse /home/cs6400h/abcdefg/ as of '11/01/96 23:59:59'
If so, don't worry, just carry on with the instructions as normal.
3. Go to your own UniTree subdirectory by typing:
cd /unitree/[userid]
where [userid] is your own user name. This is where your old UniTree files are stored.
4. To specify the path the files are to be be recovered into (for example into your own
account) type:
relocate [dir]
If you do not use relocate to specify a path, by default the files are restored to the directory /work/[your userid] . This is only a temporary directory so you must remove the files from here fairly quickly.
Note: It is recommended that you create this directory yourself before starting, if it does not already exist, by typing:
mkdir /work/[your userid]
The reason for this is that, if the Backup system creates the directory for you, it is created with only execute permission, and this does not allow you to list the contents of the directory. You can, however, copy the file(s) from there to another directory (for example, to your home directory) using:
cp /work/[your userid]/[filename(s)]$HOME
5. View the files available via the command ls, and from those shown, choose the ones you wish to copy into your account by adding them to the list. For example:
add bigfile.data
6. To check the list of files marked for recovery type list.
7. When you are happy with the list, type
recover
It will take some time to recover the files requested.
If you have problems archiving or recovering your data, please check you have followed the instructions carefully, and check the on-line man pages for further details.