Index of /~liangli/ZMTk

      Name                    Last modified       Size  Description


[DIR] Parent Directory        21-Feb-2005 01:51      -  
[   ] ZMTk-install.pl         21-Feb-2005 01:34    20k  
[   ] ZMTk.2.3.tgz            21-Feb-2005 01:51    22k  GZIP compressed tar ar>
[TXT] autossh.html            21-Feb-2005 00:52     3k  
[DIR] bin/                    21-Feb-2005 00:07      -  
[DIR] doc/                    10-Jun-2004 04:17      -  

Copyleft 2003-2005 Liang Li (liangli@mail.desy.de)

General information about ZENITH-job Management Toolkit (ZM/Tk)

Version 2.3

1. UP TO DATE INFORMATION
   
   please visit http://www-zeus.desy.de/~liangli/ZMTk

2. WHY?

   Do you use ZENITH/EAZE/ORANGE? If yes, then ZMTk can be a big help.
   Most of us (especially new people) have to spend lots of time
   on making suitable 'control.cards', submitting them to the ZENITH PC FARM, and waiting 
   them to be finished. Sometimes the nutples were too big, sometimes too small. 
   After many tuning, you thought you got the best choice and submitted all 
   your nicely tailored ntuples. But just after you got your first ntuple done, 
   you realized you 'accidently' made some mistake in the code (oops!) so you need to 
   restart the whole process... Finally, your jobs were 'sucessfully' processed
   and you actually saw nutples generated, you couldn't wait to 'grab' them all.
   But wait..., you have been sitting in front of the computer and typing "jobsub..."
   for several days now and never typed "jobpurge...", you have no idea which job
   is wrong or right ?! Search the millon jobs you submitted one by one? or shall we
   restart from the scratch? Tough choice...
   
   Physists SHOULD NOT waste their time fighting with these problems while there are
   more important tasks to do.

   ZM/Tk is specifically designed to avoid these 'non-physics' problems -- 
   a powerful computering tool which can feed any technical needs related
   to control cards formatting, ZENITH-job submission & retrive and a lot more (see below). 
   Everything is done automatically by the shell and perl script, all you need to do 
   is to type a few commands. 
   
   ZM/Tk is distributed AS IS without any warranty. You are allowed to freely distribute
   and modify it for educational and acdamic purposes. It has been so far tested 
   by several groups and many users at ZEUS under variant Unix/Linux environments.

3. HOW TO ... ?

   For installing/upgrading ZM/Tk, please see 'doc/INSTALL'

   ZM/Tk contains several small programs for different functionalities. Each program 
   communicate with each other and integrate as one big program.
   By typing the name of individual program, you get detailed introductions for
   how to use them (self-documented).
   
   General guideline here: 
   First, you use 'zeusinfi' to generate a set of control cards files. You may
   skip this step if you already have your own cards files, but remember, your cards
   filename have to be 'something.extension.cards', like data99.100.cards or ard.abc.cards,
   for the submission script to work. 'zeusinfi' can also duplicate your set of control
   cards and regenerate files with suitable filenames, only takes seconds. 
   
   Second, you use 'sub' or 'suba' to submit them in a row, like 'sub data99 1 M' or 
   'suba data99 1 100 M'.
 
   Third, you use 'joba' (main program) to automatically check the jobs status, 
   fetch the finished jobs, search for any specific jobs and kill & purge those you 
   don't want to keep anymore.

   That's it, simple as it is.

4. TROUBLESHOOTING
  
   a) "Permission denied" or "No remote authentication"
      First make sure there is no general network problem, otherwise no need for additional checks. 
      The error may be due to two reasons: file permission set wrong or automatic login failed. 
      To solve that:
      First, issue a 'klog' command to make sure your AFS token is still valid, then check if 
      you have the write permission in your ntuple destination (no typos etc.) 
      Second, start a new terminal, rerun 'zmtk-intall.pl' to test your automatic login. 
      Third, if 'rsh' does not work for you, see http://www-zeus.desy.de/~liangli/ZMTk/autossh.html 
      for automatic ssh login. All you need is to be able to automatically login to 
      'zenith' from the machine where you run ZM/Tk, and from 'zenith' automatically 
      login to the machine where you store your ntuples (may or may not be the machine 
      where you run ZM/Tk). 
      Two simple test commands:
      ssh -vv zenith (from the machine where you want to run ZM/Tk)
      ssh -vv machine-where-the-ntuples-are (from zenith)
      If both of them works (i.e. 'ssh' succeeds without asking for password) then you are done,
      otherwise you need to look into the debug output and change machine's ssh settings.
   
   b) ZM/Tk Prompt for password very frequently
      This is a similar problem to a) but mostly happens when you do a remote login
      (e.g. 'ssh' etc.) and run ZM/Tk from there (instead of running it 'locally').
      To solve that, first try 'klog', if still doesn't work then try to start a new 
      terminal and login to another machine (some machine's ssh settings are so screwed up
      that it might not be possible to do a automatic ssh login from/to that machine).
            
   c) "Warning: No xauth data; using fake authentication data for X11 forwarding."
      This is just a warning message, you can basically ignore it unless it gets annoying.
      It usually happens when you 'ssh' from a remote machine that cannot contact your 
      local display to create an authorization using the MIT-MAGIC-COOKIE-1 protocol.
      The command
      xauth list
      displays all your xauth data. If there is no entry for your local display - 
      something like localhost/unix:0 MIT-MAGIC-COOKIE-1 026666083d4e480d3e77544949281b49
      you can generate one using the command
      xauth generate :0 .  
      The period "." is part of the command, type 'man xauth' to see details.
      ZM/Tk usually uses 'ssh -q' to turn off the warnings. Please check the 'joba.export'
      configuration file.
 
    d) "/usr/X11R6/bin/xauth:  error in locking authority file /home/.Xauthority"
       This is similar to c) but more severe. It ususally happens when ZM/Tk exits abnormally 
       (ctrl+c ?). The command
       xauth -b
       should take care of the problem (may need to do 'klog' first), type 'man xauth' to see details.

    e) ZMTk keeps complaining "no matches found /jobspool..." and couldn't get ntuples.
      This is a ZENITH problem (usually caused by a major change in the ZENITH server environment, 
      like what happened on Feb 9, 2005). 
      Normally when a ZENITH job is submitted, it is stored at '/jobspool0/usr#jobid/' 
      or '/jobspool1/77/usr#jobid' etc. If for any reason, this pattern is changed, you need to 
      get the updated information from ZENITH people and change the corresponding line in
      'joba.export' or individual files (search for 'JOB_PATH').


    
5. HELP & SUGGESTION

   Still having questions? want to help to improve this utility?
   Send me an email -- liangli@mail.desy.de


   Hope you will enjoy ZM/Tk.

   -- Liang Li  Feb 20, 2005