Scenarios where Nimrod may Assist your Experiment

These all relate to running computational models. If the model is computationally intensive then the Nimrod tools are particularly useful as they perform runs concurrently. But even for models which run in milliseconds, the tools can be helpful for organizing your experiment, optimizing output or determining the importance of each input. Most importantly, for nearly all scenarios there is no need to modify your existing code.

1. You need to run your computational model many times and average the results because the model is stochastic
   
   These types of Monte Carlo simulations are greatly facilitated by Nimrod/G. It will generate multiple jobs, furnishing them with different random number seeds and execute them concurrently on the grid. For a simple example see here.
2. You wish to explore the effect of varying the inputs to your model.
   
   Inputs may be in the form of command line parameters or of values in some input file; in Nimrod these are both called parameters. Either way Nimrod/G will let you specify values for each parameter, will generate the jobs for all possible combinations of these values and execute the jobs on the Grid. Examples are here.
3. You wish to explore the effect of input parameters but for certain combinations are not allowed.
   
   There are several possible ways to do this. If the disallowed combinations give jobs that quickly fail then you may schedule all possible combinations and let those bad jobs fail. However, if those jobs will use substantial resources then it would be better to filter them out first. This can be done in Nimrod/G by processing the run file. Alternatively Nimrod/O may be used in sweep mode as it provides the functionality to add formulas for constraints on the parameters. Examples of all three are solutions are here.
4. You are running several different experiments. They may produce some jobs with the same inputs and you don't want to waste resources duplicating these jobs.
   
   Nimrod/O can be invoked with a persistent cache shared by the experiments, so avoiding duplications.
5. You wish to find the combination of input parameters that produce an optimal output.
   
   Nimrod/O allows you to specify the inputs that vary, any constraints on those inputs, and the output to be maximized or minimized. It offers a selection of search methods for finding the optimum. Multiple searches, perhaps using different optimization methods, may be run in parallel. An example is shown here.
6. You want to find the inputs that generate the best output. There is no numerical measure of how good the output is but you know good when you see it.
   
   Nimrod/O allows optimization where human input may be used to assess the quality of an output. This is especially useful for complex outputs such as images or animations.
7. There are several aspects of your model output that you wish to optimize.
   
   Nimrod/O is currently being modified to facilitate such multi-objective optimization. Contact Tom Peachey at tcp@infotech.monash.edu.au for a progress report.
8. You wish to perform multiple optimizations on the ouput of your model, one optimization for each combination of some parameter settings.
   
   This can be done using Nimrod/G to sweep over the setting and having it call Nimrod/O to do each optimization.
9. You wish to run your model in reverse. In other words you want to find the input values that will produce a given output.
   
   Nimrod/O is commonly used for this task. You need to have a way of measuring the discrepency between your output and the desired one. Then run an optimization to minimize this discrepency.
10. You have developed your own optimization method. You wish to use the method, taking advantage of the distributed execution of jobs and the cacheing supplied in Nimrod/O.
    
    This can be done by calling the Nimrod/O library functions from your code. Nimrod/O can then be co-scheduled with your optimization.
11. Your model has many input parameters of interest. Exploring all combinations of these generates far too many computationally intensive jobs.
    
    Using Nimrod/E to design an appropriate subset of the jobs should vastly reduce the number of jobs. The reduced set will be handled by Nimrod/G and then Nimrod/E post-processing can be used to fill in results for the other jobs.
12. Before performing a sweep over the inputs you wish to determine which inputs have little effect and may be disregarded.
    
    Nimrod/E will produce an experiment that will recommend which parameters to sweep and which to ignore.
13. You wish to run your model many times but each run takes weeks. If there are say, memory problems or node failures, and a job fails, then all that time is lost.
    
    A job with very long execution times is best staged, with files saving the results at each stage. Nimrod/G can be used perform sweeps wherein jobs are performed in stages. It ensures that a stage will commence only when the previous one is complete.
14. Different parts of your model need to run on different machines because of data files perhaps or licence requirements.
    
    Nimrod/G can schedule such workflow experiments and perform the appropriate communication between the parts. Where possible a downstream component may start processing its input file before an upstream component has completed that file.
15. Some of your model executions require prior execution of other models.
    
    Nimrod/G can handle such dependencies.