Brandon Barker is a Computational Scientist working at the Cornell Center for Advanced Computing with research interests in safety-critical programming, parallel computing and linear modelling of metabolic systems.
It is not uncommon to have computational analyses running for many days or weeks. Software, hardware, and power failures all present the possibility that a significant amount of work could be lost. In some cases, the programmer can incrementally save data at specific intervals, but this is an error-prone process, and it is time-consuming to implement for each application. If a failure occurs, all data not saved will need to be regenerated again, and the researcher can only hope that another failure won't occur.
There is general solution known as Checkpoint/Restart (C/R) that can work for serial and a large variety of parallel programming applications. The primary advantage of all C/R implementations is the automation of saving program state at specified increments and allowing the program to be resumed. There are numerous C/R frameworks and implementations, each offering various advantages and disadvantages; despite some implementations being very mature, C/R remains an area of open and active research as no single solution covers every application type. By knowing the capabilities and drawbacks of each C/R solution, as well as the requirements and specifications of your application, it should be straightforward to choose a C/R framework that is right for you.
In this talk, we discuss in more detail what checkpointing is, and several scenarios where one would and would not want to use it. Next we discuss several popular checkpointing solutions, giving examples when pertinent of software that would not work well with each solution. Finally, we give some simple examples of how checkpointing can be implemented on your own system (Linux currently required).
Date: Dec 8, 2014
Time: 11:00 AM
Location: Weill hall, Room 121
Webex (webcast)
Slides
Showing posts with label HPC. Show all posts
Showing posts with label HPC. Show all posts
Thursday, December 4, 2014
Monday, October 27, 2014
Date: Nov 3, 2014. Steve Lantz: Parallel MATLAB: the Parallel Computing Toolbox, MDCS, and Red Cloud
Steve Lantz is a Senior Research Associate working at the Cornell Center for Advanced Computing with research interests in numerical modeling, fluid dynamics and parallel computing.
MATLAB can be very useful as a tool for data analysis and interaction. In the typical scenario, you simply run it on a single-user computing resource like a laptop, controlling it through some convenient combination of scripts, commands, and the GUI. But what happens when your intended analysis starts to take days to run, instead of hours? Or what if your script starts crashing because you have exceeded your local memory?
The Parallel Computing Toolbox (PCT) gives you features that can help you overcome these performance and memory limitations. First, it allows you to write code that may take better advantage of the multiple cores on your local machine, or perhaps even its GPU. If that is insufficient, PCT also allows you to connect your local MATLAB client to remote resources based on the MATLAB Distributed Computing Server (MDCS) software. These remote resources become an extension of your local client, so that you can do large-scale, batch-style processing straight from your laptop. Many of the same PCT strategies that you use to enhance local execution are also able to exploit MDCS; furthermore, you can use a whole cluster’s memory in an aggregated fashion.
In this talk, Dr. Lantz will give a quick overview of the various capabilities provided by PCT. He will look at how scripts can be scaled up from your multi-core laptop all the way to cluster-scale MDCS resources. Finally, he will present CAC’s Red Cloud with MATLAB service as an on-campus source of the extra cycles and memory that you may sometimes require to get your work done. The process of connecting your client to CAC (or any similar MDCS-based service) will also be described.
Date: Nov 3, 2014
Time: 11:00 AM
Location: Weill hall, Room 121
Slides
MATLAB can be very useful as a tool for data analysis and interaction. In the typical scenario, you simply run it on a single-user computing resource like a laptop, controlling it through some convenient combination of scripts, commands, and the GUI. But what happens when your intended analysis starts to take days to run, instead of hours? Or what if your script starts crashing because you have exceeded your local memory?
The Parallel Computing Toolbox (PCT) gives you features that can help you overcome these performance and memory limitations. First, it allows you to write code that may take better advantage of the multiple cores on your local machine, or perhaps even its GPU. If that is insufficient, PCT also allows you to connect your local MATLAB client to remote resources based on the MATLAB Distributed Computing Server (MDCS) software. These remote resources become an extension of your local client, so that you can do large-scale, batch-style processing straight from your laptop. Many of the same PCT strategies that you use to enhance local execution are also able to exploit MDCS; furthermore, you can use a whole cluster’s memory in an aggregated fashion.
In this talk, Dr. Lantz will give a quick overview of the various capabilities provided by PCT. He will look at how scripts can be scaled up from your multi-core laptop all the way to cluster-scale MDCS resources. Finally, he will present CAC’s Red Cloud with MATLAB service as an on-campus source of the extra cycles and memory that you may sometimes require to get your work done. The process of connecting your client to CAC (or any similar MDCS-based service) will also be described.
Date: Nov 3, 2014
Time: 11:00 AM
Location: Weill hall, Room 121
Slides
Subscribe to:
Posts (Atom)