ANL's OSG BOSCO Job Submission sw. Because we still need grids, in addition to clouds
So what is the difference between Grids and Clouds, and why we still need (badly) grids
Grids
assume a fixed size of resources and the management software (LSF, PBS,
GE, and so on) assumes there is more user demand, especially at peaks,
- than the resources available. As a result there are a lot of policies
based on priorities, urgency, set from the top down. Users expect
different levels of SLA's depending on how many want to access the grid
simultaneously. For low level users, the service levels decay
substantially.
A cloud is a grid with elasticity and billing. The elasticity means the service levels can be maintained, as resources are added or withdrawn automatically as needed. The billing means one can can always get what s/he pays for and does not depend on top-down policies. This creates an entire new business model.
A cloud is a grid with elasticity and billing. The elasticity means the service levels can be maintained, as resources are added or withdrawn automatically as needed. The billing means one can can always get what s/he pays for and does not depend on top-down policies. This creates an entire new business model.
Another key difference from Grids and Clouds,
is that in clouds the utilization of virtual resources is only 17% for
small # of instances and it can drop to 2% for very large # of
instances .The fact that one rents a virtual node, it does not mean that it
can not sit idle for days and weeks. This is a bombshell slide titled "Under-utilization is rampant" from Cloudyn regarding Clouds.
In grids,- manged by DRM (Distributed Management Software) such as Grid Engine (Univa, Oracle Grid Engine, Son of Grid Engine,) , LSF (now with IBM), Condor or PBS Professional, - the software knows exactly what is going on on every single node, physical or virtual and this makes possible the maximum utilization closer to 100%
Sure we can use clouds, but the clouds must have configured clusters (Grids), identified as a resource within a cloud. Normally the grids are stand alone, and multiple grids are accessed at the time.
Sure we can use clouds, but the clouds must have configured clusters (Grids), identified as a resource within a cloud. Normally the grids are stand alone, and multiple grids are accessed at the time.
A new Job Submission software, called Bosco. Why we need it?
My ex colleague in Sun, Dan Fraser is a Principal Solutions Architect with Argonne National Lab and a Production Coordinator with Open Science Grid (OSG)
In Sun Dan created the first ever web portal to access a grid the easy way.Those familiar with DRM software, know it is not easy to use , unless you are an experienced system administrator. And faced with a myriad of grids managed with different DRM brands, why not use a single tool to access them all ? In the cloud, outside the cloud, where grids access is permitted
 |
| Dr. Dan Fraser, Argonne National Labs |
Bosco Job Submission software is probably the ideal product to implement the Campus grid concept. Imagine writing only one script, and then the software does the rest: it talks with GE or LSF or Condor or PBS, without bothering the user to change anything. This is huge. Don't be deluded of the appearance that a Job Submission software is a legacy product. OSG proves this tool is needed more than ever,
When one says; "Hmm, how I didn't think of this before?", this is a sign we bumped into a great idea
Bosco v1.0 is a beta release
Q. Why OSG decided to there a need for BOSCO?
We are trying to make it as easy as possible for a researcher (who in most cases is not a system administrator) to access more resources than the single cluster they may be currently using. Using this desktop tool, they can gradually expand to use other resources on their campus, or even utilize resources on a collaborators campus. Bosco provides a stepping stone to enable the automatic distribution and management of jobs across multiple resources. The beauty of this model is that once they start accessing multiple resources in this manner, they can even use Bosco to access resources from the OSG itself.
Q Are there any similar BOSCO software Job Submission schedulers?
Most batch scheduler systems have software that schedules jobs between clusters of the same flavor. In other words PBS users can schedule and distribute jobs across different PBS systems if it is configured properly. Grid Engine has this feature, as does Condor, and LSF. Bosco allows users to schedule jobs across different kinds of batch schedulers at the same time. The only other capabilities I am aware of that do something similar are CREAM (an EGI product) and Globus-GRAM2/5, and these products are akin to what is used in the Open Science Grid.
The biggest difference between these products and Bosco is the security model. Both CREAM and GRAM use the Grid Security Interface requiring users to obtain and manage grid certificates and software. For new grid users however, this has proven to be an unfamiliar and daunting requirement. Bosco is designed to utilize whatever security the Campus uses. This is both familiar and readily understandable to Bosco users.
Q. Do they track downloads?
Currently they are not tracking downloads.
Q. Is Bosco in use only for Open Science Grid researchers? Any external users?
Bosco was designed for researchers that may not yet be using the Open Science Grid. They can start by using resources within their campus, or perhaps accessing resources on another campus. Once users use this model, it is very straightforward to start using the national cyberinfrastructure such as OSG.
Q. How many open grid researchers work High Throughput?
The Open Science Grid is 100% dedicated to high throughput computing.
Q. Who is responsible for project Bosco?
Dan Fraser is the Project Lead. In addition to Argonne, he also has a joint appointment with the University of Chicago. Brooklin Gore is the Project Manager for Bosco. He works at the University of Wisconsin, Madison.
Q. Is support free for outside users? Or only community support?
Currently, as for many open source projects, support is free for outside users as well as community users.
Bosco will have a dedicated product URL. Until then you can download the open source, play with it, and perhaps make money with it.
Comments