AWS adds support for batch processing in its managed EKS Kubernetes solution

Primarily relating to programs that can be run with minimal human interaction, batch processing is generally considered an important feature of high performance computing.

To meet this need, Amazon Web Services Inc. recently announced that it has integrated AWS Batch and Amazon EKS services with each other to help businesses more easily run batch workloads in the cloud.

“We recently announced our batch support for EKS, our managed solution Kubernetes offering at AWS,” said Ian Colle (pictured), general manager of HPC at AWS. “And so batch processing is still a big part HPC workloads.

Colle spoke to industry analysts theCUBE David Nicholson and Paul Gillin at SC22, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s live streaming studio. They discussed the use cases of batch processing and its connection to HPC. (*Disclosure below.)

IT areas ready to take advantage of the update

With batch processing, a computer processes a number of tasks simultaneously as a group, and adding the functionality to EKS is poised to benefit niches such as autonomous vehicle simulations, according to Colle.

“We see a lot of distributed machine learning, autonomous vehicle simulation and traditional HPC workloads leveraging AWS batch processing,” he said.

A trait that adds versatility and cost-effectiveness to an AWS implementation is the ability to dynamically scale computing resources based on a workload’s queue depth. Clients can go from “seemingly nothing” to thousands of nodes, Colle explained.

“While they perform their work, they only pay for the times while they are working,” he said. “And then when the queue depth starts to decrease and the number of pending jobs in the queue begins to fall; then we start to dynamically reduce those resources. So it is extremely powerful.

In terms of the physical location of the K8 workload and batch processing cluster, it’s entirely up to customer preference, according to Colle.

“We have fully-fledged workflows in a single region, so where they could have some of, say, the traditional HPC workflow, in that region as well as the batch, and they save the results to shared storage file system,” he said. “Or you can have customers that have some sort of multi-regional orchestration layer.”

Here’s the full video interview, some of SiliconANGLE and theCUBE’s coverage of the Event SC22:

(*Disclosure: This is an unsponsored editorial segment. However, theCUBE is a paid media partner for SC22. Neither Dell Technologies Inc., the presenting sponsor of theCUBE event coverage, nor other sponsors n have editorial control over the content of theCUBE or SiliconANGLE.)

Photo: SiliconANGLE

Show your support for our mission by joining our Cube Club and our Cube Event community of experts. Join the community that includes Amazon Web Services and CEO Andy Jassy, ​​Dell Technologies Founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many other luminaries and experts.

Comments are closed.