We have been working closely with our customers on their requests to bring the power of the Amazon Web Services cloud closer to their existing on-premises compute infrastructures. The Amazon Virtual Private Cloud extends on-premises compute with all the power of AWS, making it elastic, scalable and highly reliable. AWS Identity and Access Management brings together on-premises and cloud identity management. VM Import allows our customers to move virtual machine images from their datacenters to the Cloud and Amazon Direct Connect makes the network latencies and bandwidth between on-premises and AWS more predictable. With the launch of the AWS Storage Gateway our customers can now integrate their on-premises IT environment with AWS’s storage infrastructure.

The AWS Storage Gateway is a service connecting an on-premises software appliance with cloud-based storage. Once the AWS Storage Gateway’s software appliance is installed on a local host, you can mount Storage Gateway volumes to your on-premises application servers as iSCSI devices, enabling a wide variety of systems and applications to make use of them. Data written to these volumes is maintained on your on-premises storage hardware while being asynchronously backed up to AWS, where it is stored in Amazon S3 in the form of Amazon EBS snapshots. Snapshots are encrypted to make sure that customers do not have to worry about encrypting sensitive data themselves. When customers need to retrieve data, they can restore snapshots locally, or create Amazon EBS volumes from snapshots for use with applications running in Amazon EC2.

Here are three example use cases that we envision for the AWS Storage Gateway. The first one is using the AWS Storage Gateway to back up your data to Amazon S3’s highly reliable storage environment. Amazon S3 is designed to sustain the concurrent loss of data in two facilities, redundantly storing your data on multiple devices across multiple facilities in an AWS Region. So, backing up your data to Amazon S3 means a lot less headaches worrying about your local storage environment.

The second use case is where customers want to move data between local infrastructure and the Amazon Web Services cloud to provide access to applications and other computations running in Amazon EC2. The use of the Amazon EBS snapshot format means the data that was on-premises can be restored as an Amazon EBS volume mounted to an Amazon EC2 instance.

The third use case, cloud-based Disaster Recovery, is a specific variation of the previous two. If there is a failure in your local infrastructure, you can quickly launch a DR environment in Amazon EC2 which will have full access to the data snapshots backed up into Amazon S3 by the AWS Storage Gateway.

For more information on the AWS Storage Gateway, you can visit the detail page Jeff Barr over at the AWS Developer Blog has more details.

View full post on All Things Distributed

Today, Amazon Web Services is expanding its worldwide coverage with the launch of a new AWS Region in Sao Paulo, Brazil. This new Region has been highly requested by companies worldwide, and it provides low-latency access to AWS services for those who target customers in South America.

South America is one of the fastest growing economic regions in the world. In particular, South American IT-oriented companies are seeing very rapid growth. Case in point: over the past 10 years IT has risen to become 7% of the GDP in Brazil. With the launch of the South America (Sao Paolo) Region, AWS now provides companies large and small with infrastructure that allows them to get to market faster while reducing their costs which enables them to focus on delivering value, instead of wasting time on non-differentiating tasks.

Local companies have not been the only ones to frequently ask us for a South American Region, but also companies from outside South America who would like to start delivering their products and services to the South American market. Many of these firms have wanted to enter this market for years but had refrained due to the daunting task of acquiring local hosting or datacenter capacity. These companies can now benefit from the fact that the new Sao Paulo Region is similar to all other AWS Regions, which enables software developed for other Regions to be quickly deployed in South America as well.

Several prominent South American customers have been using AWS since the early days. The new Sao Paulo Region provides better latency to South America, which enables AWS customers to deliver higher performance services to their South American end-users. Additionally, it allows them to keep their data inside of Brazil. In the words of Guilherme Horn, the CEO of ÓRAMA, a Brazilian financial services firm and AWS customer: “The opening of the South America Sao Paulo Region will enable greater flexibility in developing new services as well as guarantee that we will always be compliant to the needs of the regulations of the financial markets.”

You can learn more about our growing global infrastructure footprint at http://aws.amazon.com/about-aws/globalinfrastructure. Please also visit the AWS developer blog for more great stories from our South American customers.

View full post on All Things Distributed

AWS Import/Export transfers data off of storage devices using Amazon’s high-speed internal network and bypassing the Internet. With this new functionality AWS Import/Export now supports importing data directly into Amazon EBS snapshots. Once loaded into an Amazon EBS snapshot, The customer can create a volume based on that snapshot and attach it to an Amazon EC2 instance, or they can share that snapshot with others.

Amazon Import/Export is an important tool for customers to accelerate moving large amounts of data into the AWS storage systems. It is the proverbial “Do not underestimate the bandwidth of a Fedex box” but it are not only bandwidth constrained customers that are using the service, also those who have complex object collection layouts and who feel more comfortable writing it to a disk than transferring it over the network. With Import into EBS customers can now develop arbitrary complex layouts as the import service is doing a full binary copy of the disk into Amazon EBS and is not interpreting file system layouts, etc.

More information on AWS Import/Export can be found at their detail page.

View full post on All Things Distributed

The concept of regions gives AWS customers control over the placement of their resources and services. Next to GovCloud (US) there are five general purpose regions; two in the US (one on the west coast and one on the east coast), one in the EU (in Ireland) and two in APAC (in Singapore and Tokyo). There are different considerations when deciding where to allocate resources with latency and cost being the two obvious ones, but compliance sometimes plays an important role as well. For example a number of our European customers are subject to data residency requirements when it comes to PII data and they use the EU Region to meet to those requirements.

Our government customers sometimes have an additional layer of regulatory requirements given that they at times deal with highly sensitive information, such as defense-related data. These customers are satisfied with the general security controls and procedures in AWS but in these more sensitive cases they often need assurances that only personnel that meet certain requirements, e.g. citizenship or permanent residency, can access their data. AWS GovCloud (US) implements specific requirements of the US government such that agencies at the federal, state and local levels can use the AWS cloud for their more sensitive workloads.

Cloud First

The US Federal Cloud Computing Strategy lays out a “Cloud First” strategy which compels US federal agencies to consider Cloud Computing first as the target for their IT operations:

To harness the benefits of cloud computing, we have instituted a Cloud First policy. This policy is intended to accelerate the pace at which the government will realize the value of cloud computing by requiring agencies to evaluate safe, secure cloud computing options before making any new investments

By leveraging shared infrastructure and economies of scale, cloud computing presents a compelling business model for Federal leadership Organizations will be able to measure and pay for only the IT resources they consume, increase or decrease their usage to match requirements and budget constraints, and leverage the shared underlying capacity of IT resources via a network Resources needed to support mission critical capabilities can be provisioned more rapidly and with minimal overhead and routine provider interaction.

Given the current economic climate, reducing cost within the US federal government is essential — and an aggressive move to cloud will have a substantial positive impact on the governments IT budget. The move to the cloud is projected by 2015 see a reduction of 30% in IT infrastructure costs, which amounts to $7.2 billion. The application of the Cloud First strategy across all agencies will see many cost savings similar to what the GSA saw when they moved their main portal to the cloud: a savings of $1.7M on a yearly basis while greatly improving uptime and maintainability.

With AWS’s strategy of continuous price reduction as additional economies of scale are achieved, many of these cost saving may become even more substantial without the agencies have to do anything.

Many US federal agencies are already migrating existing IT infrastructure onto the cloud using Amazon Web Services. The Cloud First strategy is most visible with new Federal IT programs, which are all designed to be “Cloud Ready”; many of these applications are launching on AWS from the start, and a number can be found on the AWS Federal use case list.

There were however a number of programs that really could benefit from the Cloud but which had unique regulatory requirements, such as ITAR, that blocked migration to AWS. ITAR is the International Traffic in Arms Regulatory framework which stipulates for example that data must be stored in an environment where physical and logical access is restricted to US Persons. There is no formal ITAR certification process, but a review of the ITAR compliance program for AWS GovCloud (US) has been conducted and resulted in a favorable letter of attestation with respect to the stated ITAR objects. This clears the path for agencies that have IT programs that need to be ITAR-compliant to start using AWS GovCloud (US) for these applications.

This new region, like all other AWS regions, provides FISMA Moderate controls and supports existing AWS security controls and certifications such as SAS-70, ISO 27001 and PCI DSS Level 1.

Government and Big Data

One particular early use case for AWS GovCloud (US) will be massive data processing and analytics. Several agencies of very different parts of the government have needs for data analytics that really put the Big in Big-Data, sometimes several orders of magnitude larger than commonly found in industry. Examples here are certain agencies that work on national security and those that work on economic recovery; their incoming data streams are exploding in size and their needs for collecting, storing, organizing, analyzing and sharing are changing rapidly. It is very difficult for an on-premise IT infrastructure to effectively address the needs of these agencies and the time scales at which they need to operate. The scalability, flexibility and the elasticity of AWS makes it an ideal environment for the agencies to run their analytics.

Often the data streams that they operate on are not classified in nature, but the combination and aggregation of these streams using complex new algorithms may fall for example under the controls of ITAR. AWS GovCloud (US) will be used by several of these agencies to help them with their Bigger-than-Big-Data needs.

More information

As with all AWS services and regions, information on GovCloud is publicly available on the AWS website, However, given the restrictive nature of this new AWS Region, customers will need to sign an AWS GovCloud Enterprise agreement that requires a manual step beyond the usual self-service signup process. To make use of the services in this region, customers will use the Amazon Virtual Private Cloud (VPC) to organize their AWS resources.

As the name of the region already suggests, we do not envision that over time GovCloud will address only the needs of the US Government and contractors. We are certainly interested in understanding whether there are opportunities in other governments with respect to their specific regulatory requirements that could be solved by a specialized region.

For more details on the AWS GovCloud (US) visit the Federal Government section of the AWS website and the posting on the AWS developer blog.

View full post on All Things Distributed

Caching has become a standard component in many applications to achieve a fast and predictable performance, but maintaining a collection of cache servers in a reliable and scalable manner is not a simple task. These efforts clearly fall into the category of “operational muck”, but given the widespread usage of caching, maintenance of cache servers is no longer a differentiator and everyone will have to uptake it as the “costs of doing business”. Amazon ElastiCache takes away many of the headaches of deploying, operating and scaling the caching infrastructure. A Cache Cluster, which is a set of collaborating Cache Nodes, can be started in minutes. Scaling the total memory in the Cache Cluster is under complete control of the customers as Caching Nodes can be added and deleted on demand. Amazon Cloudwatch can be used to get detailed metrics about the performance of the Cache Nodes. Amazon ElastiCache automatically detects and replaces failed Cache Nodes to protect the cluster from those failure scenarios. Access to the Cache Cluster is controlled using Cache Security Groups giving customers full control over which application components can access which Cache Cluster.

Amazon ElastiCache is compliant with Memcached, which makes it easy for developers who are already familiar with that system to start using the service immediately. Existing applications, tools and libraries that are using a Memcached environment can simply switch over to using Amazon ElastiCache without much effort.

For more details on Amazon ElastiCache visit the detail page of the service. For more hands-on information and to get started right away, see Jeff Barr’s posting on the AWS Developer Blog. Please note that Amazon ElastiCache is currently available in the US East (Virginia) Region. It will be available in other AWS Regions in the coming months.

View full post on All Things Distributed

As a big music fan with well over 100Gb in digital music I am particularly excited that I now have access to all my digital music anywhere I go.

Order in the Chaos

The number of digital objects in our lives is growing rapidly. What used to be only available in physical formats now often has digital equivalents and this digitalization is driving great new innovations. The methods for accessing these objects is also rapidly changing; where in the past you needed a PC or a Laptop to access these objects, now many of our electronic devices have become capable of processing them. Our smart phones and tablets are obvious examples, but many other devices are quickly gaining these capabilities; TV Sets and Hifi systems are internet enabled, and soon our treadmills and automobiles will be equally plugged into the digital world.

Managing all these devices, along with the content we store and access on them, is becoming harder and harder. We see that with our Amazon customers; when they hear a great tune on a radio they may identify it using the Shazam or Soundhound apps on their mobile phone and buy that song instantly from the Amazon MP3 store. But now this mp3 is on their phone and not on the laptop that they use to sync their iPod with and not on the Windows Media Center PC that powers their HiFi TV set. That’s frustrating – so much so that customers tell us they wait to buy digital music until they are in front of the computer they store their music library on, which brings back memories of a world constrained by physical resources.

The launch of Amazon Cloud Drive, Amazon Cloud Player and Amazon Cloud Player for Android will help to bring order in this chaos and will ensure that customers can buy, access and play their music anywhere. Customers can upload their existing music library into Amazon Cloud Drive and music purchased from the Amazon MP3 store can be added directly upon purchase. Customers then use Amazon Cloud Player Web application to easily manage their music collections with download and stream options. The Amazon Cloud Player for Android is integrated with the Amazon MP3 app and gives customers instant access to all the music they have stored in Amazon Cloud Drive on their mobile device. Any purchases that customers make on their Android devices can be stored in Amazon Cloud Drive and are immediately accessible from anywhere.

A Drive in the Cloud

To build Amazon Cloud Drive the team made use of a number of cloud computing services offered by Amazon Web Services. The scalability, reliability and durability requirements for Cloud Drive are very high which is why they decided to make use of the Amazon Simple Storage Service (S3) as the core component of their service. Amazon S3 is used by enterprises of all sizes and is designed to handle scaling extremely well; it stores hundreds of billions of objects and easily performs several hundreds of thousands of storage transaction a second.

Amazon S3 uses advanced techniques to provide very high durability and reliability; for example it is designed to provide 99.999999999% durability of objects over a given year. Such a high durability level means that if you store 10,000 objects with Amazon S3, you can on average expect to incur a loss of a single object once every 10,000,000 years. Amazon S3 redundantly stores your objects on multiple devices across multiple facilities in an Amazon S3 Region. The service is designed to sustain concurrent device failures by quickly detecting and repairing any lost redundancy, for example there may be a concurrent loss of data in two facilities without the customer ever noticing.

Cloud Drive also makes extensive use of AWS Identity and Access Management (IAM) to help ensure that objects owned by a customer can only be accessed by that customer. IAM is designed to meet the strict security requirements of enterprises and government agencies using cloud services and allows Amazon Cloud Drive to manage access to objects at a very fine grained level.

A key part of the Cloud Drive architecture is a Metadata Service that allows customers to quickly search and organize their digital collections within Cloud Drive. The Cloud Player Web Applications and Cloud Player for Android make extensive use of this Metadata service to ensure a fast and smooth customer experience.

Making it simple for everyone

Amazon Cloud Drive and Amazon Cloud Player are important milestones in making sure that customer have access to their digital goods at anytime from anywhere. I am excited about this because it is already making my digital music experience simpler and I am looking forward to the innovation that these services will drive on behalf of our customers.

If you are an engineer interested in working on Amazon Cloud Drive and related technologies the team has a number of openings and would love to talk to you! More details at http://www.amazon.com/careers.

View full post on All Things Distributed

A well know application area that makes use of the advanced network infrastructure in Japan is social gaming, and it shouldn’t be surprising that some of the early customers of the new AWS Tokyo Region are large gaming companies like Gumi and Zynga Japan. Gumi Inc. is one of the largest social gaming companies in Japan with well over 10 million users per day. Zynga Japan is the Japanese subsidiary of the world’s leading social gaming company, with localized versions of their popular Farmville and Treasure Isle games.

Several Japanese conglomerates are also evaluating Amazon Web Services as a way to radically change the way IT is organized within the enterprise. These corporations are using AWS to improve organizational agility, radically transform resource allocation models, reduce time-to-market, and remove the need to pour financial and intellectual capital into their IT operations that do not meaningfully differentiate their businesses from those of other global corporations.

A local AWS Region enables companies to deploy a global cloud platform for their operations. With the launch of the new Tokyo Region Japanese corporations can now start to benefit from the uniform application deployment that AWS offers world-wide: applications configured and built to run in one Region can be launched in the same fashion in other Regions around the world with minimal reconfiguration.

With the new Tokyo Region companies that are required to meet certain compliance, control, and data locality requirements can now achieve these certifications: customers can now choose to keep their data entirely within the Tokyo Region.

With the launch of this new Region, the AWS Support team is also offering support in Japanese for any customer with a Japanese language preference: AWS Basic Support, which is free to all users, and AWS Premium Support, which provides a 24x7x365 fast-response support channel, are now both available in Japanese.

For more details on AWS in Japan see http://aws.amazon.com/jp

View full post on All Things Distributed

To battle this complexity, developers who do not need
control over the whole software stack often use development platforms that help
them manage their application development, deployment and monitoring. There are
some excellent platforms running on AWS that do precisely this; Ruby on Rails
developers have Heroku
and Engine Yard, Springsource
users have CloudFoundry
Drupal developers can use Acquia,
and PHP aficionados can sign up for class=SpellE>phpfog, just to name a few. These platforms take away
much of the "muck" of software development to the extent that most class=SpellE>RoR developers these days will choose to run on a platform
instead of managing the whole stack themselves.

Developers have continuously asked us to create
similar platforms to simplify development on AWS. However, given that there are
probably as many different approaches to development as there are developers,
instead of creating a particular platform, we are launching href="http://aws.amazon.com/elasticbeanstalk">AWS Elastic Beanstalk, an
application development container that can be the basis for the development of
many different development platforms. It targets both the application developer
by providing a set of simple tools to get started quickly and the platform
developer by giving control over the underlying technology stack.

Elastic Beanstalk makes it easy for developers to deploy and
manage scalable and fault-tolerant applications on the AWS cloud. It takes just
minutes to get started and deploy your first application. AWS Elastic Beanstalk
automatically creates the AWS resources and application stack needed to run the
application, freeing developers from worrying about server capacity, load
balancing, scaling their application, and version control. There is no charge
to use Elastic Beanstalk and developers only pay for the AWS resources used to
run their applications.  Elastic Beanstalk stays true to the AWS
principles by not locking customers into a black box; Elastic Beanstalk creates
resources on behalf of the developer providing transparency and control over
application operations as well making it easy to move applications out of the
container at any time. An Elastic Beanstalk container comprises an application
software stack running on Amazon EC2 compute resources with an Elastic Load
Balancer, pre-configured EC2 Auto-Scaling, monitoring with Amazon class=SpellE>CloudWatch, the ability to store data in Amazon S3, and
multiple database options.

Developers who want even more control have access to the AWS
resources supporting their application and can easily select more advanced
deployment options such as using multiple Availability Zones for higher
availability, logging into their Amazon EC2 servers, opening specific network
ports for use, or taking control of Elastic Load Balancer or Auto-Scaling
settings. The public beta release of AWS Elastic Beanstalk supports a container
for Java developers using the familiar Linux / Apache Tomcat application stack.
We plan to make additional containers available over time including support for
customers and solution providers to develop and share their own containers.

AWS Elastic Beanstalk has been developed in such a way that
other programming platforms can be created relatively easily. This is extremely
important as the AWS developer ecosystem has always been very rich and we want
to keep it that way. Our goal is to ensure every developer’s favorite platform
is always available on AWS so they can stop worrying about deploying and
operating scalable and fault-tolerant application and focus on application
development. In a nutshell, we want to let a thousand platforms bloom on AWS.

Last week I ran into an AWS customer at CES who was
enthusiastic about how his digital production workflow and video encoding is
now running reliably in the cloud. When discussing how AWS could improve to
serve him even better he finished with “I have a bunch of smaller java apps
that I really want to run in AWS but I just can’t be bothered with picking the
right instance size and setting up the load-balancing, etc.” This is exactly
where Elastic Beanstalk will help: to make it even simpler to get started and
to run applications in the AWS cloud. “Easy to begin and impossible to outgrow”
is an excellent characterization of Elastic Beanstalk which handles deployment,
scaling and reliability such that its customers don’t have to.

To get started using AWS Elastic Beanstalk, visit http://aws.amazon.com/elasticbeanstalk.
More information on the launch can be found on the href="http://aws.typepad.com/aws/2011/01/introducing-elastic-beanstalk.html">AWS
developer blog.

View full post on All Things Distributed

Some fundamentals on Naming

Naming is one of the fundamental concepts in Distributed Systems. Entities in a system are identified through their name, which is separate from the way that you would choose to access that entity, the address that the access point resides at and what route to take to get to that address.

A simple example is the situation with Persons and Telephones; a person has a name, a person can have one or more telephones and each phone can have one or more telephone numbers. To reach an individual you will look up him or her in your address book, and select a phone (home, work, mobile) and then a number to dial. The number will be used to route the call through the myriad of switches to its destination. The person is the entity with its name, the phones are access points and the phones numbers are addresses.

Names do not necessarily need to be unique, but it makes life a lot easier if that is the case. There is more than one Werner Vogels in this world and although I never get emails, snail mail or phones calls for any of my peers, I am sure they are somewhat frustrated if they type in our name in a search engine .

In distributed systems we use namespaces to ensure that we can create rich naming without having to continuously worry about whether these names are indeed globally unique. Often these namespaces are hierarchical in nature such that it becomes easier to manage them and to decentralize control, which makes the system more scalable.
The naming system that we are all most familiar with in the internet is the Domain Name System (DNS) that manages the naming of the many different entities in our global network; its most common use is to map a name to an IP address, but it also provides facilities for aliases, finding mail servers, managing security keys, and much more. The DNS namespace is hierarchical in nature and managed by organizations called registries in different countries. Domain registrars are the commercial interface between the DNS registries and those wishing to manage their own namespace.

DNS is an absolutely critical piece of the internet infrastructure. If it is down or does not function correctly, almost everything breaks down. It would not be a first that a customer thinks that his EC2 instance is down when in reality it is some name server somewhere that is not functioning correctly.

DNS looks relatively simple on the outside, but is pretty complex on the inside. To ensure that this critical component of the internet scales and is robust in the face of outages, replication is used pervasively using epidemic style techniques. The DNS is one of those systems that rely on Eventual Consistency to manage its globally replicated state.

While registrars manage the namespace in the DNS naming architecture, DNS servers are used to provide the mapping between names and the addresses used to identify an access point. There are two main types of DNS servers: authoritative servers and caching resolvers. Authoritative servers hold the definitive mappings. Authoritative servers are connected to each other in a top down hierarchy, delegating responsibility to each other for different parts of the namespace. This provides the decentralized control needed to scale the DNS namespace.

But the real robustness of the DNS system comes through the way lookups are handled, which is what caching resolvers do. Resolvers operate in a completely separate hierarchy which is bottoms up, starting with software caches in a browser or the OS, to a local resolver or a regional resolver operated by an ISP or a corporate IT service. Caching resolvers are able to find the right authoritative server to answer any question, and then use eventual consistency to cache the result. Caching techniques ensure that the DNS system doesn’t get overloaded with queries.

The Domain Name System is a wonderful practical piece of technology; it is a fundamental building block of our modern internet. As always there are many improvements possible, and many in the area of security and robustness are always in progress.

Amazon Route 53

Amazon Route 53 is a new service in the Amazon Web Services suite that manages DNS names and answers DNS queries. Route 53 provides Authoritative DNS functionality implemented using a world-wide network of highly-available DNS servers.
Amazon Route 53 sets itself apart from other DNS services that are being offered in several ways:

A familiar cloud business model: A complete self-service environment with no sales people in the loop. No upfront commitments are necessary and you only pay for what you have used. The pricing is transparent and no bundling is required and no overage fees are charged.

Very fast update propagation times: One of the difficulties with many of the existing DNS services are the very long update propagation times, sometimes it may even take up to 24 hours before updates are received at all replicas. Modern systems require much faster update propagation to for example deal with outages. We have designed Route 53 to propagate updates very quickly and give the customer the tools to find out when all changes have been propagated.

Low-latency query resolution The query resolution functionality of Route 53 is based on anycast, which will route the request automatically to the DNS server that is the closest. This achieves very low-latency for queries which is crucial for the overall performance of internet applications. Anycast is also very robust in the presence of network or server failures as requests are automatically routed to the next closest server.

No lock-in. While we have made sure that Route 53 works really well with other Amazon services such as Amazon EC2 and Amazon S3, it is not restricted to using it within AWS. You can use Route 53 with any of the resources and entities that you want to control, whether they are in the cloud or on premise.

We chose the name “Route 53″ as a play on the fact that DNS servers respond to queries on port 53. But in the future we plan for Route 53 to also give you greater control over the final aspect of distributed system naming, the route your users take to reach an endpoint. If you want to learn more about Route 53 visit http://aws.amazon.com/route53 and read the blog post at the AWS Developer weblog.

View full post on All Things Distributed

From financial processing and traditional oil & gas
exploration HPC applications to integrating complex 3D graphics into online and
mobile applications, the applications of GPU processing appear to be
limitless. We believe that making these GPU resources available for everyone
to use at low cost will drive new innovation in the application of highly
parallel programming models.

From CPU to GPU

Building general purpose architectures has always been hard;
there are often so many conflicting requirements that you cannot derive an
architecture that will serve all, so we have often ended up focusing on one
side of the requirements that allow you to serve that area really well. For
example, the most fundamental abstraction trade-off has always been latency
versus throughput. These trade-offs have even impacted the way the lowest level
building blocks in our computer architectures have been designed. Modern CPUs strongly
favor lower latency of operations with clock cycles in the nanoseconds and we
have built general purpose software architectures that can exploit these low
latencies very well. Now that our ability to generate higher and higher clock
rates has stalled and CPU architectural improvements have shifted focus towards
multiple cores, we see that it is href="http://www.cccblog.org/2008/08/26/the-multicore-challenge/">becoming
harder to efficiently use these computer systems.

One trade-off area where our general purpose CPUs were not
performing well was that of massive fine grain parallelism. Graphics processing
is one such area with huge computational requirements, but where each of the
tasks is relatively small and often a set of operations are performed on data
in the form of a pipeline. The throughput of this pipeline is more important
than the latency of the individual operations. Because of its focus on latency,
the generic CPU yielded rather inefficient system for graphics processing. This
lead to the birth of the Graphics Processing Unit (GPU) which was focused on
providing a very fine grained parallel model, with processing organized in
multiple stages, where the data would flow through. The model of a GPU is that
of task parallelism describing the different stages in the pipeline, as well as
data parallelism within each stage, resulting in a highly efficient, high
throughput computation architecture.

The early GPU systems were very vendor specific and mostly
consisted of graphic operators implemented in hardware being able to operate on
data streams in parallel. This yielded a whole new generation of computer
architectures where suddenly relatively simple workstations could be used for
very complex graphics tasks such as Computer Aided Design. However these fixed
functions for vertex and fragment operations eventually became too restrictive
for the evolution of next generation graphics, so new GPU architectures were
developed where user specific programs could be run in each of the stages of
the pipeline. As each of these programs was becoming more complex and demand
for new operations such as geometric processing increased, the GPU architecture
evolved into one long feed-forward pipeline consisting of generic 32-bit
processing units handling both task and data parallelism. The different stages were
then load balanced across the available units.

General Purpose GPU programming

Programming the GPU evolved in a similar fashion; it started
with the early APIs being mainly pass-through to the operations programmed in
hardware. The second generation APIs to GPU systems were still graphics-oriented
but under the covers implemented dynamic assignments of dedicated tasks over
the generic pipeline. A third generation of APIs, however, left the graphics
specifics interfaces behind and instead focused on exposing the pipeline as a
generic highly parallel engine supporting task and data parallelism.

Already with the second generation APIs researchers and
engineers had started to use the GPU for general purpose computing as the
generic processing units of the modern GPU were extremely well suited to any system
that could be decomposed into fine grain parallel tasks. But with the third generation
interfaces, the true power of General Purpose GPU programming was unlocked. In the
taxonomy of traditional parallelism, the programming of the pipeline is a
combination of SIMD (single instruction, multiple data) inside a stage and SPMD
(single program, multiple data) for how results get routed between stages. A
programmer will write a series of threads each defining the individual SIMD
tasks and then an SPMD programs to execute those threads and collect and store/combine
the results of those operations. The input data is often organized as a Grid.

NVIDIA’s CUDA SDK provides a higher level interface with
extensions in the C language that supports both multi-threading and data
parallelism. The developer writes single c functions dubbed a “kernel”
that operate on data and are executed by multiple threads according to an execution
configuration. To easily facilitate different input models, threads can be
organized into thread-blocks that are hierarchies for one-, two- and
three-dimensional processors of vectors, matrices and volumes. Memories are
organized into global memory, per-thread-block memory and per-thread private
memory.

This combination of very basic primitives drives a whole
range of different programming styles: map & reduce, scatter & gather
& sort, as well as stream filtering and stream scanning. All running at
extreme throughputs as high-end GPUs such as those supporting the Tesla “Fermi”
CUDA architecture have close to 500 cores generating well over 500 GigaFLOPS
per GPU.

The NVIDIA “Fermi” architecture as implemented in the NVIDIA
Tesla 20-series GPUs (where we are providing instances with Tesla M2050 GPUs) are
a major step up from the earlier GPUs as they provide high performance double
precision floating point operations (64FP) and ECC GDDR5 memory.

The Amazon EC2 Cluster GPU instance

Last week it was revealed that the world’s fastest
supercomputer is now the Tianhe-1A
with a peak performance of 4.701 PetaFLOPS. The Tianhe-1A runs on
14,336 Xeon X5670 processors and 7,168 Nvidia Tesla M2050
general purpose GPUs. Each node in the system consists of two Xeon processors
and one GPU.

The EC2 Cluster GPU instance provides even more power in
each instance: the two Xeon X5570 processors are combined with two NVIDIA Tesla
M2050 GPUs. This gives you more than a TeraFLOPS processing power per instance.
By default we allow any customer to instantiate clusters of up to 8 instances
making the incredible power of an 8 TeraFLOPS available for anyone to use.
This instance limit is a default usage limit, not a technology limit. If you
need larger clusters we can make those available on request via the href="http://aws.amazon.com/contact-us/ec2-request/">Amazon EC2 instance
request form. If you are willing to switch to single precision floating the
Tesla M2050 will even give you a TeraFLOP performance per GPU, doubling the
overall performance.

We have already seen early customers out of the life
sciences, financial, oil & gas, movie studios and graphics industries becoming
very excited about the power these instances give them. Although everyone in
the industry has known for years that General Purpose GPU processing is a
direction with amazing potential, making major investments has been seen as
high-risk given how fast moving the technology and programming was.

Cluster GPU programming in the Cloud with the Amazon Web
Services changes of all of that. The power of world’s most advanced GPUs is now
available for everyone to use without any up-front investment, removing the
risks and uncertainties that owning your own GPU infrastructure would involve. We
have already seen with the EC2 Cluster Compute instances that “traditional” HPC
has been unlocked for everyone to use, but Cluster GPU instances take this one
step further making innovative resources that were even outside the reach of
most professionals now available for everyone to use at very low cost. An 8
TeraFLOPS HPC cluster of GPU-enabled nodes will now only cost you about $17 per
hour.

CPU and/or GPU

As exciting as it is to make GPU programming available for
everyone to use, unlocking its amazing potential, it certainly doesn’t mean
that this is the start of the end of CPU based High Performance Computing. Both
GPU and CPU architectures have their sweet spots and although I believe we will
see a shift in the direction of GPU programming, CPU based HPC will remain very
important.

GPUs work best on problem sets that are ideally solved using
massive fine-grained parallelism, using for example at least 5,000 – 10,000
threads. To be able build applications that exploit this level of parallelism
one needs to enter a very specific mindset of kernels, kernel functions,
threads-blocks, grids of threads-blocks, mapping to hierarchical memory, etc.
Configuring kernel execution is not a trivial exercise and requires GPU device
specific knowledge. There are a number of techniques that every programmer has
grown up with, such as branching, that are not available, or should be avoided
on GPUs if one wants to truly exploit its power.

HPC programming for CPUs is very convenient compared to GPU
programming as the power of traditional serial programming can be combined with
that of using multiple powerful processors. Although efficient parallel
programming on CPUs absolutely also requires a certain level of expertise its
models and capabilities are closer to that of traditional programming. Where
kernel functions on GPUs are best written as simple data operations combined
with specific math operations, CPU based HPC programming can take on any level
of complexity without any of the restrictions of for example the GPU memory
models. Applications, libraries and the tools for CPU programming are plentiful
and very mature, giving developers a wide range of options and programming
paradigms.

One area where I expect progress will be made with the
availability of the Cluster GPU instances is a combination of both HPC
programming models which combines the power of CPUs and GPUs, as after all the
Cluster GPU instances are based on the Cluster Compute Instances with their
powerful quad core i7 processors.

Some good insight into the work that is needed to convert
certain algorithms to run efficiently on GPUs is the UCB/NVIDIA “ href="http://mgarland.org/files/papers/gpusort-ipdps09.pdf">Designing Efficient
Sorting Algorithms for Manycore GPUs” paper.

Cluster Computer, Cluster GPU and Amazon EMR

href="http://aws.amazon.com/elasticmapreduce/">Amazon Elastic MapReduce (EMR) makes it very easy to run Hadoop’s (MapReduce) massively parallel processing tasks. Amazon EMR will handle workload parallelization, node configuration and scaling, and cluster management, such that our customers can focus on writing the actual HPC programs.

Starting today Amazon EMR can take advantage of the Cluster Compute and Cluster GPU instances, giving customers ever more powerful components to base the large scale data processing and analysis on. These programs that rely on significant network I/O will also benefit from the low latency, full bisection bandwidth 10Gbps Ethernet network between the instances in the clusters.

Where to go from here?

For more information on the new Cluster GPU Instances for
Amazon EC2 visit the High
Performance Computing with Amazon EC2 page. For more information on using
the HPC Cluster instances with Amazon Elastic MapReduce see the href="http://aws.amazon.com/elasticmapreduce/">Amazon EMR detail page. Also
more details can be found on the AWS Developer
blog. James Hamilton has some interestign insights on GPGPU.

View full post on All Things Distributed