Amazon Web Services

Java SDK

Outline

  • Introduction & Installation
  • General concepts
  • Zoom on EC2, S3 and ELB
  • Alternatives

Introduction & Installation

History

  • Java API
  • Integrated with Eclipse
  • Supports
    • EC2 (auto-scaling, ELB, CloudWatch)
    • S3, Glacier, DataPipeline, EMR
    • CloudFront, CloudSearch
    • RDS, DynamoDB
    • SQS, SNS, SES
    • Route53, ElasticBeanstalk

Installation

Documentation

  • Detailed installation instruction:

http://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/setup-install.html

  • API reference:

http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/index.html

General concepts

Principles

  • Main package: com.amazonaws
    • Authentication: com.amazonaws.auth
    • Services: com.amazonaws.services.<service>
  • Each connection to a service has its own interface (e.g. AmazonEC2)
    • AmazonXXXClient: synchronous interface
    • AmazonXXXClientAsync: asynchronous interface

Credentials

  • Interface com.amazonaws.auth.AWSCredentials
  • Implementations:
    • BasicAWSCredentials(String accessKey, String secretKey)
    • PropertiesCredentials(File file)

Credentials Providers

  • Interface com.amazonaws.auth.AWSCredentialsProvider
  • Implementations:
    • AWSCredentialsProviderChain(AWSCredentialsProvider ...credentialsProviders) (a sort of wallet for credentials)
    • ClasspathPropertiesFileCredentialsProvider (uses a Java properties file in the CP)
    • SystemPropertiesCredentialsProvider (uses Java properties
    • EnvironmentVariableCredentialsProvider (uses env. vars)
    • ProfileCredentialsProvider (uses same profiles as AWS CLI)

Getting credentials

  • Through the provider chain:
    • AWSCredentials = new DefaultAWSCredentialsProviderChain();
      (looks in a number of usual locations)
    • Specifying a specific provider
      • import com.amazonaws.auth.AWSCredentials;
      • import com.amazonaws.auth.profile.ProfileCredentialsProvider;

AWSCredentials credentials = new ProfileCredentialsProvider().getCredentials();

Client

  • Get a <service>Client object for a given region and manipulate it
  • Common constructor arguments:
    • AWSCredentials or AWSCredentialsProvider
    • ClientConfiguration: configures the underlying REST client (proxy, retries, timeout, protocols (HTTP/HTTPS)

Examples:

  • ec2 = new AmazonEC2Client(credentials);
  • s3 = new AmazonS3Client(credentials);

Request interface

A Request object represents a HTTP request through which you provide arguments

  • then provided to a client that will submit to AWS
  • Classes are named after what they allow to manipulate
    • Create<what>Request, Describe<what>Request, Delete<what>Request

Examples:

  • CreateSecurityGroupRequest
  • DeleteInstanceRequest`

Result interface

A Result object represents a HTTP response

  • from which you get the result of your request
  • Classes are name the same way as requests

Examples:

  • CreateSecurityGroupResult
  • CreateInstanceResult

Models

Package com.amazonaws.services.<service>.model

  • Contains the object-oriented representations of AWS entities:
    • Instances, Buckets, SecurityGroups, etc.

Regions

Package com.amazonaws.regions

  • The Enum regions.Region lists all regions
  • Classes regions.Region/regions.RegionUtils:
    • List regions
    • Get a region object from a string id (e.g. eu-west-1)
    • Get a service endpoint
    • Create a client for a service

Documentation

http://aws.amazon.com/fr/sdkforjava/

EC2

Connection to EC2

1AWSCredentials credentials = null;
2try {
3  credentials = new ProfileCredentialsProvider().getCredentials();
4} catch (Exception e) {
5   throw new AmazonClientException (
6     "Cannot load credentials", e);
7}
8ec2 = new AmazonEC2Client(credentials);

Select the region

1Region usWest2 = new Region.getRegion(Regions.US_WEST_2);
2ec2.setRegion(usWest2);

List AMIs

1DescribeImagesRequest request = new DescribeImagesRequest();
2
3request.setImageIds(Arrays.asList('ami-xxxxxxx', 'ami-yyyyyyy');
4DescribeImagesResults res = ec2.describeImages(request);
5List<Images> images = res.getImages();

Run an instance

1RunInstancesRequest request = new RunInstancesRequest(<ami>, <min count>, <max count>);
2RunInstancesResult res = ec2.runInstances(request);
3Reservation reservation= res.getReservation();
4List<Instance> instances = reservation.getInstances();

Query an instance state

1InstanceState state = instance.getState();
2String dns = instance.getPublicDnsName();
3String ip = instance.getPublicIpAddress();
4Date launchDate = instance.getLaunchTime();

Terminate an instance

 1// Get the reservation
 2DescribeInstancesRequest request = new DescribeInstancesRequest();
 3Filter filter = new Filter('tag:name', Arrays.asList('prod'));
 4request.setFilters(Arrays.asList(filter));
 5DescribeInstancesResult res = ec2.describeInstances(request);
 6List<Reservation> reservations = result.getReservations();
 7Reservation resa = reservations[0];
 8// Get the instance
 9instance = resa[0];
10instance.terminate();
11// Or terminate all instances
12resa.terminate_all();

Create a security group

1CreateSecurityGroupRequest request = new CreateSecurityGroupRequest('ssh-access', 'open ssh port');
2ec2.createSecurityGroup(request);

Configure a security group

1// Add a rule
2IpPermission ssh = new IpPermission();
3ssh.withFromPort(22).withToPort(22).withIpRanges('0.0.0.0/0');
4IpPermission http = new IpPermission();
5http.withFromPort(80).withToPort(80).withIpRanges('0.0.0.0/0');
6List<IpPermission> perms = Arrays.asList(ssh, http);
7AuthorizeSecurityGroupIngressRequest request = new authorizeSecurityGroupIngressRequest('ssh-access', perms);

S3

Connection to S3

1AmazonS3 s3 = new AmazonS3Client(
2  new ClasspathPropertiesFileCredentialsProvider());

Create a bucket

1CreateBucketRequest request =
2  new s3.CreateBucketRequest("<name>", "<region>");
3Bucket b = s3.createBucket(request);

List buckets

 1ListBucketsRequest request = new ListBucketsRequest();
 2List<Bucket> buckets = s3.listBuckets(request);
 3
 4for (Bucket b: buckets) {
 5  String s = String.format("====\n" +
 6                           "name: %s\n" +
 7                           "Date: %s\n",
 8                           b.getName(),
 9                           b.getCreationDate());
10  System.out.println(s);
11}

Store objects

1// Store an image from a file
2PutObjectRequest request =
3  new PutObjectRequest("<bucket name>", "<object name>",
4    new File('rihanna.jpg'));
5s3.putObject(request);

Get objects

1// Download to a file
2GetObjectrequest request = new GetObjectRequest("<bucket name>", "<object name>");
3S3Object res = s3.getObject(request);
4S3ObjectInputStream stream = res.getObjectContent();
5IOUtils.copy(stream, new FileOutputStream("morerihanna.jpg"));

List a bucket content

1ListObjectRequest request = new ListObjectRequest().withBucketName("<bucket name>");
2ObjectListing res = S3.listObject(request);
3List<S3ObjectSummary> summaries = res.getObjectSummaries();

ELB

Concept

An Elastic Load Balancer is a server that receives client connections and forwards them to a pool of instances based on defined rules.

Instances can be added or removed from the pool based on their health and auto-scaling policies.

Connection to ELB

1AmazonElasticLoadBalancing elb =
2  new AmazonElasticloadBalancing(provider);

Define a HealthCheck

  • Every 20 seconds
  • With a 15-second timeout
  • On the resource: HTTP:8080/health
  • 3 successful checks: resource is enabled
  • 5 failed checkes: resource is disabled
1HealthCheck hc =
2  new HealthCheck('HTTP:8080/health', 20, 15, 5, 3);

A web server must serve http://<ip>:8080/health

Creating a load balancer

1List<String> zones = Arrays.asList('eu-west-1a', 'eu-west-1b');
2Listener http = new Listener('http', 80, 8080);
3Listener https = new Listener('https', 443, 8443);
4List<Listener> listeners = Arrays.asList(http, https);
5CreateLoadBalancerRequest request =
6  new CreateLoadBalancerRequest('lb1', listeners, zones);
7CreateLoadBalancerResult res = elb.createLoadBalancer(request);

The listener provides network translation from the client to an instance. Its frontend is limited to ports 25, 80, 443, 465, 587 and 1024-65535. Its backend (instance) can listen to any port.

Creating a load balancer (2)

1// Get the load balancer DNS
2String dns = res.getDNSName();
3// Configure the healthcheck
4ConfigureHealthCheckRequest request =
5  new ConfigureHealthCheckRequest('lb1', hc);
6elb.configureHealthCheck(request);

Add instances to a LB

1List<Instances> instances =
2  Arrays.asList(new Instance('i-xxx'), new Instance('i-yyy'));
3
4RegisterInstancesWithLoadBalancerRequest request =
5  new RegisterInstancesWithLoadBalancerRequest('lb1', instances);
6Elb.registerInstancesWithLoadBalancer(request);

Remove instances from a LB

1DeregisterInstancesFromLoadBalancerRequest request =
2  new DeregisterInstancesFromLoadBalancerRequest('lb1', instances);
3elb.deregisterInstancesFromLoadBalancer(request);

Remove a load balancer

1DeleteLoadBalancerRequest request = new DeleteLoadBalancerRequest('lb1');
2elb.deleteLoadBalancer(request);

Alternatives

Boto

  • Python library
  • Official Python SDK for AWS
  • (A lot) less verbose!

https://aws.amazon.com/sdk-for-python/

Libcloud

  • Python library by Apache
  • Broker for multiple IaaS stacks (more than 50!)
  • Supports
    • Virtual machines management
    • Storage service
    • Load balancing service
    • DNS service

https://libcloud.apache.org/

Deltacloud

Deltacloud arcitecture

  • A Broker in the form of a REST proxy
  • Developed by Apache
  • Library bindings (Ruby, Python, C, C++) and REST (via Curl, for example)
  • Supports 15 providers