Using Glue Job ETL from REST API Source to Amazon S3 Bucket Destination

   Go back to the Task List
  [« Create S3 Bucket ]    [Create Glue Job »]

Create Glue Connection

Creating AWS Glue Connection is a very important step in completing this excercise. The connection is used to create an ENI in the private subnet of the VPC. AWS Glue job uses ENI to make call to the internet based REST API. It is a dummy connection with sole purpose to create an ENI for the AWS Glue Job. It is actually not going to make any real connection.

  1. Go to the AWS Glue Console, select Connections in left menu and click on the Add connection button.

    Add Connection

  2. You will use JDBC connection type to create a dummy connection. Enter Connection name as dojoconnection, select JDBC as the Connection type and click on the Next button.

    Connection Name

  3. On the next screen, you use dummy values for the JDBC URL, Username and Password fields. Enter JDBC URL as jdbc:http://thehost:80/dev, Enter Username as dummyuser and Password as dummypassword. For the VPC, select the VPC created in the previous steps. For Subnet, select the private subnet of the VPC. Then select the default for the Security groups. Finally click on the Next button.

    Connection Details

  4. On the next screen, click on the Finish button. It will create a dummy connection. Please don’t test the connection as it is dummy and it will fail for sure. But it will serve the purpose of creating an ENI for the Glue job.