Using AWS Glue ETL Job with Streaming Data

   Go back to the Task List
  « 3: Create Kinesis Data Stream    5. Register the Device »

4. Create S3 Bucket

In this task, you create an Amazon S3 bucket dojo-data-stream which is used as destination for the Glue ETL job. The bucket will also have a folder script which will keep the generated script of the ETL job. We see more of this later.

  1. Goto the S3 Management Console and create a S3 bucket with name dojo-data-stream. Please make sure you select the region of the bucket the same which you choose for the workshop. If this bucket name is not available, kindly create bucket with another name and use that in the configuration in the remaining part of the workshop.

    S3 Bucket

  2. Create script folder in the bucket dojo-data-stream using the + Create folder button.

    S3 Bucket

  3. The bucket is ready as well. The next task is to configure the IoT device thing which can send streaming data to the Kinesis data stream.