Developer endpoint provides development environment to create Glue Job using languages and frameworks like PySpark. In this task, you create a developer endpoint which you will use to code with PySpark.
-
Goto the AWS Glue console, click on the Dev endpoints option in the left menu and then click on the Add endpoint button.
-
On the next screen, type in dojoendpoint as the name. Select dojogluerole as the IAM role. Then click on the Next button.
-
On the next screen, select Skip networking information as the option and click on the Next button.
-
On the next Add an SSH public key (Optional) screen, click on the Next button.
-
On the next Review screen, click on the Finish button. The endpoint creation will start.
-
It will take some 8-10 mins for the developer endpoint to be ready. Wait till the status changes to READY.
-
Once the developer endpoint is ready, select it and click on Create Sagemaker notebook under the Action dropdown menu.
-
On the next screen, enter dojonotebook as notebook name, select Create an IAM role as the option, select dojosagemakerrole as the IAM role and then click on the Create notebook button.
-
The notebook creation will start.
-
Wait till the notebook status changes to Ready.
-
The development environment is ready. Let’s do PySpark programming in notebook which then you use later to create a Glue job.