Partitioning is an important technique for organizing datasets so they can be queried efficiently. It organizes data in a hierarchical directory structure based on the distinct values of one or more columns. By default, a DynamicFrame is not…
Amazon SageMaker notebook instance is a managed ML compute instance that runs the Jupyter Notebook Application. The Jupyter notebook enables you to fetch raw files and download them, and even exposes a download button. Due to security and compliant…
It the previous blog post, we discussed on how to Publish and Monitor Metrics from a SageMaker Notebook Instance. There is one caveat while implementing this approach. If we restart the notebook the changes won’t persist. Only the changes made to…
SageMaker Notebook Instances do not publish any metrics to CloudWatch unlike other SageMaker components like Endpoints. This prevents us from observing any metrics and in turn creating alarms on those metrics. However, considering the fact that we…
Glue is an Amazon provided and managed ETL platform that uses the open source Apache Spark behind the back. When you write a DynamicFrame ton S3 using the method, it will internally call the Spark methods to save the file. Since Spark uses the…
When you deploy a SageMaker Endpoint, the following operations occur at the backend. An ML compute instance is provisioned in a service managed account The Model image is downloaded on the instance and the container is run. The ML compute instance…
The instance fleets configuration for EMR clusters allows us to provision core nodes with different purchasing options (On-Demand/Spot). When a job is submitted to EMR, it may run the application master process in any of the available core nodes…
Docker allows us to package and run applications as an isolated process on a shared operating system, acting as a lighter weight alternative to virtual machines. The reason I chose to dockerise my blog was the same as everybody else, 'speeding up the…
Amazon CloudWatch Events enables you to react selectively to events in the cloud as well as in your applications. Using simple rules you can easily route each type of event to one or more targets including but not limited to AWS Lambda functions…
Amazon CloudWatch Events enables you to react selectively to events in the cloud as well as in your applications. Using simple rules you can easily route each type of event to one or more targets including but not limited to AWS Lambda functions…