Here at Cortexica we have made a number of our Machine Learning models available on the SageMaker marketplace. These include:

Some of these models have been optimised to work with AWS CPU instances, and others have been optimised to work with GPU instances for enhanced performance. When deploying these models, only the instances that they have been optimised for will be available for selection, so you do not need to worry about selecting an inappropriate instance type.

In this article we will discuss how to use AWS marketplace SageMaker models and, in particular, how to use the models made available by Cortexica to annotate an input set of images.

Using SageMaker Marketplace Models

To get started using SageMaker models from the AWS Marketplace, you will need an AWS account with the appropriate permissions set. The SageMaker marketplace is currently only available in the Ohio (us-east-2) region, so switch to that, and then head over to the SageMaker dashboard.

All SageMaker marketplace models are made available by subscribing to them as a “model package”. A model package is a way of bundling the inference image (the model) with allowed run-time parameters (like supported instance types, and acceptable input/output MIME types).

In order to use one of the marketplace models, we must first subscribe to that model. From the SageMaker dashboard follow the link to the AWS Marketplace. Browse the listings, and select a model that you would like to run. For this example we will chose Cortexica’s CPU Fashion localisation model.

Fashion localisation

Click the “Continue to Subscribe” button and, if you agree to the terms, select “Accept Offer”.

If we now navigate back to the SageMaker dashboard, and look at the available model packages, we will see that we have a package available under the AWS Marketplace subscriptions section.

This model can now be deployed like any other SageMaker model in your account. There are generally two ways to utilise a model:

  • Create an endpoint – this will provide you with an API to enable you to call the inference model in real-time.
  • Create a batch transform job – this will spool up the inference model and run multiple calls to the engine for an input data set (hosted on S3). The output data is saved onto S3, and the inference model is deleted once the batch job has completed.

In this example we will look at creating an endpoint with our Fashion Localisation model so that we can make real-time calls to it. There are three elements that are required to create an endpoint:

  1. A model
  2. An endpoint configuration
  3. An endpoint

1) To create a model, on the SageMaker dashboard click Models -> Create Model. Give your model a name, and in the Container input options, choose “Use a model package subscription from AWS Marketplace”. Select the model from the marketplace you wish to use, and click Create Model.

2) To create an endpoint configuration, click the button on the endpoint configurations page. Give the endpoint configuration a name, and then click “Add Model” to add the model that you created in step 1 to the endpoint configuration. You can specify the specific type of instance that the inference model will be run on. As this is a simple example, I will choose a cost-effective instance type here (ml.m5.large), but if you are running heavier workloads you will need to choose your instance type and number of instances accordingly. Once you have selected the correct model and instance type, click Create endpoint configuration.

3) Finally, to create the endpoint, click the Create endpoint button on the endpoints page. Give your endpoint a name and choose the endpoint configuration that you created in step 2. Click Create endpoint.

The endpoint will take a few minutes to spool up. After it does, the status will say InService.

 

Invoking A SageMaker Endpoint

Once you have a SageMaker endpoint running, invocation is easiest via the AWS CLI. Here I am invoking the “fashion-loc” endpoint using the CLI from a bash command prompt:

$ aws --region=us-east-2 sagemaker-runtime invoke-endpoint \
--endpoint-name "fashion-loc" \
--body fileb://myimage.jpg \
outfile.json
{
"InvokedProductionVariant": "variant-name-1", 
"ContentType": "application/json"
}

 

The default output content type is application/json. Here is an example of the returned JSON:

{
    "status": 0,
    "result": [
        {
            "confidence": 0.7777002453804016,
            "xEnd": 0.8701398372650146,
            "categoryClass": "dress",
            "xStart": 0.12617331743240356,
            "yEnd": 0.9873287677764893,
            "yStart": 0.0
        }
    ]
}

 

You can see that the model found one fashion item (a dress) in the image, with a confidence of around 78%. The location of the dress can be found within the bounding box coordinates x/yStart – x/yEnd – these are normalised values between 0 and 1.

As well as application/json output, our models also support outputting image data, which is the original input image with the bounding boxes overlaid on it. This provides a way to get instant results, and also helps to quickly fine-tune the usage of the model. To request an image output, invoke the endpoint with the image/jpeg accept type:

$ aws --region=us-east-2 sagemaker-runtime invoke-endpoint \
--endpoint-name "fashion-loc" \
--body fileb://myimage.jpg \
--accept image/jpeg \
outfile.jpg

 

This time a JPEG file will be returned. Here is an example the output from our fashion localisation model:

Our models also support changing the default score threshold via custom attributes:

$ aws --region=us-east-2 sagemaker-runtime invoke-endpoint \
--endpoint-name "fashion-loc" \
--body fileb://myimage.jpg \
--accept image/jpeg \
--custom-attributes '{"threshold": 0.4}' \
outfile.jpg

 

Setting a higher threshold will ensure that only items with a higher score are output. Setting a lower threshold will result in more results from an image, with a greater leniency for false-positives.

 

Our other models behave in much the same way, and can all be invoked from the AWS CLI, once an endpoint has been deployed. Here is an example of the output from our interiors localisation model:

About Amazon SageMaker

Amazon SageMaker is a fully-managed platform that enables developers and data scientists to quickly and easily build, train, and deploy machine learning models at any scale. SageMaker has recently extended its capabilities by providing a marketplace where model makers can offer their pre-trained machine learning (ML) models for anyone to deploy and use.