remyx_ai_classifier(Python)

Import Notebook

Classify Images in DataFrames with Remyx AI

In this notebook, we walk through the process of downloading a previously trained model from the Remyx Engine to help classify unknown samples of images in a data lake. If you haven't already, sign up here and train a classification model of your choice! Once that model shows a FINISHED status, continue with steps described below.

For databricks dependencies, ensure you are configuring a cluster with a Databricks ML Runtime >= 12.2 LTS.

Setup - Configure the CLI

We can access our models programmatically using the Remyx AI CLI! 🎉

Install it from pypi along with other dependencies.

%pip install remyxai
%pip install onnx
%pip install onnxruntime
Show result
import io
import os
import ast

import onnx
import onnxruntime as ort

import mlflow
import numpy as np
import pandas as pd
from PIL import Image

To use the API or CLI, we'll need to set a personal access token. Navigate to your account view to access or create a new token. For security, we store the token as a databricks secret so we can safely use it in a notebook context.

Extract the API key with dbutils secrets and set the environment variable

os.environ["REMYXAI_API_KEY"] = dbutils.secrets.get(scope="remyx", key="api-key")

Let's download a previously trained model we called interiorspacesid trained to differentiate between rooms in a home.

from remyxai.api import download_model
model_name = "interiorspacesid"
model_format = "onnx"
print(download_model(model_name, model_format))
The file interiorspacesid.zip was saved successfully

To access the model files, unzip the package.

%sh unzip interiorspacesid.zip -d /tmp/
Archive: interiorspacesid.zip inflating: /tmp/interiorspacesid.onnx inflating: /tmp/metadata.csv

Let's load the metadata.csv to access more information about the model, including labels and image shape requirements

metadata_df = (
    spark.read.format('csv')
              .options(header='true', inferSchema='true')
              .load('file:/tmp/metadata.csv')
)
display(metadata_df)
 
name
input_shape
labels
1
interiorspacesid
(None, 224, 224, 3)
bedroom|bathroom|kitchen
1 row

Store the metadata into variables for use later.

metadata_data = metadata_df.collect()[0]
input_shape = ast.literal_eval(metadata_data["input_shape"])
labels = metadata_data["labels"].split("|")

Custom PyFunc wrapper

By extending the mlflow.pyfunc.PythonModel class, we can create a custom model structure that can capture any image preprocessing and inference logic we'll need to use to call out model. This wrapper can then be used as a function that can be applied to a dataframe of images to produce a column of predictions

The following is an example on creating a custom pyfunc for calling an ONNX formatted model.

class RemyxPyFunc(mlflow.pyfunc.PythonModel):
    """
    Custom PyFunc class to enable prediction with tflite model against a binary encoded images
    """
    def load_context(self, context):
        """
        Loads artifacts to be used by `PythonModel.predict` when evaluating inputs.
        When loading an MLflow model with `.load_model`, this method is called as soon as the 
        `PythonModel` is constructed.
        
        Args:
            context (`mlflow.pyfunc.PythonModelContext`):
                A `PythonModelContext` instance containing artifacts that the model can use to 
                perform inference.
        """
        
        # Path to logged model
        model_path = context.artifacts['model_path'] 
        # Load onnx model
        onnx_model = mlflow.onnx.load_model(model_path)
        self.model = ort.InferenceSession(onnx_model.SerializeToString())

        # Set image params
        self.img_height = input_shape[1]
        self.img_width = input_shape[2]

        self.input_name = self.model.get_inputs()[0].name
        self.output_name = self.model.get_outputs()[0].name
    

    def predict(self, context, model_input: pd.Series) -> np.array:
        """
        Take a pd.Series of binary encoded images (BinaryType 'contents' colummn from Binary file data source),
        preprocess images, and apply model

        See https://docs.databricks.com/data/data-sources/binary-file.html#binary-file for info on data source
                
        Args:
            context: 
                instance containing artifacts that the model can use to perform inference
            model_input (pd.Series): 
                pd.Series of binary encoded images
                
        Returns: 
            np.array of predicted class labels 
        """
        model_input_np = np.array(model_input)
        # Preprocess binary images
        preprocessed_arr = np.array(list(map(self.preprocess, model_input_np))).astype(np.float32)
        # Predictions
        pred_arr = self.model.run([self.output_name], {self.input_name:preprocessed_arr})     
        # Predicted indices
        pred_idx = np.argmax(pred_arr[0], axis=1)
        # Predicted class labels
        pred_classes = np.take(labels, pred_idx, axis=0)

        return pred_classes      
  
    def preprocess(self, img_bytes: bytes) -> np.array:   
        """
        Take a single binary encoded image, decode image to numpy array
        and resize according to image height and size set in metadata.
        
        Args:
            img_bytes (bytes): 
                A single binary encoded image.
                
        Returns: 
            np.array of shape [img_height, img_width].
        """
        # When pyfunc is applied as SparkUDF byte format is converted to str
        # We require bytes format        
        if type(img_bytes) is not bytes:
            img_bytes = ast.literal_eval(str(img_bytes[0]))
        
        img = Image.open(io.BytesIO(img_bytes)).convert('RGB')
        img = img.resize([self.img_height, self.img_width])

        img = np.asarray(img, dtype='float32') / 255
        # Return a scaled array between -1 and 1
        return img * 2 - 1

Let's use mlflow to log our original model and log a RemyxPyFunc wrapped model.

mlflow.set_experiment("/Shared/remyx-models/")

with mlflow.start_run() as mlflow_run:
    onnx_model_file = "/tmp/interiorspacesid.onnx"
    onnx_model = onnx.load(onnx_model_file)
    mlflow.onnx.log_model(onnx_model, model_name)

    # onnx model
    model_path = f'runs:/{mlflow_run.info.run_id}/{model_name}'         

    pyfunc_artifacts = {'model_path': model_path}
    # Log MLflow pyfunc Model
    mlflow.pyfunc.log_model('pyfunc_model',
                            python_model=RemyxPyFunc(), 
                            artifacts=pyfunc_artifacts)
/databricks/python/lib/python3.9/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils. warnings.warn("Setuptools is replacing distutils.")

Running Inference

Let's load our logged model so we can apply it to a test dataset next.

# Get run_id from the run object executed above
run_id = mlflow_run.info.run_id
model_uri = f'runs:/{run_id}/pyfunc_model'
print(model_uri)

loaded_model = mlflow.pyfunc.load_model(model_uri)
runs:/347327214acc48db810fe7a0d7d48099/pyfunc_model

We've hosted a small test dataset of freely licensed stock images on the Remyx AI blog. Download it and decompress it into a directory of images.

%sh wget https://blog.remyx.ai/assets/interiorspacesid_test_dataset.zip
--2023-05-02 17:59:02-- https://blog.remyx.ai/assets/interiorspacesid_test_dataset.zip Resolving blog.remyx.ai (blog.remyx.ai)... 18.161.6.124, 18.161.6.10, 18.161.6.31, ... Connecting to blog.remyx.ai (blog.remyx.ai)|18.161.6.124|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 29915937 (29M) [application/zip] Saving to: ‘interiorspacesid_test_dataset.zip’ 0K .......... .......... .......... .......... .......... 0% 8.44M 3s 50K .......... .......... .......... .......... .......... 0% 8.81M 3s 100K .......... .......... .......... .......... .......... 0% 7.66M 3s 150K .......... .......... .......... .......... .......... 0% 16.4M 3s 200K .......... .......... .......... .......... .......... 0% 9.28M 3s 250K .......... .......... .......... .......... .......... 1% 8.73M 3s 300K .......... .......... .......... .......... .......... 1% 12.3M 3s 350K .......... .......... .......... .......... .......... 1% 16.1M 3s 400K .......... .......... .......... .......... .......... 1% 8.53M 3s 450K .......... .......... .......... .......... .......... 1% 66.2M 3s 500K .......... .......... .......... .......... .......... 1% 20.0M 2s 550K .......... .......... .......... .......... .......... 2% 20.5M 2s 600K .......... .......... .......... .......... .......... 2% 18.7M 2s 650K .......... .......... .......... .......... .......... 2% 17.5M 2s 700K .......... .......... .......... .......... .......... 2% 23.2M 2s 750K .......... .......... .......... .......... .......... 2% 19.2M 2s 800K .......... .......... .......... .......... .......... 2% 9.02M 2s 850K .......... .......... .......... .......... .......... 3% 144M 2s 900K .......... .......... .......... .......... .......... 3% 28.4M 2s 950K .......... .......... .......... .......... .......... 3% 45.5M 2s 1000K .......... .......... .......... .......... .......... 3% 42.6M 2s 1050K .......... .......... .......... .......... .......... 3% 22.8M 2s 1100K .......... .......... .......... .......... .......... 3% 19.7M 2s 1150K .......... .......... .......... .......... .......... 4% 26.9M 2s 1200K .......... .......... .......... .......... .......... 4% 29.7M 2s 1250K .......... .......... .......... .......... .......... 4% 15.8M 2s 1300K .......... .......... .......... .......... .......... 4% 31.3M 2s 1350K .......... .......... .......... .......... .......... 4% 65.3M 2s 1400K .......... .......... .......... .......... .......... 4% 20.9M 2s 1450K .......... .......... .......... .......... .......... 5% 37.8M 2s 1500K .......... .......... .......... .......... .......... 5% 22.3M 2s 1550K .......... .......... .......... .......... .......... 5% 45.4M 2s 1600K .......... .......... .......... .......... .......... 5% 21.9M 2s 1650K .......... .......... .......... .......... .......... 5% 21.0M 2s 1700K .......... .......... .......... .......... .......... 5% 34.2M 2s 1750K .......... .......... .......... .......... .......... 6% 7.33M 2s 1800K .......... .......... .......... .......... .......... 6% 19.6M 2s 1850K .......... .......... .......... .......... .......... 6% 19.5M 2s 1900K .......... .......... .......... .......... .......... 6% 19.2M 2s 1950K .......... .......... .......... .......... .......... 6% 19.1M 2s 2000K .......... .......... .......... .......... .......... 7% 18.0M 2s 2050K .......... .......... .......... .......... .......... 7% 29.8M 2s 2100K .......... .......... .......... .......... .......... 7% 30.4M 1s 2150K .......... .......... .......... .......... .......... 7% 41.2M 1s 2200K .......... .......... .......... .......... .......... 7% 19.0M 1s 2250K .......... .......... .......... .......... .......... 7% 42.8M 1s 2300K .......... .......... .......... .......... .......... 8% 46.2M 1s 2350K .......... .......... .......... .......... .......... 8% 22.9M 1s 2400K .......... .......... .......... .......... .......... 8% 68.4M 1s 2450K .......... .......... .......... .......... .......... 8% 22.7M 1s 2500K .......... .......... .......... .......... .......... 8% 30.7M 1s 2550K .......... .......... .......... .......... .......... 8% 4.58M 1s 2600K .......... .......... .......... .......... .......... 9% 11.6M 1s 2650K .......... .......... .......... .......... .......... 9% 145M 1s 2700K .......... .......... .......... .......... .......... 9% 152M 1s 2750K .......... .......... .......... .......... .......... 9% 17.8M 1s 2800K .......... .......... .......... .......... .......... 9% 84.7M 1s 2850K .......... .......... .......... .......... .......... 9% 21.0M 1s 2900K .......... .......... .......... .......... .......... 10% 46.4M 1s 2950K .......... .......... .......... .......... .......... 10% 16.5M 1s 3000K .......... .......... .......... .......... .......... 10% 128M 1s 3050K .......... .......... .......... .......... .......... 10% 11.0M 1s 3100K .......... .......... .......... .......... .......... 10% 51.7M 1s 3150K .......... .......... .......... .......... .......... 10% 15.0M 1s 3200K .......... .......... .......... .......... .......... 11% 22.3M 1s 3250K .......... .......... .......... .......... .......... 11% 48.0M 1s 3300K .......... .......... .......... .......... .......... 11% 29.6M 1s 3350K .......... .......... .......... .......... .......... 11% 42.8M 1s 3400K .......... .......... .......... .......... .......... 11% 26.9M 1s 3450K .......... .......... .......... .......... .......... 11% 26.0M 1s 3500K .......... .......... .......... .......... .......... 12% 39.8M 1s 3550K .......... .......... .......... .......... .......... 12% 73.7M 1s 3600K .......... .......... .......... .......... .......... 12% 15.0M 1s 3650K .......... .......... .......... .......... .......... 12% 35.1M 1s 3700K .......... .......... .......... .......... .......... 12% 21.7M 1s 3750K .......... .......... .......... .......... .......... 13% 47.8M 1s 3800K .......... .......... .......... .......... .......... 13% 60.1M 1s 3850K .......... .......... .......... .......... .......... 13% 45.4M 1s 3900K .......... .......... .......... .......... .......... 13% 30.9M 1s 3950K .......... .......... .......... .......... .......... 13% 32.2M 1s 4000K .......... .......... .......... .......... .......... 13% 71.3M 1s 4050K .......... .......... .......... .......... .......... 14% 67.5M 1s 4100K .......... .......... .......... .......... .......... 14% 14.6M 1s 4150K .......... .......... .......... .......... .......... 14% 29.1M 1s 4200K .......... .......... .......... .......... .......... 14% 96.2M 1s 4250K .......... .......... .......... .......... .......... 14% 21.6M 1s 4300K .......... .......... .......... .......... .......... 14% 40.6M 1s 4350K .......... .......... .......... .......... .......... 15% 42.5M 1s 4400K .......... .......... .......... .......... .......... 15% 25.3M 1s 4450K .......... .......... .......... .......... .......... 15% 146M 1s 4500K .......... .......... .......... .......... .......... 15% 17.5M 1s 4550K .......... .......... .......... .......... .......... 15% 141M 1s 4600K .......... .......... .......... .......... .......... 15% 145M 1s 4650K .......... .......... .......... .......... .......... 16% 28.7M 1s 4700K .......... .......... .......... .......... .......... 16% 115M 1s 4750K .......... .......... .......... .......... .......... 16% 26.3M 1s 4800K .......... .......... .......... .......... .......... 16% 27.8M 1s 4850K .......... .......... .......... .......... .......... 16% 16.5M 1s 4900K .......... .......... .......... .......... .......... 16% 48.7M 1s 4950K .......... .......... .......... .......... .......... 17% 142M 1s 5000K .......... .......... .......... .......... .......... 17% 28.5M 1s 5050K .......... .......... .......... .......... .......... 17% 150M 1s 5100K .......... .......... .......... .......... .......... 17% 24.1M 1s 5150K .......... .......... .......... .......... .......... 17% 138M 1s 5200K .......... .......... .......... .......... .......... 17% 9.35M 1s 5250K .......... .......... .......... .......... .......... 18% 43.3M 1s 5300K .......... .......... .......... .......... .......... 18% 44.9M 1s 5350K .......... .......... .......... .......... .......... 18% 56.2M 1s 5400K .......... .......... .......... .......... .......... 18% 130M 1s 5450K .......... .......... .......... .......... .......... 18% 28.1M 1s 5500K .......... .......... .......... .......... .......... 18% 35.3M 1s 5550K .......... .......... .......... .......... .......... 19% 24.6M 1s 5600K .......... .......... .......... .......... .......... 19% 18.6M 1s 5650K .......... .......... .......... .......... .......... 19% 24.4M 1s 5700K .......... .......... .......... .......... .......... 19% 57.5M 1s 5750K .......... .......... .......... .......... .......... 19% 25.4M 1s 5800K .......... .......... .......... .......... .......... 20% 56.9M 1s 5850K .......... .......... .......... .......... .......... 20% 46.9M 1s 5900K .......... .......... .......... .......... .......... 20% 28.3M 1s 5950K .......... .......... .......... .......... .......... 20% 15.8M 1s 6000K .......... .......... .......... .......... .......... 20% 28.1M 1s 6050K .......... .......... .......... .......... .......... 20% 27.9M 1s 6100K .......... .......... .......... .......... .......... 21% 49.4M 1s 6150K .......... .......... .......... .......... .......... 21% 129M 1s 6200K .......... .......... .......... .......... .......... 21% 27.2M 1s 6250K .......... .......... .......... .......... .......... 21% 27.2M 1s 6300K .......... .......... .......... .......... .......... 21% 63.9M 1s 6350K .......... .......... .......... .......... .......... 21% 48.5M 1s 6400K .......... .......... .......... .......... .......... 22% 8.59M 1s 6450K .......... .......... .......... .......... .......... 22% 5.27M 1s 6500K .......... .......... .......... .......... .......... 22% 16.5M 1s 6550K .......... .......... .......... .......... .......... 22% 16.4M 1s 6600K .......... .......... .......... .......... .......... 22% 60.0M 1s 6650K .......... .......... .......... .......... .......... 22% 23.1M 1s 6700K .......... .......... .......... .......... .......... 23% 36.8M 1s 6750K .......... .......... .......... .......... .......... 23% 36.1M 1s 6800K .......... .......... .......... .......... .......... 23% 67.7M 1s 6850K .......... .......... .......... .......... .......... 23% 43.1M 1s 6900K .......... .......... .......... .......... .......... 23% 30.1M 1s 6950K .......... .......... .......... .......... .......... 23% 37.4M 1s 7000K .......... .......... .......... .......... .......... 24% 49.7M 1s 7050K .......... .......... .......... .......... .......... 24% 32.7M 1s 7100K .......... .......... .......... .......... .......... 24% 43.5M 1s 7150K .......... .......... .......... .......... .......... 24% 18.1M 1s 7200K .......... .......... .......... .......... .......... 24% 38.3M 1s 7250K .......... .......... .......... .......... .......... 24% 5.65M 1s 7300K .......... .......... .......... .......... .......... 25% 24.7M 1s 7350K .......... .......... .......... .......... .......... 25% 42.5M 1s 7400K .......... .......... .......... .......... .......... 25% 44.9M 1s 7450K .......... .......... .......... .......... .......... 25% 34.3M 1s 7500K .......... .......... .......... .......... .......... 25% 45.5M 1s 7550K .......... .......... .......... .......... .......... 26% 34.7M 1s 7600K .......... .......... .......... .......... .......... 26% 31.5M 1s 7650K .......... .......... .......... .......... .......... 26% 19.7M 1s 7700K .......... .......... .......... .......... .......... 26% 20.1M 1s 7750K .......... .......... .......... .......... .......... 26% 62.3M 1s 7800K .......... .......... .......... .......... .......... 26% 31.5M 1s 7850K .......... .......... .......... .......... .......... 27% 27.8M 1s 7900K .......... .......... .......... .......... .......... 27% 32.9M 1s 7950K .......... .......... .......... .......... .......... 27% 3.43M 1s 8000K .......... .......... .......... .......... .......... 27% 23.0M 1s 8050K .......... .......... .......... .......... .......... 27% 31.8M 1s 8100K .......... .......... .......... .......... .......... 27% 11.4M 1s 8150K .......... .......... .......... .......... .......... 28% 21.6M 1s 8200K .......... .......... .......... .......... .......... 28% 9.35M 1s 8250K .......... .......... .......... .......... .......... 28% 10.6M 1s 8300K .......... .......... .......... .......... .......... 28% 41.0M 1s 8350K .......... .......... .......... .......... .......... 28% 11.3M 1s 8400K .......... .......... .......... .......... .......... 28% 40.1M 1s 8450K .......... .......... .......... .......... .......... 29% 13.6M 1s 8500K .......... .......... .......... .......... .......... 29% 26.8M 1s 8550K .......... .......... .......... .......... .......... 29% 13.4M 1s 8600K .......... .......... .......... .......... .......... 29% 15.3M 1s 8650K .......... .......... .......... .......... .......... 29% 26.6M 1s 8700K .......... .......... .......... .......... .......... 29% 39.8M 1s 8750K .......... .......... .......... .......... .......... 30% 8.26M 1s 8800K .......... .......... .......... .......... .......... 30% 28.5M 1s 8850K .......... .......... .......... .......... .......... 30% 22.2M 1s 8900K .......... .......... .......... .......... .......... 30% 12.0M 1s 8950K .......... .......... .......... .......... .......... 30% 20.9M 1s 9000K .......... .......... .......... .......... .......... 30% 39.9M 1s 9050K .......... .......... .......... .......... .......... 31% 15.1M 1s 9100K .......... .......... .......... .......... .......... 31% 21.6M 1s 9150K .......... .......... .......... .......... .......... 31% 24.4M 1s 9200K .......... .......... .......... .......... .......... 31% 27.4M 1s 9250K .......... .......... .......... .......... .......... 31% 89.0M 1s 9300K .......... .......... .......... .......... .......... 32% 27.5M 1s 9350K .......... .......... .......... .......... .......... 32% 80.6M 1s 9400K .......... .......... .......... .......... .......... 32% 16.3M 1s 9450K .......... .......... .......... .......... .......... 32% 32.1M 1s 9500K .......... .......... .......... .......... .......... 32% 65.0M 1s 9550K .......... .......... .......... .......... .......... 32% 16.1M 1s 9600K .......... .......... .......... .......... .......... 33% 53.3M 1s 9650K .......... .......... .......... .......... .......... 33% 23.8M 1s 9700K .......... .......... .......... .......... .......... 33% 40.1M 1s 9750K .......... .......... .......... .......... .......... 33% 54.2M 1s 9800K .......... .......... .......... .......... .......... 33% 35.5M 1s 9850K .......... .......... .......... .......... .......... 33% 23.2M 1s 9900K .......... .......... .......... .......... .......... 34% 79.0M 1s 9950K .......... .......... .......... .......... .......... 34% 35.2M 1s 10000K .......... .......... .......... .......... .......... 34% 18.0M 1s 10050K .......... .......... .......... .......... .......... 34% 36.6M 1s 10100K .......... .......... .......... .......... .......... 34% 25.7M 1s 10150K .......... .......... .......... .......... .......... 34% 24.1M 1s 10200K .......... .......... .......... .......... .......... 35% 36.1M 1s 10250K .......... .......... .......... .......... .......... 35% 23.6M 1s 10300K .......... .......... .......... .......... .......... 35% 36.4M 1s 10350K .......... .......... .......... .......... .......... 35% 23.9M 1s 10400K .......... .......... .......... .......... .......... 35% 117M 1s 10450K .......... .......... .......... .......... .......... 35% 28.0M 1s 10500K .......... .......... .......... .......... .......... 36% 126M 1s 10550K .......... .......... .......... .......... .......... 36% 12.7M 1s 10600K .......... .......... .......... .......... .......... 36% 38.4M 1s 10650K .......... .......... .......... .......... .......... 36% 129M 1s 10700K .......... .......... .......... .......... .......... 36% 17.2M 1s 10750K .......... .......... .......... .......... .......... 36% 129M 1s 10800K .......... .......... .......... .......... .......... 37% 23.3M 1s 10850K .......... .......... .......... .......... .......... 37% 22.0M 1s 10900K .......... .......... .......... .......... .......... 37% 29.0M 1s 10950K .......... .......... .......... .......... .......... 37% 24.6M 1s 11000K .......... .......... .......... .......... .......... 37% 70.7M 1s 11050K .......... .......... .......... .......... .......... 37% 13.9M 1s 11100K .......... .......... .......... .......... .......... 38% 81.3M 1s 11150K .......... .......... .......... .......... .......... 38% 37.9M 1s 11200K .......... .......... .......... .......... .......... 38% 23.0M 1s 11250K .......... .......... .......... .......... .......... 38% 18.2M 1s 11300K .......... .......... .......... .......... .......... 38% 36.0M 1s 11350K .......... .......... .......... .......... .......... 39% 22.1M 1s 11400K .......... .......... .......... .......... .......... 39% 86.6M 1s 11450K .......... .......... .......... .......... .......... 39% 27.8M 1s 11500K .......... .......... .......... .......... .......... 39% 33.5M 1s 11550K .......... .......... .......... .......... .......... 39% 22.9M 1s 11600K .......... .......... .......... .......... .......... 39% 29.4M 1s 11650K .......... .......... .......... .......... .......... 40% 22.4M 1s 11700K .......... .......... .......... .......... .......... 40% 35.7M 1s 11750K .......... .......... .......... .......... .......... 40% 33.5M 1s 11800K .......... .......... .......... .......... .......... 40% 44.1M 1s 11850K .......... .......... .......... .......... .......... 40% 44.6M 1s 11900K .......... .......... .......... .......... .......... 40% 36.7M 1s 11950K .......... .......... .......... .......... .......... 41% 19.0M 1s 12000K .......... .......... .......... .......... .......... 41% 24.9M 1s 12050K .......... .......... .......... .......... .......... 41% 59.5M 1s 12100K .......... .......... .......... .......... .......... 41% 19.3M 1s 12150K .......... .......... .......... .......... .......... 41% 28.6M 1s 12200K .......... .......... .......... .......... .......... 41% 49.3M 1s 12250K .......... .......... .......... .......... .......... 42% 109M 1s 12300K .......... .......... .......... .......... .......... 42% 10.7M 1s 12350K .......... .......... .......... .......... .......... 42% 123M 1s 12400K .......... .......... .......... .......... .......... 42% 14.7M 1s 12450K .......... .......... .......... .......... .......... 42% 44.9M 1s 12500K .......... .......... .......... .......... .......... 42% 125M 1s 12550K .......... .......... .......... .......... .......... 43% 143M 1s 12600K .......... .......... .......... .......... .......... 43% 22.2M 1s 12650K .......... .......... .......... .......... .......... 43% 135M 1s 12700K .......... .......... .......... .......... .......... 43% 143M 1s 12750K .......... .......... .......... .......... .......... 43% 137M 1s 12800K .......... .......... .......... .......... .......... 43% 140M 1s 12850K .......... .......... .......... .......... .......... 44% 11.1M 1s 12900K .......... .......... .......... .......... .......... 44% 125M 1s 12950K .......... .......... .......... .......... .......... 44% 39.7M 1s 13000K .......... .......... .......... .......... .......... 44% 137M 1s 13050K .......... .......... .......... .......... .......... 44% 128M 1s 13100K .......... .......... .......... .......... .......... 45% 26.9M 1s 13150K .......... .......... .......... .......... .......... 45% 138M 1s 13200K .......... .......... .......... .......... .......... 45% 124M 1s 13250K .......... .......... .......... .......... .......... 45% 127M 1s 13300K .......... .......... .......... .......... .......... 45% 143M 1s 13350K .......... .......... .......... .......... .......... 45% 145M 1s 13400K .......... .......... .......... .......... .......... 46% 106M 1s 13450K .......... .......... .......... .......... .......... 46% 152M 1s 13500K .......... .......... .......... .......... .......... 46% 133M 1s 13550K .......... .......... .......... .......... .......... 46% 142M 1s 13600K .......... .......... .......... .......... .......... 46% 135M 1s 13650K .......... .......... .......... .......... .......... 46% 129M 1s 13700K .......... .......... .......... .......... .......... 47% 147M 1s 13750K .......... .......... .......... .......... .......... 47% 143M 1s 13800K .......... .......... .......... .......... .......... 47% 7.46M 1s 13850K .......... .......... .......... .......... .......... 47% 138M 1s 13900K .......... .......... .......... .......... .......... 47% 136M 1s 13950K .......... .......... .......... .......... .......... 47% 11.7M 1s 14000K .......... .......... .......... .......... .......... 48% 132M 1s 14050K .......... .......... .......... .......... .......... 48% 132M 1s 14100K .......... .......... .......... .......... .......... 48% 146M 1s 14150K .......... .......... .......... .......... .......... 48% 145M 1s 14200K .......... .......... .......... .......... .......... 48% 108M 1s 14250K .......... .......... .......... .......... .......... 48% 137M 1s 14300K .......... .......... .......... .......... .......... 49% 145M 1s 14350K .......... .......... .......... .......... .......... 49% 142M 1s 14400K .......... .......... .......... .......... .......... 49% 148M 1s 14450K .......... .......... .......... .......... .......... 49% 116M 1s 14500K .......... .......... .......... .......... .......... 49% 151M 1s 14550K .......... .......... .......... .......... .......... 49% 8.64M 1s 14600K .......... .......... .......... .......... .......... 50% 118M 1s 14650K .......... .......... .......... .......... .......... 50% 118M 1s 14700K .......... .......... .......... .......... .......... 50% 122M 1s 14750K .......... .......... .......... .......... .......... 50% 143M 1s 14800K .......... .......... .......... .......... .......... 50% 123M 1s 14850K .......... .......... .......... .......... .......... 51% 120M 1s 14900K .......... .......... .......... .......... .......... 51% 144M 1s 14950K .......... .......... .......... .......... .......... 51% 128M 1s 15000K .......... .......... .......... .......... .......... 51% 123M 1s 15050K .......... .......... .......... .......... .......... 51% 133M 1s 15100K .......... .......... .......... .......... .......... 51% 133M 1s 15150K .......... .......... .......... .......... .......... 52% 116M 1s 15200K .......... .......... .......... .......... .......... 52% 4.28M 1s 15250K .......... .......... .......... .......... .......... 52% 31.9M 1s 15300K .......... .......... .......... .......... .......... 52% 113M 1s 15350K .......... .......... .......... .......... .......... 52% 13.2M 1s 15400K .......... .......... .......... .......... .......... 52% 146M 1s 15450K .......... .......... .......... .......... .......... 53% 143M 1s 15500K .......... .......... .......... .......... .......... 53% 18.6M 1s 15550K .......... .......... .......... .......... .......... 53% 135M 1s 15600K .......... .......... .......... .......... .......... 53% 126M 1s 15650K .......... .......... .......... .......... .......... 53% 17.7M 1s 15700K .......... .......... .......... .......... .......... 53% 129M 0s 15750K .......... .......... .......... .......... .......... 54% 17.9M 0s 15800K .......... .......... .......... .......... .......... 54% 130M 0s 15850K .......... .......... .......... .......... .......... 54% 139M 0s 15900K .......... .......... .......... .......... .......... 54% 18.4M 0s 15950K .......... .......... .......... .......... .......... 54% 146M 0s 16000K .......... .......... .......... .......... .......... 54% 14.9M 0s 16050K .......... .......... .......... .......... .......... 55% 90.9M 0s 16100K .......... .......... .......... .......... .......... 55% 122M 0s 16150K .......... .......... .......... .......... .......... 55% 144M 0s 16200K .......... .......... .......... .......... .......... 55% 18.5M 0s 16250K .......... .......... .......... .......... .......... 55% 140M 0s 16300K .......... .......... .......... .......... .......... 55% 16.4M 0s 16350K .......... .......... .......... .......... .......... 56% 133M 0s 16400K .......... .......... .......... .......... .......... 56% 15.5M 0s 16450K .......... .......... .......... .......... .......... 56% 49.5M 0s 16500K .......... .......... .......... .......... .......... 56% 30.5M 0s 16550K .......... .......... .......... .......... .......... 56% 129M 0s 16600K .......... .......... .......... .......... .......... 56% 161M 0s 16650K .......... .......... .......... .......... .......... 57% 17.7M 0s 16700K .......... .......... .......... .......... .......... 57% 155M 0s 16750K .......... .......... .......... .......... .......... 57% 115M 0s 16800K .......... .......... .......... .......... .......... 57% 15.3M 0s 16850K .......... .......... .......... .......... .......... 57% 21.9M 0s 16900K .......... .......... .......... .......... .......... 58% 99.8M 0s 16950K .......... .......... .......... .......... .......... 58% 152M 0s 17000K .......... .......... .......... .......... .......... 58% 15.1M 0s 17050K .......... .......... .......... .......... .......... 58% 43.3M 0s 17100K .......... .......... .......... .......... .......... 58% 65.8M 0s 17150K .......... .......... .......... .......... .......... 58% 122M 0s 17200K .......... .......... .......... .......... .......... 59% 14.8M 0s 17250K .......... .......... .......... .......... .......... 59% 110M 0s 17300K .......... .......... .......... .......... .......... 59% 121M 0s 17350K .......... .......... .......... .......... .......... 59% 133M 0s 17400K .......... .......... .......... .......... .......... 59% 16.6M 0s 17450K .......... .......... .......... .......... .......... 59% 25.1M 0s 17500K .......... .......... .......... .......... .......... 60% 109M 0s 17550K .......... .......... .......... .......... .......... 60% 31.1M 0s 17600K .......... .......... .......... .......... .......... 60% 140M 0s 17650K .......... .......... .......... .......... .......... 60% 12.2M 0s 17700K .......... .......... .......... .......... .......... 60% 147M 0s 17750K .......... .......... .......... .......... .......... 60% 38.3M 0s 17800K .......... .......... .......... .......... .......... 61% 31.7M 0s 17850K .......... .......... .......... .......... .......... 61% 124M 0s 17900K .......... .......... .......... .......... .......... 61% 18.3M 0s 17950K .......... .......... .......... .......... .......... 61% 125M 0s 18000K .......... .......... .......... .......... .......... 61% 141M 0s 18050K .......... .......... .......... .......... .......... 61% 17.5M 0s 18100K .......... .......... .......... .......... .......... 62% 121M 0s 18150K .......... .......... .......... .......... .......... 62% 15.0M 0s 18200K .......... .......... .......... .......... .......... 62% 38.3M 0s 18250K .......... .......... .......... .......... .......... 62% 23.5M 0s 18300K .......... .......... .......... .......... .......... 62% 115M 0s 18350K .......... .......... .......... .......... .......... 62% 134M 0s 18400K .......... .......... .......... .......... .......... 63% 116M 0s 18450K .......... .......... .......... .......... .......... 63% 16.5M 0s 18500K .......... .......... .......... .......... .......... 63% 138M 0s 18550K .......... .......... .......... .......... .......... 63% 16.1M 0s 18600K .......... .......... .......... .......... .......... 63% 20.7M 0s 18650K .......... .......... .......... .......... .......... 64% 122M 0s 18700K .......... .......... .......... .......... .......... 64% 128M 0s 18750K .......... .......... .......... .......... .......... 64% 18.7M 0s 18800K .......... .......... .......... .......... .......... 64% 108M 0s 18850K .......... .......... .......... .......... .......... 64% 140M 0s 18900K .......... .......... .......... .......... .......... 64% 14.9M 0s 18950K .......... .......... .......... .......... .......... 65% 28.7M 0s 19000K .......... .......... .......... .......... .......... 65% 131M 0s 19050K .......... .......... .......... .......... .......... 65% 18.7M 0s 19100K .......... .......... .......... .......... .......... 65% 115M 0s 19150K .......... .......... .......... .......... .......... 65% 121M 0s 19200K .......... .......... .......... .......... .......... 65% 17.6M 0s 19250K .......... .......... .......... .......... .......... 66% 132M 0s 19300K .......... .......... .......... .......... .......... 66% 15.8M 0s 19350K .......... .......... .......... .......... .......... 66% 15.5M 0s 19400K .......... .......... .......... .......... .......... 66% 134M 0s 19450K .......... .......... .......... .......... .......... 66% 22.2M 0s 19500K .......... .......... .......... .......... .......... 66% 23.0M 0s 19550K .......... .......... .......... .......... .......... 67% 94.6M 0s 19600K .......... .......... .......... .......... .......... 67% 145M 0s 19650K .......... .......... .......... .......... .......... 67% 158M 0s 19700K .......... .......... .......... .......... .......... 67% 18.8M 0s 19750K .......... .......... .......... .......... .......... 67% 27.7M 0s 19800K .......... .......... .......... .......... .......... 67% 93.7M 0s 19850K .......... .......... .......... .......... .......... 68% 123M 0s 19900K .......... .......... .......... .......... .......... 68% 18.6M 0s 19950K .......... .......... .......... .......... .......... 68% 156M 0s 20000K .......... .......... .......... .......... .......... 68% 157M 0s 20050K .......... .......... .......... .......... .......... 68% 8.64M 0s 20100K .......... .......... .......... .......... .......... 68% 64.7M 0s 20150K .......... .......... .......... .......... .......... 69% 12.5M 0s 20200K .......... .......... .......... .......... .......... 69% 38.9M 0s 20250K .......... .......... .......... .......... .......... 69% 72.1M 0s 20300K .......... .......... .......... .......... .......... 69% 37.4M 0s 20350K .......... .......... .......... .......... .......... 69% 55.6M 0s 20400K .......... .......... .......... .......... .......... 69% 28.8M 0s 20450K .......... .......... .......... .......... .......... 70% 27.3M 0s 20500K .......... .......... .......... .......... .......... 70% 24.0M 0s 20550K .......... .......... .......... .......... .......... 70% 19.6M 0s 20600K .......... .......... .......... .......... .......... 70% 51.4M 0s 20650K .......... .......... .......... .......... .......... 70% 24.7M 0s 20700K .......... .......... .......... .......... .......... 71% 30.3M 0s 20750K .......... .......... .......... .......... .......... 71% 36.7M 0s 20800K .......... .......... .......... .......... .......... 71% 31.8M 0s 20850K .......... .......... .......... .......... .......... 71% 51.2M 0s 20900K .......... .......... .......... .......... .......... 71% 36.5M 0s 20950K .......... .......... .......... .......... .......... 71% 142M 0s 21000K .......... .......... .......... .......... .......... 72% 34.9M 0s 21050K .......... .......... .......... .......... .......... 72% 138M 0s 21100K .......... .......... .......... .......... .......... 72% 9.49M 0s 21150K .......... .......... .......... .......... .......... 72% 8.25M 0s 21200K .......... .......... .......... .......... .......... 72% 144M 0s 21250K .......... .......... .......... .......... .......... 72% 76.8M 0s 21300K .......... .......... .......... .......... .......... 73% 132M 0s 21350K .......... .......... .......... .......... .......... 73% 128M 0s 21400K .......... .......... .......... .......... .......... 73% 150M 0s 21450K .......... .......... .......... .......... .......... 73% 128M 0s 21500K .......... .......... .......... .......... .......... 73% 153M 0s 21550K .......... .......... .......... .......... .......... 73% 4.48M 0s 21600K .......... .......... .......... .......... .......... 74% 28.3M 0s 21650K .......... .......... .......... .......... .......... 74% 88.7M 0s 21700K .......... .......... .......... .......... .......... 74% 18.7M 0s 21750K .......... .......... .......... .......... .......... 74% 23.0M 0s 21800K .......... .......... .......... .......... .......... 74% 119M 0s 21850K .......... .......... .......... .......... .......... 74% 134M 0s 21900K .......... .......... .......... .......... .......... 75% 11.9M 0s 21950K .......... .......... .......... .......... .......... 75% 134M 0s 22000K .......... .......... .......... .......... .......... 75% 132M 0s 22050K .......... .......... .......... .......... .......... 75% 128M 0s 22100K .......... .......... .......... .......... .......... 75% 117M 0s 22150K .......... .......... .......... .......... .......... 75% 131M 0s 22200K .......... .......... .......... .......... .......... 76% 127M 0s 22250K .......... .......... .......... .......... .......... 76% 117M 0s 22300K .......... .......... .......... .......... .......... 76% 143M 0s 22350K .......... .......... .......... .......... .......... 76% 142M 0s 22400K .......... .......... .......... .......... .......... 76% 136M 0s 22450K .......... .......... .......... .......... .......... 77% 133M 0s 22500K .......... .......... .......... .......... .......... 77% 2.42M 0s 22550K .......... .......... .......... .......... .......... 77% 9.19M 0s 22600K .......... .......... .......... .......... .......... 77% 151M 0s 22650K .......... .......... .......... .......... .......... 77% 114M 0s 22700K .......... .......... .......... .......... .......... 77% 137M 0s 22750K .......... .......... .......... .......... .......... 78% 155M 0s 22800K .......... .......... .......... .......... .......... 78% 123M 0s 22850K .......... .......... .......... .......... .......... 78% 160M 0s 22900K .......... .......... .......... .......... .......... 78% 140M 0s 22950K .......... .......... .......... .......... .......... 78% 6.58M 0s 23000K .......... .......... .......... .......... .......... 78% 17.0M 0s 23050K .......... .......... .......... .......... .......... 79% 46.0M 0s 23100K .......... .......... .......... .......... .......... 79% 19.0M 0s 23150K .......... .......... .......... .......... .......... 79% 69.1M 0s 23200K .......... .......... .......... .......... .......... 79% 12.8M 0s 23250K .......... .......... .......... .......... .......... 79% 14.0M 0s 23300K .......... .......... .......... .......... .......... 79% 91.6M 0s 23350K .......... .......... .......... .......... .......... 80% 45.3M 0s 23400K .......... .......... .......... .......... .......... 80% 20.1M 0s 23450K .......... .......... .......... .......... .......... 80% 22.5M 0s 23500K .......... .......... .......... .......... .......... 80% 29.6M 0s 23550K .......... .......... .......... .......... .......... 80% 19.7M 0s 23600K .......... .......... .......... .......... .......... 80% 22.9M 0s 23650K .......... .......... .......... .......... .......... 81% 45.4M 0s 23700K .......... .......... .......... .......... .......... 81% 94.7M 0s 23750K .......... .......... .......... .......... .......... 81% 12.8M 0s 23800K .......... .......... .......... .......... .......... 81% 46.3M 0s 23850K .......... .......... .......... .......... .......... 81% 14.7M 0s 23900K .......... .......... .......... .......... .......... 81% 28.6M 0s 23950K .......... .......... .......... .......... .......... 82% 17.0M 0s 24000K .......... .......... .......... .......... .......... 82% 14.8M 0s 24050K .......... .......... .......... .......... .......... 82% 25.0M 0s 24100K .......... .......... .......... .......... .......... 82% 48.0M 0s 24150K .......... .......... .......... .......... .......... 82% 137M 0s 24200K .......... .......... .......... .......... .......... 83% 22.4M 0s 24250K .......... .......... .......... .......... .......... 83% 23.7M 0s 24300K .......... .......... .......... .......... .......... 83% 106M 0s 24350K .......... .......... .......... .......... .......... 83% 147M 0s 24400K .......... .......... .......... .......... .......... 83% 129M 0s 24450K .......... .......... .......... .......... .......... 83% 119M 0s 24500K .......... .......... .......... .......... .......... 84% 147M 0s 24550K .......... .......... .......... .......... .......... 84% 128M 0s 24600K .......... .......... .......... .......... .......... 84% 9.29M 0s 24650K .......... .......... .......... .......... .......... 84% 146M 0s 24700K .......... .......... .......... .......... .......... 84% 142M 0s 24750K .......... .......... .......... .......... .......... 84% 103M 0s 24800K .......... .......... .......... .......... .......... 85% 143M 0s 24850K .......... .......... .......... .......... .......... 85% 131M 0s 24900K .......... .......... .......... .......... .......... 85% 125M 0s 24950K .......... .......... .......... .......... .......... 85% 12.4M 0s 25000K .......... .......... .......... .......... .......... 85% 112M 0s 25050K .......... .......... .......... .......... .......... 85% 2.93M 0s 25100K .......... .......... .......... .......... .......... 86% 35.4M 0s 25150K .......... .......... .......... .......... .......... 86% 121M 0s 25200K .......... .......... .......... .......... .......... 86% 118M 0s 25250K .......... .......... .......... .......... .......... 86% 5.33M 0s 25300K .......... .......... .......... .......... .......... 86% 6.99M 0s 25350K .......... .......... .......... .......... .......... 86% 129M 0s 25400K .......... .......... .......... .......... .......... 87% 137M 0s 25450K .......... .......... .......... .......... .......... 87% 125M 0s 25500K .......... .......... .......... .......... .......... 87% 5.79M 0s 25550K .......... .......... .......... .......... .......... 87% 48.7M 0s 25600K .......... .......... .......... .......... .......... 87% 162M 0s 25650K .......... .......... .......... .......... .......... 87% 10.5M 0s 25700K .......... .......... .......... .......... .......... 88% 129M 0s 25750K .......... .......... .......... .......... .......... 88% 77.8M 0s 25800K .......... .......... .......... .......... .......... 88% 149M 0s 25850K .......... .......... .......... .......... .......... 88% 31.3M 0s 25900K .......... .......... .......... .......... .......... 88% 133M 0s 25950K .......... .......... .......... .......... .......... 88% 133M 0s 26000K .......... .......... .......... .......... .......... 89% 13.0M 0s 26050K .......... .......... .......... .......... .......... 89% 138M 0s 26100K .......... .......... .......... .......... .......... 89% 130M 0s 26150K .......... .......... .......... .......... .......... 89% 132M 0s 26200K .......... .......... .......... .......... .......... 89% 122M 0s 26250K .......... .......... .......... .......... .......... 90% 123M 0s 26300K .......... .......... .......... .......... .......... 90% 133M 0s 26350K .......... .......... .......... .......... .......... 90% 31.3M 0s 26400K .......... .......... .......... .......... .......... 90% 31.6M 0s 26450K .......... .......... .......... .......... .......... 90% 131M 0s 26500K .......... .......... .......... .......... .......... 90% 128M 0s 26550K .......... .......... .......... .......... .......... 91% 33.0M 0s 26600K .......... .......... .......... .......... .......... 91% 130M 0s 26650K .......... .......... .......... .......... .......... 91% 145M 0s 26700K .......... .......... .......... .......... .......... 91% 139M 0s 26750K .......... .......... .......... .......... .......... 91% 122M 0s 26800K .......... .......... .......... .......... .......... 91% 131M 0s 26850K .......... .......... .......... .......... .......... 92% 12.4M 0s 26900K .......... .......... .......... .......... .......... 92% 144M 0s 26950K .......... .......... .......... .......... .......... 92% 103M 0s 27000K .......... .......... .......... .......... .......... 92% 132M 0s 27050K .......... .......... .......... .......... .......... 92% 146M 0s 27100K .......... .......... .......... .......... .......... 92% 144M 0s 27150K .......... .......... .......... .......... .......... 93% 116M 0s 27200K .......... .......... .......... .......... .......... 93% 4.80M 0s 27250K .......... .......... .......... .......... .......... 93% 136M 0s 27300K .......... .......... .......... .......... .......... 93% 138M 0s 27350K .......... .......... .......... .......... .......... 93% 120M 0s 27400K .......... .......... .......... .......... .......... 93% 26.4M 0s 27450K .......... .......... .......... .......... .......... 94% 139M 0s 27500K .......... .......... .......... .......... .......... 94% 9.06M 0s 27550K .......... .......... .......... .......... .......... 94% 119M 0s 27600K .......... .......... .......... .......... .......... 94% 148M 0s 27650K .......... .......... .......... .......... .......... 94% 147M 0s 27700K .......... .......... .......... .......... .......... 94% 135M 0s 27750K .......... .......... .......... .......... .......... 95% 114M 0s 27800K .......... .......... .......... .......... .......... 95% 147M 0s 27850K .......... .......... .......... .......... .......... 95% 134M 0s 27900K .......... .......... .......... .......... .......... 95% 146M 0s 27950K .......... .......... .......... .......... .......... 95% 119M 0s 28000K .......... .......... .......... .......... .......... 96% 153M 0s 28050K .......... .......... .......... .......... .......... 96% 8.01M 0s 28100K .......... .......... .......... .......... .......... 96% 196M 0s 28150K .......... .......... .......... .......... .......... 96% 169M 0s 28200K .......... .......... .......... .......... .......... 96% 192M 0s 28250K .......... .......... .......... .......... .......... 96% 191M 0s 28300K .......... .......... .......... .......... .......... 97% 200M 0s 28350K .......... .......... .......... .......... .......... 97% 209M 0s 28400K .......... .......... .......... .......... .......... 97% 184M 0s 28450K .......... .......... .......... .......... .......... 97% 12.5M 0s 28500K .......... .......... .......... .......... .......... 97% 181M 0s 28550K .......... .......... .......... .......... .......... 97% 138M 0s 28600K .......... .......... .......... .......... .......... 98% 244M 0s 28650K .......... .......... .......... .......... .......... 98% 165M 0s 28700K .......... .......... .......... .......... .......... 98% 201M 0s 28750K .......... .......... .......... .......... .......... 98% 193M 0s 28800K .......... .......... .......... .......... .......... 98% 170M 0s 28850K .......... .......... .......... .......... .......... 98% 1.73M 0s 28900K .......... .......... .......... .......... .......... 99% 103M 0s 28950K .......... .......... .......... .......... .......... 99% 124M 0s 29000K .......... .......... .......... .......... .......... 99% 32.2M 0s 29050K .......... .......... .......... .......... .......... 99% 139M 0s 29100K .......... .......... .......... .......... .......... 99% 126M 0s 29150K .......... .......... .......... .......... .......... 99% 153M 0s 29200K .......... .... 100% 62.5M=1.0s 2023-05-02 17:59:03 (27.9 MB/s) - ‘interiorspacesid_test_dataset.zip’ saved [29915937/29915937] 
%sh unzip interiorspacesid_test_dataset.zip  -d /tmp/
Archive: interiorspacesid_test_dataset.zip creating: /tmp/interiorspacesid_test_dataset/ inflating: /tmp/interiorspacesid_test_dataset/guest-rooms-1201571.jpg inflating: /tmp/interiorspacesid_test_dataset/.DS_Store inflating: /tmp/interiorspacesid_test_dataset/guest-room-1201691.jpg inflating: /tmp/interiorspacesid_test_dataset/bedroom-interior-design-1206189.jpg inflating: /tmp/interiorspacesid_test_dataset/marble-bathroom-1203017.jpg inflating: /tmp/interiorspacesid_test_dataset/kitchen-1934302.jpg inflating: /tmp/interiorspacesid_test_dataset/kichen-1416309.jpg inflating: /tmp/interiorspacesid_test_dataset/kitchen-1215308.jpg inflating: /tmp/interiorspacesid_test_dataset/70-s-toilet-1467536.jpg inflating: /tmp/interiorspacesid_test_dataset/kitchen-2317108.jpg inflating: /tmp/interiorspacesid_test_dataset/toilet-closet-for-men-1638641.jpg inflating: /tmp/interiorspacesid_test_dataset/bathroom-2-1201519.jpg inflating: /tmp/interiorspacesid_test_dataset/guest-rooms-1201624.jpg inflating: /tmp/interiorspacesid_test_dataset/modern-bedroom-1892517.jpg inflating: /tmp/interiorspacesid_test_dataset/georgia-island-kitchen-1207296.jpg

We'll use the binaryFile file type to load images into a properly formatted dataset. 

# load a dataset to run inference
inference_df = (
    spark.read.format('binaryFile')
              .option('pathGlobFilter', '*.jpg')
              .load('file:/tmp/interiorspacesid_test_dataset/')
              )
inference_pdf = inference_df.toPandas()

# call our pyfunc
inference_pdf["prediction"] = loaded_model.predict(inference_pdf['content'])
display(inference_pdf[["path", "prediction"]])
 
path
prediction
1
2
3
4
5
6
7
8
9
10
11
12
13
14
file:/tmp/interiorspacesid_test_dataset/guest-room-1201691.jpg
bedroom
file:/tmp/interiorspacesid_test_dataset/bathroom-2-1201519.jpg
bathroom
file:/tmp/interiorspacesid_test_dataset/modern-bedroom-1892517.jpg
bedroom
file:/tmp/interiorspacesid_test_dataset/kitchen-1934302.jpg
kitchen
file:/tmp/interiorspacesid_test_dataset/toilet-closet-for-men-1638641.jpg
bathroom
file:/tmp/interiorspacesid_test_dataset/kitchen-1215308.jpg
kitchen
file:/tmp/interiorspacesid_test_dataset/guest-rooms-1201571.jpg
bedroom
file:/tmp/interiorspacesid_test_dataset/kitchen-2317108.jpg
kitchen
file:/tmp/interiorspacesid_test_dataset/marble-bathroom-1203017.jpg
bathroom
file:/tmp/interiorspacesid_test_dataset/guest-rooms-1201624.jpg
bedroom
file:/tmp/interiorspacesid_test_dataset/bedroom-interior-design-1206189.jpg
bedroom
file:/tmp/interiorspacesid_test_dataset/70-s-toilet-1467536.jpg
bathroom
file:/tmp/interiorspacesid_test_dataset/kichen-1416309.jpg
kitchen
file:/tmp/interiorspacesid_test_dataset/georgia-island-kitchen-1207296.jpg
kitchen
14 rows
# optional: to display with image thumbnails

inference_output = spark.createDataFrame(inference_pdf).select("path", "prediction")
display(inference_df.join(inference_output, inference_df.path == inference_output.path).select("content", "prediction"))
 
content
prediction
1
2
3
4
5
6
7
8
9
10
11
12
bedroom
bathroom
bedroom
kitchen
bathroom
kitchen
bedroom
kitchen
bathroom
bedroom
bedroom
bathroom
14 rows

Conclusion

The Remyx AI platform is another handy tool in the Machine learning practitioner toolkit for quickly creating machine learning models.

This example demonstrated how to use a previously created model to analyze new datasets. These models can be used alone or served as the foundations for more complex applications. Check out more tips, projects, and breakdowns on the New Spin 📀, a Remyx AI blog!