Salesforce provides many cool Einstein features out of the box to improve sales, marketing, service. All company and customer needs are unique on or the other way. You may not get everything from out of the box features and you need a developer to build custom Einstein feature.

Developers can use Einstein Platform Services APIs to build AI-enabled apps with image recognition and natural language processing.

Einstein Vision

You can build your own deep learning models for use cases including visual search, brand detection, and object identification with Einstein Vision using standard REST API.

Let’s say, XYZ shoe company wants to increase their order value and it has social media details of their customers. In such case, whenever a customer posts a picture, they can detect the shoes and the brand (self or competitor’s). This helps to understand the customer preferences.

You can train your own image classifiers to solve a vast array of image recognition use cases.

Einstein Vision includes these APIs:

Einstein Image Classification—Enables developers to train deep learning models to recognize and classify images at scale.

Einstein Object Detection—Enables developers to train models to recognize and count multiple distinct objects within an image, providing granular details like the size and location of each object.

Use cases:

  • Allow customers to take photos of your products to discover where they can make purchases online or in-store indirectly to increase sales.
  • Monitor your brand across all your channels to increase your marketing reach and maintain brand integrity.
  • Better understand customer preferences and lifestyle through their social media images.
  • Identify product issues before sending out a field technician to increase case resolution time to streamline customer service.

Einstein Image Classification

Here’s the scenario: you’re a developer who works for a company that sells outdoor sporting gear. The company has automation that monitors social media channels. When someone posts a photo, the company wants to know whether the photo was taken at the beach or in the mountains. Based on where the photo was taken, the company can make targeted product recommendations to its customers.

Your task is straightforward: create a model that can identify whether an image is of the beach or the mountains. Then test the model with an image of a beach scene.

Einstein Vision Terminology

Dataset: The training data, which consists of inputs and outputs. Training the dataset creates the model used to make predictions. For example, in the above scenario, you create a model named Beach and Mountain Model from a binary training dataset consisting of two labels: Beaches (images of beach scenes) and Mountains (images of mountain scenes). A non-binary dataset contains three or more labels.

Label:A label references the output name you want your model to predict. For example, for our Beach and Mountain model, the training data contains images of beaches and that label is “Beaches.” Images of mountains have a label of “Mountains.”

Model:A machine learning construct used to solve a classification problem. Developers design a classification model by creating a dataset and then defining labels and providing positive examples of inputs that belong to these labels. When you train the dataset, the system then determines the commonalities and differences between the various labels to generalize the characteristics that define each label. The model predicts which class a new input falls into based on the predefined classes specified in your training dataset.

Training:The process through which a model is created and learns the classification rules based on a given set of training inputs (dataset).

Prediction: The results that the model returns as to how closely the input matches data in the dataset.


  1. Setup your Einstein Platform account:

  1. Install apex wrapper code from github via sfdx as mentioned:

NOTE: This is a custom library built more like a testing app which gives overall introduction. You must explore on your own to build custom functionality.

  1. In your org, locate “Einstein Settings” custom setting => click “Manage” => Click “New” above Default Organizational Level Value => In the Einstein Email field, enter the email address you used when you signed up for an Einstein Platform account => Click Save.

Make sure you have assigned “Einstein_Platform_Playground” permission set to yourself.

  1. Download and observe how the inputs are classified. There must be two folders named “Beaches” and “Mountains” (folder names indicate labels) containing sample images (i.e., called as examples) to be used as inputs in the dataset.

Switch to lightning experience and launch “Einstein Platform” app via app launcher.

Observe the tabs under Playground tab. For Einstein vision 3 tabs are built.

  1. Let’s start with IMAGE CLASSIFICATION.

Create the dataset:

Under the “Playground” tab, switch to IMAGE CLASSIFICATION tab and then DATASET CREATION under it. You upload your dataset either from web URL or zip file. Let’s go with url approach and enter the input box and click “Create”. This creates a dataset.

Move to DATASETS AND MODELS tab, click “Refresh Datasets” to see the dataset. It may look like this:

Mountainvsbeach is the dataset name and two labels with certain number of examples.

  1. Train the dataset to create a model. In the above image, models size is 0. Click on “Train” to create one.

Switch to MODELS sub tab under mountainvsbeach card and check the progress of model creation.

Click on “Refresh Models” to see the updated progress %. Once it becomes 100% with success status, it is available for prediction.


Pick the mountainvsbeach and upload an image to get the prediction. You can also use web url of the image. Once you upload the image, after some delay you can see the prediction under Response column. As per below image, 99% probability is Mountains and 1% Beaches which is accurate.

Stay tunes and wait for my next blog. Meanwhile explore IMAGE CLASSIFICATION(MULTI) tab on your own to learn further.