Navigating IBM’s Data and AI Products
Originally published on: https://developer.ibm.com/blogs/putting-ai-to-work-with-watson/
Many enterprises are now looking at artificial intelligence (AI) and how to use their own data. Enterprise needs vary. Depending on what data is being stored and sent, the data might be required to be kept on-premises. IBM is a great fit for enterprises with strict business needs (like HIPPA or GDPR) because IBM keeps these requirements in mind as the software is written. IBM also provides on-ramps with free-to-use offerings, learnings paths, tutorials, and videos.
In this blog, I’ll look at these sets of data and AI products from a developer’s point of view:
Before looking at each set of products, let’s take a quick detour and see how they can be used.
Deployment options
Most of the IBM data and AI products can be accessed through:
- IBM Cloud: The IBM public cloud offering that offers lite and pay-as-you-go plans
- IBM Cloud Pak for Data: A bundle of data and AI add-ons that can be installed on Red Hat OpenShift
The following image shows a representation of each stack for consuming the data and AI offerings on both IBM Cloud Pak for Data and IBM Cloud.
| IBM Cloud | IBM Cloud Pak for Data |
|---|---|
 |
 |
NOTE: The Watson Machine Learning Accelerator product suite requires Power hardware.
Watson APIs
These services are the most easily understood, and knowing AI concepts is not required to use them. They each have a REST API that can be called, and are meant to be called at the application level by SDKs. These services use pre-built models and provide a user-friendly way to create user custom models.
Watson APIs:
- Include services such as Assistant and Discovery
- Have well-maintained APIs
- Have SDKs for popular languages (such as the Java language, Node, Swift, and Go)
- Have generous free tier plans that let you get your hands dirty without excessive costs
- Are available on the IBM Cloud and as add-ons to IBM Cloud Pak for Data
- Are truly a great place for developers to start their AI journey
Watson Assistant
The main “chatbot” offering and so much more. Watson Assistant comes pre-trained with industry-relevant content. It has an intuitive editor; provides middleware for integrating with Slack, Facebook, and many other platforms; and can even be integrated with Watson Voice Gateway so that you can talk to your assistant over a phone.
Watson Discovery
The “AI search” offering. Simply put, use the Discovery tool or API to load a bunch of documents (such as .pdf or .doc files) and have Discovery build a model for you. You can query this model with natural language, and Discovery pulls out relevant parts of the documents. You can make Discovery smarter by using “Smart Document Understanding” to ensure documents are read properly or by using “Watson Knowledge Studio” to build a domain or industry-specific model.
And more
- Watson Text to Speech: Text goes in, audio comes out
- Watson Speech to Text: User’s voice goes in, text comes out
- Watson Translator: Text goes in, select your source and destination languages, more text comes out
- Watson Natural Language Understanding: A short string of text goes in, and concepts, entities, keywords, emotions, and sentiment comes out
- Watson Natural Language Classifier: Build text classifiers based on training data, and then classify text against the models that are built
Tabular representation
| Offering | On Cloud | On Prem | Free tier (on Cloud) | SDK support |
|---|---|---|---|---|
| Assistant | ✓ | ✓ | ✓ | ✓ |
| Discovery | ✓ | ✓ | ✓ | ✓ |
| Text to Speech | ✓ | ✓ | ✓ | ✓ |
| Speech to Text | ✓ | ✓ | ✓ | ✓ |
| Translator | ✓ | ✓ | ✓ | ✓ |
| Natural Language Understanding | ✓ | ✓ | ✓ | ✓ |
| Natural Language Classifier | ✓ | ✓ | ✓ | ✓ |
Now onto the hard stuff, no more pre-made models. So, let’s move on to Watson Studio.
Watson Studio
Watson Studio gives you an environment and tools to collaboratively work with data. Data can be imported (through connectors), viewed, refined, and analyzed. Models can then be created with the Watson Studio Deep Learning and Machine Learning tools.
Watson Studio:
- Is based on various open source technology such as Jupyter Notebooks and R Studio
- Provides tools based on popular open source frameworks such as Tensorflow, Keras, and Pytorch
- Includes 2 GB of free storage on IBM Cloud Object Storage
- Is available on a public cloud and as add-ons to IBM Cloud Pak for Data
- Is an excellent platform to begin your data science journey
Jupyter Notebooks
Watson Studio provides support for running Jupyter Notebooks. There are several environments to choose from (Python 3.x, R3.4, and Scala 2.11), and each has the option to add a Spark kernel as well. You can share your Notebook collaboratively, add data from your Object Store as a data frame, publish your Notebook, and use many other of the features you’ve come to expect.
Watson Machine Learning
The Watson Machine Learning service is a major part of Watson Studio. It provides a set of APIs that can be called to interact with a machine learning model. The model can be created using a Jupyter Notebook, SPSS Modeler, or AutoAI, and then deployed to an instance of the Watson Machine Learning service. After it’s deployed, you can score data by using a REST call against an API that the service provides.
OpenScale
Watson OpenScale allows monitoring and management of machine learning models that are built on various platforms: Watson Studio, Amazon Sagemaker, Azure Machine Learning, and other popular open source frameworks.
And even more
- AutoAI: A graphical tool in Watson Studio that automatically analyzes your data and generates candidate model pipelines that are customized for your predictive modeling problem.
- SPSS Modeler: Watson Studio offers a variety of modeling methods taken from machine learning, artificial intelligence, and statistics.
- Data visualization: Watson Studio provides the ability to visualize data with Data Refinery.
- Cognos Dashboards: A dashboard for viewing and analyzing data, instead of using pandas or pixiedust.
- Connections: Watson Studio provides connections to import data from IBM Services (Db2, Cognos, Cloudant, Cloud Object Storage, and many more) and for third-party services (Amazon S3, Hortonworks HDFS, Microsoft SQL Server, Salesforce, MySQL, Oracle, Google BigQuery, and many more).
Watson Machine Learning Accelerator
IBM Watson Machine Learning Accelerator is geared toward enterprises. It is a software bundle that is optimized to run on Power hardware. It bundles IBM PowerAI, IBM Spectrum Conductor, IBM Spectrum Conductor Deep Learning Impact, and support from IBM for the whole stack including the open source deep learning frameworks. It provides an end-to-end, deep learning platform for data scientists.
IBM Maximo Visual Inspection
IBM Maximo Visual Inspection provides a video and image analysis platform that offers built-in deep learning models that learn to analyze images and video streams for classification and object detection. Maximo Visual Inspection is built on open source frameworks and provides sophisticated methods to create models with an easy to understand user interface.
One last note
- Spectrum Conductor: Deploys modern computing frameworks and services for an enterprise environment, both on-premises and in the cloud.
- Snap ML: A library developed by IBM Research for training generalized linear models with the intent on removing training time as a bottleneck. Snap ML scales gracefully to data sets with billions, and offers distributed training and GPU acceleration.