artificial intelligence, Semantics

To Bot, Or Not To Bot


Are Bots the Future of Computing?

Bots — simple computer-based [micro] services that you interact with conversationally — are being hailed by some as the next wave of computing, a profound platform shift and the most exciting technology since the iPhone.

Microsoft this week became the latest technology provider to strategically embrace bots and offer a framework for developing them. Microsoft joins Facebook, Slack, WeChat, Kik and many others. In Microsoft’s case, as with several others, bots are a part of a continued push into a broad set of technologies centered around artificial intelligence or AI.

But let’s forget AI for a moment. This trend towards functional capabilities delivered as well-bounded, discrete services goes back at least as far as IT concepts such as object-oriented programming and  component-based development (think of a method or web service like “CheckBalance” or “MakeDeposit”). To some degree this basic idea is also manifest in app connection frameworks such as IFTTT and associated app integration platforms like Zapier. The rapid trend towards hosting services in cloud environments and making them easily accessible through Application Programming Interfaces (APIs) has now provided much of the infrastructure or connective tissue to enable the rise of bots.

If you’re a business looking at where technology is headed as part of your product strategy, you may be asking if bots are likely to truly transform digital products and whether that’s a good or a bad thing. Personally I think they have the potential to be transformative and that they will indeed be a good thing. As with other major tech disruptions, the transformation will almost certainly not happen overnight and as it goes mainstream it won’t necessarily look like this first wave of the technology. But I’m convinced bots are coming (some are here already) and smart businesses should start now exploring the opportunities that will come from building products using a bot model.

Wait, I thought Bots Were Bad????

We’re not talking here about hijacked PCs running programs whose sole purpose is to spew SPAM, or other automated programs that ‘mindlessly’ push pre-programmed content. The bots coming onto the scene now are helpful software agents, with many of them embodying at least some form of intelligence. By that I mean they have the ability to interpret a request for a service and to respond appropriately with relevant information, content or actions. Here’s the clincher in my mind: the services these bots perform need to offer real identifiable value to the user who is interacting with the bot.

Why Is This Happening Now? You Might Ask

In a world where everything is becoming an endpoint, bots are the little engines that interpret incoming signals — in context– and provide responses or initiate outbound actions. Think about the streams of data generated by Internet of Things (IoT) based sensors and all the data traffic traveling through messaging services.  This data is not just valuable en masse for historical or predictive analytics. It can be put to more immediate, localized and personalized use. Whether pinged directly through a single trigger word or command, or more passively activated using AI to interpret a chunk of data in a message, bots are driven by actionable data [intelligently] understood within a context. The ubiquity of devices such as sensors, beacons and smartphones, along with omnipresent network connections, always-on communications channels, vast sources of content and publicly-accessible APIs have all reached a tipping point of availability and maturity to support human interaction with the world in a way that is augmented by automated software agents – bots.

While the bots may be stand-alone services, to support a critical mass of them they need to be hosted within a platform that either resides in or interfaces with our existing communications channels. Today that takes the form of smart phone OSs, SMS/text messaging services, social messaging services such as Facebook Messenger, Kik, WeChat and Line, and/or virtual personal assistants such as Google Now, Microsoft Cortana or Amazon Echo’s Alexa. Virtual personal assistants can be thought of as universal or generalized bots that orchestrate or intermediate more specialized sub-bots. Mobile payment services such as Apple Pay are also contributing to the trend. Although simple text is sufficient as a basic user interface to bots, advances in natural language interpretation and understanding, and in voice recognition and speech synthesis provide additional, richer ways for humans and bots to interact. The technology then is wholly or largely existent today. Given the sheer volume of data and APIs, we need technology just to handle that volume. Technology also offers the opportunity to both simplify and augment our personal and business communications and transactions. So arguably the demand side of the equation exists now, too.

Should My Business Dive In Now or Wait?

It’s early days and a lot remains to be ironed out, particularly in terms of bot platforms. Every technology provider seems to be doing their own platform, there are few or no standards and little if any integration across platforms. There will be successes and failures and shakeouts. So it’s too early to jump in, right? Absolutely not! Any business that isn’t venturing into this space risks missing the boat as the wave of bots washes in. There are ways to venture into this space that mitigate the risk, while providing valuable revenue and experience to businesses and real value to their customers. Here are a few tips for getting started:

  1. Start by creating bots from services that your business already has exposed as APIs or single-function apps
  2. Focus on bots that are easy for your customers to understand and use, and that provide real, identifiable value to your customers. It’s okay if the service is simple and the value is small, like providing limited sets of information or content in response to specific customer requests or focusing on a commonly performed task with a small, fixed number of options
  3. Design your bots conceptually, then implement them initially on 1 or 2 platforms within communications channels where your customers are most active and where your business has experience, presence and familiarity. That could be SMS/text messaging for a single telecom carrier or mobile OS, or an in-message bot command embedded in one of the messaging services like Kik or Skype
  4. Let your customers know they are part of something new and that you value their input and feedback. Make the experience fun for your customers, and for your product team, too
  5. Don’t get too far ahead in setting goals and expectations. Monitor your initial bot experiences, using your own tools or through the analytics that are provided as part of a bot platform or delivery channel. Don’t be hesitant to iterate, pivot or rethink. You’re not betting your business on these, at least to start with, so failures don’t have a large downside. Look for where you are getting initial success, try to understand the success drivers, and replicate those as you add to the volume and complexity of your bots.

Keep in mind that the bot-driven era of computing will be like a marathon, or at least a 10K run, not a sprint. It’s not too soon to start venturing with bots. Along the way you’ll get a better sense for how they can benefit your customers and your business. And you’ll help determine the shape they take as they come into the mainstream as the next wave of computing.


The Postive, Sunny Side of AI

Recently I had the pleasure of presenting at DATAVERSITY’s 2015 Smart Data Conference as part of their track on artificial intelligence (AI). My presentation covered various AI technologies and products and how they are used in ‘smart applications’ in markets ranging from customer care to retail, healthcare, education, entertainment and smart homes. I also delved into intelligent virtual assistants and other forms of software agents, which if you have read many of my blog posts here, you’ll know is a passion of mine. A video of the presentation is available, courtesy of DATAVERSITY. If you don’t already have an account with DATAVERSITY, it’s just a quick registration and they’ll send you on your way to viewing the video.

Sunny Side of AI Presentation Video

Sunny Side of AI Video

The DATAVERSITY website has lots of other great resources (articles, white-papers, slides, webinars and other videos) on a wide range of topics around semantic technologies, including artificial intelligence and cognitive computing. Speaking of cognitive computing, I also participated at the conference on a panel discussing the burgeoning commercialization of cognitive computing technologies. An article about the panel session and the full video are here:

Cognitive Computing 201 Panel

And here’s another related article that provides a good introduction to cognitive computing.

The audience in both cases really seemed engaged and interested in the topics, and the discussions both in the sessions and on the sidelines afterwards were stimulating. I ran into a lot of familiar faces who have been working in this field for some time and who, like me, are encouraged to see the high levels of interest on the part of those developing AI technologies and products, and those using them for smart applications. There were lots of new faces, too, both fresh vendors unveiling their products and enterprises interested in exploring practical applications of AI. While there were lots of references to the fear factor around AI, to a large degree the attendees seemed to realize that much of what’s written about AI, the killer robot stories for example, is mostly sensationalism. What doesn’t always get written about — unfortunately — are the hundreds if not thousands of everyday uses of AI that provide real benefits to businesses and consumers. That’s the positive, sunny side of AI. Hopefully my presentation helps get that word out. Attendees who are supportive of AI now have more positive material to help counter all of the negative coverage. Perhaps we’ll get more balanced media coverage going forward, even if the positive uses aren’t always as dramatic as sentient AIs rising up and enslaving the human race!

I hope you’ll enjoy the presentation(s), too. Please feel free to contact me if you have questions or comments, or if you want to discuss any of the topics or technologies covered.


artificial intelligence, Semantics

What Netflix Could Do with Its Recommendation Engine to Excite Me as a Customer

You Might Like House of Cards, But I Couldn’t Possibly Comment

My colleague Peter Sweeney, the founder of Primal, and I were talking recently about Netflix using AI, specifically deep learning algorithms, as part of efforts to further improve its recommendation engine. I’ll admit, instead of being excited at the prospect of more insights being gleaned from my viewing history, my first reaction was concern about yet-another bubble along the lines of, or even worse than, the infamous search engine filter bubble, only this time for recommendation engines.

True, we learned earlier this winter that Netflix has re-engineered Hollywood. Netflix has a very rich and extensive categorization scheme — the product of analyzing movies and TV shows from an amazing number of angles, and presumably also following the trails of relationships that customers perceive among the films and TV shows that they view. I think Netflix provides good recommendations, certainly better than they did a few years ago. But frankly, the recommendations still hit dead ends quite often and its easy to get stuck in the rut of more of the same old thing. So my fear upon reading about what Netflix is doing was basically this: deeper mining equals even deeper rut — even more of the same old thing. And that could easily be the case. Just going deeper into analyses of the content itself, as well as my past preferences for it, might well add more categories to their classification scheme, but it doesn’t endear me more to Netflix if I still just get recommendations based largely on my past preferences, only now using more specialized categories. I’m still stuck in a rut.

What I want to experience is more of what I like to call ‘designed serendipity’. If Netflix or Amazon or one of their peers are truly uncovering deeper and more nuanced patterns, particularly within the content itself, but also about my viewing preferences, then they could use that new data to make the recommendation experience more interesting and more compelling for me — giving me something I could actually get excited about. How could they do that? They could start by proposing content from adjacent categories based on walking their classification scheme. Because there would presumably be more, finer-grained categories, exploring some of the neighboring ones could add some fun while still keeping the risk low of jarring me with an off-the-mark recommendation. They could also take those lower-level elements and apply them in somewhat different contexts, preserving the elements that I like, but also mixing in some new twists. They could even try combinations of the lower-level elements, as they’ve done fairly successfully already at higher levels of their classification scheme.

Let me use some examples to illustrate the sorts of things I’d like to see. The West Wing and House of Cards are both political dramas. But at a deeper level, The West Wing is much more about the camaraderie of the White House staff as a team, with politics and political intrigue as more of a plot device for the personal interaction. House of Cards on the other hand is more of a psychological thriller set in a political context. The political maneuvering and bold back-stabbing are core to the show — and for me at least, that’s what makes it fun to watch. Those are fairly subtle, but significant differences that if deep learning can expose, would establish its customer value. Put another way, just because I like House of Cards (the Netflix version, but even more so the original BBC version), does not mean The West Wing would be a good recommendation for me, since such a recommendation is based on more superficial similarities between the shows. I’m a friendly, collaborative, team-oriented person in my real-life, so I’d rather see ego-maniacal scheming and back-stabbing as part of my diversionary viewing!

If Netflix could ‘context lift’ those elements of House of Cards that I do like and then reapply them in different contexts, that would excite me. For example, because I like House of Cards, I might like The Tudors better than The West Wing, even though The Tudors is a historical drama. The Tudors has more of that scheming and back-stabbing (or head chopping!) that I like. While it’s a political drama of sorts, it isn’t in the same sense as The West Wing and House of Cards, so it might not come up as an obvious recommendation. To make that recommendation is more deep and subtle. I also happen to like The Americans, a drama that is political only in an espionage context, but again also a thriller with lots of unexpected twists and turns. And I hate to admit it, but I also like Revenge. Revenge has almost nothing to do with politics, but shares the dark scheming and plotting of House of Cards. Would Netflix be able to recommend either of them to me based on House of Cards? If they get to that level, they’d have my customer loyalty.

At that point, the only thing still missing for me would be adding even more pleasant surprises — turning down the ‘designed’ aspect and turning up the ‘serendipity’. What if I want something that’s conceptually related to a past viewing interest, but still quite different? If I watched Planet of the Apes (particularly the original), wouldn’t Jane Goodall‘s documentary for Animal Planet, Almost Human, be an interesting recommendation? It would for me! Or what if I want to broaden my horizons and try something completely different than what I’ve been watching? Can Netflix put my past preferences in a blender and recommend something really novel and out-of-the-ordinary? Or alternatively can I just throw at Netflix some topics that I’ve been thinking about or have a point-in-time interest in and get a recommendation made-to-order at that particular moment, based on what I just provided, or perhaps subtly influenced by my viewing history? Do those things, Netflix, and I might become a loyal customer for life!


Enhanced by ZemantaOther Related Articles:

Is CIA Behind Your Smart Phone’s Virtual Assistant?

No, I don’t mean the CIA as in Central Intelligence Agency  – although based on the recent revelations about the US government’s intrusion into our electronic lives, they might well be. I mean use of central(ized) intelligent agents (CIA). For example, does the intelligent virtual assistant app on your smart phone require all the data about you as an individual, and all the knowledge rules about what to do with that data, to reside in some centralized repository that embodies (or maybe, disembodies) you and your life? Put more bluntly, does the agent capability or agency depend upon central or centralized intelligence?

So far at least, based on popular intelligent virtual assistants like Google Now or Apple’s Siri, the answer would seem to be yes. Clearly companies like Google and Apple have a vested business interest in being the one place where all the data about your daily life is collected, analyzed and utilized to enable agents like Google Now and Siri to help make your life a little easier. And of course that data happens to be useful for targeting ads and services at you, too. In all fairness though, Google, Apple and others who may be taking a centralized approach to empowering intelligent agents aren’t necessarily doing so purely out of their own self interest. Arguably it makes things a lot easier from a technical perspective to have all that data and logic in one spot, harmonized and maintained by one company. It solves or at least mitigates myriad integration problems that would otherwise have to be addressed with the alternative, ‘distributed intelligence agency’ approach.

What are some of those problems? Well, to illustrate, here’s a (simple?) example.  If my intelligent virtual assistant doesn’t know that I drove to a place near my current geo-location and parked my car in a parking garage that closes at 10:00 PM, it might not know whether to recommend that I walk or drive to a nearby restaurant that it suggested for dinner based on my interest in Asian Fusion cuisine. Is it likely too far to walk (given my health, weight and normal walking habits)? Is it a safe neighborhood to walk in? At what time of day would I be walking and when does it get dark at my location? If I made reservations for more than myself, might others be walking with me? Will the weather be conducive to walking? If walking doesn’t seem reasonable, what are the alternatives (public transit, Uber/Lyft, or my car that’s presently in the parking garage)? If I decide to walk, am I likely to make it back to the parking garage before it closes? If it is too far to walk, but I prefer not to drive or take transit, are there other restaurants nearby with variants of cuisine that are similar to Asian Fusion? Or are there other restaurants with altogether different cuisine that would meet other dining interests or goals that I have expressed (e.g., “Someday I’d like to try one of those restaurants where everything on the menu contains garlic”)? What are my plans for after dinner? Do I have an early meeting or an early flight in the morning? How far is my home or my hotel? Moving seamlessly and quickly across the data and apps, including times, activities, tasks, places, people, personal preferences and other contexts without loss of data or context might well be easier if there’s one spot where all that data lives and one entity that manages it for me. But that comes at the price of “lock-in” and rigidity.

I’m a big fan of distributed systems in general, and so here in the specific case of intelligent software agents, my preference would be for distributed data and a distributed agent framework to enable collaboration across the various data sources, apps and entities that might be involved in agent-based transactions of this nature. That will likely take standards – de facto and de rigueur – or at least agreements among groups of vendors working in the intelligent software agent space. That includes vendors of both specialized ‘vertical agents’ and more general, ‘horizontal agents’. Will the W3C step up to this challenge? Will some other organization or body? Is the community of Android developers powerful enough to pressure Google to open things up at least when it comes to agents on Android? What about similarly for Apple and iOS? Without such action, the intelligent software agent space is likely to be driven entirely by a few big, well-known players who will compete through their own proprietary technologies built on the model of central(ized) intelligence agency. As technologists and/or consumers, is that what we really want?

Where do you stand? Are you for or against a CIA? Keep in mind, your intelligent virtual assistant might be listening to your answer!



Secret Agent Action

This blog post isn’t about some superhero or secret agent code-named Action. It’s about enabling intelligent software agents to take action. As I’ve been writing periodically about intelligent software agents or virtual personal assistants, I’ve not shied away from saying there are significant challenges to making them commonplace in our everyday lives. But that doesn’t mean we shouldn’t be starting to build intelligent agents today.

One challenge is providing software agents knowledge about the domains in which they are intended to operate, in other words making software agents ‘intelligent’. In my last post, “Teaching a Martian to Make a Martini”, I tried to provide a sense of the scale of the challenge involved in making software agents intelligent, and pointed to ways to get started using semantic networks (whether constructed, induced, blended or generated). There are at least two other significant challenges: describing or modeling the actions to be performed and specifying the triggers/conditions, inputs, controls and expected outputs from those actions. These are of course intertwined with having underlying knowledge of the domain, as these challenges involve putting domain knowledge in a form and context such that the software agent can make use of it to perform one or more tasks for their human ‘employer’.

Here’s the first secret to producing agents capable of action: create the models of the actions or processes (at least in rough form) through ‘simply’ capturing instructions via natural language processing, perhaps combined with haptic signals (such as click patterns) from the manual conduct of the tasks on devices such as smart phones. Thinking of the example from my previous post, this is the equivalent of you telling the Martian the steps to be executed to make a martini or going through the steps yourself with the Martian observing the process. In either case, this process model includes the tasks to be performed and the associated work flow (including critical path and optional or alternative steps), as well as triggers and conditions (such as prerequisites, dependencies, etc.) for the execution of the steps and the inputs, outputs and controls for those steps. Keywords extracted from the instructions can serve as the basis for building out a richer, underlying contextual model in the form of a semantic network or ontology. But there is more work to be done. A state like the existence of a certain input or the occurrence of an event such as the completion of a prior task can serve as a trigger, and the output of one process can be an input to one or more others. There can be loops, and since there can be unexpected situations, there can be other actions to take when things don’t go according to plan. Process modeling can get quite complex. Writing code can be a form of process modeling, where the code is a model or a model can be created in a form that is executable by some state machine. But we don’t want to have to do either of those in this case. The goal should be to naturally evolve these models, not to require they be developed in all their detail before they get used by an agent for the first time. And more general models that can be specialized as they get applied are the best case scenario.

I know a fair bit about complex process models. I encoded a model of the product definition (i.e., product design) process for aircraft (as developed by a team at Northrop Corporation — with a shout out here to @AlexTu) into a form executable by an artificial intelligence/expert system. My objective at the time was to test an approach to modeling of a process ontology and an associated AI technology built around it. The objective of Northrop was to be able to have a software system ‘understand’ things like the relationships among the process steps, related artifacts (e.g., input and outputs) and conditionals and to be able to optimize the process through eliminating unnecessary steps and introducing more parallelism. In other words, a major goal of the project was to enable more of what was called at the time ‘concurrent engineering’, both to shorten the time needed for product definition and to catch problems with the design of the product as early in the process as possible (since the later such problems are caught, the more they cost to correct — with the worst case being of course that a problem isn’t discovered until the aircraft has been built and is deployed and in use for its mission in the field). The project was pretty darned impressive, and the technology worked well as an ‘assistant’ to product engineers looking to improve their processes.

Many of the tasks we deal with on a regular basis in our everyday lives aren’t as complex or specialized as the product definition process for an aircraft. But even relatively simple processes can be tedious to encode if every detail has to be explicitly modeled. Here is where another secret comes in: rather than model detailed conditionals for things like triggers, why not use statistical data about the presence of certain indicative concepts in input and output data associated with the process, along with refinements to the model based on user feedback? Clearly this approach makes the most sense for non-critical processes. You don’t want to try it for brain surgery (at least not if you are one of the first patients). But virtual personal assistants and other agents aren’t intended to do jobs for us entirely on their own, so much as to help us do our job (at least at first). So if we have some patience up-front and are willing to work with ‘good enough’, I think we could see a lot more examples of such software agents. If we have expectations that the agents know everything and are right close to 100% of the time, we’ll see a lot fewer. It’s that simple, I think.

So let’s get started building and using some ‘good enough’ assistants. If other people want to wait until they’re ‘perfected’, they can join the party later, maybe after The Singularity has occurred. I think it is time to start now. And I’m convinced we’ll get farther faster if we do start now, rather than waiting until years from now to begin building such technologies en masse. Let’s refocus some of our collective efforts from yet-another social networking app or more cute cat videos onto more intelligent agents. Then intelligent, actionable software agents won’t be so secret anymore – in fact, they’ll be everywhere. And you’ll have more free time to spend with your cat.


Teaching a Martian to Make a Martini

English: Liquid nitrogen storage facility at t...

What Happens When a Martian Makes a Martini? (Photo credit: Wikipedia)

In my last blog post, I stated I felt expert systems were an important forerunner of today’s emerging digital personal assistants and any other software technologies that include an element of ‘agency’ — acting on behalf of others, in this case the humans who invoke them. For someone or something to act on your behalf effectively, they need to understand many specific things about the particular domain they are tasked with working in, along with some general knowledge of the type that cuts horizontally across many vertical domains, and of course they need to know some things about you.

Chuck Dement, the late founder of Ontek Corporation and one of the smartest people I’ve met, used to say that teaching software to understand and execute the everyday tasks that humans do was like teaching a Martian visiting here on Earth how to make a martini. His favorite Martian, George the Gasbag, like the empty shell of a computer program, didn’t know anything about our world or how it works, let alone the specifics of making a martini. Forgetting for a moment George’s physical limitations due to being a gasbag, imagine trying to explain to him (or to encode in software) the process of martini-making — starting with basically no existing knowledge.

First, George has to know something about the laws of physics. He doesn’t need to understand the full quantum model (does anyone actually understand it?), but he does need to be aware of some of the more practical aspects of physics from the standpoint of how it applies to everyday life on the surface of Earth. Much of martini-making involves combining liquid substances. Liquid substances need to be confined in a container of some sort, preferably a non-porous one. The container has to maintain a [relatively] stable and upright position during much of the process. The container holds certain quantities of the liquids. For a martini to be a martini and to taste ‘right’ to its human consumers, the liquids have to be particular substances. Their chemical properties have to meet certain criteria to be suitable (and legal) for use. The quantities of the liquid have to measured in relative proportions to one another. The total combined quantity shouldn’t (or at least needn’t) exceed the total quantity that the container can hold.

You need some ice, which involves another substance — water — its liquid form having been transformed into a solid at a certain temperature. If you are making the martini indoors in most cases or outside when the temperature is warm, the process of producing ice from water requires special devices to create the required temperature conditions within some fixed space. And so on and so forth. You can pull on any of those threads and dive into the subject. Think of having a conversation with a 4 or 5 year-old child and answering all the “Why?” and “How?” questions.

Of course there are at least two major different processes that can be used to mix the liquids along with the ice. They involve different motions — stirring the liquid within the container versus shaking the container (after putting a lid or similar enclosure on the previously ‘open part’ of the container to keep the liquid from flying out). The latter begs the question: is the open ‘part’ of the container really even a part of it, or the absence of some part?

There are allowable variations in the substances (ingredients), both in terms of kinds and specific brands (gin versus vodka, Beefeater versus Tanqueray for gin). Both the process and the ingredients often come down to the specific preferences of the intended individual consumer (take James Bond, for example), but may also be influenced by availability, business criteria such as price or terms of supplier contracts, and whether the consumer has already consumed several martinis or similar alcoholic beverages within some relatively fixed timeframe (don’t forget here to factor in the person’s gender, body size, previous night’s sleep, history of alcohol consumption, altitude, etc.). The main point here is simply if they’ve had several such drinks, their preferences may be more flexible than for the first one or two!

Whew!!! All that just to make a martini? That’s all to illustrate that encoding knowledge for everyday tasks is non-trivial. No one ever said developing intelligent agent software would be simple. But as previously mentioned, George doesn’t need to know everything about every aspect of the domains involved in martini-making. Going overboard is a sure recipe for failure. Knowing where to draw the line is the key and so a healthy serving of pragmatism is recommended. A place to start is I think even getting in the ballpark of knowledge about everyday things and applying that approximate knowledge to practical application uses. Since you don’t always know beforehand how much knowledge you need, I’m a fan of the generative approach to semantic technologies (see my related blog post on approaches to semantic technologies). The generative approach allows agility and flexibility in the production of that knowledge, as well as providing ways to tailor it for individual differences.

And speaking of individual differences: how will George recognize when I’m ready for him to make me a martini? What are the triggers and any prerequisite conditions (like being of legal drinking age in the geo-location where the drink is being made and consumed). Well, I could always ask George (or my personal, robotically-enabled, martini-making software assistant), but I trust that he knows me well enough to recognize that telling look that says, “I could sure use a drink, my friend,…especially after all the knowledge I had to encode to enable you to make one.”


Enhanced by Zemanta

Meet Clippy, Your Personal Assistant

Bill Gates recently spoke at a Microsoft Research event about the return of Microsoft Bob and by association everybody’s favorite on-screen personal assistant, Clippy. Well, he didn’t literally say that MS Bob and Clippy would be back directly incarnate, but he said he could envision them returning in some form as part of a new wave of personal agents or assistants, but with “a bit more sophistication”. A bit more? That’s sort of like saying Michelangelo’s David is like the prehistoric cave art at Lascaux Caves in France, but with “a bit more sophistication”. I mean no disrespect by the way to those cave dwelling artists, who deserve a lot of credit for being among perhaps the first humans to create art, or at least art that was preserved.

A few weeks ago, I blogged here about personal assistants. My vision for them is nothing like Microsoft Bob and his sidekick, Clippy. And in an important sense, Bill Gates and Microsoft are not even at all like the early cave artists; Bill Gates and Microsoft did not pioneer personal digital assistants.

To me, the pioneers for software-based personal assistants were the people who developed expert systems starting back in the 1970s and continuing up to about the time that Microsoft Bob debuted in 1995. I’m talking for example about things like Mycin and Eurisko. Of course the logic rules for those systems were hand-coded, something that won’t scale if personal assistants are to become commonplace in our future. Expert systems also only worked well when applied in specialized domains where specific background knowledge about the domain could be encoded without needing to pull in voluminous knowledge about the everyday world around us. Maybe Microsoft Bob’s tragic failure doomed expert systems and AI? No, at least not on their own. I think what doomed expert systems and AI was the hype gap between envisioned and expected capabilities, the latter being capabilities that far exceeded the ability of technology at that time to deliver.

For the record, I also don’t consider Apple Siri and Google Now as true personal, virtual assistants or software agents. They are flashy and fun in large part because of their natural language user interaction abilities. I do, however, like that they convey a sense that they know a little about us and our everyday world (I just wish they ‘understood’ more), and that they are trying to help us accomplish tasks in that environment. Because of that, they certainly represent steps in the right direction. I’d like to see other steps — and, yes, that includes whatever Microsoft is working on, building on the foundation it laid down with Bob and Clippy [note, later revealed to be Cortana].

Who is doing work in this important area? Tempo AI, for example, is doing some neat things within the calendaring domain. Do you know of some others, and if so, can you share without exposing intellectual property? I’d like to hear about what’s coming, and if I can help get it here faster, just have your personal assistant contact me on your behalf!


How To Get Semantified

First, let me say I’m pretty sure semantified isn’t even a ‘real word’ (yet), but I’ve seen it popping up lately, so I thought I’d help make it into a real word if it isn’t already.

Like anyone into semantics, I love defining categories and then classifying things into those categories. I guess it sort of goes with the job territory! So I’m going to share with you my category scheme for approaches to semantic technologies. I define four categories: constructive, inductive, blended or hybrid, and generative. In practice, specific instances of approaches falling within any given category may draw upon some of the elements of the other categories. In the case of the blended or hybrid approach, I’ll claim it involves a tight coupling of two of the approaches and it’s different enough to be its own category. Descriptions of each of the four categories follow.


Constructive approaches essentially hand-craft their semantic models. As a knowledge engineer, I’ve been involved in several projects using this approach and I can tell you it can be really hard work with often slow progress. Some projects using the constructive approach are done by relatively small, dedicated teams of knowledge engineers and some are more community-based or crowd-sourced type efforts. Some produce proprietary or private models and some open or public models. A few are general purpose, like Cyc/OpenCyc, but most are targeted at specific vertical domains such as finance, travel or healthcare. I view the Semantic Web’s Linked Open Data (LOD) models as being constructive models. Some constructive models are developed for internal use and some for use by and/or sale to others. Some are explicitly exposed as models – conceptual schemas or ontologies, or at least taxonomies. Some are embedded behind applications and are never made visible to their consumers.

The constructive approach is a good fit if you want to produce a relatively-static semantic model for a well-bounded and relatively-static target domain. This approach has often been used when the resulting semantic model is shared and is intended to be consistent across the set of shared users. Although hand-crafting a large, complex model may not be a wise endeavor for the faint of heart or those with not a lot of time to spare, a constructive approach may be quite tractable if there’s a large, enthusiastic community contributing to the development (and maintenance!) of the model and if the problem space lends itself to ‘divide and conquer’ tactics.


As the name implies, this approach involves inducing semantics through techniques such as topic clustering and other statistical [text] analysis applied against large volumes of corpora – think millions or hundreds of millions of documents. In other words, this approach can be described as machine learning or analytics performed over big data sets (or Big Data sets, to use buzz-worthy terminology).

Google is the star example here. Think about how Google Page Rank works with statistics based on the number of links to and from a given Web resource to some other resource, down to the keyword level in many cases. With enough data, you can create indices and associated statistical models based on the relationships among those resources and then use those to retrieve search results, suggest related topics, etc. Simple text indexing works pretty much the same way, where you extract keywords, analyze statistics about the frequency of their occurrence within a document (using for example term frequency inverse document frequency or TF-IDF algorithms), their co-occurrence with other keywords, etc. There are of course more complex algorithms for text analysis, as well as algorithms for images, voice and other multimedia types. In any case, it’s all about statistics and statistical patterns and relationships. This approach therefore works best when there are big data sets available to feed the analysis. Put another way, this approach makes sense if you have a really large amount of data and you want to be able to relate it (i.e., to index it) to other data in a relatively ad hoc, dynamic fashion. I would further describe this approach as being more actionable than reflective, so you should use this approach if you care primarily about operationally using the indices to provide results or answers, and not as much about creating and persisting specific, explicit semantic models behind those results or answers.


This approach is a blended or hybrid approach involving both constructive and inductive approaches. Typically this approach involves starting with a small-ish core or ‘upper’ ontology that’s typically comprised of quite broad classes or categories and then using that to help classify the topics or concepts that are induced via algorithms like cluster analysis. Where the topics or concepts aren’t already in the starter model, then the output of the [deeper] induction process can be used to extend the model with these new more specialized concepts.  Unlike the pure inductive method, here the model itself and richly-indexed content are both targeted outputs of the process. This process goes on in the standard “lather, rinse, repeat” fashion until you run out of compute power or money, or simply cannot statistically-induce any more semantics.

This approach is most appropriate if you want to create extensive, multi-dimensional, relatively-persistent semantic (i.e., concept) indices for large amounts of data and then use those indices to intelligently retrieve relatively-small numbers of highly-relevant results. Examples of such applications include information discovery within enterprise content management systems and question answering assistants, such as for customer care systems.  This approach may not be feasible for massive amounts of Web data that changes constantly. But for large sets of data that are relatively more persistent – like enterprise information – this approach can produce higher quality results over time. Of course given the additional processing, this approach can be slower than a pure inductive approach, so it may require the introduction of optimization techniques, particularly for real-time applications.

An example of a technology using this approach is a start-up under the umbrella of Frost Data Capital (formerly Frost Venture Partners) called MAANA, Inc. I had the opportunity to help them during the early, incubation stage of their life-cycle. Without getting into details, I can say they are doing leading-edge work in multi-dimensional semantic indexing, specifically over Hadoop/HDFS-based data stores, and that work includes innovative optimization techniques for large, enterprise-scale data sets.


The generative approach is a probabilistic approach that is essentially the opposite of the inductive approach. With this approach you start with a relatively small set of building block concepts. These are constructive primitives or atomic concepts rather than the broad or general concepts associated with the blended/hybrid approach. These get used with a set of generative rules to generate or synthesize candidate concepts that then get validated using a smaller set of reference corpora for evidential purposes.

The generative approach is applicable if you want to dynamically generate and utilize on-the-fly semantic models, particularly for highly-specialized or individualized topics that aren’t necessarily possible or feasible to model in advance. This covers two extremes, one where the volume of data is too small to lend itself to induction (for example, for new areas of data collection where there isn’t sufficient data yet) and to extremely large domains (where the sheer number of possible combinations and the cost to model those in advance using any of the other approaches would be prohibitive). In addition to the value of generating the individualized models themselves on-demand, this approach is valuable for content discovery and filtering (e.g., for applications such as personalized research or news readers) and for contextual knowledge building for personal assistants and other forms of intelligent software agents.

So far as I know, there is only one example today of a semantic technology that uses this generative approach and that’s a company I have been associated with for the past few years called Primal (

Previously, during the last generation of semantic technology in the 1990s, I worked for another company that pursued this approach, albeit somewhat differently. That company was called Ontological Technology (Ontek) Corporation. In that case, the technology – which was called the Platform for the Automated Construction of Intelligent Systems or PACIS — depended upon a very precise, formal, foundational ontology from which all the other ontologies were to be – in theory at least – automatically generated. The driver for that was this: after having tried to hand-build a complete ontology for the engineering and manufacturing domains, the visionary behind PACIS decided to define a foundational ontology from which the ontologies for engineering and manufacturing – and potentially many other domains as well – could be automatically generated. It failed for obvious reasons. Or at least they were obvious after it failed. Frankly, the precision expected of that foundational ontology and of the ontologies to be generated from it – was unrealistic to achieve at that time and likely even still today. There were in any case many valuable results produced along the way, and from failure you learn.

That’s why I became associated years later with Primal. I felt Primal’s generative-based technology – which was a working prototype at the time I got involved – was much more pragmatic and practical, and scoped towards more achievable use cases. I spent a little over 2 years up in Canada working with the talented team at Primal to progress from prototype to Minimum Viable Product (MVP), through Alpha release, and now to the first commercial product built on that core technology — an intelligent automated content service. In other words, that technology, which dynamically generates a kind of taxonomy of user interests referred to as an Interest Graph, is now commercially in use.

Picking the right approach depends on the nature of the semantic modeling work you are doing and the resources available to do the work. Using the right approach, there will in any case be hard work ahead, but you should be able to achieve your goals. Choose the wrong approach, and as my son would say, it’s destined to end in an epic FAIL.


What Makes a Good Personal Assistant?

One of my mentors from years ago — who was one of the smartest people I’ve known — had a great deal of admiration for personal assistants. The human kind I mean. Through him, I gained an appreciation for them, too. By personal assistants I mean anyone who directly helps someone else do their job or perform a similar sort of function.

A personal assistant, like other employees, works for their boss. You can say that all employees are extensions of their boss and broadly-speaking they help their boss achieve the goals of her/his organization. But most employees have specialized jobs to do that apply to particular aspects of the organization they are part of. A personal assistant is different in that their job is to help their boss do his/her job more efficiently or effectively. They take on tasks that their boss can’t do due to time or other constraints, doesn’t want to do, finds value in having someone help with, etc. They remove roadblocks, take care of details, keep things organized and stop things falling through the cracks, and sometimes even act as roadblocks themselves — preventing other people from distracting their boss or wasting her/his time. In numerous ways they augment the value and productivity of their boss and make his/her life a little bit easier.

A good personal assistant is a natural, almost seamless extension of their employer. They have a deep understanding of the organization, its goals, resources and constraints, and the nature of the tasks to be done. They understand their employer’s role in the organization, her/his motivations and his/her opinions of other people inside the organization and externally in partner, customer and competitive organizations. They know who can be trusted and who to ‘go to’ for specific needs. They are perhaps their boss’ most trusted employee, a confidant.

Really good personal assistants anticipate their boss’ next requests and are prepared to respond to those as soon as the boss gives even an indication. They know the set of possible options/next steps and the likely decision criteria that would trigger one versus another. When something unexpected comes up, they know to alert their boss. They are honest and open with their boss (but not necessarily fully open with others, as they need to know when to protect their boss and her/his interests). They don’t necessarily need to be told explicitly when to do something or what to do. They can often read between the lines or recognize triggers for actions they should be taking. They can be entrusted to represent their boss and to make appropriate decisions on their boss’ behalf, when needed (and at the same time not overstep the boundaries of their authority and responsibility).

So how does a personal assistant become capable of contributing so effectively to the execution of tasks — some more mundane, some more important — that their boss otherwise would be doing? It’s all about context. A personal assistant knows the background for tasks and decisions. They know the history. They have strong domain knowledge. They know their boss’ preferences. They know how ‘outsiders’ (anyone but them and their boss) are likely to behave in certain situations, who to trust and who to keep an eye on. Some of this information they have been told explicitly by their boss. Some of it they have picked up just from working closely with their boss for some time. They need to know the limits of their own skills and knowledge and who to bring into the process (often on their boss’ behalf) to achieve the goal. They know where and when to ‘ping’ their boss and get additional direction, clarification or confirmation.

All of this is relevant to my interest in semantics for the following reason: if we want to develop virtual personal assistants — intelligent software agents — they will need similar knowledge and capabilities. That will have to come from somewhere, and it seems unlikely or at least inefficient if all that knowledge has to be hand-curated and all that logic has to be hand-coded. Some of it at least will need to be machine- learned/generated and then validated, based on existing knowledge and through learning loops (i.e., by trial and error and/or explicit feedback loops).

Think about the so-called personal digital assistants that exist today. Do they have these capabilities and knowledge? Not the ones I’m familiar with. They do a decent job of interpreting my explicit voice commands for a relatively fixed set of tasks, such as search, creating a meeting event or calling someone from my contact list. More complex tasks, if handled at all, involve a constant series of back and forth questions between the personal assistant and me. Human personal assistants might do this early in a task to establish the framing criteria, but part of their productivity-enhancing abilities lies in the fact that they can do many things autonomously or semi-autonomously (knowing when to come back to me with questions, or to discuss options or problems). Can you imagine if they behaved like their digital counterparts do today, simply echoing back much of your input and making you connect most of the dots and orchestrate most of the workflows through repeated questions derived from some sort of decision tree? Those aren’t true personal assistants, those are at best ‘helper apps’. But you have to start somewhere I guess.

We have a long way to go to get to where we have real virtual personal assistants. I think the basic technology components exist for the most part today to at least get started. But there is much work to do to put them together in the right way and create apps that do even a little bit of what Corporal ‘Radar’ O’Reilly did for the Colonel on M*A*S*H [Note: if you don’t know that reference, by all means Google it! He’s the kind of personal assistant I want to have in automated form].