Gen AI is Way More Capable than what Google Answers suggests

Perplexity picking-up on Satire is actually quite impressive

Last week, Google rolled out “Google Answers,” which essentially uses generative AI to summarize information from websites and displays that information directly in the search results.

This has led to some ridiculous answers and has produced ammunition from critics of gen AI such as Gary Marcus.

These are some of the most ridiculous answers floating around.

Add some glue to your pizza -

Drink urine —

Eat a rock

While gen AI is not perfect and may never be, these ridiculous answers are more of a Google problem than a Gen AI problem.

I ran each of these in Perplexity.ai and here is what I got.

Why did Google decide to share the idea that one should eat rocks? Likely from a satirical article in the Onion that suggests it and is reprinted elsewhere. And, of course, the Google answer.

What is actually impressive is that Perplexity (or Opus, I chose Claude 3 Opus when running this query in Perplexity) was able to determine that the original Onion article is satirical, which points to a high degree of sophistication in the model(s).

AI scientists long held the belief that AI could never capture the nuances of language, which it clearly has (the example I always give is that it can pick-up on the nuances of how the word “crane” is used in different contexts). But it cannot just capture the nuances of language, it can also understand satire.

Some concluding notes —

(1) AI, while not perfect, is often better than some examples demonstrate.

(2) Google will likely overcome this. The technology obviously exists and the gap among companies over 6-12 months is usually not that large.

I was a very early user of Perplexity, and it is at least 10X better than it was at this point last year.

(3) Big companies like Google do need to be a bit more responsible because —

(a) Their mistakes are noticed, and it impacts how people view AI, especially gen AI.

(b) Google is starting to make a hard play for schools, and while industry is largely on-board with AI, all schools are not. If a company is going to make a big play for the education market and then tell kids to put glue on their pizza, and quote “f-smith” things aren’t going to work out, not only for Google but for others who are releasing more refined systems.

© They can probably do better.

(4) AI is not perfect, but it is useful for lots of things and can have a transformative impact on communities, including communicating with parents in a conversational manner in their own language.