An article on cnet.com recently described how Google's new smart assistant, called Google Nest Hub Max, uses facial recognition technology to tell who is talking with it.  This feature has raised privacy concerns, as Google has admitted that they reserve the right to upload facial data from it to the cloud to help improve "product experience."  But whatever Google does legitimately, a hacker might be able to do too, and so we are approaching a time when the telescreens of George Orwell's dystopian fantasy novel Nineteen Eighty-Four have become a reality—not because of the unilateral command of a totalitarian government (at least not in the U. S.), but because we want what they can do.

For those unfamiliar with the novel, Orwell's book was a warning to the free world to beware of what a dictatorship could do with communications technologies of the future.  Telescreens were two-way televisions on which propaganda by a dictator known only as Big Brother is transmitted, and through which images of whoever is watching are transmitted back to the party's central headquarters.  Orwell was simply extrapolating the efforts of regimes such as the Nazis of the 1930s and the Soviet Union of the 1940s to spy on their populace twenty-four hours a day to enforce total obedience to the regime. 

At the time the novel was published in 1949, no one took the telescreen-spying idea very seriously, because it would take a huge number of human monitors to spy on a significant number of people.  Carried to its logical extreme, the only way the government could watch everybody would be if half the population spied on the other half, and then took turns. 

But neither Orwell nor anybody else at the time reckoned on the development of advanced artificial-intelligence (AI) systems using facial recognition technology.  In China, the government is deploying many thousands of cameras and taking facial data from millions of people with the intention of developing a Social Credit rating that measures how well you measure up to the regime's model of the ideal citizen.  If you have been caught on camera by computers going to suspicious places or meetings, your Social Credit score could go in the tank, making it hard to travel, get a job, or even stay out of jail. 

None of that is happening in the U. S., but the fact that a large corporation will now have electronic access to views in millions of private residences should at least give us pause. 

Leaving the hardware aside for the moment, let's examine the difference in motives between a government, such as in Nineteen Eighty-Four, spying on its citizens for the purposes of controlling behavior, and a commercial entity such as Google using images to sell both its own services and advertising for others.  The government spying is motivated by suspicion and fear of what people might be doing while the government isn't watching them.  Whatever the regime sets out as an ideal of behavior, it watches for deviations from that ideal, and punishes those who deviate from it.  Participation is not voluntary, and people have to go to great lengths to avoid being spied on.

Now contrast that with a benign-looking thing such as the Google Nest Hub Max.  Nobody is going to make you buy one.  And if you do, there are ways of turning off the facial-recognition feature, though it will be less convenient to use.  And the device is intended to serve you, not the other way around.  It's sold with the vision portrayed in so many TV ads of people happily using it to make their lives better, not for means of social control like Orwell's telescreens. 

But maybe the differences are not as great as they first appear.  Both the telescreen and the Nest Hub Max are intended to change behavior.  If they don't, they have failed.  True, the ideal behavior that a totalitarian government wants and the ideal behavior that a company wants are two different things.  But neither ideal is the way the citizen-consumer was before the screen or Nest Hub shows up, namely, unwatched and unbenefited by the products or services that the company wants to sell.

Nobody should read this blog and then go around saying "Ahh, Stephan's saying Google is Big Brother and they're trying to take over our lives!"  That's not the comparison I'm making.  My point is that the mere fact of being watched by someone, or something that can inform someone about us, is going to change our behavior.  And that change by itself is significant.

Now, the change may not necessarily be bad.  Already, virtual audio assistant devices such as Alexa have been used in criminal cases when bad actors set them off, either by accident or on purpose, and the data thus generated has proved to be incriminating.  Though this is ancient history, I am told that in the days when some middle-class and upper-class people had servants, families tended to behave better when the servants were around, although I'm sure there were exceptions.  Alexa isn't Jeeves the butler, but as virtual assistants play a more significant role in domestic life, it's not beyond imagination to think that some of the worst behavior in homes—domestic abuse, for example—might be mitigated if the victim could call 911 by just shouting it instead of having to pull out a phone.

I'm not necessarily crying doom and gloom here.  Millions of people are already using virtual assistants with few if any problems, and adding two-way video to the mix will only increase the devices' capabilities.  But we are entering a new territory of connectivity here, and it's bound to have some effects that nobody has predicted yet.  Perhaps it's not too helpful to predict that there will be unpredictable effects, but right now that's all I can do at the moment.  Let's hope that the security features of the Nest Home Max are good enough to prevent nefarious use, and that people who buy them are truly happier with them than they were before. 

Sources:  The article "Google collects face data now.  Here's what it means and how to opt out" appeared on Sept. 11, 2019 at https://www.cnet.com/how-to/google-collects-face-data-now-what-it-means-and-how-to-opt-out/#ftag=CADf328eec.  I also referred to Wikipedia articles on Nineteen Eighty-Four and Google Home.  I thank my wife for pointing this article out to me.