Supervision Required | The Juice

Zumo Labs presents The Juice, a weekly newsletter focused on computer vision problems (and sometimes just regular problems). Get it while it’s fresh.

Week of May 24–28, 2021

____

There’s a bottleneck in training AI systems using supervised learning, and its name is labeled data. Right now, a tremendous amount of time and energy goes into data cleaning and labeling, rather than the ML project itself. There’s currently no silver bullet fix for this problem, but there are teams working on it. Here at Zumo Labs, we think that tailor-made synthetic data generated with labels from the outset will represent a significant portion of training data in the future.

Meanwhile, Facebook recently published research on self-supervised visual transformers that holds potential. The idea is to combine a couple of technologies — self-supervision, which lets the model learn from unlabeled data, and transformers, which allow it to pay attention to parts of the image that are significant. Hugo recently led a learning group on the research, and he’s written a blog post distilling the tech here. Anything that reduces the amount of human annotation required moving forward is a win in our book.

____

#Regulations

Last week Amazon extended the moratorium on police use of their facial recognition technology, Rekognition. It was a response to public pressure and a move that buys them time while formal regulations are put in place. Several bills banning or setting guidelines on facial recognition technology have already been introduced in Congress, and privacy groups are optimistic that we may see action on these soon.

We could see federal regulation on face recognition as early as next week, via MIT Technology Review.

#FirmGrip

Google bought DeepMind in 2014. Since then, the parent company has used its largess to fund pretty massive annual losses. (Turns out an AI that slays at Go doesn’t result in immediate financial returns). Meanwhile, DeepMind has been on a years-long effort to secure quasi-independence through “a legal structure used by nonprofit groups, reasoning that the powerful artificial intelligence they were researching shouldn’t be controlled by a single corporate entity.” It now seems Google has quashed the effort.

Google Unit DeepMind Tried — and Failed — to Win AI Autonomy From Parent, via The Wall Street Journal.

#U-Turn

We recently covered Elon Musk’s endeavor to eliminate radar from Tesla’s vehicles in pursuit of “pure vision.” It’s a move that nearly makes sense, if you consider how it would cut costs for the automotive company. But now, after a Model Y was spotted in Florida with a rooftop lidar array, Tesla has confirmed that it has entered into a partnership with sensor manufacturer Luminar. Seems like the lesson, as always, is to take anything Musk says with a Cybertruck-sized grain of salt.

Elon Musk called lidar a ‘crutch,’ but now Tesla is reportedly testing Luminar’s laser sensors, via The Verge.

#UniversalExpressions

Charles Darwin first hypothesized that facial expressions of emotion are universal in the late 1800s. It was a popular concept, and later research indicated that there indeed might be six or seven universal expressions. Now, a research team at Google has published a paper in Nature suggesting that sixteen facial expressions occur in similar contexts worldwide.

Understanding Contextual Facial Expressions Across the Globe, via Google AI Blog.

#SourTaste

It’s never good when a corporate Twitter account begins a Tweet with, “So we deleted this awful thread…” In this case, publicly-traded insurtech company Lemonade was apologizing for having proudly announced that its AI tech “can pick up non-verbal cues that traditional insurers can’t” when reviewing user-submitted claims videos. Now, after a public outcry, it has reversed course and suggested it does not deny claims using AI, which happens to contradict its S-1 filing.

Lemonade swears it totally isn’t using AI for phrenology, via Input.

____

📄 Paper of the Week

RepVGG: Making VGG-style ConvNets Great Again

Don’t get put off by the MAGA reference in the title, this paper is legit. These researchers from Beijing and Hong Kong created a CNN with a different structure at inference and during training (implemented via a reparameterization trick). During inference the model comprises nothing but a stack of 3x3 convolutions with ReLU activations, similar to the now-classic VGG model architecture. This model goes on to beat some big ResNets on speed and accuracy. While I don’t think VGG will become great again, I think the concept of decoupling training-time and inference-time architecture will be worth keeping an eye on.

____

Think The Juice was worth the squeeze? Sign up here to receive it weekly.