Neural Rendering | The Juice
Zumo Labs presents The Juice, a weekly newsletter focused on computer vision problems (and sometimes just regular problems). Get it while it’s fresh.
Week of May 3–7, 2021
____
We often compare traditional datasets to practical effects in filmmaking. It was the way folks did things for a long time largely because it was the only option. Then, along comes computer generated imagery and suddenly you can create effects that were simply not possible before. It was technology that grabbed the attention of auteurs like George Lucas, who were suddenly like, “Wait, wait, I can do better. Look! Dewbacks on Tatooine!”
Synthetic data is a lot like CGI. For one, it has made exciting new things possible. But more concretely, it’s created exactly the same way, often using the very same software. An artist creates a model from scratch, essentially a polygon mesh with a texture. The more complex the scene, the more challenging and time consuming that modeling process. Which begs the question, what technological breakthrough will revolutionize classical computer graphic generation?
Hugo might have an answer. Last week, he gave us a tour of an emerging class of image generation approaches called neural rendering. It’s really exciting, state of the art stuff, and we convinced him to distill that learning group into a blog post on neural rendering — just for you.
____
#NAII
This week, the White House launched AI.gov in an apparent effort to make ongoing artificial intelligence research more accessible to the public. The site aggregates all of the distinct governmental agencies that are using artificial intelligence and, in many cases, links directly to each agency’s respective documentation. If you’ve been trying to figure out the difference between the NAIIO and the NAIAC, now’s your chance.
White House launches new artificial intelligence website, via Axios.
#EQ
Emotion recognition technology is built on shaky foundations. “A 2019 systematic review of the scientific literature on inferring emotions from facial movements… found there is no reliable evidence that you can accurately predict someone’s emotional state in this manner.” And yet start-ups and industry giants have designed and widely deployed these technologies. That makes me feel, well, like this.
Artificial Intelligence Is Misreading Human Emotion, via The Atlantic.
#ASL
The American Society for Deaf Children and creative studio Hello Monday have created a computer vision-powered web app that will teach you American Sign Language. From spelling to fine finger positioning, the app seems like the next best thing to working with an actual tutor.
This web app uses computer vision to teach you the ASL alphabet, via Engadget.
#CO2
Last year, researchers at UMass Amherst published a report suggesting that training especially large machine learning models can have absolutely massive carbon footprints. Now, Google has published some research of their own refuting those numbers, suggesting that they’re off by an order of magnitude. This VentureBeat piece breaks it down, and outlines the obvious conflict of interest in Google — who sells cloud computer services — finding more palatable numbers.
Google-led paper pushes back against claims of AI inefficiency, via VentureBeat.
#UNESCO
At least 43% of the estimated 6000 languages spoken in the world are presently endangered, per UNESCO. Google has launched an app called Woolaroo in an attempt to preserve a small handful of these languages. It works sort of like Google Lens, in that you point your camera at an object and it uses machine learning and image recognition to identify and label that thing in the selected language.
Google’s AI photo app uses crowdsourcing to preserve endangered languages, via Engadget.
#AAPL
We covered the story when Samy Bengio, a high ranking researcher on Google’s Brain team, announced his departure in the wake of the Dr. Timnit Gebru dismissal. It now appears that Apple has capitalized on the shakeup, bringing him aboard to purportedly lead a new AI research unit.
Apple hires ex-Google AI scientist who resigned after colleagues’ firings, via Reuters.
____
📄 Paper of the Week
Emerging Properties in Self-Supervised Vision Transformers
This paper DINO and its equally adorably named pair PAWS have been making a splash in the ML community. Self-supervision is when a model is trained with minimal or zero labeled data examples. One way to do this is to perform two random transformations of the same image, and instruct the network to bring the two resulting feature vectors closer to each other. Do this with enough training data (and a fancy teacher-student training structure), and you will eventually get feature vectors that are pretty darn good. This paper boasts a 78.3% top-1 on ImageNet when training a k-NN classifier on these features. Self-supervision is likely to replace the feature encoders we see today (which are usually just pre-trained on COCO or ImageNet), since they can be trained on even more data and *should* result in more general features. Check out this great summary article for more.
____
Think The Juice was worth the squeeze? Sign up here to receive it weekly.
