The Nvidia Broadcast application uses AI for an unusual purpose
In the new update to the video conferencing software Nvidia Broadcast 1.4, the company has addressed the problem of eye contact in video calls. This and other small features included in the update can help creators create more engaging presentations. The new Eye Contact feature estimates and changes the direction of a video participant’s gaze to give the impression that they are looking at the camera. This effect should allow viewers to maintain eye contact with the interlocutor more easily.
Nvidia is not the first company to try to simulate eye contact during video calls or video conferencing. Apple added the Eye Contact feature to FaceTime in iOS 13 in 2019 to counter the effect that occurs when users naturally look at the face on the screen instead of the camera during calls, breaking eye contact between callers. Microsoft added similar functionality to the Surface Pro X in 2020. It works in many video calling apps, such as Microsoft Teams and Skype.
However, while Apple designed its eye contact feature around two-way conversations, Nvidia’s version helps presenters who can read from script while recording or streaming. It also tries to know when to break eye contact if the speaker deliberately looks into the distance, seamlessly transitioning between real and simulated eyes.
Eye Contact is still in beta though, as Nvidia hasn’t tested it for all combinations of eye colors and lighting conditions. The company wants users to send video clips as feedback to help the AI-powered system learn.
The other primary new addition to Nvidia Broadcast is the vignette effect, which is designed to enhance virtual backgrounds by simulating bokeh depth of field. Nvidia is now also using time information to improve the way AI handles other virtual background effects such as Blur, Replacement, and Removal. This addition should make visual defects in virtual backgrounds less noticeable. Again, video feedback from users can help this feature grow.
Additionally, users can now mirror the camera and take screenshots from webcams. The icons for these functions are located below the camera preview window.
In addition, Nvidia has added its new features to Maxine – an SDK that developers can use to integrate Nvidia Broadcast features into other applications as microservices. Developers can also get early access to another new feature called Live Portrait.
Source:
Own study/TechSpot
