|
|
|
|
|
|
|
|
|
Good morning!
Back to school might look really different this year, but the Fall always brings fresh opportunities to learn and grow - and this year is no exception. We just announced our next online course coming up at RLab October 9-25: Human-Centered Design for AR & VR. If our September online course on Virtual Production (running from September 24 - October 10) is more up your alley, be sure to register asap - early bird pricing ends tomorrow, September 14! Our friends at Lehman College in the Bronx are also launching their Fall XR Training Academy program focused on Unity on September 21.
There are a growing number of jobs to be found on the other side of these training programs, and we'll be talking about where the jobs are in our next monthly XRJobs Office Hours on September 24. RSVP to join us for a conversation and Q&A with leaders from Google, Huge, The WXR Fund and more to be announced soon!
In this week's newsletter, we learn about Microsoft's deepfake detection efforts, including preventative measures like authenticated provenance (adding digital certificates to photos to establish authenticity) and post-facto methods like detecting manipulation in photos and videos. In other news, the Fortnite team announced a new concert series broadcast from a state-of-the-art studio space in LA that features "a massive LED wall and floor and robocams that can be operated remotely" (it was designed with COVID protocols in mind). We also take a brief dive into a recent Apple patent application that suggests the use of thermal or infrared imaging for dealing with moving world objects in AR, and the announcement of XR Safety Initiative's v1.0 privacy framework, which intends to help protect users' personal data in XR.
Enjoy the rest of your weekend!
- the RLab team
We're always working to improve this newsletter - let us know how we're doing and send tips, feedback or questions to info@rlab.nyc.
|
|
|
|
|
MUST-READS
|
|
|
| Microsoft Launches a Deepfake Detector Tool Ahead of US Election |
Researchers were able to catalog 96 disinformation campaigns that used synthetic media to target the governments of 30 different countries between 2013 and 2019 - and 93% of these campaigns used original content. There are even established underground services that offer disinformation as a service. To help fight the rising tide of deepfakes, Microsoft released Video Authenticator, a tool that gives a confidence score to photos and videos that may have been manipulated, "detecting the blending boundary of the deepfake and subtle fading or greyscale elements that might not be detectable by the human eye."
Microsoft acknowledges that detection tools can quickly become outdated: "We expect that methods for generating synthetic media will continue to grow in sophistication. As all AI detection methods have rates of failure, we have to understand and be ready to respond to deepfakes that slip through detection methods."
The company is also introducing a tool built into Azure that allows developers to add hashes and certificates to content to provide a means of authentication. A reader - built into, say, a browser extension - would then be able to certify content using these elements.
5 min read
|
|
|
|
| Fortnite Is Launching a Concert Series It Hopes Will Become a 'Tour Stop' for Artists |
Nate Nanzer, Fortnite's head of global partnerships, wants Epic's proto-metaverse to be a "platform for artists." Not just an occasional venue for superstars like Travis Scott and Marshmello, but a corporeal stop for touring musicians wanting to promote their songs to tens of millions of gamers, "similar to the way if you're an artist and you have a new album coming out, you might play on Fallon or Saturday Night Live," says Nanzer.
Epic has already set up the virtual stage - Party Royale island features a concert arena and a theater space (which recently showed the 1984 Apple parody). The stage has already hosted Diplo, Steve Aoki, Deadmau5, and Kenshi Yonezu for millions of players. But Epic is planning something bigger:
"Epic built what it describes as a state-of-the-art studio space in LA. It has a massive LED wall and floor and robocams that can be operated remotely.... While past events were successful, they were also limited primarily to DJs performing in their own homes. The studio is meant to make performances bigger and more impressive." You can watch the first studio performance from yesterday's Dominic Fike concert - part of a three-week series - below:
3 min read |
|
|
|
| Apple May Add Infrared Thermal Imaging to the iPhone to Help AR Accuracy |
Last week, we covered the rumor that the iPhone 12 Pro will have the same LiDAR sensor tech we saw on this year's iPad Pro. Since the iPad's launch in April, there hasn't been much commotion about the technology - VentureBeat wrote that the feature "made a narrow category of apps a little better on a narrow slice of Apple devices," but it is expected to see more widespread use on the iPhone (maybe helped along by more Apple AR easter eggs?) For a refresher on LiDAR and its importance for future AR applications, check out this Mashable explainer.
But there's a deeper accuracy problem with AR that can't be solved with LiDAR depth-sensing alone: dynamic environments. Computer vision methods used in AR tracking assume a static environment, where the only object that moves is the camera or headset itself. Apple thinks this is a fundamentally broken assumption. AR glasses that only work perfectly when world objects are still will be less than functional in a dynamic, ever-moving world. Apple addressed this issue in a recent, 22k-word patent application.
SOTA algorithms that segment dynamic objects are "usually computationally expensive and rely on motion segmentation and/or optical flow techniques," the patent application notes. Apple suggests thermal imagining or infrared as possible solutions: "[Thermal] properties of real objects in the scene could be used in order to improve robustness and accuracy of computer vision algorithms (e.g. vision based object tracking and recognition)." Relying on thermal imaging would also mean less dependence on good lighting.
3 min read |
|
|
|
| XRSI Releases VR/AR User Privacy Framework, Citing 'Urgent' Need |
The nonprofit XR Safety Initiative recently released v1.0 of its privacy framework - a rule set meant to establish accountability and trust for XR applications while providing data privacy to XR users. As with most emerging tech, regulation is falling far behind in XR, with few standards for developers and users. (This is true even in heavily scrutinized fields like gene editing - an international National Academies commission has just released a 225-page report detailing strict criteria for human genome editing... almost two years after the CRISPR baby scandal).
XRSI's framework is concerned with technology companies harvesting personal information; in XR, there's plenty of vulnerable data points: skin and eye color, age, hand position at a given second, gaze tracking and facial gestures, location, and other sensitive data. The framework emphasizes preventative measures ("content moderation, proactive actions to preserve privacy, differential privacy, decentralization, anonymization") and a more thoughtful approach (think before you act vs. "move fast and break things"). The framework was developed by a group of academics, attorneys, XR industry executives, engineers, and writers. See the 45-page framework here.
2 min read |
|
|
|
| MUST-SEE |
If you've been following Nintendo's recent physical products and collaborations, you know the company seems to have an infinite repository of bold, creative ideas it's not afraid to experiment with (see the Lego SNES and Nintendo Labo). The company recently announced Mario Kart Live: Home Circuit, an RC car with a built-in camera that lets you set up a custom course in your home, then race through it with an AR overlay using your Switch.
|
|
| COMMUNITY NEWS |
Lehman College announced its Fall XR Academy Program and Unity workshops. This affordable, intensive online six-week, 72-hour program will introduce you to Unity fundamental concepts. At the end of this training, you will be able to create your own mobile application using Unity. You will be able to design your game or application for different platforms (like iOS or Android) and learn to load them onto your test devices. Program starts on September 21. Find out more and register here.
echoAR (RLab Beta Summer '19) is running its first AR/VR hackathon next month in partnership with TOHacks. The Great AR/VR Challenge is a virtual three-week challenge for 3D/AR/VR enthusiasts where teams of students and young entrepreneurs will collaborate with industry professionals to develop AR/VR-based solutions. Are you ready to take on a one-of-a-kind challenge from the comfort of your home? Apply now.
Eric Schwertzel (RLab Mentor) announced a partnership between his company OEM Content and Raiz Media focused on creating digital twins of any physical location that can be accessed by users around the world via any PC and Chrome browser - for concerts, events, film sets, training, tours, art, and optimizing of IoT system and manufacturing processes. Raiz New Media creates state-of-the-art 4K/8K/16K digitization and content production for 3D spatial computing and 2D platforms. Clients include Google, Deloitte, NASA, museums, venues, and factories. Contact Eric to learn more
|
|
| TWEET OF THE WEEK |
Playcrafting is hosting an Oculus Start community game jam with a touching tribute to Brooklyn VR developer Jose Zambrano. The jam starts September 14 - register here.
 |
|
|
TECH TALK
|
Assistant professor at Virginia Tech Jia-Bin Huang and his research team recently published Flow-edge Guided Video Completion, detailing a new method that removes objects and watermarks, or expands field-of-view from casually captured videos. The pseudocode is pretty readable too, if you're curious.
6D.ai (sold to Niantic) co-founder Matt Miesnieks sums up the applications for this tech nicely in his tweet: "Remove distractions, ads, people we don't like… our social media filter bubbles will be even scary(er)."
Diminished Reality is a powerful concept for XR: VentureBeat defined it in 2017 as "the conceptual opposite" of AR, "erasing traces of reality." Besides the erasing "people we don't like" part, there are practical business applications, such as retail. Take any retail AR app that allows you to decorate your home. If you want to completely redesign an existing room, imagine being able to erase furniture and change wall colors to start with a completely blank canvas, instead of just adding things. Check out Huang and co.'s work in action below (and for some old-school augmented/diminished reality, check out this video from 2010.).
 |
|
| EVENTS & OPPORTUNITIES |
|
|
UPCOMING CLASSES AT RLAB
Online Intensive: Virtual Production
September 24 - October 10
Learn virtual production tools and techniques for remote multi-user collaboration and content creation in the Unreal Engine. This two-week online intensive is designed to help digital filmmakers, directors, producers and other creators address remote production challenges and develop the skills and processes that are most relevant to their projects and pipelines. Register Here. Early bird pricing ends September 14.
Online Intensive: Human Centered-Design for AR & VR
October 9-25
Learn a human-centered design approach for creating experiences across the mixed reality spectrum. Create your own WebVR, WebAR, social AR and spatial computing experiences in this flexible, two-week online intensive, which combines learn-at-your-own-pace video lectures, workshops, instructor office hours and group webinars. Register Here. Early bird pricing ends September 28. |
|
Event: Ethics and Privacy: Terms of Usage
September 14, 10AM - 11:30AM
As part of the Data Science Institute's flagship annual event, Eric Schmidt will join for a discussion with Jeannette M. Wing.
Event: Facebook Connect
September 16
Facebook Reality Labs' first ever fully digital AR/VR-focused conference (rebranded from Oculus Connect), airing on Facebook Live.
Course: XR Academy Program
September 21 - November 3
Lehman College's intensive online 6-week program will introduce you to the core fundamentals of Unity that are essential for developing 2D apps, 3D apps or XR apps.
Workshop: Introduction to Unity Platform
September 21 - 24
Lehman College's 12-hour course covering Unity's interface, asset store and customization features, with a focus on VR projects.
Event: AWE Nite NYC
September 22, 7-8PM
Monthly online meetup for AR enthusiasts features talks, demos and networking.
Event: XR Jobs Office Hours
September 24, 12-1PM
Monthly office hours hosted by RLab featuring a panel and Q&A on jobs in XR. This month's topic is: Where are the jobs?
Event: NYC Media Lab Summit 2020
October 7-9
1,000+ media and tech executives, university faculty, students, investors, and entrepreneurs come together virtually to explore the future of media and tech in New York City and beyond.
Opportunity: The Great AR/VR Challenge
Deadline: October 3
From echoAR (RLab Beta Summer '19) and TOHacks, a virtual 3-week challenge where teams of students and young entrepreneurs will collaborate with industry professionals to develop AR/VR-based solutions.
Opportunity: Techstars Anywhere
Deadline: October 11
While anyone from anywhere building anything can apply to Techstars' remote program, they're specifically looking for founders building products and services catalyzed by a remote-first world.
Opportunity: Kaleidoscope Grants
Deadline: October 30
Grants for projects in development include Activist Lens for documentary projects, Femme Futures for female artists, and more. Note: all grants are now awarded quarterly, except for Black Realities for projects by Black artists, which is awarded monthly.
|
|
|
|
| OTHER NEWS |
|
|
NVIDIA's DLSS 2.1 Supersampling Now Supports VR, Could Significantly Boost Fidelity
Voice Assistants Don't Work for Kids: The Problem With Speech Recognition in the Classroom
Mobile Volumetric Capture Unit, the Polymotion Stage Truck, Unveiled
BFI London Film Festival Unveils New Virtual Reality Strand
Miners in China Work From Home Using 5G-Enabled Machinery to Do Heavy Lifting |
|
|
|
|
|
|
