Facebook has been hiring an army of third-party developers to design its new Camera Effects Platform that aims at offering unique augmented reality image filters and similar kind of interactive experiences. The announcement was made on the April 18th F8 conference and anyone using Facebook Camera on their smartphone will be able to use the filter soon enough. The Camera Effects technology is however meant for a bigger goal and will be made compatible with augmented reality hardware like Eyeglasses.
The first clues to a redefined camera feature in Facebook-owned apps like Instagram, Messenger and WhatsApp had been there for months. However, critics believed that Facebook was just mindlessly copying the features from Snapchat. In the F8 conference, Mark Zuckerberg clarified that these were just the beginnings of the Camera Effects platform.
It’s true that even if AR glasses like Eyeglasses are developed and available in the market, people will still be using their camera phones to click and share pictures. The first AR glass isn’t going to be too affordable either. Zuckerberg believes that unlike apps like Snapchat that keep the AR experience secretive, Facebook would be pushing an open platform where AR experience would be available to the masses. Further, developers from across the globe will be able to work on the Camera Effects platform and consequently add thousands of effects to the app. Zuckerberg believes that keeping the platform open source allows for a much wider diversity in experiences.
Facebook has already started allowing developers apply for the closed Beta version of Camera Effects AR Studio tool. The tool will allow developers work with precise location, depth detection, and object recognition to create special effects. Consequently, the new Facebook camera will be able to recognize a cup of coffee and compliment a picture with effects that include sharks swimming in the cup or steam coming off the cup and similar creativity.
Facebook’s Frame Studio, on the other hand, allows anyone with the basic designing knowledge to create face filters. Anyone will be able to upload static and overlaid images in the app and it will appear on their friend’s pages or be accessed by Page pans. Working with Frame Studio doesn’t require any kind of coding knowledge but you will be credited for your contribution in the app preview and your News Feed.
As of now, Facebook isn’t inviting any kind of advertising or branding but it will eventually become the tool to monetize the platform. As of this point in time Facebook is
“focused on making this product great for people before introducing paid ways for businesses to participate”.
Facebook believes that AR experiences can take three useful forms. Firstly, they can be about augmenting objects in a physical space with additional and helpful information like directions on a road or reviews on a storefront. Secondly, AR can be used by layering virtual objects onto real space and see how things fit in. Thirdly, AR could help enhance objects using extra effects and this can be anything from adding a castle turret to your apartment or adding a glitter beard on your face. Facebook is already working on the tech that will allow developers to get creative with their filters and effects. Consequently, the products of Facebook AR would include interactive games, mind bending arts and informational guides that would have been impossible to achieve otherwise. Even regular users of Facebook will contribute to the trend by adding AR notes in specific locations, be it about the best food in a restaurant or risky lanes.
Developers, at this point in time, have three creative tools to trigger the AR filters. There’s Face Tracker, Sensor Data (location, gyroscope) and Scripting APIs that pull in data from other apps and respond to commands in real time. Launch partner for Camera Effects includes Manchester United that allows for pulling in match data and an image overlay of “GOOAALL!” with addition crowd noises. EA Sports too has created a helmet mask filter that will pull in your stats from the game and offer a mass themed effect. Similarly, Nike would be overlaying a sweatband on your forehead and pull in your recent running/jogging route, overlaid in a map.
Developers working with AR Studio will also be able to add effects for Facebook Live. An effect that was launched at the conference included an overlay viewer polling called “This or That” and allowed comments that include Giphy Live Animated GIFs and hash tags. According to Zuckerberg, “the next phase of AR on camera phones won’t require special hardware”. The vision is exciting. With AR based Eyeglasses or even special AR embedded contact lenses, you can have a TV put up virtually on your walls and have a $1 app installed to see programs rather than investing $500 in the actual hardware! In this case, even if the AR device costs about $500, you are saving a whole lot of money by not buying anything else!