Microsoft just built an all-new iPhone camera app that according to the Redmond giant can make Apple’s phone click better photos. It’s in fact better than the camera app that ships with the iPhone, because apparently it is smarter. The app in question is called Pix and is the brain-child of Microsoft’s Computational Photography Group.
So how does Pix make the iPhone camera smarter? Well, for one, it gets to business even before someone has tapped the shutter button and second, it makes use of artificial intelligence to make sure even the least knowledgeable person in town can click awesome photos with the bare minimum tools and settings. The Pix app essentially works by taking a burst shot of 10 frames with each shutter tap and then intelligently selects up to a maximum of three shots that look the best while deleting the others. It however churns through every deleted shot to remove noise, brightening faces, beautifying skin and adjusting the photo’s color and tone. The best part is everything is done within seconds so there’s minimum post-processing delay.
In addition, another set of algorithms intelligently sorts through every frame to find whether a Live Image is possible by looking for any motion that would make for an interesting looping video.
Interestingly, the app starts capturing frames from the time the viewfinder is loaded and continues with the process till the time the shutter is tapped. Should there be faces in the shot; the whole process then starts revolving around them. The app is also said to intelligently omit shots wherein there’s blur, or instances where the subject’s eyes might be closed.
The Pix app is available through Microsoft.com and Apple’s iTunes Store. There is however no word yet when the app would be coming to Android.
Microsoft tries building a smarter iPhone camera app in Pix
By using AI and other software tricks, Microsoft asserts that its camera app outdoes Apple’s at taking pictures of people. An Android version is on its way.
Your iPhone has a built-in camera app, but Microsoft hopes to lure you to a new alternative called Pix that’s packed with computing smarts.
Pix combines several advanced technologies: artificial intelligence, computational photography and video stabilization. But Pix isn’t something like Photoshop with hundreds of options and abilities for you to consider. Instead, it’s designed to improve your photos and videos without you doing any work.
The app, released Wednesday in Apple’s App Store, embodies Microsoft’s new ethos under Chief Executive Satya Nadella. Microsoft’s products once served to reinforce the power of its core Windows software. But with Microsoft’s failure to win over the smartphone market, the company now is bringing key programs like Office and Outlook to rival mobile products powered by Apple’s iOS and Google’s Android software.
When you shoot a photo with Pix, it actually takes 10 photos in a half second and presents you with the one it believes is best. If the scene changes significantly, it will present you with a second option and, rarely, three.
The app isn’t just for still photos. It can speed up videos into smooth fast-forward versions with the approach used in Microsoft’s Hyperlapse app for Android. It can also combine the images into distinctive shots called “cinemagraphs” in which parts of the image are still while other parts are in motion.
“Apple has a strong brand in photography,” said Josh Weisberg, principal program manager in Microsoft Research’s Computational Photography Group. But Microsoft believed it could help the iPhone take better photos. An Android version of Pix is on the way. “Now we feel we can do that on any platform,” he said.
The sophistication comes at a cost. It does take up to two seconds to perform all the image processing, and people exactly don’t like waiting for their photos to appear on screen.
Microsoft Pix is designed to be a better camera app.
How it works
Pix, at this stage, is geared toward taking pictures of people. It tracks faces and tries to set brightness to expose them properly. That’s just the first phase, though. From the 10 photos it takes when you tap the shutter button, it deletes ones in which eyes are shut, faces are blurred or people aren’t smiling, Weisberg said.
When you’re shooting photos with Pix, it’s actually taking photos continuously. Microsoft keeps seven photos from before you tapped the button and three from after, because its tests showed that people often missed the moment.
Even though Microsoft discards most of the shots, it first uses them to reduce image noise, or the colored speckles that degrade photos taken in dim or dark conditions. With several frames of the same scene, Pix can zero in on original colors.
In cases where Pix detects a combination of moving and stationary elements, it creates a cinemagraph. For example, it can show people standing still in front of a moving river, or a person’s face unmoving except for blinking. Done right, cinemagraphs can be very eye-catching.
“We intelligently detect when to make them and only make them when it’s interesting,” Weisberg said.
Microsoft isn’t the only company trying to blur the lines between photos and videos. Apple’s newer phones take Live Photos that record a smidgen of video for a more immersive view. Google Motion Stills can stabilize videos and create cinemagraph-like looped animations. And the new Polaroid Swing app captures one second’s worth of your life in what it calls “moving photos.”
Bring on the brains
That intelligence stems from the hot field of machine learning, a type of AI in which computers operate with a digital relative of the human brain called a neural network. Training neural networks to produce good results can take a lot of time. But once a model for making the right choices is built, it can run on an iPhone 5S or later.
AI brings a new facet to a technology called computational photography that’s been steadily gaining in importance. Camera makers have strived to improve their hardware for years, but computational photography processes images after the fact to go beyond the hardware. It combines multiple images into one, corrects lens flaws, gets rid of blurring from a shaky camera and changes the point of focus after the photo is taken.
Deciding when to build a cinemagraph is one situation where Microsoft takes the AI approach. It also uses that approach for understanding a scene to set exposure correctly, to pick the best of the 10 frames and to detect if a person has taken a photo of something stationary like an office whiteboard.
The app is headed for Android phones, though Microsoft won’t say when it’s due out. “Android has more complexity in the variety of camera modules, chipsets and versions of Android,” Weisberg said. Source: http://www.cnet.com/news/microsoft-pix-uses-ai-to-challenge-apple-iphone-camera-app/