Snapchat is getting a funky new augmented reality (AR) feature today in the form of a new World Lens. Continue reading “Snapchat’s new 3D Friendmojis put virtual versions of you and a friend in an AR world”
Snapchat is getting a funky new augmented reality (AR) feature today in the form of a new World Lens. Continue reading “Snapchat’s new 3D Friendmojis put virtual versions of you and a friend in an AR world”
The New York Times has just announced it would begin incorporating augmented reality in its journalism. The Times prominently featured the announcement on its website’s front page, speaking to the publisher’s commitment to offer its readers and subscribers the highest quality news content by investing in new digital content technologies. Continue reading “The NYT is boarding the AR train — here’s what that means for storytelling”
During Apple’s fourth-quarter earnings call with analysts, CEO Tim Cook said, “AR is going to change everything.” He wasn’t exaggerating. Augmented reality (AR) is shaping an entirely new paradigm for mass technology use. We’ve quickly evolved from typing on our PC keyboards, to the point-and-click of the mouse, to the smartphone’s tap or swipe, to simply asking Alexa or Siri to do things for us. Now AR brings us to the age of holographic computing. Along with animojies and Pokémon and face filters, a fresh and futuristic user interface is emerging. Continue reading “Move over, voice: Holograms are the next user interface”
Surrounded by darkness, the looming 20 foot skull is so close I can touch it. With the click of a mouse, the 3D model of the human head and neck pivots and I’m inside the eye socket examining this complex system from the inside out. It’s A viewpoint typically reserved for surgeons on the operating table; I’m amazed by the scale and detail of the mechanism that gives us the miracle of sight. Continue reading “VR transforms health care: 3D human model coming to a school near you”
When we think about virtual reality (VR), games such as Job Simulator and Raw Data usually spring to mind. It’s understandable that many of us view VR headsets like the Oculus Rift as glorified video game consoles; VR’s early breakout commercial successes have largely been in the entertainment genre.
In fact, VR has quietly been making significant improvements to the lives of people with issues such as chronic pain, lazy eye, and autism. In this piece, I’ve interviewed three VR entrepreneurs who are pioneering virtual solutions to improve healthcare. Continue reading “3 practical ways that VR is improving health care”
Amazon is jumping onto the augmented and virtual reality bandwagon with the launch of Sumerian, a new application that’s supposed to help make it easier for people to develop 3D experiences for a wide variety of platforms. Continue reading “Amazon launches Sumerian, a browser-based tool for building AR, VR experiences”
GUEST: The consumer electronics industry is evolving every day, and the world of cameras is no exception. We’re seeing incredible things happen with photography, and recent products like the GoPro Fusion and Ricoh Theta V are pushing the industry forward. But despite all of their advancements, there’s one thing many recent cameras are claiming to accomplish that they simply don’t: virtual reality.
As manufacturers, we are part of the group who will set the trend for how each technology is perceived by the public. It is our responsibility to help define the lexicon. But even this deep in the camera industry, there seems to be confusion at the highest levels about what virtual reality is and how it differs from other 360 degree camera technology. As a result, this confusion is being passed on to the consumer, leading many to think that the terms “virtual reality” and “360 degrees” are synonymous. They aren’t. Continue reading “360 degree cameras aren’t VR — and it’s important we say so”
As we prepare to open this new floodgate of information and engage with complex data in the context of the world around us, it’s worth pausing to look at what the potential impact of augmented and virtual reality might be in our everyday lives. While much of this impact is positive, we will have to work to make sure those effects are evenly distributed.
Screeds about the malignant impact of technology on society are, for the most part, somewhat misguided. But as AR/VR moves closer to the mainstream, it’s certainly fair to wonder: if people are already exceedingly glued to their phones, how much worse will our collective situational awareness become when everyone can disappear completely into their own virtual universe?
Actually, there are several reasons to be optimistic on this front. AR/VR has the potential to give us a deeper understanding of our surroundings by enabling us to merge with and better understand the information around us.
Let’s look first at how this could impact us in workplaces. Consider a worker in a car factory. As an engine block comes their way, an augmented overlay highlights the exact places where various components need to be installed before it can be passed along the assembly line. The display also flashes red if it senses that a component is out of alignment or has otherwise been improperly installed.
Meanwhile, a maintenance & operations worker strolls the factory floor, making a comprehensive check of all the equipment. With the benefit of AR, the worker has a stream of speeds and feeds popping up in their field of vision every time they look at a particular machine. They can better understand exactly how the production line is running or when something will go wrong before it happens. This type of information immersion is much more sophisticated and actionable compared to reviewing on a clipboard or scrolling through a tablet to access.
While the automotive industry provides a fruitful example, the benefits of AR/VR can be extrapolated out to any variety of industries and settings: the lifeguard at the beach with an augmented display of tides and temperatures; the construction worker viewing building plans that are overlaid onto the construction site; the surgeon receiving ultra-realistic training on how to remove an appendix; and so on.
Let’s turn now, from the workplace, to a pedestrian walking down a crowded street. Can we expect similar benefits when out on the town?
We have every reason to be hopeful that technologies like AR will in fact provide an enhanced layer of awareness that will improve our fluency with our surroundings in everyday settings. Say you’re out shopping for a new shirt or a cleaning product or a gaming console – what if simply by looking at competing items we could see beyond just cost and aesthetics, but also understand and compare their relative environmental impacts?
In this way, AR has the potential to be additive rather than subtractive, making people more connected to their surroundings than they are today.
We’ve all heard the phrase “with great power comes great responsibility.” There is a responsibility on the people who are developing AR and VR technologies to make sure that it is done in a thoughtful and well-considered way.
It’d be all too easy for developers to unwittingly create a deeper digital divide in society by accidentally limiting who has access to these new technologies and who can benefit from them. Do we want a world where only the wealthy or the tech-savvy are able to enjoy the benefits of a virtual or augmented environment?
Of course not.
AR and VR are only going to become more commonplace in the years ahead. Those who adopt and embrace these new tools will have an information fluency that those who don’t have access to it will be lacking as the 21st century marches forward. If we want to prevent people from being left behind, we need to make sure this technology is accessible—meaning, both affordable and user friendly.
It’s still early and we don’t have it all figured out, but I’d like to propose three guiding principles for AR/VR hardware and software developers:
The truth is, we’ll need to learn as we go. And that’s okay. But be sure to prepare yourself for the ride because our augmented future will be here before you know it.
Brian Pene is Director of Emerging Technology for the Office of the CTO at Autodesk, Inc.
Quartz is known for its news app that delivers the day’s events in short conversational pieces, allowing readers to dive into more detail if they want or move on to the next story. It’s structured like a text conversation with a friend, parceling out each piece of information in message bubbles and aping the format of the chat bots that were all the rage this time last year. Now it’s incorporating another tech trend into its app — augmented reality.
Apple’s been hyping its iOS 11 AR capabilities since it unveiled its software developer kit ARKit at the Worldwide Developers Conference earlier this year. And now that iOS 11 has rolled out to all users with compatible devices, users are starting to see AR pop up in various places, such as Major League Baseball’s At Bat and Fifth Star Lab’s Sky Guide. Quartz has also taken the opportunity to incorporate some lightweight AR into its app, first with a 3D rendering of the Cassini probe that recently entered an early, permanent retirement on Saturn.
“We currently illustrate our news stories with not only images and videos, but also emoji and animated GIFs,” said John Keefe, Quartz’s product manager, in a phone call with VentureBeat. “We decided that it would also be cool to incorporate 3D news objects into it. Since Apple made it pretty easy for us, we got to it right away. We have a developer on staff who started playing with it.”
Keefe says the Quartz team has been working on the AR feature for a few weeks now, and the main issue they’ve encountered is actually finding the 3D models to begin with. They’ve found marketplaces where people create 3D models for others to buy or download, and they’ve experimented with using their own scanning device. Their Cassini model came from NASA, which Keefe says has a number of models available.
Finding the models is only the first step. Keefe says that they want the models to be as realistic as possible, so lighting is often a consideration. He says that ARKit does take care of some of the work, but they still have to make adjustments, such as how reflective certain surfaces are.
“You’ll look at something and be like, hey, that’s not supposed to be shiny, or it is supposed to be shiny,” said Keefe. “That’s usually what we’re trying to adjust. As more and more people are making these 3D models, however they make them, it’s going to get easier to incorporate them.”
Keefe says that they’re aiming to always have at least one AR story in the app, but of course, that depends on what’s happening in the news at any given time. The benefit for readers, he says, is that it gives them a new way of experiencing the news.
“One of the neat things about the Cassini model to me is I never really had a sense of how big Cassini was,” said Keefe. “You see pictures of it and animations that are usually with Saturn in the background, but you have no clue. Is this the size of a building, a car, a lunch box? It turns out that with AR you can put it in your living room and be like, OK, now I get a sense of the size of it.”
Right now, the interaction with the 3D models is limited to simply walking around them or resizing them, but Keefe says that the team is working on adding other features. They don’t expect users to spend a lot of time with the AR feature, since their whole model is to deliver bite-sized bits of information that can be quickly digested. That being said, Keefe said that they’ve received requests from users for an AR gallery so that they can revisit past models, and that’s something they’re looking into.
In the past, other news publications have tackled new tech, such as The New York Times‘ use of virtual reality and 360-degree experiences. Keefe says that they’re at an advantage with AR because it’s a lot less labor than creating VR experiences, which he likens to creating a documentary.
“With AR, we have the benefit of taking advantage of the world that’s actually around you,” said Keefe. “We don’t have to re-create your world, which saves us a lot of work. And then we can add to that. That’s the augmented reality part. The additive concept is just an easier lift to begin with. It allows us to get a bit more experimental and playful.”
So Microsoft’s annual Build developer conference came and went, and the computing giant unveiled a number of initiatives aimed at the software engineers and web developers of the world.
From Windows 10 to Cortana and bots, here’s a quick recap of everything announced at Build 2017.
A core theme to emerge from Build 2017 was Microsoft’s tacit acknowledgement that it needs to embrace competing platforms if it’s to thrive as a software and services company.
Though iTunes has been available on Windows for more than a decade, Microsoft revealed that Apple’s media-focused software would be landing in the Windows Store later this year, a key launch given the impending arrival of Windows 10 S which can only use apps available in the Windows Store.
Elsewhere, Microsoft also announced that its cloud storage service, OneDrive, would soon work with Apple’s iMessage, letting users share documents and photos with friends without leaving their iMessage chat. As part of the same announcement, Microsoft revealed that it was opening offline access to folders within OneDrive on Android, with support for iOS users coming later this year.
Microsoft also unveiled Xamarin Live Player in preview, which is effectively a “live coding environment” for developers to debug Android and iOS apps without SDKs or emulators.
Additionally, Microsoft’s integrated development environment (IDE), Visual Studio, was also launched out of preview for Mac. And as another swift reminder that Microsoft has been increasingly prioritizing the “big 2” mobile operating systems over its own, the company finally revealed that Visual Studio Mobile Center was finally getting Windows support — seven months after debuting with support only for Android and iOS.
In a show of support for developers and fans of Linux, Microsoft also revealed that Ubuntu, Suse Linux, and Fedora are all coming to the Windows Store, making it easier to run Linux apps on Windows 10 devices.
A little more than a year after launching its Bot Framework, Microsoft revealed that it’s now used by over 130,000 developers. With that in mind, the company used Build 2017 to announce that Bing search results now include Skype bots, in addition to myriad chat apps including Slack, Facebook Messenger, and Telegram. Bots will also soon be able to join video conversations in Skype, with developers able to use the BotBuilder RealTimeMediaCalling extension on Github, which just entered preview.
Microsoft also revealed that Cognitive Services, a conduit for third-parties to access its AI algorithms for “vision, speech, language, and knowledge,” was arriving on a bot-to-bot communication platform Interbot, a product from Gupshup. Related to this, Microsoft announced it was releasing four new customizable artificial intelligence services via Cognitive Services, allowing developers to build even more AI into their own products.
A number of announcements were also made relating to Cortana, Microsoft’s voice-enabled digital assistant. The company announced a public preview of its Cortana Skills Kit that enables developers to create voice apps for Cortana, and more than 20 such apps made their debut at Build. The software giant also revealed that it was working with both Intel and HP to bring more Cortana-powered devices to market.
Microsoft announced some interesting tidbits about its core bread and butter services, in addition to making a few surprise announcements.
The company gave a glimpse into how it wants to tie its various apps, products, and platforms together with the Microsoft Fluent Design System, which is effectively guidelines to enable Microsoft to evolve its Metro/Modern UI design language, replete with rules for developers creating software to run on Windows 10.
Microsoft also demonstrated Windows 10 running on devices with ARM processors, with full support for existing Win32 apps. And it also revealed that Windows 10 is now installed on more than 500 million devices, a feat that took around 21 months to achieve.
Looking to the future, Microsoft announced a new Windows 10 Fall Creators Update, which is coming later this year, with the company teasing a new creative app called Windows Story Remix that uses the Microsoft Graph to transform and combine your photos and videos.
As part of the Windows 10 Fall Creators Update, Microsoft also outlined plans to launch OneDrive files on-demand, a feature that lets users access their files online without having to download them and consume valuable storage space on their devices.
Elsewhere, Microsoft had a little news to share around its cloud computer service Azure. With Azure Cosmos DB, Microsoft is offering a globally distributed database with five consistency choices, rather than forcing developers to choose between strong and eventual consistency.
Other notable announcements to emerge from Build include: developers can now publish Microsoft Teams apps to the Office Store, while Microsoft unveiled its first Windows Mixed Reality motion controllers, alongside a mixed reality developer kit.
Finally, one of the more interesting early-stage products to emerge at Build was the Emma Watch, a wearable that counters Parkinson’s tremors through using vibrations.
Oculus chief scientist Michael Abrash believes that we may soon see the proliferation of augmented reality glasses, describing it as possibly being “one of the great transformational technologies of the next 50 years.” But he doesn’t think that a copy of Google Glass isn’t going to cut it, that AR glasses must be see-through, stylish, and socially acceptable, among other things.
At this year’s F8 developer conference, Facebook put forth the next phase of its push towards augmented reality, introducing its camera effects platform in closed beta which would let anyone create their own Snapchat-like masks and display contextual information inside images. Powered through its camera app, company chief executive Mark Zuckerberg described it as the “first augmented reality platform.”
While Zuckerberg and chief technology officer Mike Schroepfer touted the benefits of Facebook’s newest platform, Abrash sought to impart wisdom on attendees about AR’s potential. While bringing the technology into smartphones, the camera you’re going to have on you practically at all times, there have been efforts made to make them more convenient, or at least that was the intent. And even while none have really succeeded, Abrash suggested that in the long-term an AR-enabled wearable will become available outside of the phone.
“Twenty or thirty years from now, I predict that instead of carrying stylish smartphones everywhere, we’ll be wearing stylish glasses,” he said during his F8 keynote address. “Those glasses will offer virtual reality, AR, and everything in-between, and we will wear them all day and use them in almost all aspects of our lives. The distinction between AR and VR will vanish. The real and virtual world will mix and match throughout the day according to our needs.”
When it comes to VR, the norm is that you’re wearing a headset like for the Oculus Rift or Samsung Gear VR, but for AR and mixed reality, “few people are absolutely comfortable socializing in-person with someone whose eyes they can’t see and social acceptability is an absolute requirement for wearing it in public,” Abrash declared.
Having see-through AR glasses is part of the journey to achieving what he said was “full AR” which is augmentation that enhances the vision and hearing of the wearer seamlessly to make them smarter and more capable, while also being “light, comfortable, stylish, power efficient, and socially acceptable enough to accompany you everywhere you go.”
Even if he didn’t mention Google by name, it’s evident that Abrash was referencing the sentiment Google Glass received years ago.
Some of the use cases he alluded to around AR glasses include being able to see in low light, communicating with people in other parts of the world, being able to see better, quickly translate words, display information about the food you’re eating, mute random noises in the background, run a diagnostic test on someone, and even prompt you when you run into someone you don’t remember the name of.
Abrash’s speech seemed designed to cast a different light on AR, something that many are probably attributing to what Snapchat and Facebook is doing. The move towards bringing this technology to more devices and applications could cause people to have different ideas about what it is, so his talk may have been intended to inspire developers to think more creatively and strategically versus simply churning out things like useless bots.
Perhaps it was intended to spark the imagination of developers. The Information reports (subscription required) that Facebook’s engineering director Ficus Kirkpatrick was repeatedly asked by developers about why they should care about the camera effects platform, causing confusion about its actual usefulness. Abrash’s talk may have been aimed at putting a spotlight on the bigger picture.
This talk comes a year after the Oculus chief scientist also pontificated on stage about why people will care about virtual reality, offering up another TED-like talk about the new frontier.
Naturally while these words of wisdom are inspiring, it’s also an effort to get developer support behind what Facebook is doing. He remarked that “unless AR glasses made for broad consumers are literally impossible, which I doubt, they will become a reality in the not too distant future” suggesting that it could be at least five years before we see AR having its Macintosh moment.
But in order to have AR glasses be as ubiquitous as smartphones, full AR is needed and the devices cannot be for “occasional or special case” usage. “It’s the always-on helper, continually aware of your surroundings, your context and your history, constantly mixing the real and virtual worlds to serve your needs to keep you connected,” Abrash said.
Much more work is needed, including in the areas of optics and displays, audio, interaction, computer vision, artificial intelligence, system design, and user experience. It’s possible that as Facebook gets more involved with the technology, acquisition of startups working in these areas might be possible.
“The tipping point will come with full, always-on, go everywhere audio/visual AR glasses, and that will certainly take five years, although it could take 10 years,” Abrash claimed.
Facebook has unveiled a host of new features centered around curating a global community. At the company’s 2017 F8 developer conference, attendees were introduced to not only Facebook’s push into augmented reality, but also the next phase of its Messenger platform, bot support within Workplace, a new virtual reality experience, and more.
If you missed the keynote, never fear because we have you covered at VentureBeat. Here’s everything Facebook announced at the conference, as of day 1:
In his opening keynote, Facebook chief executive Mark Zuckerberg announced the closed beta of a new platform centered around augmented reality. Called the camera effects platform, it’ll enabled developers to build and create effects and masks around the Facebook camera, taking its competition against Snapchat to another level.
Doubling down on augmented reality, Facebook revealed that it was working on developing games around the technology later this year. Such games will help define whole new genres, Zuckerberg claimed. The tech depends on the ongoing revolution in computer vision, artificial intelligence, and a number of other technologies. Zuckerberg said that AR is a tech that can make both digital and physical reality better.
As for Facebook’s traditional platform, the company introduced a program which establishes local support groups worldwide for developers to interact and learn from one another. Additionally, the company revealed updates to Facebook Analytics, Login, Account Kit, and launched its Place Graph API, giving developers access to location data for 140 million places.
Facebook’s Slack competitor is growing up, with new file-sharing and document management tools, third-party partnerships, and is now opening up its groups and work chat feature to curated bot developers.
After giving developers the tools to build on top of the Messenger platform, Facebook turns its attention towards discovery, so users can find brands and bots more easily. It also launched chat extensions, which lets third-party apps more natively be integrated into conversations.
While Facebook’s M artificial assistant has become available to the masses through Suggested Responses in Messenger, the company revealed it has begun a pilot program to see what the experience would be like if third-party service was integrated. The first partner to trial this is Delivery.com.
Facebook has provided updates on its progress with its major gaming initiatives: Instant Games, Gameroom, and Gaming Video. That’s all part of its vision to become the place where people play, watch, and share games.
Facebook has hit a new milestone that indicates just how large its community of gamers has become: 800 million monthly active users. That number includes the people who play games on Facebook, but it also includes those who are merely logged in to Facebook while playing games on other platforms, such as smartphones.
Facebook has open sourced Caffe2, a framework for deep learning. Deep learning generally involves training artificial neural networks on lots of data, like photos, and then getting them to make inferences about new data.
More than a year after beginning to support chatbots, Facebook Messenger has more than 100,000 on its platform, up from the 33,000 in September.
With the bot, Spotify will now automatically offer playlist recommendations based on factors such as mood, activity, and genres directly in Facebook Messenger conversations.
Apple will be introducing an Apple Music chat extension to Facebook Messenger, making it possible for people to bring the music streaming app into conversations with friends and family members and share music.
The NBA’s Golden State Warriors launched a Facebook Messenger bot to act as a personal playoff assistant to fans. The bot helps you cheer on the team, trash talk to friends and coworkers, and follow along as the Warriors seek to reach the NBA Finals for the third consecutive year.
Money transfer company Western Union launched a Facebook Messenger bot today that allows people within the United States to transfer money to more than 200 countries around the world.
Yahoo has launched its Captain bot on Facebook Messenger, becoming one of the first third-party programs to integrate with Messenger’s chat extensions. Now you can track what needs to be done within your group conversations.
Giphy has launched three more products aimed at making it easier to use GIFs in more ways: in creative camera effects, in live video, and more natively in Messenger as a chat extension.
Bot maker Octane.ai will now allow bot creators to generate Facebook Messenger parametric codes to guide a user to their bot and customize the consumer’s experience.
There’s more to come from Facebook as the conference concludes on Wednesday. VentureBeat will be there and reporting on all the news. Please check back later for an update or you can view an updated list of F8 stories right here.