Update Release Notes

❄️ A Snap OS Update - Designed to Bring you Closer to Others This Holiday Season ❄️

• 🎲 3 New shared games to play with your friends and family
  • 👨‍🍳 Clash of Cuisines - a board game where you compete to take over a city with your restaurants
  • 🎭 Guess It - A holiday favorite re-imagined for AR glasses by WabiSabi
  • 🐹 Spatial Whack-a-mole - get moving with others while whacking moles coming out of portals in space by Benny Paruzynski
• 📖 Star Tale (coming Dec 24)- a magical 3D holiday pop-up book experience
• 🎄 Project Holiday - immerse yourself in decorating a virtual tree in your space
• ❄️ Holiday themed updates to Beat Boxer & Make Believe to get you in the spirit of the season

• 📦 The new Spectacles Sync Kit - a revamp of our framework for developing shared AR experiences with a brand new Typescript package
• 💻 A new realtime Connected Lens monitor in Lens Studio to optimize your debugging experience of multi-player AR Lenses
• 🪪 An update to Spectacles captures to include a new Lens info end card showcasing the Lens icon and developer name, as well as a new capture settings to support additive or blending modes
• 🤝 A new guided mode to boot your device into a single Lens experience for demos and events making it easy to demo the same Lens over and over without having to launch it from Lens Explorer
• 👌 A new hand input API to get the hand velocity for interactive experiences that include use a throwing or poking interactions
• 🖐️ Reduced hand tracking jitter and improved pinch robustness.
• 🌐 Support for captive portal internet so you can connect to the internet and enjoy your Spectacles at hotels, events, and other venues
• 👩‍💻 A new Git repository with sample projects to help you learn how to build Lenses
• 🚄 Improved motion to photon latency

Introducing new interactive shared experiences to enjoy with family and friends - Clash of Cuisines, Charades & Whack-a-mole

This holiday season, we havemade 3 new Lenses that you can enjoy with your family as a shared AR experience. Play a risk like a game celebrating the cuisines of the world, or a holiday favorite of charades with your friends and family. If you want to enjoy a fun game with movement, try the new spatial game of whack-a-mole, these Lenses show how Spectacles are designed to bring you together with those you care about the most.

We are also releasing 2 new Lens to spread the magic of the holidays in AR including Star Tale - a magical story telling experience featuring a holiday pop-up book, and Project Holiday - a festive holiday experience where you can immerse yourself in decorating a virtual tree in AR.

Bring People Together with the Spectacles Platform

Inspired by all these different ways to bring people together and want to try yourself? We are also releasing our new Spectacles Sync Kit - a revamp of our Connected Lenses framework to make it easier to build shared experience. Those include a re-write in Typescript with improved stability. We also added 4 new sample projects and documentation to make it easier for you to follow along and build your own multi-player shared AR experiences. These are accessible from our git repository, with more projects to be added in the future.

Connected Lenses enable multiple users with Spectacles to view and interact with the same content in real time. Spectacles and a friend are the only requirements for play; no additional items like a printed marker are needed. Each device operates within its own unique coordinate space, determining how Spectacles perceive their position relative to the world. In a co-located connected Lens environment, you must align the coordinate spaces of the devices to synchronize digital content. Spectacles simplify this process to take seconds and feel seamless and natural. Begin by opening a shared / connected Lens, look around your environment, instruct your friend to join the same session by accessing the Active Nearby category in Lens Explorer, then have them also look around your environment. Within seconds, synchronization occurs, and you are ready to play!

Spectacles enable ease of use of Connected Lenses and the development process is simplified as well. This release introduces new platform capabilities to expedite development time. The Spectacles Connected Lenses package in the Asset Library is now the Spectacles Sync Kit, featuring:

A complete re-write in TypeScript for modularity, allowing easier navigation of its code base with type completion.

Additional examples demonstrating basic functionalities such as:

• Synchronizing moving objects across participants
• Synchronizing numeric values, such as scores
• Synchronizing material color values to maintain visual consistency

Whether using the Spectacles Sync Kit or example projects, Lens Studio facilitates fast iteration and development for Connected Lenses. Lens Studio simulates a Connected Lenses session with multiple players by creating several preview panels, each acting as an independent player. 

Additionally, we are introducing a new feature, the Connected Lens Monitor, which enhances debugging capabilities. With this, you can:

• Identify the session host, who originally created the Connected Lenses
• Monitor the frequency of messages sent in the session
• Determine the order of message transmission
• Track changes in object values and their sources of change

Review our developer documentation for more details.

With the Spectacles Sync Kit, resources for Sample Projects, and the Connected Lens Monitor now available, we look forward to seeing how you utilize these tools to create your next Connected Lens experience.

Giving Credit

Many of you are having fun sharing captures of your early concepts online. To make them more personal, we are adding some improvements including:

• An end card that showcases the icon, name, and developer name for your Lens. As your Lenses get shared around the web, they will easily be recognized as your work.
• We also added some new visual transparency treatment options to make your AR capture pop more and be more true to what you experience on the glasses.

Captive Portal Support for Internet on the Go

We added support for connecting to the internet using Captive portals common at hotels, airports, and public venues. You can now use the Browser to authenticate when connecting to those types of networks, perfect on your travels this holiday season.

More Control over Hand Input

In this release, we are introducing a new API to help you build more refined hand based interactions in your Lens. The hand velocity API gives you access to the velocity of the hand - useful when building interactive experiences that make use of fast hand movement. Like how hard you punch in the boxing Lens.

Guided Mode for Streamlining Demos at Events and Activations

For conferences and events where you want to focus your audience on a single experience. We are providing Guided Mode, which allows you to lock the system to a single Lens that the system will show every time you turn on the device. This allows you to have a more controlled experience focused on your Lens and not worry about your users going to other Lenses.

Make your experiences Snappy with Web Sockets

In this release, we are introducing support for web sockets. Using web sockets you can connect to backend servers and establish a real-time connection to exchange data. Unlocking more responsive real-time experiences that connect to backend servers including real-time exchanges with LLMs in the cloud for a more responsive and low latency experience. To learn about how to use web sockets, please see samples and documentation here.

 

Versions

Please update to the latest version of Snap OS and the Spectacles App. Follow these instructions to complete your update.

Please confirm that you got the latest versions

OS Version: v5.59.218

Spectacles App iOS: v0.59.1.1

Spectacles App Android: v0.59.1.1

❗Important Note Regarding Lens Studio Compatibility

To ensure proper functionality with this SnapOS update, please use Lens Studio version v5.4 exclusively. Avoid updating to newer Lens Studio versions unless they explicitly state compatibility with Spectacles, Lens Studio is updated more frequently than Spectacles and getting on the latest early can cause issues with pushing Lenses to Spectacles. We will clearly indicate the supported Lens Studio version in each release note.

Checking Compatibility

You can now verify compatibility between Spectacles and Lens Studio. To determine the minimum supported SnapOS version for a specific Lens Studio version, navigate to the About menu in Lens Studio (Lens Studio -> About Lens Studio).

Pushing Lenses to Outdated Spectacles

When attempting to push a Lens to Spectacles running an outdated SnapOS version, you will be prompted to update your Spectacles to improve your development experience.

Feedback

Please share any feedback or questions on Reddit.

 

Previous Updates

Our First Snap OS Update with new Lenses, Platform Features & APIs

• 🧠  2 New ML Lenses and 7 New Platform Features
• 🎹  Piano Tutor: Learn to play piano with interactive lessons and real-time feedback.
• ⚽  Ball Game: Interact with digital content using a physical ball.
• 📍  Spatial Anchors API (Beta): Persist content between Lens sessions.
• 🌐  Fetch and Base64 APIs: Simplify web endpoint calls and camera frame exchange.
• 📸  Image Spatialization API: Convert 2D images to 3D using generative AI.
• 🗺️  Location API (Beta): Access device GPS coordinates for location-based experiences.
• 💻  Web View Component: Embed web pages directly into your Lenses.
• 🔓  Lens Unlock: Share published Lens URLs for easy access.
• 🚀  New interactive sample scene: showcasing Spectacles Interaction Kit (SIK) capabilities

In this November 2024 update, we’ll be rolling out some of our many planned updates! We're introducing exciting new Lenses that showcase the power of the Spectacles camera and SnapML. We've also added 7 new platform features and tools to empower you to create even more innovative and engaging Lens experiences. 

 

Introducing Piano Tutor and Ball Games

With the new Piano Tutor Lens, you have a personal piano instructor right at your fingertips. Learn to play through interactive lessons or practice your favorite songs. Piano Tutor uses a custom model powered by SnapML to adapt to different pianos all you need to do is sit down and play, eliminating the need for cumbersome manual setup and calibration, and uses another custom ML model for to detect if you played the right note and provide real-time feedback on your accuracy. 

The Ball Game Lens transforms a physical ball into a controller in a virtual ball game, making it fun to practice kicking a ball through challenging virtual courses, no setup required. Using a custom tracking model, the Lens follows the ball's movements, allowing you to interact with both physical and digital elements together in a truly immersive way.

 

New Platform Features and APIs

We're excited to introduce the beta version of our Spatial Anchors API, enabling developers to anchor digital objects to specific locations in your surrounding and keeping them there so you can return to them later. Like a digital post it notes reminders of tasks and chores around your house. Now you can have the content of your Lens persist between sessions for a more immersive experience and ensure users can use your Lens for multiple sessions. (see examples and documentation)

 

 

We're also introducing a groundbreaking Image Spatialization API that leverages generative AI to convert common 2D image formats into 3D. Developers can now incorporate this API into their Lenses to create stunning 3D effects. In this release, we've updated the Layout Lens, allowing you to import images from your phone using the Spectacles app and spatialize them for a captivating 3D experience. (see examples and documentation)

 

 

In this release, we are unlocking your ability to use rich content from the internet in your Lens, or experiment with using camera input for experimenting with multi-modal AI. The new Fetch API and Base64 APIs simplify the process of calling web endpoints and exchanging camera frames with those endpoints in extended permissions developer mode.  (see examples and documentation, and read more about experimental extended permissions)

 

 

 

Spectacles are the leading see through AR glasses designed for the outdoors, and with this release, we're introducing the beta version of the basic location API. This API grants access to the device's GPS coordinates, opening up a world of possibilities for location-based Lens experiences. In this release, we focused on increasing the reliability and speed of acquiring GPS coordinates, this will be a beta release followed by improvements (see examples and documentation)

 

We understand the value of web content, which is why we've added the new Web View component. This drag-and-drop component allows you to seamlessly embed web pages directly into your Lenses. (see examples and documentation)

 

 

Sharing your Lens creations is now easier than ever with the new Lens Unlock feature. Simply share the URL of a published Lens, and others can enter it into the Spectacles app to unlock and experience it. Unlocked Lenses remain in the 'All Lenses' section for 24 hours and if you want to come back to it frequently, you can favorite it for quick access in Lens Explorer.

 

 

Improvements and Bug Fixes

In addition to these great features, we also made some improvements and bug fixes, including:

• 🤖 Support for annotations in the Video calling Lens when calling an Android user (rolling out on Snapchat Android in the next several weeks)
• 🛠️ New encoding support in Video calling that makes it more power-efficient so you can use Video calling for longer
• 🛜 Improvements to MyAI in poor internet conditions, reduced voice response latency, and adjustment to more easily center on the region of interest
• 🪫 Added device time-out settings in mobile which enables you to adjust the time before the device shuts-down when not in use. 

Please update to the latest version of Snap OS and the Spectacles App. Follow these instructions to complete your update (link)

 

Once update, please confirm that you got the latest versions

 

OS Version: v5.58.621 

Spectacles App iOS: v0.58.1

Spectacles App Android: v0.58.1.0

 

❗A note on Lens Studio

Please only use version v5.3 of Lens Studio. Please DO NOT update to a later version of Lens Studio unless it explicitly mentions current support for Spectacles. Lens Studio is updated more frequently than Spectacles and getting on the latest early can cause issues with pushing to Spectacles. We’ll explicitly mention the supported version of Lens Studio with every release note

 

If you have any feedback or questions, please head over to reddit.com/r/spectacles and reply to the thread.

 

 

 

🎉 Spectacles Interim Update 10.16.2024

In this interim update, we have added bug fixes to improve the reliability of transferring captures from Spectacles to the phone among other bug fixes.

 

Please update to the latest version of Snap OS and the Spectacles App. Follow these instructions to complete your update and confirm you have the latest versions below:

 

OS Version: v5.57.0449

Spectacles App iOS: v0.57.9.0

Spectacles App Android: v0.57.54

❗A note on Lens Studio

Please only use version v5.1.1 of Lens Studio. Please DO NOT update to the latest version of Lens Studio unless it explicitly mentions current support for Spectacles. Lens Studio is updated more frequently than Spectacles and getting on the latest early can cause issues with pushing to Spectacles. We’ll explicitly mention the supported version of Lens Studio with every release note

 

If you have any feedback or questions, please respond to the thread on Reddit.