1. Spectacles Support
  2. Spectacles '24
  3. Spectacles Overview

Update Release Notes

Our First Snap OS Update with new Lenses, Platform Features & APIs

  • 🧠  2 New ML Lenses and 7 New Platform Features
  • 🎹  Piano Tutor: Learn to play piano with interactive lessons and real-time feedback.
  • ⚽  Ball Game: Interact with digital content using a physical ball.
  • 📍  Spatial Anchors API (Beta): Persist content between Lens sessions.
  • 🌐  Fetch and Base64 APIs: Simplify web endpoint calls and camera frame exchange.
  • 📸  Image Spatialization API: Convert 2D images to 3D using generative AI.
  • 🗺️  Location API (Beta): Access device GPS coordinates for location-based experiences.
  • 💻  Web View Component: Embed web pages directly into your Lenses.
  • 🔓  Lens Unlock: Share published Lens URLs for easy access.
  • 🚀  New interactive sample scene: showcasing Spectacles Interaction Kit (SIK) capabilities

In this November 2024 update, we’ll be rolling out some of our many planned updates! We're introducing exciting new Lenses that showcase the power of the Spectacles camera and SnapML. We've also added 7 new platform features and tools to empower you to create even more innovative and engaging Lens experiences. 

 

Introducing Piano Tutor and Ball Games

With the new Piano Tutor Lens, you have a personal piano instructor right at your fingertips. Learn to play through interactive lessons or practice your favorite songs. Piano Tutor uses a custom model powered by SnapML to adapt to different pianos all you need to do is sit down and play, eliminating the need for cumbersome manual setup and calibration, and uses another custom ML model for to detect if you played the right note and provide real-time feedback on your accuracy. 

The Ball Game Lens transforms a physical ball into a controller in a virtual ball game, making it fun to practice kicking a ball through challenging virtual courses, no setup required. Using a custom tracking model, the Lens follows the ball's movements, allowing you to interact with both physical and digital elements together in a truly immersive way.

 

New Platform Features and APIs

We're excited to introduce the beta version of our Spatial Anchors API, enabling developers to anchor digital objects to specific locations in your surrounding and keeping them there so you can return to them later. Like a digital post it notes reminders of tasks and chores around your house. Now you can have the content of your Lens persist between sessions for a more immersive experience and ensure users can use your Lens for multiple sessions. (see examples and documentation)

 

 

We're also introducing a groundbreaking Image Spatialization API that leverages generative AI to convert common 2D image formats into 3D. Developers can now incorporate this API into their Lenses to create stunning 3D effects. In this release, we've updated the Layout Lens, allowing you to import images from your phone using the Spectacles app and spatialize them for a captivating 3D experience. (see examples and documentation)

 

 

In this release, we are unlocking your ability to use rich content from the internet in your Lens, or experiment with using camera input for experimenting with multi-modal AI. The new Fetch API and Base64 APIs simplify the process of calling web endpoints and exchanging camera frames with those endpoints in extended permissions developer mode.  (see examples and documentation, and read more about experimental extended permissions)

 

 

 

Spectacles are the leading see through AR glasses designed for the outdoors, and with this release, we're introducing the beta version of the basic location API. This API grants access to the device's GPS coordinates, opening up a world of possibilities for location-based Lens experiences. In this release, we focused on increasing the reliability and speed of acquiring GPS coordinates, this will be a beta release followed by improvements (see examples and documentation)

 

We understand the value of web content, which is why we've added the new Web View component. This drag-and-drop component allows you to seamlessly embed web pages directly into your Lenses. (see examples and documentation)

 

webview-in-lens-studio.gif

 

Sharing your Lens creations is now easier than ever with the new Lens Unlock feature. Simply share the URL of a published Lens, and others can enter it into the Spectacles app to unlock and experience it. Unlocked Lenses remain in the 'All Lenses' section for 24 hours and if you want to come back to it frequently, you can favorite it for quick access in Lens Explorer.

 

 

Improvements and Bug Fixes

In addition to these great features, we also made some improvements and bug fixes, including:

  • 🤖 Support for annotations in the Video calling Lens when calling an Android user (rolling out on Snapchat Android in the next several weeks)
  • 🛠️ New encoding support in Video calling that makes it more power-efficient so you can use Video calling for longer
  • 🛜 Improvements to MyAI in poor internet conditions, reduced voice response latency, and adjustment to more easily center on the region of interest
  • 🪫 Added device time-out settings in mobile which enables you to adjust the time before the device shuts-down when not in use. 

Please update to the latest version of Snap OS and the Spectacles App. Follow these instructions to complete your update (link)

 

Once update, please confirm that you got the latest versions

 

OS Version: v5.58.621 

Spectacles App iOS: v0.58.1

Spectacles App Android: v0.58.1.0

 

❗A note on Lens Studio

Please only use version v5.3 of Lens Studio. Please DO NOT update to a later version of Lens Studio unless it explicitly mentions current support for Spectacles. Lens Studio is updated more frequently than Spectacles and getting on the latest early can cause issues with pushing to Spectacles. We’ll explicitly mention the supported version of Lens Studio with every release note

 

If you have any feedback or questions, please head over to reddit.com/r/spectacles and reply to the thread.

 

 

Previous Updates

🎉 Spectacles Interim Update 10.16.2024

In this interim update, we have added bug fixes to improve the reliability of transferring captures from Spectacles to the phone among other bug fixes.

 

Please update to the latest version of Snap OS and the Spectacles App. Follow these instructions to complete your update and confirm you have the latest versions below:

 

OS Version: v5.57.0449

Spectacles App iOS: v0.57.9.0

Spectacles App Android: v0.57.54



❗A note on Lens Studio

Please only use version v5.1.1 of Lens Studio. Please DO NOT update to the latest version of Lens Studio unless it explicitly mentions current support for Spectacles. Lens Studio is updated more frequently than Spectacles and getting on the latest early can cause issues with pushing to Spectacles. We’ll explicitly mention the supported version of Lens Studio with every release note

 

If you have any feedback or questions, please respond to the thread on Reddit.

 

Related articles