Your Apple Watch Is Now Listening: Series 12, Ultra 4 Get AI-Powered Audio Intelligence

Apple has introduced a new suite of AI-powered "Audio Intelligence" features with the Apple Watch Series 12 and Apple Watch Ultra 4, marking one of the biggest upgrades to the smartwatch lineup beyond health and fitness tracking.

The new features allow the watches to recognise environmental sounds, identify music, summarise conversations and even generate a transcript of anything said around the wearer in the previous 15 seconds. Apple, however, is stressing that privacy remains at the centre of the experience, with the company claiming the watches never store or expose raw audio recordings.

Four New Audio Intelligence Features

The update brings four opt-in capabilities powered by the Apple Watch's microphones:

- Sound Recognition detects sounds such as doorbells, sirens, alarms and even a baby crying before alerting the wearer.
- Music Recognition integrates with Shazam to identify songs playing nearby.
- Siri Recap can automatically summarise conversations throughout the day or during scheduled periods.
- Live Rewind lets users retrieve a text transcript of the previous 15 seconds of speech by double-pressing the Digital Crown.

Together, the features transform the Apple Watch from a wearable fitness companion into a context-aware AI assistant capable of understanding what's happening around the user.

Powered by the New S11 Chip

Apple says the new capabilities are made possible by the S11 chip, which introduces a dedicated protected memory area known as the Secure Exclave.

Instead of storing recordings, the Secure Exclave temporarily buffers audio in an isolated environment that remains inaccessible to apps, the operating system, users and even Apple itself.

For example, when identifying music, the watch creates a digital fingerprint of a song inside the Secure Exclave and sends only that fingerprint to Shazam's servers. The original audio never leaves the device and is deleted immediately after processing.

Conversation Summaries Without Keeping Recordings

The most ambitious addition is Siri Recap.

When enabled, the Apple Watch first determines whether people are speaking using an on-device AI model. If it detects a conversation, encrypted audio is securely transferred to the paired iPhone, where speech recognition and Apple Intelligence models generate a transcript.

The original audio is then deleted from both devices.

Before a summary is created, Apple says the transcript is cleaned up by removing filler words and repeated phrases. Another safety model filters potentially harmful terms and strips sensitive information such as financial details or personal identifiers.

Apple also sends limited contextual information, such as whether the user is at home, work or school, along with broad location labels like city or state, to improve the quality of summaries. The company says it does not use precise location or exact points of interest.

Live Rewind Stores Only 15 Seconds

Another standout feature is Live Rewind, which continuously maintains a rolling 15-second audio buffer.

Users must manually activate the feature by double-pressing the Digital Crown, after which the buffered audio is securely transferred to the paired iPhone and converted into text. The recording itself is immediately deleted once transcription is complete.

To prevent covert recording, Apple says the watch emits an audible chime whenever Live Rewind is activated, even if the watch is on silent mode or connected to headphones.

Privacy Still at the Centre

Apple is positioning Audio Intelligence as an example of AI that can be useful without compromising user privacy.

The company says raw audio is never accessible to applications or cloud services. Where cloud processing is required, only encrypted and pre-processed information is sent through Apple's Private Cloud Compute infrastructure instead of the original recordings.

The announcement comes as AI assistants become increasingly capable of understanding not just what users type, but also what they see and hear. While Apple argues its privacy architecture sets it apart from rivals, the new features are also likely to reignite broader debates over AI-powered devices that continuously monitor their surroundings, even if recordings are never permanently stored.

Also Read: iPhone Duo: Apple Folds For The First Time Ever In An Engineering Marvel | WATCH