Google has announced a major update to its Gemini AI assistant for the macOS platform. New features radically change the approach to working with text and on-screen data: from automatically editing voice notes to deep analysis of open applications. However, expanding AI capabilities also brings new challenges for cybersecurity.

Smart Dictation: From Sound to Ready Text

The first and most noticeable innovation is the "smart dictation" feature. It is already enabled by default for all Gemini users on Mac. Unlike standard voice inputs that simply transcribe speech into text, Google's new development works as an intelligent editor.

The system does not just record words; it automatically improves the text right at the cursor's location. The algorithm instantly filters out speech "noise": removing pauses, filler words, and extra sounds, leaving only clean, edited content.

The main advantage of the solution is its versatility. The function works in any application on Mac. To start dictation, simply press the Fn key and speak. This allows you to create drafts of emails, messages, or documents without switching to a chatbot or spending time manually correcting errors.

Screen Analysis: AI with Administrator Rights

The second major update is called "screen-aware reasoning." Unlike dictation, this option is not enabled automatically — users must manually allow it in the settings.

The function provides Gemini with access to the content of windows open on the monitor. This allows the assistant to perform complex tasks requiring context:

  • Note Summarization: If a notes application is open on the screen, you can ask the AI: "Make a brief summary for the daily meeting." Gemini will read the text, process it, and insert the ready result.
  • Cascading Commands: You can assign the drafting of an email, and the AI will independently pull necessary data from open files.
  • On-the-fly Corrections: The system understands the context of changes. If during dictation you say: "No, move the meeting from 18:30 to 19:00," the AI will make corrections to the final text without requiring a complete rewrite.

Privacy Risks and Prompt Injections

Providing algorithms with access to everything happening on the desktop inevitably raises security questions. The main threat for such tools with extended privileges is so-called prompt injections.

Attackers can hide malicious commands on websites or within documents. If a user asks the AI to process such a page, the assistant may perceive the hidden command as an instruction from the computer owner. As a result, the AI could perform a dangerous action: download a malicious file or attempt to access saved data.

At the moment, the updated Gemini for macOS is being rolled out globally only in English. Support for other languages is planned to be added in the future.