Tool use is what turns a voice AI from a clever talker into something that gets things done. When the LLM decides it needs to perform an action like sending an email, creating a calendar event, or searching the web, it emits a structured function call. The voice OS routes that call to the appropriate backend, executes it, returns the result, and gives the LLM another turn to summarize the outcome to the user. Tool use is the bridge between language and action, and it is what makes voice AI a productivity tool rather than a conversation toy.
WHAT TO LOOK FOR
Function calling protocol
Modern LLMs support a structured function call format where the model emits JSON describing the call rather than free-form text. This is more reliable than parsing natural language because the function name and arguments are validated against a schema before execution.
Tool router
A backend component that receives function calls from the LLM, validates them, dispatches to the right backend, and returns the result. The router enforces permissions, rate limits, and input validation. Without it, the LLM could be talked into executing dangerous operations.
Confirmation for write actions
Tools that modify external state, like sending email or creating calendar events, surface a confirmation step. The LLM emits the call, the router asks the user for confirmation, and only then executes. This is the safety baseline for action-taking voice AI.
TLDR:Lucy OS1 ships a curated set of tools rather than exposing every possible API. The default tool set includes calendar create, calendar update, email draft, email send with confirmation, web search, reminder create, and memory write. Each tool has tight input validation and clear failure modes. The LLM is given just the tool definitions relevant to the current conversation, which keeps the context window small and the tool selection reliable. Adding new tools requires explicit design work, not just an API key, which is why Lucy's tool use is reliable enough to trust with real actions.
Modern LLMs support a structured function call format where the model emits JSON describing the call rather than free-form text. This is more reliable than parsing natural language because the function name and arguments are validated against a schema before execution.
A backend component that receives function calls from the LLM, validates them, dispatches to the right backend, and returns the result. The router enforces permissions, rate limits, and input validation. Without it, the LLM could be talked into executing dangerous operations.
Tools that modify external state, like sending email or creating calendar events, surface a confirmation step. The LLM emits the call, the router asks the user for confirmation, and only then executes. This is the safety baseline for action-taking voice AI.
After a tool returns, the LLM gets another turn to summarize the result conversationally. The user hears 'Done, the meeting is on your calendar for Thursday at 3 pm' rather than raw JSON. The summarization turn is where tool use becomes a natural conversation.
Not every tool is relevant to every conversation. The voice OS dynamically selects which tool definitions to expose based on the current topic, which keeps the context window small and the LLM's tool choice accurate.
Tools fail. The router translates failures into clear messages the LLM can relay to the user, distinguishing transient errors that can be retried from permanent errors that need user action. Silent failures are the worst possible outcome.
QUICK COMPARISON
| Capability | Lucy OS1 | Most AI tools |
|---|---|---|
| Memory across sessions | ✓ Permanent, never resets | ✗ Resets after every session |
| Voice quality | ✓ Lucy OS1 Natural Voice (best-in-class) | ✗ Basic STT, struggles with noise |
| Calendar awareness | ✓ Reads Google Calendar in real time | ✗ No calendar access |
| Available 24/7 | Always on, any device | Available but stateless each time |
| Gets personal over time | ✓ Builds your context continuously | ✗ Starts from zero every session |
Voice-first AI with memory and calendar integration. Free to try.
Start TalkingFree tier available. No credit card required.
GET STARTED
Create your free account
No credit card required. Sign in with your Google account and you're inside in under a minute.
Connect your Google Calendar
Lucy reads your upcoming events before every conversation, so it already knows your day before you say a word.
Start talking about tool use in voice os
Speak naturally. Lucy listens, responds by voice, and begins building context from your very first exchange. The more you use it, the better it gets.
Welcome