# Building an AI-Powered Personal Companion Mobile App - Gerald Versluis - NDC Copenhagen 2026

## Executive summary

The talk demonstrates how to integrate advanced AI functionalities into cross-platform mobile applications using .NET MAUI. The session emphasizes building robust, context-aware companion apps by leveraging abstraction layers and standardized APIs (like the Essentials API). Key architectural patterns include supporting multimodal input (voice, image), utilizing cloud services like Azure AI Foundry for LLMs, and implementing local model support via Onyx for sensitive data scenarios.

## Key takeaways

- AI Integration is Cross-Platform: The implementation of AI features can be abstracted across platforms (iOS, Android, Windows) using .NET MAUI's abstraction layer, ensuring a consistent user experience while allowing platform-specific native behavior.
- Context Awareness and Memory: AI applications can be significantly enhanced by incorporating context from various sources—including location (geolocation), calendar data, biometrics, and conversation history (memory)—to provide highly personalized user experiences.
- Multimodal Input Handling: The system supports multiple input types beyond text, including voice recording and image analysis. This requires transforming raw data (audio/image) into a format consumable by Large Language Models (LLMs), such as using speech-to-text transcription.
- Model Agnosticism via Interfaces: By relying on interfaces (e.g., `I ChatClient`), developers can easily swap out concrete AI implementations—such as those from Microsoft Foundry, OpenAI, or local models like Apple Intelligence/Onyx—without changing core application logic.

## Technical details

- Cross-Platform Framework: .NET MAUI is used for building cross-platform apps (iOS, Android, Windows) based on C# and .NET. It provides abstraction layers for UI controls and device APIs.
- AI Service Abstraction: The `Microsoft Extensions AI` library provides common abstractions (like the chat client) that allow developers to manage different LLM providers (e.g., OpenAI, Azure AI Foundry, local models) through a unified interface.
- Core APIs: The 'Essentials' API provides common non-UI functionalities across platforms (e.g., `Geolocation`, `MediaPicker`, `Battery`), which are crucial for gathering context data.
- Implementation Flow: The sample app demonstrates a flow: Capture input (text, voice, image) -> Gather Context (location/calendar) -> Initialize `I ChatClient` with credentials and system prompt -> Send request to LLM endpoint for analysis.

## Practical implications

- Design mobile applications with modularity, ensuring that AI logic and external service calls are handled through interfaces to facilitate easy swapping of LLM providers (e.g., moving from cloud-based models to on-device local models).
- Build robust context pipelines by integrating multiple data sources—location services, calendar APIs, and device sensors—to enrich the prompts sent to the AI.
- When handling sensitive user data (like medical or personal records), prioritize using local model runtimes (e.g., Onyx) to ensure processing never leaves the device.

## Topics

AI, .NET MAUI, Cross-Platform Development, LLMs, Context Awareness, Build Architecture, Multimodal AI, Azure AI Foundry, .NET MAUI Essentials API, Microsoft Extensions AI, Onyx Runtime

Source: https://www.youtube.com/watch?v=bxuU-jtHJbY