# Turning My Obsidian Vault Into a Local AI Engineer — Filip Makraduli, Superlinked

## Executive summary

This talk details building a private, cost-efficient, and highly adaptable AI workflow using an Obsidian vault as a knowledge base. The solution, built around Superlinked's open-source model inference engine and an MCP (Model Communication Protocol) integration, allows users to offload sensitive document processing (like OCR and redaction) to a self-controlled Virtual Private Cloud (VPC). This approach addresses critical concerns regarding data privacy, high token costs associated with large models, and the need for task-specific model adaptation (LoRAs) in sensitive enterprise environments.

## Key takeaways

- Privacy and Cost Control: The system offloads sensitive document processing (e.g., OCR, redaction) to a self-hosted VPC, ensuring that raw, sensitive data never leaves the controlled environment, thus mitigating privacy risks and reducing token costs compared to sending all data to external APIs.
- Modular and Adaptable Inference: Superlinked provides an open-source inference engine that supports multiple models (including task-specific LoRAs) running on dedicated GPU worker pools. This allows for model hot-swapping and testing different models for optimal performance and cost.
- Infrastructure Control (MCP): The MCP setup establishes a clear separation of responsibilities: the client/host interacts with the MCP server, which coordinates with the dedicated GPU worker pools. This architecture provides granular control over resource allocation (e.g., picking specific worker pools or GPU types) and ensures the entire process runs within the user's private cloud.

## Technical details

- Architecture Components: The system uses a three-tiered architecture: the MCP client (running in Cloud Code/Co-work), the MCP server, and the dedicated Superlinked cluster (containing GPUs and worker pools). Communication is managed via JSON RPC.
- Document Processing Workflow: The workflow involves pre-processing documents (e.g., using open-source OCR models) to generate smaller, structured artifacts (e.g., Markdown). These artifacts are then passed to the main agent/harness for tasks like redaction, significantly reducing context size and token expenditure.
- Scalability and Deployment: The infrastructure is designed for enterprise deployment, supporting auto-scaling, multiple worker pools, and the ability to deploy the entire stack using tools like Terraform (Terraform apply).
- Model Optimization: The inference layer supports advanced techniques like smarter batching per worker and model hot-swapping across different GPUs, optimizing performance and cost, especially when running multiple fine-tuned LoRAs.

## Practical implications

- For build engineers, the architecture demonstrates a robust pattern for building secure, data-sensitive AI applications by enforcing data processing within a private, controlled cloud environment (VPC).
- The use of Terraform and Kubernetes/Helm charts suggests a highly containerized and reproducible deployment model, ideal for CI/CD pipelines.
- The modularity of the MCP allows for integrating diverse tools (OCR, NER, summarization) as discrete services, simplifying the development of complex agentic workflows.
- The focus on open-source models and self-hosting provides a blueprint for achieving AI capability while maintaining strict compliance and data sovereignty requirements.

## Topics

Generative AI, Model Inference, Data Privacy, Cloud Architecture, Containerization, Natural Language Processing (NLP), Superlinked Sci Repo, Superlinked Website

Source: https://www.youtube.com/watch?v=Ndp5VaqFrl0