Turning My Obsidian Vault Into a Local AI Engineer — Filip Makraduli, Superlinked
Summary
This talk details building a private, cost-efficient, and highly adaptable AI workflow using an Obsidian vault as a knowledge base. The solution, built around Superlinked's open-source model inference engine and an MCP (Model Communication Protocol) integration, allows users to offload sensitive document processing (like OCR and redaction) to a self-controlled Virtual Private Cloud (VPC). This approach addresses critical concerns regarding data privacy, high token costs associated with large models, and the need for task-specific model adaptation (LoRAs) in sensitive enterprise environments.
Key takeaways
-
Privacy and Cost Control
The system offloads sensitive document processing (e.g., OCR, redaction) to a self-hosted VPC, ensuring that raw, sensitive data never leaves the controlled environment, thus mitigating privacy risks and reducing token costs compared to sending all data to external APIs.
-
Modular and Adaptable Inference
17:20
Superlinked provides an open-source inference engine that supports multiple models (including task-specific LoRAs) running on dedicated GPU worker pools. This allows for model hot-swapping and testing different models for optimal performance and cost.
-
Infrastructure Control (MCP)
15:11
The MCP setup establishes a clear separation of responsibilities: the client/host interacts with the MCP server, which coordinates with the dedicated GPU worker pools. This architecture provides granular control over resource allocation (e.g., picking specific worker pools or GPU types) and ensures the entire process runs within the user's private cloud.
Technical details
-
Architecture Components
430s
The system uses a three-tiered architecture: the MCP client (running in Cloud Code/Co-work), the MCP server, and the dedicated Superlinked cluster (containing GPUs and worker pools). Communication is managed via JSON RPC.
-
Document Processing Workflow
229s
The workflow involves pre-processing documents (e.g., using open-source OCR models) to generate smaller, structured artifacts (e.g., Markdown). These artifacts are then passed to the main agent/harness for tasks like redaction, significantly reducing context size and token expenditure.
-
Scalability and Deployment
1040s
The infrastructure is designed for enterprise deployment, supporting auto-scaling, multiple worker pools, and the ability to deploy the entire stack using tools like Terraform (Terraform apply).
-
Model Optimization
1040s
The inference layer supports advanced techniques like smarter batching per worker and model hot-swapping across different GPUs, optimizing performance and cost, especially when running multiple fine-tuned LoRAs.
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.