AI Engineer NYC: OpenAI, Blackstone, Ramp, Coinbase, Apollo, Citadel, Basis, Runway, Compute Trading
Summary
The industry is rapidly moving toward autonomous, reliable, and secure AI agents capable of handling complex, long-running tasks, particularly in regulated sectors like finance. Key architectural shifts include designing 'Software Factories' that prioritize quality over volume, implementing policy-driven workflows using Directed Acyclic Graphs (DAGs), and establishing robust, multi-layered security controls (e.g., sandboxing, mutual authentication, and deterministic governance). The focus is shifting from simple co-pilots to 'autopilots' that can operate persistently and autonomously, requiring new methods for managing context, verifying outputs, and ensuring cost-effective compute operations.
Key takeaways
-
Agentic Software Factories require systemic quality controls.
18:40
To build reliable agentic systems, organizations must move beyond simple 'human-in-the-loop' checks, which do not scale. Instead, they must engineer confidence through four components: clear intent, 'guide rails' (not guardrails), independent verification, and verifiable evidence. (Tar Shaat, Sonar)
-
Workflow automation must be modeled using Directed Acyclic Graphs (DAGs).
23:50
Policy-driven workflows should be represented as DAGs or Directed Cyclic Graphs (DCGs) to manage dependencies and ensure work flows through specific, gated directions. This allows for parallelizing independent reviews and maintaining necessary checks. (Evan Cormos, Coinbase)
-
AI security requires a layered, deterministic defense approach.
29:20
Security cannot rely solely on better sandboxes. A layered defense must include: workload identity (short-lived, scoped credentials), prompt/input defense (isolation, provenance tagging), tool governance (least privilege, deterministic human approval), and robust observability (audit trails, circuit breakers). (Andre Ripka, Bloomberg)
-
The value of AI is determined by 'Operating Capability,' not just hardware.
39:10
In compute commodity markets, the cost of useful intelligence is determined by the operator's ability to maximize output (e.g., tokens per second) and reliability, not just the raw GPU power. Better software and operational processes significantly lower the cost per acceptable token. (Oliver Kenan, Semi Analysis)
Technical details
-
Agentic Capabilities & Optimization
850s
Advanced techniques like post-training and KV cache tricks (e.g., 'cartridges') are used to improve agent performance and reduce latency/cost. For instance, RAMP used post-training to create a financial question answering model that outperformed frontier models while serving at high throughput. (Alexander Shiftchenko, RAMP)
-
Software Factory Principles
1120s
Modern software development principles (like those from the Toyota Production System) emphasize building systems engineered for quality and repeatability, rather than just volume. This translates to building 'Software Factories' that deliver confidence. (Tar Shaat, Sonar)
-
Process Modeling and State Management
1430s
Workflows should be modeled using DAGs/DCGs. The process requires defining a policy boundary and managing state transitions, including exception flows, to ensure the work is 'good enough' to pass to the next step. (Evan Cormos, Coinbase)
-
AI Security Architecture
1760s
Key security requirements include: 1) Workload identity (short-lived, scoped credentials); 2) Input/Prompt defense (isolation, provenance tagging); 3) Tool governance (least privilege, deterministic human approval); 4) Sandboxing (default deny for all outbound traffic, including DNS). (Andre Ripka, Bloomberg)
-
Context and Data Management
2100s
For institutional finance, context is massive (e.g., 15,000 files, 20MB data rooms). Solutions must address this by using agent swarms to read files (rather than indexing everything in a vector DB) and developing proprietary engines (like rebuilding Office) to handle complex, deep data structures accurately and efficiently. (EJI, Quant AI)
Mentioned resources
- RAMP
- Sonar
- Coinbase
- OpenAI / ChatBT
- Bloomberg
- Quant AI
- Semi Analysis
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.