# AI Engineer NYC: OpenAI, Blackstone, Ramp, Coinbase, Apollo, Citadel, Basis, Runway, Compute Trading

## Executive summary

The industry is rapidly moving toward autonomous, reliable, and secure AI agents capable of handling complex, long-running tasks, particularly in regulated sectors like finance. Key architectural shifts include designing 'Software Factories' that prioritize quality over volume, implementing policy-driven workflows using Directed Acyclic Graphs (DAGs), and establishing robust, multi-layered security controls (e.g., sandboxing, mutual authentication, and deterministic governance). The focus is shifting from simple co-pilots to 'autopilots' that can operate persistently and autonomously, requiring new methods for managing context, verifying outputs, and ensuring cost-effective compute operations.

## Key takeaways

- Agentic Software Factories require systemic quality controls.: To build reliable agentic systems, organizations must move beyond simple 'human-in-the-loop' checks, which do not scale. Instead, they must engineer confidence through four components: clear intent, 'guide rails' (not guardrails), independent verification, and verifiable evidence. (Tar Shaat, Sonar)
- Workflow automation must be modeled using Directed Acyclic Graphs (DAGs).: Policy-driven workflows should be represented as DAGs or Directed Cyclic Graphs (DCGs) to manage dependencies and ensure work flows through specific, gated directions. This allows for parallelizing independent reviews and maintaining necessary checks. (Evan Cormos, Coinbase)
- AI security requires a layered, deterministic defense approach.: Security cannot rely solely on better sandboxes. A layered defense must include: workload identity (short-lived, scoped credentials), prompt/input defense (isolation, provenance tagging), tool governance (least privilege, deterministic human approval), and robust observability (audit trails, circuit breakers). (Andre Ripka, Bloomberg)
- The value of AI is determined by 'Operating Capability,' not just hardware.: In compute commodity markets, the cost of useful intelligence is determined by the operator's ability to maximize output (e.g., tokens per second) and reliability, not just the raw GPU power. Better software and operational processes significantly lower the cost per acceptable token. (Oliver Kenan, Semi Analysis)

## Technical details

- Agentic Capabilities & Optimization: Advanced techniques like post-training and KV cache tricks (e.g., 'cartridges') are used to improve agent performance and reduce latency/cost. For instance, RAMP used post-training to create a financial question answering model that outperformed frontier models while serving at high throughput. (Alexander Shiftchenko, RAMP)
- Software Factory Principles: Modern software development principles (like those from the Toyota Production System) emphasize building systems engineered for quality and repeatability, rather than just volume. This translates to building 'Software Factories' that deliver confidence. (Tar Shaat, Sonar)
- Process Modeling and State Management: Workflows should be modeled using DAGs/DCGs. The process requires defining a policy boundary and managing state transitions, including exception flows, to ensure the work is 'good enough' to pass to the next step. (Evan Cormos, Coinbase)
- AI Security Architecture: Key security requirements include: 1) Workload identity (short-lived, scoped credentials); 2) Input/Prompt defense (isolation, provenance tagging); 3) Tool governance (least privilege, deterministic human approval); 4) Sandboxing (default deny for all outbound traffic, including DNS). (Andre Ripka, Bloomberg)
- Context and Data Management: For institutional finance, context is massive (e.g., 15,000 files, 20MB data rooms). Solutions must address this by using agent swarms to read files (rather than indexing everything in a vector DB) and developing proprietary engines (like rebuilding Office) to handle complex, deep data structures accurately and efficiently. (EJI, Quant AI)

## Practical implications

- Architect agents with explicit policy boundaries and state machines (DAGs) to ensure predictable, auditable, and controllable workflows.
- Implement security controls at the execution layer (e.g., circuit breakers, least privilege tools, default deny egress) rather than relying solely on model guardrails.
- Focus development efforts on proprietary data handling and specialized engines (e.g., for Excel/PowerPoint) to overcome the limitations of general-purpose LLMs when dealing with complex, structured enterprise data.
- Measure and optimize for 'Operating Capability' (useful output per dollar spent) rather than just raw compute power (GPU hours).

## Topics

AI Agents, Software Development Lifecycle (SDLC), Financial Technology (FinTech), Cybersecurity, Compute Economics, Workflow Automation, RAMP, Sonar, Coinbase, OpenAI / ChatBT, Bloomberg, Quant AI, Semi Analysis

Source: https://www.youtube.com/watch?v=jiUC6IUMdvc