The Problem with Jev: Can You Trust AI Decisions?
Summary
The video examines the trustworthiness of AI decisions, contrasting Large Language Models (LLMs) with Jev, a system designed for structured decision-making. While Jev provides structured outputs and confidence probabilities (e.g., 97%), the speaker emphasizes that confidence does not equate to evidence or proof. For high-stakes automation, it is critical to implement robust observability, audit trails, and break down complex decisions into multiple, verifiable steps to ensure accountability and prevent corrupted or manipulated inputs from leading to confidently wrong outcomes.
Key takeaways
-
Jev vs. LLMs: Decision vs. Generation
LLMs generate text, providing a response that can be inspected for evidence. Jev, conversely, makes structured decisions and provides a probability of confidence, which is useful for automation but requires deeper scrutiny.
-
Confidence is Not Evidence
The probability of confidence (e.g., 97%) only indicates the model's certainty given its input; it does not explain the reason for the decision or provide proof of correctness. Confidence is distinct from evidence and proof.
-
The Need for Observability and Audit Trails
To ensure accountability, systems must provide an audit trail that allows investigators to reconstruct what happened, what data was consulted, and which rules fired, rather than treating the decision as a black box.
-
Refining Decisions for Verification
Instead of asking one large question, it is better engineering practice to break down the problem into a series of smaller, sequential questions (stepwise refinement). This allows for the observation and verification of individual decisions that contribute to the final outcome.
Technical details
-
LLM Output Inspection
0s
LLMs provide text output that can be inspected, challenged, or reviewed by another model, allowing access to the tools and data used for training, creating a chain of evidence.
-
Jev Functionality
0s
Jev takes input and outputs a structured decision along with a probability of confidence. This structured approach is beneficial for automation workflows.
-
Attack Surface Concerns
0s
The input and the model itself are part of the attack surface. Attackers can manipulate inputs (e.g., altering logs or poisoning data sources) to force a consistent, yet corrupted, decision.
-
Verification Principle
0s
The core security principle must be 'Verify then Trust,' not 'Trust then Verify.' The goal is to independently verify the intelligence that produced the decision, not just the output format.
Mentioned resources
Channel & topics
Watch on YouTube · Back to latest
This independent, AI-assisted summary is provided for commentary and informational purposes. It may contain errors or omit important context. Please watch the original video for the creator's complete presentation. Video, thumbnail, and related copyrights belong to their respective owners.