# The Problem with Jev: Can You Trust AI Decisions?

## Executive summary

The video examines the trustworthiness of AI decisions, contrasting Large Language Models (LLMs) with Jev, a system designed for structured decision-making. While Jev provides structured outputs and confidence probabilities (e.g., 97%), the speaker emphasizes that confidence does not equate to evidence or proof. For high-stakes automation, it is critical to implement robust observability, audit trails, and break down complex decisions into multiple, verifiable steps to ensure accountability and prevent corrupted or manipulated inputs from leading to confidently wrong outcomes.

## Key takeaways

- Jev vs. LLMs: Decision vs. Generation: LLMs generate text, providing a response that can be inspected for evidence. Jev, conversely, makes structured decisions and provides a probability of confidence, which is useful for automation but requires deeper scrutiny.
- Confidence is Not Evidence: The probability of confidence (e.g., 97%) only indicates the model's certainty given its input; it does not explain the reason for the decision or provide proof of correctness. Confidence is distinct from evidence and proof.
- The Need for Observability and Audit Trails: To ensure accountability, systems must provide an audit trail that allows investigators to reconstruct what happened, what data was consulted, and which rules fired, rather than treating the decision as a black box.
- Refining Decisions for Verification: Instead of asking one large question, it is better engineering practice to break down the problem into a series of smaller, sequential questions (stepwise refinement). This allows for the observation and verification of individual decisions that contribute to the final outcome.

## Technical details

- LLM Output Inspection: LLMs provide text output that can be inspected, challenged, or reviewed by another model, allowing access to the tools and data used for training, creating a chain of evidence.
- Jev Functionality: Jev takes input and outputs a structured decision along with a probability of confidence. This structured approach is beneficial for automation workflows.
- Attack Surface Concerns: The input and the model itself are part of the attack surface. Attackers can manipulate inputs (e.g., altering logs or poisoning data sources) to force a consistent, yet corrupted, decision.
- Verification Principle: The core security principle must be 'Verify then Trust,' not 'Trust then Verify.' The goal is to independently verify the intelligence that produced the decision, not just the output format.

## Practical implications

- Treat AI systems (like Jev) as untrusted components and implement security controls around them.
- Implement input validation to ensure the integrity of data entering the AI system.
- Require independent signals and sources of truth, rather than relying on a single AI output.
- Maintain a human 'in the loop' for high-risk decisions, ensuring human oversight and independent verification.
- Audit logs and monitor for drift in the model's behavior to detect subtle inaccuracies.

## Topics

AI Security, Cybersecurity, AI Observability, Zero Trust, Structured Decision Making, Audit Trails, New Frontier AI Models, IBM Technology YouTube Channel

Source: https://www.youtube.com/watch?v=_yO_Q7qo0HQ