# Processing Documents: Jev vs OSS Models

## Executive summary

This video compares several model architectures (Jev, Quen, Leia, Jeff) for automating decision-making and classification tasks within document processing pipelines. The core finding is that the open decoder model, Quen, performs remarkably close to the generalized classifier, Jev, across various tasks (language detection, document classification, document routing). The approach demonstrates that robust classification can be achieved without requiring pre-training on specific labels, making it highly valuable for building flexible, automated document pipelines.

## Key takeaways

- Quen's Performance in Classification: The Quen decoder model, which is a standard Language Model (LM) and not specifically trained for classification, showed accuracy very close to Jev in tasks like language detection and document routing, suggesting its utility for general document understanding.
- Model Architecture Comparison: Jev is a generalized classifier trained with reinforcement learning, optimized for confidence scoring. Quen is a decoder LM. Leia and Jeff are encoder models (Leia predicts a mask token; Jeff is good for entity/relation extraction). While encoders are smaller and easier to fine-tune, they were less accurate than the decoders in this demonstration.
- Document Processing Tasks Covered: The models were tested on five key document pipeline tasks: Language Detection (using Lingua as baseline), Orientation Detection (using Tesseract as baseline), Document Classification, Document Splitting, and Document Routing/Triage.

## Technical details

- Input Preparation (LightParse): The open-source parser, LightParse, is used to gather necessary inputs for classification, including detecting document complexity, text coverage, running OCR, and generating per-page screenshots for orientation detection.
- Classification Input Format: All models are normalized to accept a 'state' (the input text) and a set of 'questions' (the possible choices/labels), allowing for generalized classification.
- Document Routing/Triage: This task uses metadata (e.g., OCR confidence, text length, complexity stats) as input to classify whether a page requires an upgrade to a more advanced pipeline (e.g., LlamaParse).

## Practical implications

- Build engineers can use this comparative analysis to select the optimal model (e.g., Quen or Jev) for specific pipeline stages, balancing accuracy, speed, and resource cost.
- The framework demonstrates how to build robust, multi-stage document processing pipelines by treating classification tasks (like routing or splitting) as inputs for subsequent processing steps.
- The ability to classify documents without extensive pre-training on labels significantly reduces the overhead of developing new document processing features.

## Topics

AI, Natural Language Processing (NLP), Document Processing, Machine Learning, Classification, Pipeline Engineering, LlamaIndex, jev_vs_oss

Source: https://www.youtube.com/watch?v=MLZ1dUOZG74