Hy4 preview for office analysis: can it audit 72 files?

The useful office model is not the one that makes a paragraph sound polished. It is the one that can find the policy that is currently in force, apply it to messy source files, and show enough evidence for another person to review the decision. Tencent’s Hy4 preview announcement gives a concrete example: checking invoice compliance across 72 files while identifying the effective rules in three policy documents.

That task is harder than summarization. It requires document classification, date-aware policy selection, field extraction, rule mapping, and an evidence-backed decision. A plausible answer with the wrong policy version is still a failed audit.

Key takeaway

Invoice review is a date-aware evidence task, not ordinary document summarization.

Require file IDs, page citations, effective-rule versions, missing evidence, and next actions.

Test conflicts and boundary dates to measure false approvals and false rejections.

A model can accelerate review; a human still owns the final financial decision.

A reproducible evaluation

Put the files in a read-only directory, assign stable IDs, and record publication, effective, and retirement dates for each policy. Ask the model to build a rule timeline before it evaluates an invoice. For every result, require the invoice ID, conclusion, rule version, cited file and page, missing evidence, and recommended next action.

Then add adversarial cases: an old policy with a different threshold, an invoice that crosses the effective date, and an attachment missing an approval signature. Measure false approvals, false rejections, unsupported citations, and time to review. A large context window can reduce manual chunking, but it cannot replace evidence checks.

What a production hand-off needs

A useful output has more than a compliant/non-compliant label. It should include the source fields used, the exact rule text, uncertainty, confidence, and a clear request for human input when evidence is missing. Save the original files, prompt, model version, tool calls, and final decision so the result can be replayed.

SandBase or another provider gateway can keep keys server-side and record request IDs, token usage, latency, and retries. Whether financial documents may leave an organization’s boundary is a separate compliance decision; a gateway does not remove that responsibility.

Preview boundaries

Tencent positions Hy4 preview for cross-file collaboration and end-to-end delivery of documents, spreadsheets, and presentations. The supplied case is a strong evaluation lead, not an independent audit: the full corpus, prompts, and scoring procedure are not public. Tencent also lists overthinking and excessive self-verification as known issues, so production workflows should enforce a time budget, retry limit, and human escalation rule.

Initial assessment

Hy4 preview is worth testing on office workflows when every conclusion must be traceable. Treat the 72-file case as a benchmark you reproduce, not a guarantee you inherit.

A decision record beats a confident answer

For each invoice, keep a structured decision record rather than only the model’s prose. Store the normalized supplier, amount, date, currency, approval state, selected policy version, and a list of supporting pages. Add a reason code for every rejection and an “insufficient evidence” state that cannot be silently converted into approval. This makes sampling and later appeals much easier.

The policy timeline deserves its own test. Give the evaluator two nearly identical rules with different effective dates and place them in separate files. Ask it to explain which rule wins and why. Then repeat with a scanned page, a superseded attachment, and a document whose filename is misleading. Measure not only accuracy but citation completeness and the number of times a reviewer must open the source file.

Privacy and human review

Financial documents often contain personal and banking data. Minimize the corpus, redact fields that do not affect the decision, and define retention before sending anything to a hosted endpoint. Keep access logs separate from the model prompt log, and ensure reviewers can revoke a run or delete its artifacts.

Hy4 can accelerate triage, but the approval authority should remain explicit. Route borderline amounts, missing signatures, conflicting policies, and low-confidence extraction to a human queue. The acceptance metric is therefore end-to-end: fewer manual minutes without increasing unsupported approvals or hiding uncertainty.

A practical sampling plan

Start with a small, representative corpus rather than sending every historical invoice to the model. Include routine approvals, high-value exceptions, missing attachments, duplicate invoices, and cases where the policy changed during the reporting period. Have two reviewers label the expected outcome and the exact supporting rule before the model sees the files. Their disagreement is useful evidence: it shows where the policy itself needs clarification instead of pretending the model can resolve ambiguity.

Run the same corpus in two modes. In the first, the model may only retrieve and cite evidence. In the second, it may draft a recommendation for a reviewer. Compare review time, citation completeness, and the rate at which reviewers accept or correct the recommendation. Do not allow the model to submit a payment, edit an accounting system, or notify a supplier during this evaluation. Those side effects need a separate approval path and a tested rollback.

When a result is wrong, preserve the complete trace: the file IDs presented, the rule timeline, the extracted fields, the cited pages, and the prompt version. Classify the failure as retrieval, date selection, extraction, reasoning, or policy ambiguity. This taxonomy is more useful than a single error rate because each class requires a different fix. Better OCR will not repair a rule that has two active versions, and a larger context window will not repair a missing approval signature.

The final hand-off should state what the model is allowed to decide, what it may only recommend, and which cases must always reach a person. Include an owner for policy updates and a date for re-running the evaluation after a new model or prompt is introduced. That turns a one-time demo into an operating control that can be audited when the inputs, rules, or provider change.

Bottom line: use Hy4 preview to reduce routine review effort only when the evidence chain, escalation rules, privacy controls, and human approval boundary are explicit and measurable.