Build for the Memo, Not the Demo — Shawn Chan, China Resources Holdings
This talk contrasts 'demos' (polished, impressive marketing presentations) with 'memos' (deeply scrutinized documents that survive intense financial review). The speaker argues that most AI products are built for the demo—designed to impress for five minutes. However, for real-world applications involving significant capital, the product must pass the 'memo test,' which requires absolute verifiability and accountability. Key architectural requirements include ensuring every claim has a traceable source (provenance), reconciling conflicting data points, and maintaining clear separation between established facts and speculative guesses.
Key takeaways
-
The Demo vs. Memo Test
9:06
A demo aims for fluency and confidence; a memo must survive an argument and prove its accuracy under scrutiny. The moment real money is watching, every sentence becomes a memo sentence, meaning there is no safe demo anymore.
-
Source Trust Hierarchy
15:45
AI systems must differentiate between sources of varying trust levels (e.g., an audited filing vs. a group chat rumor). Treating all sources equally leads to unreliable outputs.
-
Data Reconciliation is Mandatory
18:42
The system must automatically check that figures agree across all sections of the document (e.g., page one vs. table on page eleven). Failure to reconcile numbers signals a critical flaw.
-
Contradictions are Signals, Not Bugs
24:19
Instead of smoothing over conflicts (e.g., CEO's number vs. official filing), the AI must surface contradictions. The gap between conflicting numbers is often the most important piece of information.
-
Accountability and Provenance
Every claim must be linked directly to its source paragraph (provenance), not just a citation tab. Furthermore, the final decision requires an auditable human sign-off gate.