The New Era of Knowledge Distillation: From Programming to Demonstration

For decades, capturing human knowledge meant programming it. Domain experts explained their knowledge to programmers, who translated it into explicit rules and algorithms. This approach powered the digital revolution but has fundamental limitations.

Artificial intelligence now enables a different approach: knowledge distillation through demonstration. Instead of programming every rule, we show AI systems what we do. The AI observes, learns patterns, and applies knowledge contextually.

The Limits of Traditional Knowledge Capture

Traditional programmed systems share common limitations. They handle anticipated scenarios well but fail with unexpected situations—every edge case must be explicitly programmed. Creating these systems requires domain experts, programmers, and analysts in a three-way translation that's slow and expensive, meaning much expertise never makes it into systems.

Michael Polanyi observed, "We can know more than we can tell" (The Tacit Dimension, 1966). Pattern recognition and intuition are nearly impossible to program because experts cannot fully articulate their decision-making rules. Meanwhile, knowledge evolves constantly, but programmed systems are expensive to update. Organizations maintain legacy systems with outdated knowledge because updating is prohibitively expensive.

Most human expertise remains locked in people's heads, lost when they leave, impossible to scale.

AI-Enabled Knowledge Distillation

AI enables knowledge capture through demonstration. Experts work while AI observes and learns patterns—both explicit rules and implicit patterns revealed through examples. The AI reasons about context and intent, and when encountering novel situations, it applies principles from similar cases rather than simply failing.

Knowledge distillation becomes an ongoing conversation where corrections refine understanding and the system improves through use. By observing many demonstrations, AI learns tacit knowledge experts cannot articulate—consistent patterns, follow-up questions, and context-dependent standards that would be impossible to program explicitly.

A Real-World Example

My workflow for working on browser illustrates this paradigm. I've built a hybrid system where scripts perform well-defined tasks like bug triage, data extraction, browser automation, and calendar parsing. Markdown documents capture domain expertise on topics like window management, keyboard handling, and memory diagnostics.

The AI reads this knowledge base, executes scripts, and applies knowledge contextually to provide prioritized summaries with reasoning. Crucially, the AI writes prompts, documentation, and scripts for itself—each successful workflow becomes both a reusable pattern and executable automation, creating a self-improving knowledge loop.

The AI learned through demonstration and correction, not programming. This hybrid approach leverages both paradigms—scripts provide reliable execution, the knowledge base captures expertise, and AI provides contextual understanding.

Implications and Opportunities

Programming expertise remains valuable, but the balance is shifting. As AI improves, demonstrating, documenting, and curating knowledge becomes increasingly important. Programmers are evolving from code writers to system growers. Building knowledge systems becomes accessible beyond traditional programmers—domain experts can create systems by demonstrating work and refining AI understanding.

AI-enabled systems evolve continuously. As conditions change, experts demonstrate new approaches and systems adapt, reducing traditional reprogramming cycles. Each person can distill knowledge into customized systems—AI assistants understanding their particular expertise and contexts. Companies can capture institutional knowledge through AI observation, so when employees retire, their judgment patterns remain accessible.

Conclusion

We're witnessing a fundamental shift in knowledge capture. For the first time, tools can learn from demonstration rather than requiring explicit programming. We're not fully there yet—programming expertise still matters—but as AI improves, domain expertise and demonstration skills may become more valuable than coding ability. This paradigm has challenges: AI systems can learn incorrect patterns and perpetuate biases. They're pattern matchers, not domain experts, and experts remain essential for direction, final decisions, and novel situations.

Those who master this new form of knowledge distillation will have a profound advantage. The era of truly scalable knowledge systems is just beginning.