AI in 2025: Models, Scaling, Work and What Comes Next
The AI race and how people use models
Lex Fridman speaks with Sebastian Raschka and Nathan Lambert about developments in AI over the past year and what may come next. They discuss competition among U.S. and Chinese labs, the influence of DeepSeek’s open-weight releases, and the different strengths and business incentives of companies including Google, OpenAI and Anthropic. They see no obvious single winner: ideas move between labs, while resources, organizational culture, products and infrastructure also shape progress.
The guests compare consumer chatbots, coding tools and their own ways of choosing models. They describe using different systems for fast queries, long-context work, coding and research, and discuss how product interfaces and familiarity influence which models people return to.
Open models and how they are built
The conversation surveys open-weight models and the growing range of organizations releasing them. The guests explain why open models can support local use, customization and research, and why permissive licenses and low-cost access matter. They also discuss tool use: models can call search or code tools instead of relying only on memorized information, though tool access raises security and reliability concerns.
Raschka gives an accessible account of transformer architecture and changes such as mixture of experts, attention variants and KV caching. The guests distinguish architectural tweaks from advances driven by training methods and systems. They outline pre-training, mid-training and post-training, and discuss scaling laws, compute costs, data quality, synthetic data, data selection and copyright questions around training material.
Reinforcement learning, learning and work
Lambert and Raschka describe reinforcement learning with verifiable rewards (RLVR), in which models attempt tasks such as math and coding and receive rewards based on checkable results. They discuss how this can encourage longer reasoning and tool use, while noting concerns about benchmark contamination and what the training actually teaches. They compare RLVR with reinforcement learning from human feedback (RLHF), which can shape style and preferences but compresses varied human preferences into training signals.
The guests recommend building small language models from scratch as a way to understand their components, then using reference implementations and tests to study more complex systems. They also consider how AI affects programming enjoyment and expertise: it can remove tedious work and help people build, but relying on it for every step may reduce opportunities to practice and learn.
Research careers, organizational culture and social effects
The discussion weighs academic research, open-model labs, frontier companies and startups, including trade-offs in pay, publishing, compute and career risk. The guests talk about demanding work cultures, competition and the possibility of burnout, as well as the risk that a concentrated tech bubble may overlook perspectives beyond its own environment.
They consider model character, safety and the tension between useful, engaging systems and guardrails intended to prevent harm. They discuss the importance of human agency, the potential effects of AI on work and learning, and the value of human judgment in reviewing AI-generated material.
What may come next
The guests examine alternatives to autoregressive transformers, including text diffusion models, which may generate multiple tokens in parallel but face trade-offs in quality and tool use. They discuss recursive language models, tool integration, continual learning, memory and longer context windows, along with world models and possible connections to robotics. They describe robotics as facing distinct challenges, particularly adaptation to varied real-world settings and safety.
On AGI timelines, they question whether the label is useful and differ in their expectations for automating software and research. They discuss possible economic effects, specialized AI systems, advertising, acquisitions and the future of companies and open models. Lambert describes the Atom Project and argues for investment in genuinely open U.S. models; the guests also discuss the roles of NVIDIA, compute and influential technology leaders.
The episode closes with reflections on technology, human connection, work and the future. The guests consider how computing and communication may be remembered, how people might preserve agency and community amid automation, and what gives them hope about humanity’s ability to respond to AI.