Python Packaging, Query Syntax, and Apple Silicon

Dependency Management Innovations

The episode opens by discussing PipChill, a powerful tool designed to simplify package management. Unlike standard pip freeze, which lists every sub-dependency in a virtual environment, PipChill intelligently filters the output to show only the top-level packages you explicitly installed.

• It features a verbose mode that displays sub-dependencies as commented-out notes, helping developers understand why specific libraries were included.
• The hosts also discuss the ongoing challenges with Dependabot, specifically the frustration of manual merge conflicts when multiple dependency updates arrive simultaneously.

Data Science and System Hurdles

Windows Updates and NumPy

A critical issue was highlighted regarding the recent Windows 10 2004 update, which broke OpenBLAS, causing widespread failures in NumPy and related data science libraries.

"The problem can be summarized as when calling FMOD, something is changed so that much later calling OpenBlaz assembly routine fails."

Developers are currently advised to pin NumPy to version 1.19.3 to avoid these unstable registers until a long-term fix is released in 2021.

Developer Productivity and Tools

Fluent Query Syntax with Link-It

The hosts dive into the world of in-memory data manipulation, exploring the Link-It project. Inspired by LINQ from the C# world, it allows developers to write SQL-like queries directly on Python lists, enabling readable operations such as where, select, take, and skip.

Plugin Architectures with Pluggy

For those interested in building extensible software, the discussion turned to Pluggy, the underlying library that powers PyTest's plugin system. The hosts suggest walking through existing codebases to understand the hooks and specs required to implement a robust, plugin-based architecture for custom CLI applications.

Media Processing and Notebooks

• Clio: A framework by Spotify built on Apache Beam, designed for large-scale processing of binary data like audio or images at scale.
• Jupyter Reports: A straightforward method involving custom JavaScript to collapse code cells in notebooks, perfect for creating clean, article-style reports that hide complex logic until requested by the reader.