Python Dependency Management, Profiling, and Testing

Python Dependency Changes and Optimization

Pip Dependency Resolver

• Dependency management is evolving with the upcoming default change to the new pip resolver in October 2020.
• Users should test their packages using the --use-features=2020-resolver flag to avoid potential conflicts.
• Lock files generated with the old resolver may face incompatibilities; it is advised to regenerate them using the new resolver.

Python Import Profiling

• Startup times are often hindered by slow imports; the import-profiler tool helps diagnose which modules contribute to delay.
• It is crucial to monitor heavy import chains to ensure performant application startup, especially for CLI tools.

Advanced Testing and Data Analysis

Django Testing Toolbox

• Maintaining a fast test cycle in Django is improved by techniques such as using in-memory SQLite databases, implementing Factory Boy for data generation, and utilizing Django Test Plus.
• Avoiding expensive operations—like complex password hashing—during testing significantly accelerates development velocity.

Exploratory Data Analysis with Pandas

• The pandas-profiling library automates the exploratory data analysis (EDA) process by generating comprehensive HTML reports containing correlations, histograms, and missing data alerts.

"This is one line of code to get this output. This is great. This like replaces a couple interns at least."

Architectural Concepts

Modern OOP in Python

• The show explores interfaces, mix-ins, and abstract base classes within a Python-centric context, emphasizing that Python's OOP philosophy differs significantly from languages like Java or C#.
• Developers are encouraged to use built-in abc modules for clearer structure while avoiding the unnecessary complexity of over-engineering classes.

The Dangers of Pickle

• Ned Batchelder’s Pickle's Nine Flaws serves as a critical reminder that pickle is insecure, slow, and specific to Python, making it unsuitable for cross-language data interchange or untrusted inputs.