Advanced Python Development: Using Powerful Language Features in Real-World Applications

Table of Contents
About the Author
About the Technical Reviewers
Acknowledgments
Introduction
Chapter 1: Prototyping and environments
	Prototyping in Python
		Prototyping with the REPL
		Prototyping with a Python script
		Prototyping with scripts and pdb
			Post-mortem debugging
				The breakpoint function
		Prototyping with Jupyter
			Notebooks
		Prototyping in this chapter
	Environment setup
	Setting up a new project
		Prototyping our scripts
		Installing dependencies
	Exporting to a .py file
	Building a command-line interface
		The sys module and argv
		argparse
		click
	Pushing the boundaries
		Remote kernels
		Developing code that cannot be run locally
	The completed script
	Summary
		Additional resources
Chapter 2: Testing, checking, linting
	Testing
		When to write tests
		Creating formatting functions for improved testability
		pytest
			Unit, integration, and functional testing
			Pytest fixtures
				Categorizing test functions
			Coverage
	Type checking
		Installing mypy
		Adding type hints
		Subclasses and inheritance
		Generic types
		Debugging and overuse of typing
		When to use typing and when to avoid it
		Keeping type hints separate from code
	Linting
		Installing flake8 and black
		Fixing existing code
		Running automatically
		Running on pull requests
	Summary
		Additional resources
Chapter 3: Packaging scripts
	Terminology
	Directory structure
	Setup scripts and metadata
	Dependencies
	Declarative configurations
		Things to avoid in setup.py
			Conditional dependencies
			Readme in metadata
			Version numbers
		Using setup.cfg
	Custom index servers
		Setting up pypiserver
		Durability
		Confidentiality
		Integrity
		Wheel formats and executing code on installation
			Creating wheels from existing distributions
	Installing the console script using entrypoints
	README, DEVELOP, and CHANGES
		Markdown format
		reStructured text format
		README
		CHANGES.md and versioning
			Semantic versioning
			Calendar versioning
	Upstream dependency version pins
		Loose pins
		Strict pins
		Which pinning scheme to use
	Uploading a version
		Configuring twine
	Summary
		Additional resources
Chapter 4: From script to framework
	Writing a sensor plugin
		Developing the plugin
	Adding a new command option
		Subcommands
		Command options
		Error handling
		Off-loading parsing to Click with argument types
		Custom click argument types
		Canned options
	Allowing third-party sensor plugins
		Plugin detection using fixed names
		Plugin detection using entrypoints
		Configuration files
		Environment variables
		Approach for apd.sensors vs. similar programs
	Summary
		Additional resources
Chapter 5: Alternative interfaces
	Web microservices
		WSGI
		API design
			Authentication
		Flask
		Python decorators
			Closures
				Modifying variables in parent scopes
			Basic decorators
			Decorators with arguments
			Decorator-based security
		Testing the view function
		Deployment
	Extending software as a third party
		Agreeing on an ad hoc signature with peers
		Abstract base classes
		Fallback strategies
			Adapter pattern
			Dynamic class generation
			Other serialization formats
		Bringing it all together
	Fixing the serialization problem in our code
		Tidying up
	Versioning APIs
		Testability
	Summary
		Additional resources
Chapter 6: Aggregation process
	Cookiecutter
		Creating a new template
	Creating the aggregation package
		Database types
		Our example
		Object-relational mappers
		Versioning the database
			Other useful alembic commands
		Loading data
	New technologies
		Databases
		Custom attribute behavior
		Generators
	Summary
		Additional resources
Chapter 7: Parallelization and async
	Nonblocking IO
		Making our code nonblocking
	Multithreading and multiprocessing
		Low-level threads
		Bytecode
			The GIL
		Locks and deadlocks
			Deadlocks
		Avoiding global state
			Collating data
			Passing data
		Other synchronization primitives
			Reentrant locks
			Conditions
			Barriers
			Event
			Semaphore
		ProcessPoolExecutors
		Making our code multithreaded
	AsyncIO
		async def
		await
		async for
		async with
		Async locking primitives
		Working with synchronous libraries
		Making our code asynchronous
	Comparison
	Making a choice
	Summary
		Additional resources
Chapter 8: Advanced asyncio
	Testing async code
		Testing our code
			Test servers and pytest fixtures with teardown
			Fixture scoping
		Mocking objects for easier unit testing
			Mocks with branching logic
			Data classes
				contextlib
			Test methods
	Asynchronous databases
		Classic SQLAlchemy style
			Uncompiled
			mssql
			mysql
			Postgresql
			sqlite
		Using run_in_executor
		Querying data
		Avoiding complex queries
			Querying against views
		Alternatives
	Global variables in asynchronous code
	Summary
		Additional resources
Chapter 9: Viewing the data
	Query functions
		Filtering data
		Multilevel iterators
		Additional filters
		Testing our query functions
			Parameterized tests
	Displaying multiple sensors
	Processing data
	Interactivity with Jupyter widgets
		Multiply nested synchronous and asynchronous code
		Tidying up
	Persistent endpoints
	Charting maps and geographic data
		New plot types
		Supporting map type charts in apd.aggregation
			Backward compatibility in data classes
		Drawing a custom map using the new configs
	Summary
		Additional resources
Chapter 10: Speeding things up
	Optimizing a function
		Profiling and threads
		Interpreting the profile report
		Other profilers
			timeit
			line_profiler
			yappi
			Tracemalloc
			New Relic
	Optimizing control flow
		Visualizing profiling data
		Caching
			Cached properties
	Summary
		Additional resources
Chapter 11: Fault tolerance
	Error handling
		Getting items from a container
			Abstract base classes
			Exception types
				TypeError and ValueError
				RuntimeError and SystemExit
				AssertionError
		Custom exceptions
			Creating new exception types
			Additional metadata
		Tracebacks involving multiple exceptions
			Exception in except or finally block
			raise from
		Testing for exception handling
			New behaviors
			Advanced mocking with unittest.Mock
	Warnings
		Warning filters
	Logging
		Nested loggers
		Custom actions
			Extra metadata
				Logging adapter
				LogRecord factory
				Logging filters
		Logging configuration
		Other handlers
			Audit logs
	Designing around problems
		Scheduling sensor lookups
			APIs and filtering
	Summary
		Additional resources
Chapter 12: Callbacks and data analysis
	Generator data flow
		Generators that consume their own output
		Enhanced generators
			Using classes
			Using an enhanced generator to wrap an iterable
			Refactoring functions with excessive return values
		Queues
		Choosing a control flow
	Structure for our actions
		Analysis coroutines
		Ingesting data
		Running the analysis process
	Process status
		Callbacks
	Extending the actions available
	Summary
		Additional resources
		Epilogue
Index