Immediate download Python Real-World Projects: Crafting your Python Portfolio with Deployable Applications Steven F. Lott ebooks 2024
Immediate download Python Real-World Projects: Crafting your Python Portfolio with Deployable Applications Steven F. Lott ebooks 2024
com
https://ebookmass.com/product/python-real-world-projects-
crafting-your-python-portfolio-with-deployable-applications-
steven-f-lott/
OR CLICK BUTTON
DOWNLOAD NOW
https://ebookmass.com/product/fortran-with-python-integrating-legacy-
systems-with-python-bisette/
ebookmass.com
https://ebookmass.com/product/publishing-python-packages-test-share-
and-automate-your-projects-1st-edition-dane-hillard/
ebookmass.com
https://ebookmass.com/product/applied-numerical-methods-with-python-
for-engineers-and-scientists-steven-c-chapra/
ebookmass.com
Python Programming and SQL: 10 Books in 1 - Supercharge
Your Career with Python Programming and SQL Andrew Reed
https://ebookmass.com/product/python-programming-and-sql-10-books-
in-1-supercharge-your-career-with-python-programming-and-sql-andrew-
reed/
ebookmass.com
https://ebookmass.com/product/c-and-python-applications-embedding-
python-code-in-c-programs-sql-methods-and-python-sockets-philip-joyce/
ebookmass.com
https://ebookmass.com/product/learning-scientific-programming-with-
python-hill/
ebookmass.com
https://ebookmass.com/product/r-and-python-for-oceanographers-a-
practical-guide-with-applications-hakan-alyuruk/
ebookmass.com
https://ebookmass.com/product/algorithmic-essentials-trading-with-
python-your-comprehenive-guide-for-2024-van-der-post/
ebookmass.com
Content
1. Preface
1. Who this book is for
2. What this book covers
3. A note on skills required
4. To get the most out of this book
1. Complete the extras
2. Download the example code files
5. Conventions used
6. Get in touch
7. Share your thoughts
8. Download a free PDF copy of this book
2. Chapter 1: Project Zero: A Template for Other Projects
1. 1.1 On quality
1. 1.1.1 More Reading on Quality
2. 1.2 Suggested project sprints
1. 1.2.1 Inception
2. 1.2.2 Elaboration, part 1: define done
3. 1.2.3 Elaboration, part 2: define components and tests
4. 1.2.4 Construction
5. 1.2.5 Transition
3. 1.3 List of deliverables
4. 1.4 Development tool installation
5. 1.5 Project 0 – Hello World with test cases
1. 1.5.1 Description
2. 1.5.2 Approach
3. 1.5.3 Deliverables
4. 1.5.4 Definition of done
6. 1.6 Summary
7. 1.7 Extras
1. 1.7.1 Static analysis - mypy, flake8
2. 1.7.2 CLI features
3. 1.7.3 Logging
4. 1.7.4 Cookiecutter
3. Chapter 2: Overview of the Projects
1. 2.1 General data acquisition
2. 2.2 Acquisition via Extract
3. 2.3 Inspection
4. 2.4 Clean, validate, standardize, and persist
5. 2.5 Summarize and analyze
6. 2.6 Statistical modeling
7. 2.7 Data contracts
8. 2.8 Summary
4. Chapter 3: Project 1.1: Data Acquisition Base Application
1. 3.1 Description
1. 3.1.1 User experience
2. 3.1.2 About the source data
3. 3.1.3 About the output data
2. 3.2 Architectural approach
1. 3.2.1 Class design
2. 3.2.2 Design principles
3. 3.2.3 Functional design
3. 3.3 Deliverables
1. 3.3.1 Acceptance tests
2. 3.3.2 Additional acceptance scenarios
3. 3.3.3 Unit tests
4. 3.4 Summary
5. 3.5 Extras
1. 3.5.1 Logging enhancements
2. 3.5.2 Configuration extensions
3. 3.5.3 Data subsets
4. 3.5.4 Another example data source
5. Chapter 4: Data Acquisition Features: Web APIs and Scraping
1. 4.1 Project 1.2: Acquire data from a web service
1. 4.1.1 Description
2. 4.1.2 Approach
3. 4.1.3 Deliverables
2. 4.2 Project 1.3: Scrape data from a web page
1. 4.2.1 Description
2. 4.2.2 About the source data
3. 4.2.3 Approach
4. 4.2.4 Deliverables
3. 4.3 Summary
4. 4.4 Extras
1. 4.4.1 Locate more JSON-format data
2. 4.4.2 Other data sets to extract
3. 4.4.3 Handling schema variations
4. 4.4.4 CLI enhancements
5. 4.4.5 Logging
6. Chapter 5: Data Acquisition Features: SQL Database
1. 5.1 Project 1.4: A local SQL database
1. 5.1.1 Description
2. 5.1.2 Approach
3. 5.1.3 Deliverables
2. 5.2 Project 1.5: Acquire data from a SQL extract
1. 5.2.1 Description
2. 5.2.2 The Object-Relational Mapping (ORM) problem
3. 5.2.3 About the source data
4. 5.2.4 Approach
5. 5.2.5 Deliverables
3. 5.3 Summary
4. 5.4 Extras
1. 5.4.1 Consider using another database
2. 5.4.2 Consider using a NoSQL database
3. 5.4.3 Consider using SQLAlchemy to define an ORM layer
7. Chapter 6: Project 2.1: Data Inspection Notebook
1. 6.1 Description
1. 6.1.1 About the source data
2. 6.2 Approach
1. 6.2.1 Notebook test cases for the functions
2. 6.2.2 Common code in a separate module
3. 6.3 Deliverables
1. 6.3.1 Notebook .ipynb file
2. 6.3.2 Executing a notebook’s test suite
4. 6.4 Summary
5. 6.5 Extras
1. 6.5.1 Use pandas to examine data
8. Chapter 7: Data Inspection Features
1. 7.1 Project 2.2: Validating cardinal domains — measures, counts,
and durations
2. 7.1.1 Description
3. 7.1.2 Approach
4. 7.1.3 Deliverables
5. 7.2 Project 2.3: Validating text and codes — nominal data and
ordinal numbers
6. 7.2.1 Description
7. 7.2.2 Approach
8. 7.2.3 Deliverables
9. 7.3 Project 2.4: Finding reference domains
1. 7.3.1 Description
2. 7.3.2 Approach
3. 7.3.3 Deliverables
10. 7.4 Summary
11. 7.5 Extras
1. 7.5.1 Markdown cells with dates and data source information
2. 7.5.2 Presentation materials
3. 7.5.3 JupyterBook or Quarto for even more sophisticated
output
9. Chapter 8: Project 2.5: Schema and Metadata
1. 8.1 Description
2. 8.2 Approach
1. 8.2.1 Define Pydantic classes and emit the JSON Schema
2. 8.2.2 Define expected data domains in JSON Schema notation
3. 8.2.3 Use JSON Schema to validate intermediate files
3. 8.3 Deliverables
1. 8.3.1 Schema acceptance tests
2. 8.3.2 Extended acceptance testing
4. 8.4 Summary
5. 8.5 Extras
1. 8.5.1 Revise all previous chapter models to use Pydantic
2. 8.5.2 Use the ORM layer
10. Chapter 9: Project 3.1: Data Cleaning Base Application
1. 9.1 Description
1. 9.1.1 User experience
2. 9.1.2 Source data
3. 9.1.3 Result data
4. 9.1.4 Conversions and processing
5. 9.1.5 Error reports
2. 9.2 Approach
1. 9.2.1 Model module refactoring
2. 9.2.2 Pydantic V2 validation
3. 9.2.3 Validation function design
4. 9.2.4 Incremental design
5. 9.2.5 CLI application
3. 9.3 Deliverables
1. 9.3.1 Acceptance tests
2. 9.3.2 Unit tests for the model features
3. 9.3.3 Application to clean data and create an NDJSON
interim file
4. 9.4 Summary
5. 9.5 Extras
1. 9.5.1 Create an output file with rejected samples
11. Chapter 10: Data Cleaning Features
1. 10.1 Project 3.2: Validate and convert source fields
1. 10.1.1 Description
2. 10.1.2 Approach
3. 10.1.3 Deliverables
2. 10.2 Project 3.3: Validate text fields (and numeric coded fields)
1. 10.2.1 Description
2. 10.2.2 Approach
3. 10.2.3 Deliverables
3. 10.3 Project 3.4: Validate references among separate data sources
1. 10.3.1 Description
2. 10.3.2 Approach
3. 10.3.3 Deliverables
4. 10.4 Project 3.5: Standardize data to common codes and ranges
1. 10.4.1 Description
2. 10.4.2 Approach
3. 10.4.3 Deliverables
5. 10.5 Project 3.6: Integration to create an acquisition pipeline
1. 10.5.1 Description
2. 10.5.2 Approach
3. 10.5.3 Deliverables
6. 10.6 Summary
7. 10.7 Extras
1. 10.7.1 Hypothesis testing
2. 10.7.2 Rejecting bad data via filtering (instead of logging)
3. 10.7.3 Disjoint subentities
4. 10.7.4 Create a fan-out cleaning pipeline
12. Chapter 11: Project 3.7: Interim Data Persistence
1. 11.1 Description
2. 11.2 Overall approach
1. 11.2.1 Designing idempotent operations
3. 11.3 Deliverables
1. 11.3.1 Unit test
2. 11.3.2 Acceptance test
3. 11.3.3 Cleaned up re-runnable application design
4. 11.4 Summary
5. 11.5 Extras
1. 11.5.1 Using a SQL database
2. 11.5.2 Persistence with NoSQL databases
13. Chapter 12: Project 3.8: Integrated Data Acquisition Web Service
1. 12.1 Description
1. 12.1.1 The data series resources
2. 12.1.2 Creating data for download
2. 12.2 Overall approach
1. 12.2.1 OpenAPI 3 specification
2. 12.2.2 RESTful API to be queried from a notebook
3. 12.2.3 A POST request starts processing
4. 12.2.4 The GET request for processing status
5. 12.2.5 The GET request for the results
6. 12.2.6 Security considerations
3. 12.3 Deliverables
1. 12.3.1 Acceptance test cases
2. 12.3.2 RESTful API app
3. 12.3.3 Unit test cases
4. 12.4 Summary
5. 12.5 Extras
1. 12.5.1 Add filtering criteria to the POST request
2. 12.5.3 Use Celery instead of concurrent.futures
3. 12.5.4 Call external processing directly instead of running a
subprocess
14. Chapter 13: Project 4.1: Visual Analysis Techniques
1. 13.1 Description
2. 13.2 Overall approach
1. 13.2.1 General notebook organization
2. 13.2.2 Python modules for summarizing
3. 13.2.3 PyPlot graphics
4. 13.2.4 Iteration and evolution
3. 13.3 Deliverables
1. 13.3.1 Unit test
2. 13.3.2 Acceptance test
4. 13.4 Summary
5. 13.5 Extras
1. 13.5.1 Use Seaborn for plotting
2. 13.5.2 Adjust color palettes to emphasize key points about the
data
15. Chapter 14: Project 4.2: Creating Reports
1. 14.1 Description
1. 14.1.1 Slide decks and presentations
2. 14.1.2 Reports
2. 14.2 Overall approach
1. 14.2.1 Preparing slides
2. 14.2.2 Preparing a report
3. 14.2.3 Creating technical diagrams
3. 14.3 Deliverables
4. 14.4 Summary
5. 14.5 Extras
1. 14.5.1 Written reports with UML diagrams
16. Chapter 15: Project 5.1: Modeling Base Application
1. 15.1 Description
2. 15.2 Approach
1. 15.2.1 Designing a summary app
2. 15.2.2 Describing the distribution
3. 15.2.3 Use cleaned data model
4. 15.2.4 Rethink the data inspection functions
5. 15.2.5 Create new results model
3. 15.3 Deliverables
1. 15.3.1 Acceptance testing
2. 15.3.2 Unit testing
3. 15.3.3 Application secondary feature
4. 15.4 Summary
5. 15.5 Extras
1. 15.5.1 Measures of shape
2. 15.5.2 Creating PDF reports
3. 15.5.3 Serving the HTML report from the data API
17. Chapter 16: Project 5.2: Simple Multivariate Statistics
1. 16.1 Description
1. 16.1.1 Correlation coefficient
2. 16.1.2 Linear regression
3. 16.1.3 Diagrams
2. 16.2 Approach
1. 16.2.1 Statistical computations
2. 16.2.2 Analysis diagrams
3. 16.2.3 Including diagrams in the final document
3. 16.3 Deliverables
1. 16.3.1 Acceptance tests
2. 16.3.2 Unit tests
4. 16.4 Summary
5. 16.5 Extras
1. 16.5.1 Use pandas to compute basic statistics
2. 16.5.2 Use the dask version of pandas
3. 16.5.3 Use numpy for statistics
4. 16.5.4 Use scikit-learn for modeling
5. 16.5.5 Compute the correlation and regression using
functional programming
18. Chapter 17: Next Steps
1. 17.1 Overall data wrangling
2. 17.2 The concept of “decision support”
3. 17.3 Concept of metadata and provenance
4. 17.4 Next steps toward machine learning
5. Why subscribe?
19. Other Books You Might Enjoy
1. Packt is searching for authors like you
2. Share your thoughts
3. Download a free PDF copy of this book
20. Index
BIRMINGHAM—MUMBAI
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted
in any form or by any means, without the prior written permission of the publisher, except in the case of
brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information
presented. However, the information contained in this book is sold without warranty, either express or
implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any
damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and
products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot
guarantee the accuracy of this information.
Published by Packt Publishing Ltd. Grosvenor House 11 St Paul’s Square Birmingham B3 1RB
ISBN 978-1-80324-676-5
www.packtpub.com
Contributors
About t he aut hor
Steven F. Lott has been programming since computers were large, expensive, and rare. Working for
decades in high tech has given him exposure to a lot of ideas and techniques; some are bad, but most
are useful and helpful to others.
Steven has been working with Python since the ‘90s, building a variety of tools and applications. He’s
written a number of titles for Packt Publishing, including Mastering Object-Oriented Python, Modern
Python Cookbook, and Functional Python Programming.
He’s a tech nomad and lives on a boat that’s usually located on the east coast of the US. He tries to live
by the words, ”Don’t come home until you have a story.”
About t he r eviewer
Chris Griffith is a Lead Software Engineer with twelve years of experience with Python. His open-
source Python projects have been downloaded over a million times, and he is the primary writer for the
Code Calamity blog. Chris enjoys studio photography in his free time as well as digitizing vintage
magazines and 8mm films.
Join our communit y Discor d space
Join our Python Discord workspace to discuss and learn more about the book:
https://packt.link/dHrHU
Table of Contents
Project Zero: A Template for Other Projects
1.1 On quality
1.2.1 Inception
1.2.4 Construction
1.2.5 Transition
1.5.1 Description
1.5.2 Approach
1.5.3 Deliverables
1.6 Summary
1.7 Extras
1.7.1 Static analysis - mypy, flake8
1.7.3 Logging
1.7.4 Cookiecutter
2.3 Inspection
2.8 Summary
3.1 Description
3.3 Deliverables
3.4 Summary
3.5 Extras
4.1.1 Description
4.1.2 Approach
4.1.3 Deliverables
4.2.1 Description
They stayd not there, but streight way in did pas, xxxvi
Whom when the Hermite present saw in place,
From his deuotion streight he troubled was;
Which breaking off[484] he toward them did pace,
With stayed steps, and graue beseeming grace:
For well it seem’d, that whilome he had beene
Some[485] goodly person, and of gentle race,
That could his good to all, and well did weene,
How each to entertaine with curt’sie well beseene.
FOOTNOTES:
[474] xl 9 vnderstand; 1596
[475] Arg. 1 Matilda] Serena corr. Hughes rightly
[476] i 2 bewrapt 1596
[477] v 4 launc’t 1609 passim
[478] xi 7 requre 1596
[479] xiii 2 en’mies 1609
[480] xxiii 9 bene] bin 1609
[481] xxviii 2 liues 1609
[482] 9 Sith 1609
[483] xxxiv 9 neighbourhoood 1596
[484] xxxvi 4 off] of 1596
[485] 7 Soome 1596
[486] xxxix 3 gree 1609
[487] 7 Squire 1596
[488] xli 2 there] their 1596
Cant. VI.
Yet were her words and lookes but false and fayned, xlii
To some hid end to make more easie way,
Or to allure such fondlings, whom she trayned
Into her trap vnto their owne decay:
Thereto, when needed, she could weepe and pray,
And when her listed, she could fawne and flatter;
Now smyling smoothly, like to sommers day,
Now glooming sadly, so to cloke her matter;
Yet were her words but wynd, and all her teares but water.
For all that night, the whyles the Prince did rest xliv
In carelesse couch, not weeting what was ment,
He watcht in close awayt with weapons prest,
Willing to worke his villenous intent
On him, that had so shamefully him shent:
Yet durst he not for very cowardize
Effect the same, whylest all the night was spent.
The morrow next the Prince did early rize,
And passed forth, to follow his first enterprize.
FOOTNOTES:
[489] vi 5 faire 1596
[490] xii 9 secret 1609 passim
[491] xvi 1 the] th’ 1596
[492] xvii 7 Calidore] Calepine corr. Hughes rightly
[493] xx 8 fight, 1596, 1609
[494] xxvi 6 Knight, 1596, 1609
[495] xxx 7 gound 1596
[496] xxxv 6 fight] right 1596
[497] xxxvi 1 since] sith 1609
[498] thy] this 1609
Cant. VII.
But that same knight would not once let him start, xxi
But plainely gan to him declare the case
Of all his mischiefe, and late lucklesse smart;
How both he and his fellow there in place
Were vanquished, and put to foule disgrace,
And how that he in lieu of life him lent,
Had vow’d vnto the victor, him to trace
And follow through the world, where so he went,
Till that he him deliuered to his punishment.
Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.
ebookmass.com