APAD course’s content

This booklet support part of the course https://e-l.unifi.it/course/view.php?id=13262 about Algorithms for Programming and Analysis of Data, taught at the University of Florence.

It is based on https://jakevdp.github.io/PythonDataScienceHandbook/ and the content taken from there, both text and code, is reported unchanged to respect the original author’s license and I deserve no merits for it. The other stuff (mainly the introduction and the last two chapters) are of mine, so blame me for errors occurring there.

The content that we will present is versioned at https://github.com/massimo-nocentini/APAD-course; moreover, it can be read either on slides

Introduction Jupyter notebooks numpy

pandas matplotlib

networkx Gotchas

or on notebooks

Introduction Jupyter notebooks Core concepts

Exercises numpy pandas

matplotlib networkx Gotchas

or, eventually, on this booklet according to the TOC

• Introduction
  • print('Hello!')
  • past and present
  • beware of shadows
  • welcome to the carousel
• Jupyter notebooks
  • IPython: Beyond Normal Python
    • Shell or Notebook?
  • Help and Documentation in IPython
    • Accessing Documentation with ?
    • Accessing Source Code with ??
    • Exploring Modules with Tab-Completion
  • IPython Magic Commands
    • Running External Code: %run
    • Timing Code Execution: %timeit
    • Help on Magic Functions: ?, %magic, and %lsmagic
  • Input and Output History
    • IPython’s In and Out Objects
    • Underscore Shortcuts and Previous Outputs
    • Suppressing Output
    • Related Magic Commands
  • IPython and Shell Commands
    • Quick Introduction to the Shell
    • Shell Commands in IPython
    • Passing Values to and from the Shell
  • Errors and Debugging
    • Controlling Exceptions: %xmode
    • Debugging: When Reading Tracebacks Is Not Enough
  • Profiling and Timing Code
    • Timing Code Snippets: %timeit and %time
    • Profiling Full Scripts: %prun
    • Line-By-Line Profiling with %lprun
  • Line # Hits Time Per Hit % Time Line Contents
    • Profiling Memory Use: %memit and %mprun
  • Line # Mem usage Increment Line Contents
• Core concepts
  • Values and their types
  • Variables
  • Operators and operands
  • Slices
  • ranges
  • lambdas
  • yields
  • Help about help
• Exercises
  • Intersection
  • (Pythagorean) tuples
  • sum_upto
  • sqrt
  • \(\pi\)
  • The Collatz’s conjecture
  • Fibonacci numbers
  • Uniform random on segmented interval
  • Bernoulli random variable
  • Russian Peasant Multiplication
  • Fixed sum
  • Some strange uses of recursion
    • Constants
    • Nats
    • Primes
    • Fibonacci, again
• numpy module
  • Introduction to NumPy
  • Understanding Data Types in Python
    • Creating Arrays from Python Lists
    • Creating Arrays from Scratch
    • NumPy Standard Data Types
    • INTERMEZZO
  • The Basics of NumPy Arrays
    • NumPy Array Attributes
    • Array Indexing: Accessing Single Elements
    • Array Slicing: Accessing Subarrays
    • Reshaping of Arrays
  • Computation on NumPy Arrays: Universal Functions
    • The Slowness of Loops
    • Introducing UFuncs
  • Aggregations: Min, Max, and Everything In Between
    • Summing the Values in an Array
    • Minimum and Maximum
  • Computation on Arrays: Broadcasting
    • Introducing Broadcasting
    • Rules of Broadcasting
  • Comparisons, Masks, and Boolean Logic
    • Comparison Operators as ufuncs
    • Boolean Arrays as Masks
  • Fancy Indexing
    • Combined Indexing
    • Example: Selecting Random Points
    • Modifying Values with Fancy Indexing
    • Example: Binning Data
  • Sorting Arrays
    • Fast Sorting in NumPy: np.sort and np.argsort
    • Partial Sorts: Partitioning
    • Example: k-Nearest Neighbors
• pandas module
  • Introducing Pandas Objects
    • The Pandas Series Object
    • The Pandas DataFrame Object
    • The Pandas Index Object
  • Data Indexing and Selection
    • Data Selection in Series
    • Data Selection in DataFrame
  • Operating on Data in Pandas
    • Ufuncs: Index Preservation
    • UFuncs: Index Alignment
    • Ufuncs: Operations Between DataFrame and Series
  • Handling Missing Data
    • Trade-Offs in Missing Data Conventions
    • Missing Data in Pandas
    • Operating on Null Values
  • Hierarchical Indexing
    • A Multiply Indexed Series
    • Methods of MultiIndex Creation
    • Indexing and Slicing a MultiIndex
    • Rearranging Multi-Indices
    • Data Aggregations on Multi-Indices
  • Combining Datasets: Concat and Append
    • Simple Concatenation with pd.concat
  • Combining Datasets: Merge and Join
    • Relational Algebra
    • Categories of Joins
    • Specification of the Merge Key
    • Specifying Set Arithmetic for Joins
    • Overlapping Column Names: The suffixes Keyword
    • Example: US States Data
  • Aggregation and Grouping
    • Planets Data
    • Simple Aggregation in Pandas
    • GroupBy: Split, Apply, Combine
  • Pivot Tables
    • Motivating Pivot Tables
    • Pivot Tables by Hand
    • Pivot Table Syntax
    • Example: Birthrate Data
  • Working with Time Series
    • Dates and Times in Python
    • Pandas Time Series: Indexing by Time
    • Pandas Time Series Data Structures
    • Frequencies and Offsets
    • Resampling, Shifting, and Windowing
    • Example: Visualizing Seattle Bicycle Counts
• matplotlib module
  • Visualization with Matplotlib
    • Importing Matplotlib
    • Setting Styles
    • Saving Figures to File
  • Simple Line Plots
  • Simple Scatter Plots
  • Visualizing Errors
  • Density and Contour Plots
  • Histograms, Binnings, and Density
    • plt.hist2d: Two-dimensional histogram
    • plt.hexbin: Hexagonal binnings
    • Kernel density estimation
  • Customizing Plot Legends
  • Customizing Colorbars
    • Choosing the Colormap
    • Color limits and extensions
    • Discrete Color Bars
  • Multiple Subplots
  • Text and Annotation
  • Customizing Ticks
  • Customizing Matplotlib: Configurations and Stylesheets
    • Default style
    • FiveThiryEight style
    • ggplot
    • Bayesian Methods for Hackers style
    • Seaborn style
  • Three-Dimensional Plotting in Matplotlib
    • Example: Visualizing a Möbius strip
  • Visualization with Seaborn
    • Histograms, KDE, and densities
    • Pair plots
    • Faceted histograms
    • Factor plots
    • Joint distributions
    • Bar plots
• networkx module
  • NetworkX
    • Creating a graph
  • Graph attributes
• Tips an’ Tricks
  • Tips, tricks and gotchas
    • A grouping pattern, avoiding quadratic time
    • The Bunch pattern
    • Python’s list.append isn’t Lisp’s cons
    • Hidden squares: concerning lists and sets
  • Counting
    • Max permutation
    • Counting Sort
• Object Orientation