1 – PYTHON FOR DATA SCIENCE
- Using modules
- Listing the methods in a module
- Creating your own modules
- List comprehension
- Dictionary comprehension
- String comprehension
- Python 2 vs. Python 3
- Sets (Python 3+)
- Python idioms
- The Python data science “ecosystem”
- NumPy
- NumPy arrays
- NumPy idioms
- pandas
- Data wrangling with the pandas DataFrame
- SciPy
- Scikit-learn
- SciPy or scikit-learn?
- Matplotlib
- Python vs. R
- Python on Apache Spark
- Python development tools and REPLs
- Anaconda
- IPython
- Visual Studio Code
- Jupyter
- Basic Jupyter commands
- Summary
2 – APPLIED DATA SCIENCE
- What is data science?
- Data science ecosystem
- Data mining vs. data science
- Business analytics vs. data science
- Data science, machine learning, AI?
- Who is a data scientist?
- Venn diagram of data science competencies
- Data scientists at work
- Examples of data science projects
- An example of a data product
- Applied data science at Google
- Pitfalls in data science
- Summary
3 – PHASES OF THE DATA ANALYTICS LIFECYCLE
- Big data analytics pipeline
- Data discovery phase
- Data harvesting phase
- Data priming phase
- Data logistics and data governance
- Exploratory data analysis
- Model planning phase
- Model building phase
- Communicating the results
- Production rollout
- Summary
4 – FIXING AND NORMALISING DATA
- Fixing and normalising data
- Dealing with missing data
- Sample dataset
- Retrieving information about null data
- Removing a column
- Interpolating missing data in pandas
- Replacing missing values with the mean
- Scaling (normalising) the data
- Data preprocessing with scikit-learn
- Scaling with the scale() function
- The MinMaxScaler object
- Summary
5 – CALCULATING DESCRIPTIVE STATISTICS IN PYTHON
- Descriptive statistics
- Non-uniformity of a probability distribution
- Using NumPy to calculate descriptive statistical metrics
- Finding the minimum and maximum in NumPy
- Using pandas to calculate descriptive statistical metrics
- Correlation
- Regression and correlation
- Covariance
- Finding pairwise correlation and covariance metrics
- Finding the minimum and maximum in a pandas DataFrame
- Summary
6 – DATA AGGREGATION AND GROUPING
- Data aggregation and grouping
- Sample dataset
- The pandas.core.groupby.SeriesGroupBy object
- Grouping by two or more columns
- Replicating SQL’s WHERE clause
- Pivot tables
- Cross-tabulation
- Summary
7 – DATA VISUALISATION WITH MATPLOTLIB
- Data visualisation
- What is matplotlib?
- Getting started with matplotlib
- The plot window
- The Figure options
- The matplotlib.pyplot.plot() function
- The matplotlib.pyplot.bar() function
- The matplotlib.pyplot.pie() function
- Subplots
- Using the matplotlib.gridspec.GridSpec object
- The matplotlib.pyplot.subplot() function
- Hands-on exercise
- Figures
- Saving figures to a file
- Visualisation with pandas
- Working with matplotlib in Jupyter notebooks
- Summary
8 – DATA SCIENCE AND ML ALGORITHMS IN SCIKIT-LEARN
- Data science, machine learning, AI?
- Types of machine learning
- Terminology: features and observations
- Continuous and categorical features (variables)
- Terminology: axis
- The scikit-learn package
- scikit-learn estimators
- Models, estimators and predictors
- Common distance metrics
- The Euclidean metric
- The LIBSVM format
- Scaling the features
- The curse of dimensionality
- Supervised vs. unsupervised machine learning
- Algorithms for supervised machine learning
- Algorithms for unsupervised machine learning
- Choosing the right algorithm
- Machine learning development lifecycles
- Splitting data into training and test datasets
- Data splitting in scikit-learn
- Hands-on exercise
- Examples of classification
- Classifying with k-Nearest Neighbors (SL)
- k-Nearest Neighbors algorithm
- The error rate
- Hands-on exercise
- Dimensionality reduction
- The benefits of dimensionality reduction
- Principal component analysis (PCA)
- Hands-on exercise
- Data blending
- Decision trees (SL)
- Decision tree terminology
- Classification with decision trees in the context of information theory
- Definition of information entropy
- The Shannon entropy formula
- The simplified decision tree algorithm
- Using decision trees
- Random forests
- SVM
- Naive Bayes classifier (SL)
- The naive Bayesian probability model in brief
- Bayes’ formula
- Classifying documents with Naive Bayes
- Type of unsupervised learning: clustering
- Examples of clustering
- k-Means clustering (UL)
- k-Means clustering in brief
- Characteristics of k-Means
- Regression analysis
- Simple linear regression model
- Linear vs. non-linear regression
- Illustrating linear regression
- Key underlying assumptions of regression analysis
- Least squares method (LSM)
- Locally weighted linear regression
- Regression models in Excel
- Multiple regression analysis
- Logistic regression
- Regression vs. classification
- Time series analysis
- Decomposing time series
- Summary
9 – LAB EXERCISES
- Lab 1 – Getting to know the lab environment
- Lab 2 – Using Jupyter Notebook
- Lab 3 – Fixing and normalising data
- Lab 4 – Calculating descriptive statistics
- Lab 5 – Grouping and aggregating data
- Lab 6 – Data visualisation with matplotlib
- Lab 7 – Splitting data
- Lab 8 – k-Nearest Neighbors algorithm
- Lab 9 – The k-Means algorithm
- Lab 10 – The random forest algorithm
