Deep Learning Projects Using TensorFlow 2: Neural Network Development with Python and Keras

Table of Contents
About the Author
About the Technical Reviewer
Acknowledgments
Preface
Chapter 1: Getting Started: Installation and Troubleshooting
	Installing Python 3
		Method 1: Direct Installation from the Official Python Website
		Troubleshooting Tips
		Method 2: Using Anaconda
		Troubleshooting Tips
	Installing Jupyter Notebook
		Dependencies
		Method 1: Using the PIP Installation Package
		Troubleshooting Tips
		Method 2: Using Anaconda
		Troubleshooting Tips
	Installing TensorFlow 2.0
		Dependencies
		Method 1: Using the PIP Installation Package
		Troubleshooting Tips
		Method 2: Using Anaconda
		Troubleshooting Tips
	Installing Keras
		Dependencies
		Using the PIP Installation Package
		Troubleshooting Tips
	Installing Python Libraries
		Installing NumPy
			Dependencies
			Using the PIP Installation Package
			Troubleshooting Tips
		Installing SciPy
			Dependencies
			Using the PIP Installation Package
			Troubleshooting Tips
		Installing Matplotlib
			Dependencies
			Using the PIP Installation Package
			Troubleshooting Tips
		Installing Pandas
			Dependencies
			Using the PIP Installation Package
			Troubleshooting Tips
		Installing Scikit-Learn
			Dependencies
			Using the PIP Installation Package
			Troubleshooting Tips
	Summary
Chapter 2: Perceptrons
	Biological Neurons
	Artificial Neurons
	Perceptrons
		Perceptron Learning Rule
		Types of Activation Functions
		The Sigmoid Activation Function
		The ReLU Function
		The Softmax Function
	Perceptrons in Action
		Stage 1: Forward Propagation of Inputs
		Stage 2: Calculation of the Net Input
			Weights
			Bias
			Net Input
		Stage 3: Activation Function
		Stage 4: Backward Propagation
		Project Description
		Important Terminology
		Required Libraries
		Procedure
		Step 1. Import Libraries
		Step 2. Declare Parameters
		Step 3. Declare the Weights and Bias
		Step 4. Define the Perceptron Function
		Step 5. Define the Loss Function and Optimizer
		Step 6. Read in the Data
		Step 7. Visualization of Labels
		Step 8. Prepare Inputs
		Step 9. Initialize Variables
		Step 10. Train the Model
		Step 11. New Values for Weights and Bias
		Step 12. View the Final Loss
		Step 13. Predicting Using the Trained Model
		Step 14. Evaluate the Model
	Summary
Chapter 3: Neural Networks
	What Is a Neural Network?
	Neural Network Components
	Advantages of Neural Networks
	Disadvantages of a Neural Networks
	How a Neural Network Works
	Forward Propagation
	Backward Propagation
	Types of Neural Networks
	Feedforward Neural Network
	Convolutional Neural Networks
	Recurrent Neural Network (RNN)
	Radial Basis Function Neural Network (RBNN)
	Project Description
	Flattening Data
	About the Dataset
	Required Libraries
	Neural Network Architecture
	Procedure
	Summary
	References
Chapter 4: Sentiment Analysis
	LSTM Review
	How an LSTM Works
		Layers in an LSTM
	Project Description
		About the Dataset
		Understanding Sentiment Analysis
			Types of Sentiment Analysis
				Fine-Grained Sentiment Analysis
				Emotion Detection
				Aspect-Based Sentiment Analysis
				Intent Analysis
				Multilingual Sentiment Analysis
			Sentiment Analysis Algorithms
			Sentiment Analysis Metrics for Evaluation
				Cross-Validation
				Precision
				Recall
				Accuracy
				Hybrid Approaches
			Parameters Affecting Model Performance
				Subjectivity and Tone
				Context and Polarity
				Irony and Sarcasm
				Comparisons
				Defining Neutral Emotions
				Tokenizer
				H5 File
				JSON File
	Required Libraries
	LSTM Architecture
	Procedure
		Step 1. Import Libraries
		Step 2. Load the Data
		Step 3. Prepare the Data
		Step 4. Clean the Data
		Step 5. Structure the Model
		Step 6. Compile the Model
		Step 7. Train the Model
		Step 8. Save the Model (Optional)
		Step 9. Import the Pretrained Model (Optional)
	Further Tests
	Troubleshooting
	Summary
	References
	Further Reading
Chapter 5: Music Generation
	GRU Overview
	How a GRU Works
		GRU Stages
			Stage 1
			Stage 2
			Stage 3
			Stage 4
		GRU Layers
		Comparing GRU and LSTM
	Project Description
		About the Dataset
		Important Terminology and Concepts
		Required Libraries
	Installation Instructions
		Using PIP
		Using Windows
		Using macOS
		Using Linux
		Installation Troubleshooting
	GRU Architecture
	Procedure
		Step 1. Import Libraries
		Step 2. Load the Data
		Step 3. Feature Extraction
		Step 4. Exploratory Data Analysis (EDA)
		Step 5. Data Preparation (Input)
		Step 6. Structure the Model
		Step 7. Train the Model
		Step 8. Prediction
		Step 9. Data Preparation (Offset)
		Step 10. Store the Output as a MIDI File
	Further Tests
	Troubleshooting
	Summary
	References
	Resources
	Further Reading
Chapter 6: Image Colorization
	Human Vision Review
		Computer Vision Review
	How a CNN Works
		Input Layer
		Convolution Layer: The Kernel
		Upsampling Layer
		DepthwiseConv2D
		Pooling Layer
		Fully Connected Layer
	Project Description
		About the Dataset
		Important Terminology
			Color Space
			Image Colorization
			VGG-16
			MAPE Loss Functions
		Required Libraries
		Installation Instructions
			Installing PIL
			Troubleshooting PIL
			Installing CV2
			Troubleshooting CV2
			Installing Scikit-Image
			Troubleshooting Scikit-Image
		CNN+VGG-16 Architecture
	Procedure
		Step 1. Import the Libraries
		Step 2. Convert the Images to Grayscale
		Step 3. Load the Data
		Step 4. Structure the Model
		Step 5. Set the Model Parameters
		Step 6. Data Preparation
		Step 7. Train the Model
		Step 8. Obtain Predictions
		Step 9. View the Results
	Troubleshooting
	Further Tests
	Summary
	References
	Further Reading
Chapter 7: Image Deblurring
	What Is a GAN?
		Types of GANs
	How a GAN Works
		The Generative Model
		Process Within the Generator
		The Discriminator Model
		Process Within the Discriminator
	Project Description
		About the Dataset
		Important Terminology and Concepts
			Image Deblurring
			Defocusing
			Motion Smudging
			Convolution
			Deconvolution
		GAN Architecture
		Required Libraries
	GAN Architecture
		Generator
		Discriminator
	Procedure
		Step 1. Import the Libraries
		Step 2. Dataset Preparation
		Step 3. Exploratory Data Analysis
		Step 4. Structure the Model
		Step 5. Input Preparation
		Step 6. View the Images
		Step 7. Save Results
	Troubleshooting
	Further Tests
	Summary
	References
	Further Reading
Chapter 8: Image Manipulation
	Project Description
		Important Terminology and Concepts
			Multimedia Forensics
				Acquisition
				Coding
				Editing
				Saving
		Copy-Move Forgeries
		About the Dataset
		Required Libraries
		Troubleshooting
	CNN Architecture
	Procedure
		Step 1. Import the Libraries
		Step 2. Preparing the Dataset
			Step 2a. Sort and Collect the Authentic Data
			Step 2b. Sort and Collect the Manipulated Data
			Step 2c. Transform and Convert the Data to an Array
			Step 2d. Create the Combined Dataset
			Step 2e. Define the Optimizer
		Step 3. Structure the Model
		Step 4. Train the Model
		Step 5. Test the Model
		Step 6. Check the Results
	Further Tests
	Summary
	References
	Further Reading
Chapter 9: Neural Network Collection
	Neural Network Zoo Primer
	Neural Networks
		Recurrent Neural Networks (RNNs)
			Reservoir Computing
		Multiplicative LSTM
		ANNs with Attention
		Transformers
		Autoencoder
		Variational Autoencoders
		Denoising Autoencoders
		Recurrent Autoencoders
		Sparse Autoencoders
		Stacked Autoencoders
		Convolutional Autoencoders
		Stacked Denoising Autoencoders
			Stochastic Corruption in SDAs
		Contractive Autoencoders
		Markov Chains
		Hopfield Networks
			How Human Memory Works
		Bidirectional Associative Memory
		Boltzmann Machines
		Restricted Boltzmann Machines
		Deep Belief Networks
		Deconvolutional Networks
		Deep Convolutional Inverse Graphics Networks
		Liquid State Machines
			Human Brain Spiking
		Echo State Networks (ESNs)
		Deep Residual Network (ResNet)
		ResNeXt
		Neural Turing Machines
			Reading
		Capsule Networks
			CAPSNet Architecture
		LeNet-5
		AlexNet
		GoogLeNet
		Xception
	Optimizers
		Stochastic Gradient Descent
		RMSProp
		AdaGrad
		AdaDelta
		Adam
		Adamax
		Nesterov Accelerated Gradient (NAG)
		Nadam
	Loss Functions
		Mean Squared Error (MSE)
		Mean Absolute Error (MAE)
		Mean Absolute Percentage Error (MAPE)
		Mean Squared Logarithmic Error (MSLE)
		Squared Hinge
		Hinge
		Categorical Hinge
		Log Cosh
		Huber Loss
		Categorical Cross-Entropy
		Sparse Categorical Cross-Entropy
		Binary Cross-Entropy
		Kullback-Leibler Divergence
		Poisson
	References
	Further Reading
Appendix: Portfolio Tips
	Data Analyst Portfolios
		LinkedIn Profile
		GitHub Profile
		Kaggle
		Blogging Platforms
	Sharing Your Portfolio
		Twitter
		Facebook
		LinkedIn Groups
		Tableau Public (Optional)
	Types of Projects
		Data Cleaning Project
		Data Storytelling Project
		Explanatory Post
		Projects to Avoid
		Selecting a Topic
	Defining Problem Statements
		Phase 1: Understanding the Goals and Expectations
		Phase 2: Translating the Goals to Data Analysis Goals
		Phase 3: Framing the Problem Statement
		Phase 4: Choosing a Success Metric
	Using Design Thinking
		Benefits of Design Thinking
		Phase 1: Planning
		Phase 2: Model Design
		Phase 3: Prototyping
		Phase 4: Black-Box Testing
			Black-Box Testing Techniques for Machine Learning Models
	Solution Implementation
		Phase 1: Data Collection
		Phase 2: Data Exploration
			Why EDA?
			Univariate EDA
			Multivariate EDA
		Phase 3: Data Handling
		Phase 4: Data Mining
		Underfitting
			Overcoming Underfitting
		Overfitting
			Overcoming Overfitting
		Phase 5: Prototyping
		Phase 6. Storytelling
		What Is Color Theory?
	Maintenance
		Uploading Your Project to GitHub
	Tips for Documenting Projects
	Appendix Checklist
	References
	Further Reading
	Resources for Building Your Portfolio
	Read.me Template
		Project Title
		Problem Statement
		Road Map Template
	Data Cleaning Checklist
Index