Deep Machine Learning – Yair Shinar

Building a NN from scratch in Python

On: November 10, 2019

In: AI, Autonomous Vehicles, Deep Machine Learning

Start by building a very simple network: import numpy as np class NeuralNetwork: def __init__(self, x,y): self.input = x self.y = y self.Weights1 = np.random.randn(self.input.shape[1],5) self.Weights2 = np.random.randn(5,1) self.output = np.zeros(self.y.shape) def sigmoid_z(self,x): #create a sigmoid function z = 1/(1 + np.exp(-x)) return z def sigmoid_z_derivative(self,x): return self.sigmoid_z(x)*(1-self.sigmoid_z(x)) def forwardpropogation(self):Continue Reading

Tensorflow Quantizaiton

On: October 8, 2019

In: AI, Computer Vision, Deep Machine Learning, NLP, Reinforcement Learning

During inference, precision in floats is not needed and can be reduced to using 8 bits instead of 32 bits this allows to bin continuous values to discrete ranges, and therefore is known as Quantization. This enables us to increase the bandwith sent (or increase the memory footprint in caseContinue Reading

Tensorflow Graph Optimizing

On: October 6, 2019

In: AI, Computer Vision, Deep Machine Learning, NLP, Reinforcement Learning

After training we can optimize a frozen graph or even a dynamic graph by removes training-specific and debug-specific nodes, fusing common operations, and removes code that isn’t used/reached. Code Example from tensorflow.python.tools import optimize_for_inference_lib inputGraph = tf.GraphDef() #read in a frozen model with tf.gfile.Open(‘frozentensorflowModel.pb’, “rb”) as f: data2read = f.read() inputGraph.ParseFromString(data2read) outputGraph = optimize_for_inference_lib.optimize_for_inference(inputGraph, [“inputTensor”], Continue Reading

Tensorflow – Graph Freezing

On: October 2, 2019

In: AI, Computer Vision, Deep Machine Learning, NLP, Reinforcement Learning

In prediction/inference mode, variable types are unnecessary, so by freezing the graph we convert all variables in a graph and checkpoint into constants. Also there are structure nodes and operations related to training that we don’t need and can be removed by the freezing process as well. The freezing processContinue Reading

Deep Q-learning(DQN) – Experience Replay

On: July 19, 2019

In: AI, Deep Machine Learning, Reinforcement Learning

To perform experience replay we store the agent’s experiences et=(st,at,rt,st+1) Then we use a random sample of these prior actions instead of the most recent action to proceed. * This removes correlations in the observation sequence and smooths changes in the data distribution. * Iterative update adjusts Q towards targetContinue Reading

Q-learning – Model-free reinforcement learning algorithm

On: July 6, 2019

In: AI, Algorithms, Deep Machine Learning, Reinforcement Learning

Learns a policy which tells an agent what action to take under what circumstances. Q-learning learns a policy that is optimal in the sense that it maximizes the expected value of the total reward over any and all successive steps, starting from the current state.Continue Reading

Markov decision process (MDP) – Reinforcement Learning decision model

On: July 6, 2019

In: AI, Deep Machine Learning, Reinforcement Learning

* Is a discrete time stochastic control process for decision making in situations where outcomes are partly random and partly under the control of a decision maker. * At each discrete time step, the process is in some state s, and the decision maker may choose any action a thatContinue Reading

Variation AutoEncoders probability distribution in hidden space

On: July 5, 2019

In: AI, Computer Vision, Deep Machine Learning, NLP, Reinforcement Learning

A variational autoencoder provides a probability distribution for describing an observation/attribute in latent/hidden space.Continue Reading

Probability Chain rule/general product rule using only conditional probabilities

On: July 5, 2019

In: AI, Deep Machine Learning

The chain rule/general product rule permits the calculation of any member of the joint distribution of a set of random variables using only conditional probabilities.Continue Reading

Reinforcement Learning – The balance between exploration and exploitation

On: July 5, 2019

In: AI, Deep Machine Learning, Reinforcement Learning

* How software agents ought to take actions in an environment so as to maximize some notion of cumulative reward. * The focus is finding a balance between exploration (of uncharted territory) and exploitation (of current knowledge). * The environment is typically formulated as a Markov decision process (MDP) utilizingContinue Reading