<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
	<channel>
		<title>Neuralnetwork on Alok Regmi</title>
		<link>https://alokregmi.com/tags/neuralnetwork/</link>
		<description>Recent content in Neuralnetwork on Alok Regmi</description>
		<generator>Hugo</generator>
		<language>en-us</language>
		
		
		
		
			<lastBuildDate>Sat, 02 May 2020 03:51:34 +0545</lastBuildDate>
		
			<atom:link href="https://alokregmi.com/tags/neuralnetwork/index.xml" rel="self" type="application/rss+xml" />
			<item>
				<title>ML Paper Questions</title>
				<link>https://alokregmi.com/ai/whattowriteabout/</link>
				<pubDate>Wed, 22 Apr 2020 00:00:00 +0000</pubDate>
				<guid>https://alokregmi.com/ai/whattowriteabout/</guid>
				<description>&lt;h2 id=&#34;problem-no-dot-1-understanding-what-pretraining-and-training-a-supervised-layer-over-it-changes-in-the-model&#34;&gt;Problem No. 1 : Understanding what pretraining and training a supervised layer over it changes in the model&lt;/h2&gt;&#xA;&lt;p&gt;Our ability to characterize exactly what aspects of the&#xA;pretrained parameters are retained during the supervised training stage is limited.&lt;/p&gt;&#xA;&lt;h2 id=&#34;problem-no-dot-2&#34;&gt;Problem No. 2 :&lt;/h2&gt;&#xA;&lt;p&gt;For example, if we train&#xA;a generative model of images of cars and motorcycles, it will need to know about&#xA;wheels, and about how many wheels should be in an image. If we are fortunate,&#xA;the representation of the wheels will take on a form that is easy for the supervised&#xA;learner to access. This is not yet understood at a mathematical, theoretical level,&#xA;so it is not always possible to predict which tasks will benefit from unsupervised&#xA;learning in this way.&lt;/p&gt;</description>
			</item>
			<item>
				<title>Efficient Nets</title>
				<link>https://alokregmi.com/braindump/2020/04/efficient-nets/</link>
				<pubDate>Sun, 19 Apr 2020 19:28:53 +0545</pubDate>
				<guid>https://alokregmi.com/braindump/2020/04/efficient-nets/</guid>
				<description>&lt;h2 id=&#34;efficient-nets&#34;&gt;Efficient Nets:&lt;/h2&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&#xA;&lt;p&gt;Why all this ?&#xA;&amp;ndash;  Scaling CNN’s only in one direction (eg depth only) will result in rapidly&#xA;deteriorating gains relative to the computational increase needed.&lt;/p&gt;&#xA;&lt;p&gt;&amp;ndash; ResNet 1000 isn’t much more accurate than ResNet152 for example,&#xA;as after 100 -150 layer’s gains rapidly drop off.&lt;/p&gt;&#xA;&lt;p&gt;&amp;ndash; Scaling depth, width and resolution all benifits quickly saturate so&#xA;not at all  possibility of sota.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&#xA;&lt;p&gt;We now know the problem so let&amp;rsquo;s do this instead:&#xA;&amp;ndash; In order to scale up efficiently, all dimensions of depth,&#xA;width and resolution have to be scaled together,&#xA;and there is an optimal balance for each dimension relative to the others.&#xA;VOILA!!!&lt;/p&gt;&#xA;&lt;p&gt;&amp;ndash;  after an extensive grid search derived the theoretically optimal formula of “compound scaling” using the following co-efficients:&lt;/p&gt;&#xA;&lt;p&gt;Depth = 1.20&lt;/p&gt;&#xA;&lt;p&gt;Width = 1.10&lt;/p&gt;&#xA;&lt;p&gt;Resolution = 1.15&lt;/p&gt;&#xA;&lt;p&gt;Only google can do this kind of computational  search !! :(&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&#xA;&lt;p&gt;By showing the existing heatmaps in different widths, depths and resolution,&#xA;they show that their model captures more real image like features than some&#xA;nonsense these NNs learn on the way to correct prediction.&#xA;At last!! We can at least have our brothers in image recognition who detects&#xA;images more naturally looking on the inside.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&#xA;&lt;p&gt;Now not just large GPUs can do it, my phone can load a model and use it properly.&#xA;Thanks to reduced number of parameters. Thanks to Gooooogogoogogogogogle&amp;rsquo;s computational&#xA;for having this grid search and the paper.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;</description>
			</item>
			<item>
				<title>Graph Neural Networks</title>
				<link>https://alokregmi.com/braindump/2020/04/graph-neural-networks/</link>
				<pubDate>Sun, 19 Apr 2020 19:24:20 +0545</pubDate>
				<guid>https://alokregmi.com/braindump/2020/04/graph-neural-networks/</guid>
				<description>&lt;h2 id=&#34;graph-neural-net-intro&#34;&gt;Graph Neural Net Intro&lt;/h2&gt;&#xA;&lt;p&gt;&amp;ndash; Information about the non euclidean space incorporated&#xA;&amp;ndash; 2D conversion of image as input vs 3D image as input&#xA;&amp;ndash; Ideas about no connectivity in cases of pixels&#xA;&amp;ndash; Atoms, molecules, SMILE notation, what it leaves behind is the actual structural information&#xA;&amp;ndash; Nodes, edges, labels, attributes, rule(function that gives another node from one node)&#xA;&amp;ndash; Static and dynamic graphs will have different ways&#xA;&amp;ndash; Graph embedding: process of representing graphs vectorically with lower dimensions with as&#xA;low information loss as possible&#xA;&amp;ndash; Adjacency Matrix are mostly used.&#xA;&amp;ndash; Laplcian matrix = Degree - adjacency matrix&#xA;&amp;ndash; Generally all three types of matrices are used in many of the approaches.&#xA;&amp;ndash; Degree Matrix are diagonal matrices with the value of degree along the diagonal.&#xA;&amp;ndash; In case of undirected graphs, the no of connections is the degree.&#xA;&amp;ndash; In case of directed graphs, degree is either  out degree or in degree&#xA;&amp;ndash; In case of weighted graphs, it is the sum of weights on all nodes&#xA;&amp;ndash; Node level, Sub graph level embeddings&lt;/p&gt;</description>
			</item>
			<item>
				<title>Markov Nets</title>
				<link>https://alokregmi.com/braindump/2020/04/markov-nets/</link>
				<pubDate>Sun, 19 Apr 2020 16:00:44 +0545</pubDate>
				<guid>https://alokregmi.com/braindump/2020/04/markov-nets/</guid>
				<description>&lt;h2 id=&#34;markov-networks&#34;&gt;Markov Networks&lt;/h2&gt;&#xA;&lt;p&gt;We study this by comparing to Bayesian networks:&#xA;Burglar, Earthquake, Alarm, He calls, She calls&lt;/p&gt;&#xA;&lt;p&gt;&amp;ndash; First, the values do not need to sum to one, that is,&#xA;the table does not define a probability distribution.&#xA;It only tells us that configurations with higher values are more likely.&#xA;Second, there is no conditioning. It is proportional to the joint distribution&#xA;of all the variables involved,&#xA;as opposed to conditional distributions in CPDs.&lt;/p&gt;</description>
			</item>
			<item>
				<title>RCNN</title>
				<link>https://alokregmi.com/braindump/2020/04/rcnn/</link>
				<pubDate>Sun, 19 Apr 2020 16:00:40 +0545</pubDate>
				<guid>https://alokregmi.com/braindump/2020/04/rcnn/</guid>
				<description>&lt;p&gt;Faster RCNN&lt;/p&gt;&#xA;&lt;p&gt;Region Proposal:&lt;/p&gt;&#xA;&lt;p&gt;Anchors: Different size bounding box of different aspect ratio and sizes&lt;/p&gt;&#xA;&lt;p&gt;Task to be done by RPN:&lt;/p&gt;&#xA;&lt;p&gt;Does this anchor contain a relevant object?&lt;/p&gt;&#xA;&lt;p&gt;How would we adjust this anchor to better fit the relevant object?&lt;/p&gt;&#xA;&lt;p&gt;It’s important to understand that even though anchors are defined based on the convolutional feature map, the final anchors reference the original image.&lt;/p&gt;&#xA;&lt;p&gt;Mathematically, if the image was w×hw × hw×h, the feature map will end up w/r×h/rw/r × h/rw/r×h/r where rrr is called subsampling ratio.&lt;/p&gt;</description>
			</item>
			<item>
				<title>Graph Neural Nets</title>
				<link>https://alokregmi.com/braindump/2020/04/graph-neural-nets/</link>
				<pubDate>Sun, 19 Apr 2020 16:00:36 +0545</pubDate>
				<guid>https://alokregmi.com/braindump/2020/04/graph-neural-nets/</guid>
				<description>&lt;p&gt;&amp;ndash; Information about the non euclidean space incorporated&#xA;&amp;ndash; 2D conversion of image as input vs 3D image as input&#xA;&amp;ndash; Ideas about no connectivity in cases of pixels&#xA;&amp;ndash; Atoms, molecules, SMILE notation, what it leaves behind is the actual structural information&#xA;&amp;ndash; Nodes, edges, labels, attributes, rule(function that gives another node from one node)&#xA;&amp;ndash; Static and dynamic graphs will have different ways&#xA;&amp;ndash; Graph embedding: process of representing graphs vectorically with lower dimensions with as&#xA;low information loss as possible&#xA;&amp;ndash; Adjacency Matrix are mostly used.&#xA;&amp;ndash; Laplcian matrix = Degree - adjacency matrix&#xA;&amp;ndash; Generally all three types of matrices are used in many of the approaches.&#xA;&amp;ndash; Degree Matrix are diagonal matrices with the value of degree along the diagonal.&#xA;&amp;ndash; In case of undirected graphs, the no of connections is the degree.&#xA;&amp;ndash; In case of directed graphs, degree is either  out degree or in degree&#xA;&amp;ndash; In case of weighted graphs, it is the sum of weights on all nodes&#xA;&amp;ndash; Node level, Sub graph level embeddings&lt;/p&gt;</description>
			</item>
			<item>
				<title>NN Interpretability</title>
				<link>https://alokregmi.com/braindump/2020/04/nn-interpretability/</link>
				<pubDate>Sun, 19 Apr 2020 16:00:20 +0545</pubDate>
				<guid>https://alokregmi.com/braindump/2020/04/nn-interpretability/</guid>
				<description>&lt;p&gt;This criteria distinguishes Whether Interpret-ability is achieved by restricting the complexity of the machine learning model (intrinsic)&#xA;or by applying methods that analyze the model after training (post hoc)&lt;/p&gt;&#xA;&lt;p&gt;Permutation feature importance is, for example, a post hoc interpretation method&lt;/p&gt;&#xA;&lt;p&gt;Contrastive explanation: explaining only the abnormalities&#xA;Rather than explaining all  features, explaining only 3 or 4.&#xA;Social context &amp;ndash;&amp;gt; who are you going to give answers to&lt;/p&gt;&#xA;&lt;p&gt;Whether explanation is required or not is also an imp question.&lt;/p&gt;</description>
			</item>
			<item>
				<title>ML Paper Questions</title>
				<link>https://alokregmi.com/braindump/2020/04/ml-paper-questions/</link>
				<pubDate>Sun, 19 Apr 2020 16:00:15 +0545</pubDate>
				<guid>https://alokregmi.com/braindump/2020/04/ml-paper-questions/</guid>
				<description>&lt;p&gt;Problem No. 1 : Understanding what pretraining and training a supervised layer over it changes in the model&lt;/p&gt;&#xA;&lt;p&gt;Our ability to characterize exactly what aspects of the&#xA;pretrained parameters are retained during the supervised training stage is limited.&lt;/p&gt;&#xA;&lt;p&gt;Problem No. 2 :&lt;/p&gt;&#xA;&lt;p&gt;For example, if we train&#xA;a generative model of images of cars and motorcycles, it will need to know about&#xA;wheels, and about how many wheels should be in an image. If we are fortunate,&#xA;the representation of the wheels will take on a form that is easy for the supervised&#xA;learner to access. This is not yet understood at a mathematical, theoretical level,&#xA;so it is not always possible to predict which tasks will benefit from unsupervised&#xA;learning in this way.&lt;/p&gt;</description>
			</item>
			<item>
				<title>Text-Text Transformers</title>
				<link>https://alokregmi.com/braindump/2020/04/text-text-transformers/</link>
				<pubDate>Sun, 19 Apr 2020 15:59:29 +0545</pubDate>
				<guid>https://alokregmi.com/braindump/2020/04/text-text-transformers/</guid>
				<description>&lt;p&gt;Catches from text to text transfer transformers:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;As an example, consider the case of English to German translation: If we have a training&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;datapoint with input sentence “That is good.” and target “Das ist gut.”, we would simply train the&#xA;model on next-step prediction over the concatenated input sequence “translate English to German:&#xA;That is good. target: Das ist gut.” If we wanted to obtain the model’s prediction for this example,&#xA;the model would be fed the prefix “translate English to German: That is good. target:” and would&#xA;be asked to generate the remainder of the sequence autoregressively.&lt;/p&gt;</description>
			</item>
			<item>
				<title>MUZero</title>
				<link>https://alokregmi.com/braindump/2020/04/muzero/</link>
				<pubDate>Sun, 19 Apr 2020 15:59:20 +0545</pubDate>
				<guid>https://alokregmi.com/braindump/2020/04/muzero/</guid>
				<description>&lt;p&gt;MuZero:&lt;/p&gt;&#xA;&lt;p&gt;&amp;ndash; Works based on AlphaZero&amp;rsquo;s search and planning space&#xA;&amp;ndash; Learns optimal policy, reward functions, value functions automatically&#xA;&amp;ndash; The main idea of the algorithm is to predict those aspects of the future that are directly&#xA;relevant for planning.&#xA;&amp;ndash; Observation image ===&amp;gt; Model ==&amp;gt; Encodes to hidden states ==&amp;gt; Hidden states updated using&#xA;previous step&amp;rsquo;s hidden state and hypothetical next action&#xA;&amp;ndash; In each step, model predicts value fn, optimal policy, and immediate reward&lt;/p&gt;</description>
			</item>
	</channel>
</rss>
