Lu, Yukuan

Common problems of Shannon entropy in information theory.

Sources:

1. The spelled-out intro to neural networks and backpropagation: building micrograd
2. Stanford CS231N, Lecture 4

Other useful resources:

1. StatQuest's Neural Networks videos

2. Efficient backprop. You need do download it via

   1	wget http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf

The python script and jupyter notebook used in this article can be found at here.

This article is a step-by-step explanation of neural networks which are extensively used in machine learning. It only involves the most basic case where the input of a neuron is a vector (1-D tensor)and output of a neuron is a scalar (0-D tensor). But the idea holds for higher, please reter to Derivatives, Backpropagation, and Vectorization for details.

Ref:

1. EE 376A: Information Theory. Winter 2018. Lecture 6. - Stanford University
2. EE 376A: Information Theory. Winter 2017. Lecture 4. - Stanford University
3. Elements of Information Theory

Sources:

1. James McClellan, Ronald Schafer & Mark Yoder. (2015). FIR Filters. DSP First (2nd ed., pp. 147-193). Pearson.

Sources:

1. Jiames McClellan, Ronald Schafer & Mark Yoder. (2015). Sinusoids. DSP First (2nd ed., pp. 9-40). Pearson.

Sources:

1. James McClellan, Ronald Schafer & Mark Yoder. (2015). Sinusoids. DSP First (2nd ed., pp. 9-40). Pearson.

Sources:

1. World Models 2018 paper
   • Training doc
2. Mastering Diverse Domains through World Models, aka Dreamer V3.
3. A Generalist Agent.
4. Learning General World Models in a Handful of Reward-Free Deployments.
5. Learning General World Models in a Handful of Reward-Free Deployments.

1. Tree of Thoughts: Deliberate Problem Solving with Large Language Models
2. Self-Consistency Improves Chain of Thought Reasoning in Language Models

1. Paul's Online Notes - Calculus

1. EE 376A: Information Theory. Winter 2018. Lecture 4. - Stanford University
2. EE 376A: Information Theory. Winter 2017. Lecture 4. - Stanford University
3. Elements of Information Theory