陆昱宽

In the previous post, we introduced TD learning algorithms. At that time, all state/action values were represented by tables. This is inefficient for handling large state or action spaces.

In this post, we will use the function approximation method for TD learning. It is also where artificial neural networks are incorporated into reinforcement learning as function approximators.

Sources:

1. Shiyu Zhao. Chapter 8: Value Function Approximation. Mathematical Foundations of Reinforcement Learning.

This post introduces TD learning--the kind of value function learning that leverages a bootstrapped target value, called TD target.

Sources:

1. Shiyu Zhao. Chapter 7: Temporal-Difference Methods. Mathematical Foundations of Reinforcement Learning.
2. Modern world model agents that adopts TD-learning:
   1. DreamerV3 paper
   2. DIAMOND paper
   3. TD-MPC paper

Sources:

1. B. P. Lathi & Roger Green. (2018). Chapter 7: Continuous-Time Signal Analysis: The Fourier Transform. Signal Processing and Linear Systems (3nd ed., pp. 701-720). Oxford University Press.

For a quick reference table, see Wikipedia page on the Fourier transform.

For more results, refer to Table of the Laplace Transform Pairs.

Sources:

1. B. P. Lathi & Roger Green. (2018). Chapter 1: Signals and Systems. Signal Processing and Linear Systems (3rd ed., pp. 97-107). Oxford University Press.

Sources:

B. P. Lathi & Roger Green. (2018). Chapter 7: Continuous-Time Signal Analysis. Signal Processing and Linear Systems (3nd ed., pp. 700-701). Oxford University Press.

Sources:

B. P. Lathi & Roger Green. (2018). Chapter 6: Continuous-Time Signal Analysis: The Fourier Series. Signal Processing and Linear Systems (3nd ed., pp. 612-620). Oxford University Press.

Sources:

B. P. Lathi & Roger Green. (2018). Chapter 7: Continuous-Time Signal Analysis. Signal Processing and Linear Systems (3nd ed., pp. 699-708). Oxford University Press.

Sources:

B. P. Lathi & Roger Green. (2018). Chapter 7: Continuous-Time Signal Analysis. Signal Processing and Linear Systems (3nd ed., pp. 704-708). Oxford University Press.

Sources:

1. B. P. Lathi & Roger Green. (2021). Chapter 4: Continuous-Time Analysis: The Fourier Transform. Signal Processing and Linear Systems (2nd ed., pp. 401-414). Oxford University Press.

Recall that in amplitude-modulated signals, the information content of the baseband (message) signal \(m(t)\) appears in the amplitude variations of the carrier. In angle modulation discussed in this article, the information content of \(m(t)\) is carried by the angle (which includes its frequency and phase) variations of the carrier.