题目2634 · 机器学习
Batch-Average Gradient 9
If the minibatch loss is the average L = (1/B) sum_{i=1}^B L_i, derive dL/dw in terms of the per-example gradients.
打开 →找到 25 个结果
中文题目题目2642 · 机器学习
BatchNorm Running Mean Update 13
A BatchNorm layer updates its running mean by mu_new = m mu_old + (1-m) mu_batch. What does this formula mean operationally?
打开 →题目2643 · 机器学习
Clipping Plus Weight Decay on a Vector 25
A parameter vector is w_t=(3,4). Its gradient is g=(6,8), whose norm is 10. Apply global-norm clipping with threshold 5, then a decoupled weight-decay step with learning rate eta=0.1 and lambda=0.1. What is the new parameter vector?
打开 →题目2640 · 机器学习
Cosine Decay Schedule 12
A learning rate decays from eta_max to eta_min over T steps using cosine annealing. What is eta_t at step t?
打开 →题目2636 · 机器学习
Decoupled Weight Decay Numerically 18
A scalar parameter has value w_t=2, gradient g_t=0.5, learning rate eta=0.1, and decoupled weight decay lambda=0.05. What is w_{t+1}?
打开 →题目2626 · 机器学习
Global-Norm Clipping Numerically 16
A gradient vector is g=(6,8), whose norm is 10. If the clip threshold is 5, what clipped gradient is produced?
打开 →题目2523 · 机器学习
Gradient of Logistic Negative Log-Likelihood 3
For one observation (x,y) with y in {0,1} and score z = w^T x, what is the gradient of the negative log-likelihood with respect to w?
打开 →题目2522 · 机器学习
Intercept From the Positive Rate 2
In an intercept-only logistic model, if the fitted probability is p_hat, what intercept b solves sigma(b)=p_hat?
打开 →题目2633 · 机器学习
Layer-Norm Shift Invariance 8
Ignoring learned affine parameters, why does adding the same constant a to every coordinate of a vector leave layer-normalized activations unchanged?
打开 →题目2624 · 机器学习
Momentum as an Unrolled Geometric Sum 3
If momentum obeys v_t = beta v_{t-1} + g_t, derive v_t in terms of v_0 and the past gradients g_1,...,g_t.
打开 →题目2623 · 机器学习
One Momentum Update 15
Suppose momentum uses v_t = beta v_{t-1} + g_t with beta=0.9, previous velocity v_{t-1}=0.5, and current gradient g_t=2. What is v_t?
打开 →题目2596 · 机器学习
Optimal Leaf Update Under Squared Loss 1
In gradient boosting for squared error, a terminal region R is assigned one constant update gamma. Derive the gamma that minimizes sum_{i in R} (r_i-gamma)^2, where r_i are the current residuals.
打开 →题目2637 · 机器学习
ReLU Local Derivative 10
For ReLU(z)=max(0,z), what derivative does backprop use when z>0 and when z<0?
打开 →题目2608 · 机器学习
Residual After Two Shrunken Updates 24
A point currently has residual 6. Two boosting rounds hit its region with leaf updates 1.5 and 0.8, using learning rate eta=0.2 in both rounds. What residual remains after the two rounds?
打开 →题目2638 · 机器学习
Residual Gradient Numerically 19
A scalar residual block has y=x+f(x) with f(x)=3x^2. What is dy/dx at x=1?
打开 →题目2639 · 机器学习
Steady-State Momentum Under a Constant Gradient 11
If v_t = beta v_{t-1} + g with constant gradient g and |beta|<1, what constant value does v_t converge to?
打开 →题目2597 · 机器学习
Weighted Region Update 2
If observations in a boosting region R carry positive weights w_i, derive the constant update gamma that minimizes sum_{i in R} w_i (r_i-gamma)^2.
打开 →题目2630 · 机器学习
Why BatchNorm Can Break Under Distribution Shift 21
Why can a network that trains well with BatchNorm behave strangely at inference when the deployment distribution shifts?
打开 →题目2524 · 机器学习
Why No Closed Form in Logistic Regression 5
Why does logistic regression usually require iterative optimization rather than a normal-equation-style closed form?
打开 →题目2628 · 机器学习
Why Residual Connections Help Train Deep Nets 20
Why do residual connections often make very deep networks easier to optimize?
打开 →题目2526 · 机器学习
Why Separable Data Pushes Coefficients Outward 7
Why do logistic-regression coefficients tend to diverge on perfectly linearly separable data if no regularization is used?
打开 →题目2515 · 机器学习
Why Small Lambda Means Weak Regularization 20
Why does a very small lambda leave the regularized solution close to OLS?
打开 →题目2635 · 机器学习
Why Warmup Helps Large-Batch Training 22
Why is learning-rate warmup often helpful when training with very large batches?
打开 →题目2541 · 机器学习
One Gradient Step on a Single Logistic Observation 22
For one observation with x = 2, y = 1, current weight w = 0, and learning rate eta = 0.4, what is one gradient-descent update on the negative log-likelihood?
打开 →题目2492 · 机器学习
Why Feature Scaling Helps Gradient Descent More Than Closed Form 22
Why is feature scaling often crucial for gradient-descent training of OLS even though the closed-form solution itself is scale-equivariant?
打开 →