機器學習之單變量線性迴歸

機器學習之單變量線性迴歸

• 簡述
• 繪圖
• 代價函數及梯度下降
• 可視化

簡述

• 單變量指的是輸入特徵只有一個。
• 根據吳恩達老師的機器學習課程推薦，機器學習研究最好使用octave/matlab作為工具，實現簡單，方便移植。
• 本文使用免費的Octave進行學習。

繪圖

plotData.m 該文件可以實現簡單的繪圖功能，寫成函數，方便後續直接調用。

function plotData (x, y)
figure; %open a new figure window
plot(x, y, 'rx', 'MarkerSize', 10);
ylabel('Profit in $10.000s');
xlabel('Population of city in 10,000');
endfunction

代價函數及梯度下降

單變量線性迴歸假設函數：

代價函數：

computeCost.m 用來計算代價函數。

function J = computeCost (X, y, theta)
m = length(y);
J = 0;
J = sum((X * theta - y).^2) / (2*m);
endfunction

求偏導：

gradientDescent.m 梯度下降函數，做梯度下降的同時用J_history記錄了每次代價函數J(θ) 的值：

function [theta, J_history] = gradientDescent (X, y, theta, alpha, num_iters)
m = length(y);
J_history = zeros(num_iters, 1);
thetas = theta;
for iter = 1 : num_iters
\ttheta(1) = theta(1) - alpha / m * sum(X * thetas - y);
\ttheta(2) = theta(2) - alpha / m * sum((X * thetas - y) .* X(:,2));
\tthetas = theta;
\tJ_history(iter) = computeCost(X, y, theta);
end
endfunction
 

可視化

現在綜合上述函數進行可視化，直觀的觀察數據ex1data1.txt，（https://github.com/peedeep/Coursera/blob/master/ex1/ex1data1.txt）此處下載。

原始訓練數據用展示如下：

%% Exercise 1: Linear Regression
%% ========== 1.Plotting ==========
data = load('ex1data1.txt');
X = data(:, 1);
y = data(:, 2);
m = length(y);
plotData(X, y);
fprintf('Program paused, Press enter to continue.\\n');
pause;

擬合出一條直線：

%% ========== 2.Cost and Gradient descent ==========
X = [ones(m, 1), data(:, 1)];
theta = zeros(2, 1);
iterations = 2000;
alpha = 0.01;
J = computeCost(X, y, theta);
fprintf('size of X = %f\\n', size(X));
fprintf('size of theta = %f\\n', size(theta));
fprintf('With theta = [0; 0]\\nCost computed = %f\\n', J);
fprintf('Expected cost value (approx) 32.07\\n');
J = computeCost(X, y, [-1; 2]);
fprintf('With theta = [-1; 2]\\nCost computed = %f\\n', J);
fprintf('Expected cost value (approx) 54.24\\n');
fprintf('Program paused, Press enter to continue.\\n');
pause
[theta, J_history] = gradientDescent(X, y, theta, alpha, iterations);
fprintf('Theta found by gradient descent:\\n');
fprintf('%f\\n', theta);
fprintf('Expected theta values (approx)\\n');
fprintf(' -3.6303\\n 1.1664\\n\\n');
hold on; % keep previous plot visible
plot(X(:,2), X * theta, '-');
legend('Training data', 'Linear regression');
hold off;

隨著迭代次數的增加代價函數J(θ) 減少：

fprintf('%f\\n', size(J_history));
figure; %open a new figure window
plot([1:size(J_history)], J_history);
pause;
predict1 = [1, 3.5] * theta;
predict2 = [1, 7] * theta;
fprintf('predict1:%f \\n', predict1);
fprintf('predict2:%f \\n', predict2);
fprintf('Program paused. Press enter to continue.\\n');
pause;

%% ========= 3.Visualizing J(theta_0, theta_1) ========== 

theta0_vals = linspace(-10, 10, 100);
theta1_vals = linspace(-1, 4, 100);
J_vals = zeros(length(theta0_vals), length(theta1_vals));
for i = 1:length(theta0_vals)
\tfor j = 1:length(theta1_vals)
\t\tt = [theta0_vals(i); theta1_vals(j)];
\t\tJ_vals(i,j) = computeCost(X, y, t);
\tend
end
J_vals = J_vals'; %由於meshgrids在surf命令中的工作方式，我們需要在調用surf之前調換J_vals，否則軸會被翻轉
% Surface plot
figure;
surf(theta0_vals, theta1_vals, J_vals); % surf命令繪製得到的是著色的三維曲面。
xlabel('\\theta_0');ylabel('\\theta_1');
% Contour plot
figure;
contour(theta0_vals, theta1_vals, J_vals, logspace(-2, 3, 20)); % contour用來繪製矩陣數據的等高線
xlabel('\\theta_0');ylabel('\\theta_1');
hold on;
plot(theta(1), theta(2), 'rx', 'MarkerSize', 10, 'LineWidth', 2);

等高線：

完。