nz::nodes::loss::MeanSquaredErrorNode Class Reference

Represents the Mean Squared Error (MSE) loss function node in a computational graph. More...

Inheritance diagram for nz::nodes::loss::MeanSquaredErrorNode:

Collaboration diagram for nz::nodes::loss::MeanSquaredErrorNode:

Public Member Functions
	MeanSquaredErrorNode (Node *input1, Node *input2)
	Constructor to initialize a MeanSquaredErrorNode for computing the Mean Squared Error (MSE) loss.
void	forward () override
	Computes the forward pass of the Mean Squared Error (MSE) loss function.
void	backward () override
	Computes the backward pass of the Mean Squared Error (MSE) loss function.
Public Member Functions inherited from nz::nodes::io::OutputNode
	OutputNode (Node *input)
	Constructor to initialize an OutputNode with a given input node.
void	forward () override
	Forward pass for the OutputNode.
void	backward () override
	Backward pass for the OutputNode.
Tensor::value_type	getLoss () const
	Retrieves the loss value stored in the OutputNode.
void	print (std::ostream &os) const override
	Prints the type, data, gradient, and loss of the node.
Public Member Functions inherited from nz::nodes::Node
void	dataInject (Tensor::value_type *data, bool grad=false) const
	Injects data into a relevant tensor object, optionally setting its gradient requirement.
template<typename Iterator >
void	dataInject (Iterator begin, Iterator end, const bool grad=false) const
	Injects data from an iterator range into the output tensor of the InputNode, optionally setting its gradient requirement.
void	dataInject (const std::initializer_list< Tensor::value_type > &data, bool grad=false) const
	Injects data from a std::initializer_list into the output tensor of the Node, optionally setting its gradient requirement.

Detailed Description

Represents the Mean Squared Error (MSE) loss function node in a computational graph.

The MeanSquaredErrorNode class computes the Mean Squared Error loss between two input tensors. The loss is calculated as the average of the squared differences between the corresponding elements of the two tensors:

MSE(x, y) = 1/n * Σ (x_i - y_i)^2

where x and y are the two input tensors, and n is the number of elements in the tensors. This class is typically used for training models, especially in regression tasks.

Key features:

• Forward Pass: Calculates the Mean Squared Error loss between the two input tensors, storing the result in the output tensor. The MSE loss is computed on a per-element basis and aggregated across the entire tensor.
• Backward Pass: Computes the gradients of the MSE loss with respect to the input tensors for use in backpropagation. The gradients are propagated only if the output tensor requires gradients.
• Shape Compatibility: Ensures that both input tensors have the same shape. An exception is thrown if the shapes do not match.
• Efficient Computation: The forward and backward passes are optimized for parallel execution using CUDA.

This class is part of the nz::nodes namespace and is useful in models where Mean Squared Error is the loss function.

Note

• The MeanSquaredErrorNode requires two input nodes, both of which must have tensors of the same shape.
• The forward pass performs the MSE calculation on the GPU, while the backward pass computes the gradient of the MSE loss.
• The loss is stored in the loss attribute, which is updated during the forward pass.
• The gradients are stored in the grad attribute of the input tensors during the backward pass.

Usage Example:

// Example: Using MeanSquaredErrorNode in a computational graph
InputNode input1({3, 3}, true);  // Create the first input node with shape {3, 3}
InputNode input2({3, 3}, true);  // Create the second input node with shape {3, 3}
 
float data1[] = {1.0f, 2.0f, 3.0f, 4.0f, 5.0f, 6.0f, 7.0f, 8.0f, 9.0f};  // Sample input1 values
float data2[] = {1.5f, 2.5f, 3.5f, 4.5f, 5.5f, 6.5f, 7.5f, 8.5f, 9.5f};  // Sample input2 values
input1.output->dataInject(data1);  // Copy data to the first input tensor
input2.output->dataInject(data2);  // Copy data to the second input tensor
 
MeanSquaredErrorNode mse_node(&input1, &input2);  // Create the Mean Squared Error node
mse_node.forward();  // Perform the forward pass and compute the MSE loss
mse_node.backward();  // Perform the backward pass and compute gradients
 
std::cout << "MSE Loss: " << mse_node.getLoss() << std::endl;  // Print the computed MSE loss

See also

forward() for the Mean Squared Error computation in the forward pass.

backward() for gradient computation in the backward pass.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/07

Definition at line 4804 of file Nodes.cuh.

Constructor & Destructor Documentation

MeanSquaredErrorNode()

nz::nodes::loss::MeanSquaredErrorNode::MeanSquaredErrorNode ( Node * input1, Node * input2 )

explicit

Constructor to initialize a MeanSquaredErrorNode for computing the Mean Squared Error (MSE) loss.

The constructor initializes a MeanSquaredErrorNode, which computes the MSE loss between two input nodes. It checks that both input nodes have the same shape and sets up the necessary internal structures for the loss calculation.

Parameters

input1	A pointer to the first input node. The output tensor of this node will be used as the predicted values.
input2	A pointer to the second input node. The output tensor of this node will be used as the ground truth values.

Exceptions

std::invalid_argument

if input1 and input2 do not have the same shape.

• The constructor verifies that the shapes of the two input tensors are the same. If they are not, an exception is thrown.
• The second input node (input2) is added to the inputs vector, while the first input node is inherited from the OutputNode base class.
• The type attribute is set to "MeanSquaredError", indicating the type of operation this node represents.

Note

• The MSE loss is only calculated if the input nodes have matching shapes. If the shapes do not match, an exception will be thrown to ensure consistency.
• The constructor initializes the internal state and prepares the node for the forward and backward passes.

See also

forward() for computing the MSE loss in the forward pass.

backward() for computing the gradients in the backward pass.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/07

Definition at line 776 of file Nodes.cu.

Member Function Documentation

backward()

void nz::nodes::loss::MeanSquaredErrorNode::backward ( )

overridevirtual

Computes the backward pass of the Mean Squared Error (MSE) loss function.

This method computes the gradients of the MSE loss with respect to the input tensors (inputs[0] and inputs[1]). The gradients are calculated only if the output tensor requires gradients, and they are used for backpropagation in training deep learning models.

The backward pass is performed using a CUDA kernel to efficiently compute the gradients in parallel on the GPU.

• The method first checks if the output tensor requires gradients by calling requiresGrad() on the output.
• If gradients are required, a CUDA kernel (MSEBackward) is launched to compute the gradients of the loss with respect to both input tensors.
• The gradients are computed by comparing the predicted values (inputs[0]) and the ground truth values (inputs[1]) in a parallel manner across all elements of the tensors.
• The result is stored in the grad attribute of the output tensor.

Note

• The method assumes that both input tensors have the same shape, as validated during the initialization phase.
• The requiresGrad() check ensures that gradients are only computed if necessary, avoiding unnecessary computations.
• This method is essential for updating the model's parameters during the training process through backpropagation.

See also

forward() for the forward pass, which computes the MSE loss.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/07

Implements nz::nodes::Node.

Definition at line 804 of file Nodes.cu.

Here is the call graph for this function:

forward()

void nz::nodes::loss::MeanSquaredErrorNode::forward ( )

overridevirtual

Computes the forward pass of the Mean Squared Error (MSE) loss function.

This method calculates the Mean Squared Error (MSE) loss between two input tensors. The loss is computed by comparing the predicted values (inputs[0]) to the ground truth values (inputs[1]) element-wise. The result is accumulated in the loss attribute of the node.

The computation is performed in parallel on the GPU using CUDA kernels for efficiency.

• The method first calls the forward() method of the OutputNode base class to handle any initialization and setup required by the base class.
• CUDA is used to compute the MSE loss across the entire tensor. A kernel is launched to calculate the squared differences between the elements of the two input tensors.
• The results from the GPU computation are copied to the host memory, and the total MSE loss is accumulated.
• The final computed loss is stored in the loss attribute.

Note

• The inputs vector must contain exactly two nodes: the predicted values and the ground truth values. Both tensors must have the same shape, and this is validated during the initialization.
• The loss is stored in the loss attribute, which is updated after each forward pass.

See also

backward() for the backward pass, which computes the gradients of the MSE loss.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/07

Implements nz::nodes::Node.

Definition at line 785 of file Nodes.cu.

Here is the call graph for this function:

---

The documentation for this class was generated from the following files:

• D:/Users/Mgepahmge/Documents/C Program/NeuZephyr/include/NeuZephyr/Nodes.cuh
• D:/Users/Mgepahmge/Documents/C Program/NeuZephyr/src/Nodes.cu