nz::nodes::loss::BinaryCrossEntropyNode Class Reference

Represents the Binary Cross-Entropy (BCE) loss function node in a computational graph. More...

Inheritance diagram for nz::nodes::loss::BinaryCrossEntropyNode:

Collaboration diagram for nz::nodes::loss::BinaryCrossEntropyNode:

Public Member Functions
	BinaryCrossEntropyNode (Node *input1, Node *input2)
	Constructor to initialize a BinaryCrossEntropyNode for computing the Binary Cross-Entropy loss.
void	forward () override
	Computes the Binary Cross-Entropy (BCE) loss in the forward pass.
void	backward () override
	Computes the gradients of the Binary Cross-Entropy (BCE) loss with respect to the inputs during the backward pass.
Public Member Functions inherited from nz::nodes::io::OutputNode
	OutputNode (Node *input)
	Constructor to initialize an OutputNode with a given input node.
void	forward () override
	Forward pass for the OutputNode.
void	backward () override
	Backward pass for the OutputNode.
Tensor::value_type	getLoss () const
	Retrieves the loss value stored in the OutputNode.
void	print (std::ostream &os) const override
	Prints the type, data, gradient, and loss of the node.
Public Member Functions inherited from nz::nodes::Node
void	dataInject (Tensor::value_type *data, bool grad=false) const
	Injects data into a relevant tensor object, optionally setting its gradient requirement.
template<typename Iterator >
void	dataInject (Iterator begin, Iterator end, const bool grad=false) const
	Injects data from an iterator range into the output tensor of the InputNode, optionally setting its gradient requirement.
void	dataInject (const std::initializer_list< Tensor::value_type > &data, bool grad=false) const
	Injects data from a std::initializer_list into the output tensor of the Node, optionally setting its gradient requirement.

Detailed Description

Represents the Binary Cross-Entropy (BCE) loss function node in a computational graph.

The BinaryCrossEntropyNode class computes the Binary Cross-Entropy loss between two input tensors. BCE is typically used in binary classification tasks to measure the difference between predicted probabilities and the true binary labels. The loss is calculated as:

BCE(x, y) = -1/n * Σ [y_i * log(x_i) + (1 - y_i) * log(1 - x_i)]

where x represents the predicted probabilities (output of the model), and y represents the true binary labels (either 0 or 1). The loss is computed element-wise for each pair of corresponding values in the tensors.

Key features:

• Forward Pass: Calculates the Binary Cross-Entropy loss between the two input tensors, storing the result in the output tensor. The BCE loss is computed for each element, and the results are aggregated to produce the final loss.
• Backward Pass: Computes the gradients of the BCE loss with respect to the input tensors for use in backpropagation. The gradients are propagated only if the output tensor requires gradients.
• Shape Compatibility: Ensures that both input tensors have the same shape. An exception is thrown if the shapes do not match.
• Efficient Computation: The forward and backward passes are optimized for parallel execution using CUDA to handle large datasets efficiently.

This class is part of the nz::nodes namespace and is used in models for binary classification tasks.

Note

• The BinaryCrossEntropyNode requires two input nodes: the predicted probabilities (input1) and the true binary labels (input2).
• Both input tensors must have the same shape, and the BCE loss is calculated element-wise across the tensors.
• The forward() method computes the BCE loss on the GPU, and the backward() method computes the gradients of the BCE loss.
• The loss is stored in the loss attribute, which is updated during the forward pass.
• The gradients are stored in the grad attribute of the output tensor during the backward pass.

Usage Example:

// Example: Using BinaryCrossEntropyNode in a computational graph
InputNode input1({3, 3}, true);  // Create the first input node (predicted probabilities)
InputNode input2({3, 3}, true);  // Create the second input node (true labels)
 
float data1[] = {0.9f, 0.2f, 0.8f, 0.1f, 0.5f, 0.7f, 0.3f, 0.9f, 0.6f};  // Sample predicted values
float data2[] = {1.0f, 0.0f, 1.0f, 0.0f, 1.0f, 1.0f, 0.0f, 1.0f, 0.0f};  // Sample true labels
input1.output->dataInject(data1);  // Copy data to the first input tensor
input2.output->dataInject(data2);  // Copy data to the second input tensor
 
BinaryCrossEntropyNode bce_node(&input1, &input2);  // Create the Binary Cross-Entropy node
bce_node.forward();  // Perform the forward pass and compute the BCE loss
bce_node.backward();  // Perform the backward pass and compute gradients
 
std::cout << "BCE Loss: " << bce_node.getLoss() << std::endl;  // Print the computed BCE loss

See also

forward() for the Binary Cross-Entropy computation in the forward pass.

backward() for gradient computation in the backward pass.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/07

Definition at line 4958 of file Nodes.cuh.

Constructor & Destructor Documentation

BinaryCrossEntropyNode()

nz::nodes::loss::BinaryCrossEntropyNode::BinaryCrossEntropyNode ( Node * input1, Node * input2 )

explicit

Constructor to initialize a BinaryCrossEntropyNode for computing the Binary Cross-Entropy loss.

The constructor initializes a BinaryCrossEntropyNode, which applies the Binary Cross-Entropy loss function to two input tensors. It verifies that both input tensors have the same shape and establishes a connection in the computational graph by storing the second input tensor. The node's type is set to "BinaryCrossEntropy".

Parameters

input1	A pointer to the first input node. This tensor represents the predicted probabilities.
input2	A pointer to the second input node. This tensor represents the true binary labels (0 or 1).

Exceptions

std::invalid_argument

If the shapes of the two input tensors do not match.

• This constructor ensures that the input tensors have the same shape; otherwise, an exception is thrown.
• The first input tensor is connected to the output node, while the second input tensor is added to the inputs vector.
• The node's type is set to "BinaryCrossEntropy" to reflect its operation.

Note

• The two input tensors must have the same shape, as the Binary Cross-Entropy loss is computed element-wise.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/07

Definition at line 813 of file Nodes.cu.

Member Function Documentation

backward()

void nz::nodes::loss::BinaryCrossEntropyNode::backward ( )

overridevirtual

Computes the gradients of the Binary Cross-Entropy (BCE) loss with respect to the inputs during the backward pass.

This method computes the gradients of the Binary Cross-Entropy loss with respect to both input tensors (input1 and input2). The gradients are computed only if the output tensor requires gradients (i.e., during the backpropagation process). The gradients are propagated back to the input nodes to update their weights during training.

The gradient of Binary Cross-Entropy with respect to the predicted probabilities (y_pred) is computed as:

dBCE/dy_pred = - ( y_true / y_pred ) + ( (1 - y_true) / (1 - y_pred) )

where y_pred is the predicted probability and y_true is the true binary label (0 or 1).

The gradient computation is parallelized on the GPU using CUDA, enabling efficient backpropagation even with large datasets.

• A kernel is launched to compute the gradients in parallel for all elements in the tensors.
• The gradients are stored in the grad attribute of the output tensor, which is propagated to the input nodes during backpropagation.
• The backward pass is only executed if the output tensor has requiresGrad() set to true, ensuring that gradients are computed only when necessary.
• The method uses GPU memory for efficient computation and returns the results to the host after calculation.

Note

• This method assumes that both input tensors have the same shape. The gradients are computed element-wise and are propagated back to the input tensors accordingly.
• The gradients with respect to the predicted probabilities (y_pred) are accumulated in the grad attribute of the output tensor.
• If the output does not require gradients, this method does nothing.

See also

forward() for the BCE loss computation in the forward pass.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/07

Implements nz::nodes::Node.

Definition at line 842 of file Nodes.cu.

Here is the call graph for this function:

forward()

void nz::nodes::loss::BinaryCrossEntropyNode::forward ( )

overridevirtual

Computes the Binary Cross-Entropy (BCE) loss in the forward pass.

This method computes the Binary Cross-Entropy loss between the predicted probabilities (from the first input tensor) and the true binary labels (from the second input tensor). The loss is calculated element-wise and accumulated. The result is stored in the loss attribute, which can be accessed after the forward pass.

The Binary Cross-Entropy loss is computed as:

BCE(y_pred, y_true) = - ( y_true * log(y_pred) + (1 - y_true) * log(1 - y_pred) )

where y_pred is the predicted probability and y_true is the true label (0 or 1).

The calculation is done in parallel on the GPU using CUDA to handle large tensor sizes efficiently.

• The forward pass involves allocating memory on the GPU, performing the BCE loss computation, and accumulating the loss.
• A kernel is launched with a grid of threads to compute the loss across all elements of the tensors.
• The result is copied back to the host memory, where the accumulated loss is added to the loss attribute.
• After computation, the allocated GPU memory is freed.

Note

• This method assumes that both input tensors have the same shape. The computation is performed element-wise, and the tensors must be compatible for this operation.
• The loss attribute will hold the accumulated Binary Cross-Entropy loss after the forward pass.

See also

backward() for the gradient computation in the backward pass.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/07

Implements nz::nodes::Node.

Definition at line 822 of file Nodes.cu.

Here is the call graph for this function:

---

The documentation for this class was generated from the following files:

• D:/Users/Mgepahmge/Documents/C Program/NeuZephyr/include/NeuZephyr/Nodes.cuh
• D:/Users/Mgepahmge/Documents/C Program/NeuZephyr/src/Nodes.cu