nz::nodes::calc::ScalarDivNode Class Reference

Represents a scalar division operation node in a computational graph. More...

Inheritance diagram for nz::nodes::calc::ScalarDivNode:

Collaboration diagram for nz::nodes::calc::ScalarDivNode:

Public Member Functions
	ScalarDivNode (Node *input, Tensor::value_type scalar)
	Constructor to initialize a ScalarDivNode for scalar division.
void	forward () override
	Forward pass for the ScalarDivNode to perform scalar division.
void	backward () override
	Backward pass for the ScalarDivNode to propagate gradients.
Public Member Functions inherited from nz::nodes::Node
virtual void	print (std::ostream &os) const
	Prints the type, data, and gradient of the node.
void	dataInject (Tensor::value_type *data, bool grad=false) const
	Injects data into a relevant tensor object, optionally setting its gradient requirement.
template<typename Iterator >
void	dataInject (Iterator begin, Iterator end, const bool grad=false) const
	Injects data from an iterator range into the output tensor of the InputNode, optionally setting its gradient requirement.
void	dataInject (const std::initializer_list< Tensor::value_type > &data, bool grad=false) const
	Injects data from a std::initializer_list into the output tensor of the Node, optionally setting its gradient requirement.

Detailed Description

Represents a scalar division operation node in a computational graph.

The ScalarDivNode class performs element-wise division of a tensor by a scalar value. It is used in computational graphs to implement normalization or scaling operations, which are fundamental in machine learning and numerical computations.

Key features:

• Forward Pass: Divides each element of the input tensor by a scalar value and stores the result in the output tensor.
• Backward Pass: Propagates gradients from the output tensor back to the input tensor by scaling the gradients with the reciprocal of the scalar value.
• Error Handling: Ensures that the scalar value is non-zero during construction to prevent division by zero errors.
• Shape Preservation: Maintains the shape of the input tensor in the output tensor.
• Gradient Management: Tracks whether gradients are required for the operation based on the properties of the input tensor.

This class is part of the nz::nodes namespace and is optimized for scalar-tensor division operations in computational graphs.

Note

• The scalar value must be non-zero. An exception is thrown during construction if this condition is not met.
• A warning is issued indicating that scalar operations do not support saving to files, and users are encouraged to use matrix operations for model persistence.

Usage Example:

// Example: Using ScalarDivNode for scalar division
InputNode input({3, 3}, true);  // Create an input node with shape {3, 3}
input.output->fill(10.0f);  // Fill the input tensor with value 10.0
 
ScalarDivNode scalar_div_node(&input, 2.0f);  // Divide the input tensor by 2.0
scalar_div_node.forward();  // Perform the forward pass
scalar_div_node.backward();  // Propagate gradients in the backward pass
 
std::cout << "Output: " << *scalar_div_node.output << std::endl;  // Print the result

See also

forward() for the forward pass implementation.

backward() for the gradient propagation in the backward pass.

Exceptions

std::invalid_argument

If the scalar value is zero, as division by zero is undefined.

Warning

• Scalar operations are not yet supported for saving to files. Use matrix operations as an alternative.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/05

Definition at line 1350 of file Nodes.cuh.

Constructor & Destructor Documentation

ScalarDivNode()

nz::nodes::calc::ScalarDivNode::ScalarDivNode	(	Node *	input,
		Tensor::value_type	scalar )

Constructor to initialize a ScalarDivNode for scalar division.

The constructor initializes a ScalarDivNode, which performs element-wise division of the output tensor of the input node by a scalar value. It validates the scalar value, sets up the node's input connections, determines the gradient tracking requirement, and prepares the output tensor with the appropriate shape and properties.

Parameters

input	A pointer to the input node. Its output tensor will be divided by the scalar value.
scalar	The scalar value to divide the input tensor by.

• The constructor verifies that the scalar value is non-zero. If the scalar is zero, an exception is thrown to prevent division by zero errors.
• The input node is added to the inputs vector to establish the connection in the computational graph.
• It initializes the output tensor with the same shape as the input tensor and determines whether the output tensor should track gradients based on the input tensor's gradient requirement.
• A warning is issued to inform the user that scalar operations currently do not support saving to files, encouraging the use of matrix operations for models requiring persistence.

Note

• The scalar value must be non-zero to ensure valid division. An exception will be thrown if this condition is not met.
• The scalar value is stored internally in the ScalarDivNode instance and used during the forward and backward passes.

Exceptions

std::invalid_argument

If the scalar value is zero, as division by zero is undefined.

Warning

• Scalar operations are not yet supported for saving to files. Use matrix operations as an alternative.

See also

forward() for the forward pass implementation.

backward() for the gradient propagation in the backward pass.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/05

Definition at line 208 of file Nodes.cu.

Member Function Documentation

backward()

void nz::nodes::calc::ScalarDivNode::backward ( )

overridevirtual

Backward pass for the ScalarDivNode to propagate gradients.

The backward() method computes the gradient of the loss with respect to the input tensor by scaling the gradient of the output tensor using the reciprocal of the scalar value. This ensures the gradients are correctly propagated back through the computational graph.

• The method first checks if the input tensor requires gradients. If true, a CUDA kernel (ScalarDiv) is launched to compute the scaled gradients.
• The gradient computation is performed as grad_input[i] = grad_output[i] / scalar for each element, where grad_output is the gradient of the output tensor.
• The resulting gradients are stored in the gradient tensor of the input node.

Note

• The backward pass uses the same scalar value as in the forward pass, ensuring consistency in gradient computation.
• The scalar value must be non-zero to avoid undefined behavior.

See also

forward() for the scalar division computation in the forward pass.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/05

Implements nz::nodes::Node.

Definition at line 227 of file Nodes.cu.

Here is the call graph for this function:

forward()

void nz::nodes::calc::ScalarDivNode::forward ( )

overridevirtual

Forward pass for the ScalarDivNode to perform scalar division.

The forward() method computes the element-wise division of the input tensor by the scalar value. It uses CUDA kernels to execute the operation efficiently on the GPU.

• A CUDA kernel (ScalarDiv) is launched to compute the division for each element in the input tensor.
• The grid and block dimensions are calculated dynamically based on the size of the output tensor to optimize parallel computation on the GPU.
• The result of the division is stored in the output tensor.

Note

• The division operation is performed as output[i] = input[i] / scalar for each element of the tensor.
• Ensure the scalar value is non-zero, as division by zero will result in undefined behavior.

See also

backward() for gradient propagation in the backward pass.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/05

Implements nz::nodes::Node.

Definition at line 221 of file Nodes.cu.

Here is the call graph for this function:

---

The documentation for this class was generated from the following files:

• D:/Users/Mgepahmge/Documents/C Program/NeuZephyr/include/NeuZephyr/Nodes.cuh
• D:/Users/Mgepahmge/Documents/C Program/NeuZephyr/src/Nodes.cu