nz::nodes::calc::ScalarMulNode Class Reference

Represents a scalar multiplication operation node in a computational graph. More...

Inheritance diagram for nz::nodes::calc::ScalarMulNode:

Collaboration diagram for nz::nodes::calc::ScalarMulNode:

Public Member Functions
	ScalarMulNode (Node *input, Tensor::value_type scalar)
	Constructor to initialize a ScalarMulNode for scalar multiplication.
void	forward () override
	Forward pass for the ScalarMulNode to perform scalar multiplication.
void	backward () override
	Backward pass for the ScalarMulNode to propagate gradients.
Public Member Functions inherited from nz::nodes::Node
virtual void	print (std::ostream &os) const
	Prints the type, data, and gradient of the node.
void	dataInject (Tensor::value_type *data, bool grad=false) const
	Injects data into a relevant tensor object, optionally setting its gradient requirement.
template<typename Iterator >
void	dataInject (Iterator begin, Iterator end, const bool grad=false) const
	Injects data from an iterator range into the output tensor of the InputNode, optionally setting its gradient requirement.
void	dataInject (const std::initializer_list< Tensor::value_type > &data, bool grad=false) const
	Injects data from a std::initializer_list into the output tensor of the Node, optionally setting its gradient requirement.

Detailed Description

Represents a scalar multiplication operation node in a computational graph.

The ScalarMulNode class performs element-wise multiplication of a tensor by a scalar value. It is used in computational graphs to implement scalar scaling of tensors, which is common in various machine learning and numerical computing tasks.

Key features:

• Forward Pass: Multiplies the input tensor's elements by a scalar value and stores the result in the output tensor.
• Backward Pass: Propagates gradients from the output tensor back to the input tensor by scaling the gradient with the scalar value.
• Scalar Handling: Handles scalar operations directly but issues a warning that scalar operations do not support saving to files, encouraging the use of matrix operations when persistence is required.

This class is part of the nz::nodes namespace and is used for scalar-tensor operations in a computational graph.

Note

• This node is designed for scalar-tensor operations. It does not support saving scalar operations to files; users should rely on matrix operations for this purpose.
• The scalar value is stored internally and used for both the forward and backward passes.

Usage Example:

// Example: Using ScalarMulNode for scalar multiplication
InputNode input({3, 3}, true);  // Create an input node with shape {3, 3}
input.output->fill(2.0f);  // Fill the input tensor with value 2.0
 
ScalarMulNode scalar_mul_node(&input, 5.0f);  // Multiply the input tensor by 5.0
scalar_mul_node.forward();  // Perform the forward pass
scalar_mul_node.backward();  // Propagate gradients in the backward pass
 
std::cout << "Output: " << *scalar_mul_node.output << std::endl;  // Print the result

See also

forward() for the forward pass computation method.

backward() for the backward pass gradient propagation method.

Warning

• Scalar operations do not yet support saving to files.
• Use matrix operations instead if model saving is required.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/05

Definition at line 1205 of file Nodes.cuh.

Constructor & Destructor Documentation

ScalarMulNode()

nz::nodes::calc::ScalarMulNode::ScalarMulNode	(	Node *	input,
		Tensor::value_type	scalar )

Constructor to initialize a ScalarMulNode for scalar multiplication.

The constructor initializes a ScalarMulNode, which performs element-wise multiplication of the output tensor of the input node by a scalar value. It sets up the node's input connections, determines the gradient tracking requirement, and prepares the output tensor with the appropriate shape and properties.

Parameters

input	A pointer to the input node. Its output tensor will be multiplied by the scalar value.
scalar	The scalar value to multiply with the input tensor.

• The constructor connects the input node to this ScalarMulNode by adding the input node to the inputs vector.
• It initializes the output tensor with the same shape as the input tensor and determines whether the output tensor should track gradients based on the input tensor's gradient requirement.
• A warning is issued to inform the user that scalar operations currently do not support saving to files, and matrix operations should be used for models requiring persistence.

Note

• The scalar value is stored internally in the ScalarMulNode instance and used during the forward and backward passes.

Warning

• Scalar operations are not yet supported for saving to files. Use matrix operations as an alternative.

See also

forward() for the forward pass implementation.

backward() for the gradient propagation in the backward pass.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/05

Definition at line 184 of file Nodes.cu.

Member Function Documentation

backward()

void nz::nodes::calc::ScalarMulNode::backward ( )

overridevirtual

Backward pass for the ScalarMulNode to propagate gradients.

The backward() method computes the gradient of the loss with respect to the input tensor by scaling the gradient of the output tensor using the scalar value. This operation ensures that the gradients are correctly propagated back through the computational graph.

• The method checks if the input tensor requires gradients. If not, no computation is performed.
• A CUDA kernel (ScalarMul) is launched to scale the gradient of the output tensor and accumulate the result in the gradient of the input tensor.
• The grid and block dimensions are calculated based on the size of the output tensor to ensure efficient GPU parallelization.

Note

• The gradient computation is performed as grad_input[i] = grad_output[i] * scalar for each element in the tensor, where grad_output is the gradient of the output tensor.

See also

forward() for the scalar multiplication computation in the forward pass.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/05

Implements nz::nodes::Node.

Definition at line 200 of file Nodes.cu.

Here is the call graph for this function:

forward()

void nz::nodes::calc::ScalarMulNode::forward ( )

overridevirtual

Forward pass for the ScalarMulNode to perform scalar multiplication.

The forward() method computes the element-wise multiplication of the input tensor by the scalar value. It uses CUDA kernels to perform the computation in parallel on the GPU, ensuring efficient execution.

• The method launches a CUDA kernel (ScalarMul) to compute the scalar multiplication.
• The output tensor is populated with the results of the multiplication.
• The grid and block dimensions are calculated based on the size of the output tensor to optimize GPU performance.

Note

• The scalar multiplication is performed as output[i] = input[i] * scalar for each element in the tensor.

See also

backward() for gradient propagation in the backward pass.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/05

Implements nz::nodes::Node.

Definition at line 194 of file Nodes.cu.

Here is the call graph for this function:

---

The documentation for this class was generated from the following files:

• D:/Users/Mgepahmge/Documents/C Program/NeuZephyr/include/NeuZephyr/Nodes.cuh
• D:/Users/Mgepahmge/Documents/C Program/NeuZephyr/src/Nodes.cu