nz::nodes::io::InputNode Class Reference

Represents an input node in a computational graph. More...

Inheritance diagram for nz::nodes::io::InputNode:

Collaboration diagram for nz::nodes::io::InputNode:

Public Member Functions
	InputNode (const Tensor::shape_type &shape, bool requires_grad=false)
	Constructor to initialize an InputNode with a specified shape and gradient requirement.
	InputNode (const Tensor &tensor)
	Constructor to initialize an InputNode with an existing Tensor.
	InputNode (const Tensor::shape_type &shape, Tensor::value_type *data, bool requires_grad=false, bool host=false)
	Constructs an InputNode object with specified tensor shape, data, gradient requirement, and data location.
	InputNode (const Tensor::shape_type &shape, const std::initializer_list< Tensor::value_type > &data, bool requires_grad=false)
	Constructs an InputNode object with a specified tensor shape, initializer list data, and gradient requirement.
void	forward () override
	Forward pass for the InputNode.
void	backward () override
	Backward pass for the InputNode.
Public Member Functions inherited from nz::nodes::Node
virtual void	print (std::ostream &os) const
	Prints the type, data, and gradient of the node.
void	dataInject (Tensor::value_type *data, bool grad=false) const
	Injects data into a relevant tensor object, optionally setting its gradient requirement.
template<typename Iterator >
void	dataInject (Iterator begin, Iterator end, const bool grad=false) const
	Injects data from an iterator range into the output tensor of the InputNode, optionally setting its gradient requirement.
void	dataInject (const std::initializer_list< Tensor::value_type > &data, bool grad=false) const
	Injects data from a std::initializer_list into the output tensor of the Node, optionally setting its gradient requirement.

Detailed Description

Represents an input node in a computational graph.

The InputNode class is a subclass of Node, representing a node that holds the input data for a neural network or computational graph. It is designed to store the input tensor and pass it forward through the graph. InputNode does not perform any computations in the forward or backward passes; its main role is to provide input data to the network.

Key features:

• Tensor Output: The InputNode stores a Tensor as its output, which is initialized either from a shape or an existing tensor.
• No Forward or Backward Operations: The forward() and backward() methods are implemented as empty, since this node only provides input data and does not modify the network during these passes.
• Shape and Gradient Support: The shape of the input tensor and whether it requires gradients is configurable during initialization.

This class is part of the nz::nodes namespace and serves as a fundamental part of the computational graph, providing input data to the network during the forward pass.

Note

• The forward() and backward() methods are implemented but do not perform any operations for the InputNode.
• This class is designed to be used as the starting point of a computational graph, where other nodes depend on the input data provided by this node.

Usage Example:

// Example 1: Creating an InputNode with a specific shape
Tensor::shape_type shape = {3, 3};  // Define a 3x3 tensor
InputNode input_node(shape, true);  // Create an InputNode with shape {3, 3} and requires_grad = true
input_node.output->fill(0.5f);  // Fill the input tensor with a value of 0.5
 
// Example 2: Creating an InputNode from an existing tensor
Tensor existing_tensor({2, 2});
existing_tensor.fill(1.0f);  // Fill the existing tensor with 1.0
InputNode input_node_from_tensor(existing_tensor);  // Create an InputNode from the existing tensor
 
// Example 3: Using InputNode in a computational graph
InputNode input_node({4, 4});  // Create an InputNode with shape {4, 4}
input_node.output->fill(2.0f);  // Fill the tensor with the value 2.0
 
// In the network, this node will pass the data to subsequent nodes during forward propagation.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/11/29

Definition at line 437 of file Nodes.cuh.

Constructor & Destructor Documentation

InputNode() [1/4]

nz::nodes::io::InputNode::InputNode ( const Tensor::shape_type & shape, bool requires_grad = false )

explicit

Constructor to initialize an InputNode with a specified shape and gradient requirement.

This constructor initializes an InputNode that holds a tensor with the specified shape. The tensor is created with the specified shape, and it can optionally track gradients if requires_grad is set to true. The InputNode does not perform any computations; its primary role is to hold and provide input data to the computational graph.

Parameters

shape	The shape of the tensor to be stored in the InputNode. This defines the dimensions of the input data.
requires_grad	A boolean flag indicating whether the tensor should track gradients. Defaults to false.

The InputNode will store a Tensor object, initialized with the given shape and gradient tracking setting. This tensor can then be used as input for subsequent nodes in the computational graph.

Note

• The InputNode class does not perform any computations during the forward or backward passes. It simply stores and provides input data.
• requires_grad determines whether the input tensor will store gradients, which is useful if the input data is part of the network's parameters.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/11/29

Definition at line 24 of file Nodes.cu.

InputNode() [2/4]

nz::nodes::io::InputNode::InputNode ( const Tensor & tensor )

explicit

Constructor to initialize an InputNode with an existing Tensor.

This constructor initializes an InputNode using an existing Tensor object. The Tensor object is directly assigned to the output of the InputNode, allowing the node to use the provided tensor as its input data. The InputNode does not perform any computations but simply holds and provides the given tensor data to the computational graph.

Parameters

tensor

The existing Tensor object to be used as the input for the InputNode. This tensor contains the input data to be passed through the network.

The InputNode stores the provided tensor in its output member, which will be used by other nodes in the graph.

Note

• The InputNode does not modify the given tensor, it simply holds a reference to it.
• This constructor is useful when you already have a tensor (e.g., loaded from a file or created elsewhere) and want to use it as input to the computational graph.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/11/29

Definition at line 29 of file Nodes.cu.

InputNode() [3/4]

nz::nodes::io::InputNode::InputNode ( const Tensor::shape_type & shape, Tensor::value_type * data, bool requires_grad = false, bool host = false )

explicit

Constructs an InputNode object with specified tensor shape, data, gradient requirement, and data location.

Parameters

shape	A reference to the shape of the output tensor of the input node (host-to-device). It defines the dimensions of the tensor.
data	A pointer to the initial data of the output tensor. The data can be either on the host or device depending on the host parameter.
requires_grad	A boolean indicating whether the output tensor requires gradient computation.
host	A boolean indicating whether the data pointed to by data is on the host or device. If true, data is on the host; otherwise, it is on the device.

Returns

None. This is a constructor.

This constructor initializes an InputNode object. It creates a new Tensor object using the provided shape, data, requires_grad, and host parameters and stores a shared pointer to this tensor in the output member variable.

In terms of memory management, the std::shared_ptr in output takes care of the memory of the Tensor object. When the last reference to the Tensor object held by a std::shared_ptr is destroyed, the Tensor object will be automatically deleted.

Regarding exception handling, this constructor does not explicitly catch any exceptions thrown by the Tensor constructor. If the Tensor constructor fails (e.g., due to insufficient memory or invalid input), the exception will propagate to the caller.

This constructor is a fundamental part of the InputNode class as it initializes the output tensor of the input node.

Exceptions

None	explicitly, but the Tensor constructor may throw exceptions, such as std::bad_alloc if memory allocation fails.

Note

• Ensure that the data pointer is valid and points to enough data to fill the tensor according to the specified shape.
• The CUDA runtime environment should be properly initialized before calling this constructor if the tensor is using CUDA memory.
• This constructor has a time complexity of O(1) for creating the std::shared_ptr and O(n) for the Tensor constructor, where n is the total number of elements in the tensor.

```cpp
#include <vector>
 
shape_type shape = {2, 3};
value_type data[] = {1.0f, 2.0f, 3.0f, 4.0f, 5.0f, 6.0f};
try {
    InputNode inputNode(shape, data, true, true);
} catch (const std::exception& e) {
    std::cerr << e.what() << std::endl;
}
```

Definition at line 34 of file Nodes.cu.

InputNode() [4/4]

nz::nodes::io::InputNode::InputNode ( const Tensor::shape_type & shape, const std::initializer_list< Tensor::value_type > & data, bool requires_grad = false )

explicit

Constructs an InputNode object with a specified tensor shape, initializer list data, and gradient requirement.

Parameters

shape	A reference to the shape of the output tensor of the input node (host-to-device). It determines the dimensions and total size of the tensor.
data	A std::initializer_list containing the initial data for the output tensor (host-to-device).
requires_grad	A boolean indicating whether the output tensor requires gradient computation.

Returns

None. This is a constructor.

This constructor initializes an InputNode object. It creates a new Tensor object using the provided shape, initializer list data, and gradient requirement, and stores a shared pointer to this tensor in the output member variable.

For memory management, the std::shared_ptr in output takes care of the Tensor object's memory. When the last reference to the Tensor object held by a std::shared_ptr is destroyed, the Tensor object will be automatically deleted.

Regarding exception handling, this constructor does not explicitly catch any exceptions thrown by the Tensor constructor. If the Tensor constructor fails (e.g., due to insufficient memory or an invalid initializer list size), the exception will propagate to the caller.

This constructor is an important part of the InputNode class as it provides a convenient way to initialize the output tensor of the input node with an initializer list.

Exceptions

None	explicitly, but the Tensor constructor may throw exceptions such as std::invalid_argument if the initializer list size is insufficient or std::bad_alloc if memory allocation fails.

Note

• Ensure that the std::initializer_list contains enough elements to fill the tensor according to the specified shape.
• The CUDA runtime environment should be properly initialized before calling this constructor if the tensor is using CUDA memory.
• The time complexity of this constructor is O(1) for creating the std::shared_ptr and O(n) for the Tensor constructor, where n is the total number of elements in the tensor.

```cpp
#include <vector>
// Assume Tensor::shape_type and Tensor::value_type are defined
using shape_type = std::vector<size_t>;
using value_type = float;
 
shape_type shape = {2, 3};
try {
    InputNode inputNode(shape, {1.0f, 2.0f, 3.0f, 4.0f, 5.0f, 6.0f}, true);
} catch (const std::exception& e) {
    std::cerr << e.what() << std::endl;
}
```

Definition at line 40 of file Nodes.cu.

Member Function Documentation

backward()

void nz::nodes::io::InputNode::backward ( )

overridevirtual

Backward pass for the InputNode.

The backward() method for the InputNode is a no-op (no operation) because the input node does not participate in the backpropagation process. It does not have any parameters or gradients to update, as its role is simply to provide the input data to the computational graph.

Since the InputNode does not modify any data during the backward pass, the backward() method does not need to perform any operations, and it is left empty. Gradients will not be propagated from this node to any other nodes, as the InputNode does not require gradients.

Note

• This method is a required implementation due to inheritance from the abstract Node class, but it does not perform any operations for InputNode. It serves as a placeholder to conform to the Node class interface.
• The InputNode is used to provide data to the computational graph, but since it doesn't have parameters, there is no need to propagate gradients through it.

See also

forward() for the forward pass computation method.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/11/29

Implements nz::nodes::Node.

Definition at line 49 of file Nodes.cu.

forward()

void nz::nodes::io::InputNode::forward ( )

overridevirtual

Forward pass for the InputNode.

The forward() method for the InputNode is a no-op (no operation) because the input node does not perform any computations during the forward pass. Its primary role is to provide the input data to the computational graph.

Since the InputNode does not modify its data or perform any calculations, the forward() method does not need to be implemented with any functionality, and it is left empty. The tensor stored in the output member will simply be passed along to the next nodes in the computational graph during the forward pass.

Note

• This method is a required implementation due to inheritance from the abstract Node class, but it does not perform any operations for InputNode. It serves as a placeholder to conform to the Node class interface.
• The InputNode only holds input data, and its forward() method ensures that the data is available for the subsequent nodes in the graph.

See also

backward() for the reverse propagation (gradient calculation) method.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/11/29

Implements nz::nodes::Node.

Definition at line 46 of file Nodes.cu.

---

The documentation for this class was generated from the following files:

• D:/Users/Mgepahmge/Documents/C Program/NeuZephyr/include/NeuZephyr/Nodes.cuh
• D:/Users/Mgepahmge/Documents/C Program/NeuZephyr/src/Nodes.cu