nz::graph::ComputeGraph Class Reference

Represents a computational graph, which manages nodes and the computation flow. More...

Public Member Functions
Constructors and Destructors
	ComputeGraph ()=default
	Default constructor for the ComputeGraph class.
	~ComputeGraph ()=default
	Destructor for the ComputeGraph class.
Graph Builders
InputNode *	addInput (const Tensor::shape_type &shape, bool requires_grad=false, const std::string &name="default")
	Adds an input node to the computational graph.
InputNode *	addInput (const Tensor &tensor, const std::string &name="default")
	Adds an input node to the computational graph using a Tensor.
InputNode *	addInput (InputNode *input, const std::string &name="default")
	Adds an existing InputNode to the computational graph.
InputNode *	addInput (const Tensor::shape_type &shape, Tensor::value_type *data, bool requires_grad, bool host, const std::string &name="default")
	Adds an input node to the computation graph and returns a pointer to the newly created InputNode.
InputNode *	addInput (const Tensor::shape_type &shape, const std::initializer_list< Tensor::value_type > &data, bool requires_grad, const std::string &name="default")
	Adds an InputNode to the ComputeGraph using a std::initializer_list for data and returns a pointer to the created node.
template<typename NodeType >
NodeType *	addNode (NodeType *node, const std::string &name="default")
	Adds a node of any type to the computational graph.
template<typename... Args>
Node *	addNode (const std::string &type, const std::string &input1, const std::string &input2, const std::string &name="default", Args... args)
	Adds a node to the computational graph based on the provided node type and inputs.
OutputNode *	addOutput (OutputNode *node, const std::string &name="default")
	Adds an output node to the computational graph.
Modifiers
void	topologicalSort ()
	Performs topological sorting on the computational graph.
void	zeroGrad () const
	Resets the gradients of all nodes in the computational graph.
void	randomize (const std::string &name, unsigned long long seed=0)
	Randomizes the output tensor of a specified node in the computational graph.
void	randomize (const Node *node, unsigned long long seed=0)
	Randomizes the output tensor of a specified node in the computational graph.
void	randomizeAll () const
	Randomizes the output tensors of all input nodes in the computational graph.
void	fill (const std::string &name, Tensor::value_type val)
	Fills the output tensor of a specified node with a given value.
void	fill (const Node *node, Tensor::value_type val)
	Fills the output tensor of a specified node with a given value.
void	fillAll (Tensor::value_type val) const
	Fills the output tensors of all input nodes with a given value.
void	setInput (const std::string &name, Tensor::value_type *data)
	Sets the input data for a specified node in the computational graph.
void	setInput (const Node *node, Tensor::value_type *data)
	Sets the input data for a specified node in the computational graph using a node pointer.
Getters
bool	isSorted () const
	Checks whether the computational graph has been topologically sorted.
Tensor::value_type *	getOutput () const
	Retrieves the output data of the first output node in the computational graph.
Tensor::value_type *	getOutputHost () const
	Retrieves the output data of the first output node in the computational graph and copies it to host memory.
OutputNode *	getOutputNode () const
	Retrieves the first output node in the computational graph.
Tensor::value_type	getLoss () const
	Retrieves the loss value from the first output node in the computational graph.
std::ostream &	print (std::ostream &os)
	Prints the details of the computational graph to the provided output stream.
Node *	operator[] (const std::string &name)
	Retrieves the node associated with the given name in the computational graph.
std::string	nodesList ()
	Generates a formatted string representing the list of nodes in the ComputeGraph.
Computation
void	forward ()
	Performs forward propagation on the computational graph.
void	backward ()
	Performs backward propagation on the computational graph.
void	update (Optimizer *optimizer) const
	Updates the parameters of the nodes that require gradients using the provided optimizer.
File Managers
void	save (const std::string &path)
	Saves the current computational graph to a JSON file.
void	load (const std::string &path)
	Loads a computational graph from a JSON file.

Friends
DL_API std::ostream &	operator<< (std::ostream &os, ComputeGraph &graph)
	Overloads the stream insertion operator to print the details of the computational graph.
DL_API void	CreateNode (ComputeGraph *graph, const std::string &type, const std::string &name, std::vector< int > pre, const Tensor::shape_type &shape, float *data, bool requires_grad, float *grad)
	Creates and adds a node to the computational graph based on the specified type.

Detailed Description

Represents a computational graph, which manages nodes and the computation flow.

The ComputeGraph class is responsible for creating, managing, and computing the flow of nodes in a neural network or any other computational graph. It handles the addition of input and output nodes, as well as performing forward and backward passes through the graph. It also supports gradient updates, randomization of node values, and node management such as saving, loading, and setting node values.

Key features:

• Graph Management: The class manages a list of nodes, input nodes, output nodes, and ensures that nodes are added and connected properly.
• Forward and Backward Passes: The forward() and backward() methods execute the forward and backward computation across all nodes in the graph.
• Topological Sort: The topologicalSort() method sorts the nodes in a way that respects their dependencies, ensuring that computation happens in the correct order.
• Randomization and Initialization: The graph supports random initialization of node values via the randomize() and randomizeAll() methods.
• Gradient Zeroing: The zeroGrad() method zeros the gradients of all nodes in the graph.
• Saving and Loading: The save() and load() methods allow for the persistence of the graph’s state.

This class is designed to be used in a computational graph where nodes represent various mathematical operations and tensors represent the data that flows through the graph.

Supported node types

Type	Reference
Input	nz::nodes::io::InputNode
Output	nz::nodes::io::OutputNode
Add	nz::nodes::calc::AddNode
MatMul	nz::nodes::calc::MatMulNode
ScalarMul	nz::nodes::calc::ScalarMulNode
ScalarDiv	nz::nodes::calc::ScalarDivNode
ScalarAdd	nz::nodes::calc::ScalarAddNode
ScalarSub	nz::nodes::calc::ScalarSubNode
Sub	nz::nodes::calc::SubNode
ReLU	nz::nodes::calc::ReLUNode
Sigmoid	nz::nodes::calc::SigmoidNode
Tanh	nz::nodes::calc::TanhNode
LeakyReLU	nz::nodes::calc::LeakyReLUNode
Swish	nz::nodes::calc::SwishNode
ELU	nz::nodes::calc::ELUNode
HardSigmoid	nz::nodes::calc::HardSigmoidNode
HardSwish	nz::nodes::calc::HardSwishNode
Softmax	nz::nodes::calc::SoftmaxNode
MeanSquaredError	nz::nodes::loss::MeanSquaredErrorNode
BinaryCrossEntropy	nz::nodes::loss::BinaryCrossEntropyNode

Note

• The graph handles nodes by their names. Each node is stored in a node roster, allowing for easy lookup.
• The nodes should be connected properly for the forward and backward passes to work correctly.

Usage Example:

// Create Graph (Method 1)
graph::ComputeGraph graph;
 
auto* input1 = graph.addInput({3, 4}, false, "Input");  // Add input data
auto* input2 = graph.addInput({4, 3}, true, "Weight");
auto* input3 = graph.addInput({3, 3}, false, "Label");
 
nodes::calc::MatMulNode matmul(input1, input2); // Add Computation nodes
graph.addNode(&matmul, "MatMul");
nodes::calc::ReLUNode relu(&matmul);
graph.addNode(&relu, "ReLU");
 
nodes::loss::MeanSquaredErrorNode loss(&relu, input3); // Add loss function
graph.addOutput(&loss, "Loss");
 
graph.randomizeAll(); // init data
 
// Create graph (Method 2)
graph::ComputeGraph graph;
 
graph.addInput({3, 4}, false, "Input");
graph.addInput({4, 3}, true, "Weight");
graph.addInput({3, 3}, false, "Label");
 
graph.addNode("MatMul", "Input", "Weight", "MatMul");
graph.addNode("ReLU", "MatMul", "", "ReLU");
graph.addNode("MeanSquaredError", "ReLU", "Label");
 
graph.randomizeAll();
 
// Perform forward and backward passes
graph.forward();
graph.backward();
std::cout << graph << std::endl; // Print result
 
// Update weights
opt::SGD optimizer(0.01); // Create optimizer
graph.update(&optimizer); // Update weights
 
graph.forward();
std::cout << graph << std::endl;
 
// Save model
graph.save("model.json");
 
// Load model
graph::ComputeGraph graph;
graph.load("model.json");
graph.forward();
graph.backward();
opt::Adam optimizer(0.01, 0.9, 0.99);
graph.update(&optimizer);
graph.forward();
std::cout << graph << std::endl;

See also

forward() for the forward pass computation method.

backward() for the backward pass gradient propagation method.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 224 of file ComputeGraph.cuh.

Constructor & Destructor Documentation

ComputeGraph()

nz::graph::ComputeGraph::ComputeGraph ( )

default

Default constructor for the ComputeGraph class.

This constructor initializes the ComputeGraph object. It sets up all internal data structures, such as the lists for nodes, input nodes, output nodes, and the node roster. The graph is initially empty and requires the addition of nodes and connections to form a complete computational graph.

The constructor doesn't require any arguments and doesn't allocate any resources other than those necessary for the internal structure of the graph.

Note

• The constructor does not perform any computations or node additions. It merely initializes the empty graph.

See also

~ComputeGraph() for the destructor that cleans up the resources used by the graph.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

~ComputeGraph()

nz::graph::ComputeGraph::~ComputeGraph ( )

default

Destructor for the ComputeGraph class.

The destructor ensures that any resources allocated by the ComputeGraph object, such as the nodes and their associated data, are properly cleaned up when the object is destroyed. It performs any necessary memory deallocation or resource release.

Note

• The destructor does not need to manually delete each node in the graph, as they are typically managed by smart pointers or are otherwise not responsible for memory deallocation.

See also

ComputeGraph() for the constructor that initializes the graph.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Member Function Documentation

addInput() [1/5]

InputNode * nz::graph::ComputeGraph::addInput	(	const Tensor &	tensor,
		const std::string &	name = "default" )

Adds an input node to the computational graph using a Tensor.

This method creates a new InputNode using the provided Tensor and adds it to the ComputeGraph object. The newly created node is added to both the inputNodes and nodes lists. Its name is stored in the nodeRoster and nodeRosterReverse maps, allowing for easy lookup by name. If no name is provided, a unique name is generated.

Parameters

tensor	The Tensor object that represents the data for the input node.
name	The name of the input node. If "default", a unique name is generated.

Returns

A pointer to the newly created InputNode.

Note

• If the name is "default", a unique name will be generated for the input node by concatenating the node's type and a reference counter.
• The node is added to both inputNodes and nodes, ensuring it is part of the computational graph.

Usage Example:

Tensor input_tensor({3, 3}, true);  // Example tensor
ComputeGraph graph;
InputNode* input = graph.addInput(input_tensor, "input_node");

See also

nodes::io::InputNode for the class definition of the input node.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 319 of file ComputeGraph.cu.

addInput() [2/5]

InputNode * nz::graph::ComputeGraph::addInput	(	const Tensor::shape_type &	shape,
		bool	requires_grad = false,
		const std::string &	name = "default" )

Adds an input node to the computational graph.

This method creates a new InputNode with the specified shape and gradient requirements and adds it to the ComputeGraph object. The newly created node is also added to the inputNodes list, and its name is recorded in the nodeRoster and nodeRosterReverse maps for easy access by name.

Parameters

shape	The shape of the input tensor for the node.
requires_grad	A boolean indicating whether the input node requires gradients for backpropagation.
name	The name of the input node. If not provided, a default name is generated.

Returns

A pointer to the newly created InputNode.

Note

• If the name is "default", a unique name is generated for the input node using the node's type and a reference counter.
• The node is added to both inputNodes and nodes, ensuring it is part of the overall computational graph.

Usage Example:

ComputeGraph graph;
InputNode* input = graph.addInput({3, 3}, true, "input_node");

See also

nodes::io::InputNode for the class definition of the input node.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 302 of file ComputeGraph.cu.

addInput() [3/5]

InputNode * nz::graph::ComputeGraph::addInput	(	const Tensor::shape_type &	shape,
		const std::initializer_list< Tensor::value_type > &	data,
		bool	requires_grad,
		const std::string &	name = "default" )

Adds an InputNode to the ComputeGraph using a std::initializer_list for data and returns a pointer to the created node.

Parameters

shape	A reference to the shape of the tensor associated with the InputNode (host-to-device). Defines the dimensions of the input tensor.
data	A std::initializer_list containing the initial values for the tensor (host-to-device).
requires_grad	A boolean indicating whether the tensor of the InputNode should require gradient computation.
name	A string representing the name of the InputNode. If set to "default", a unique name will be generated.

Returns

A pointer to the newly created InputNode.

This function is responsible for creating and adding an InputNode to the ComputeGraph. Memory management: It uses the new operator to allocate memory for the InputNode, and the ComputeGraph takes ownership of this memory. The memory should be deallocated when the ComputeGraph is destroyed.

Exception handling: This function does not explicitly catch exceptions. If memory allocation for the InputNode fails (new throws std::bad_alloc), or if there are issues with the shape or data passed to the InputNode constructor, the exceptions will propagate to the caller.

It interacts with the nodes, inputNodes, nodeRoster, and nodeRosterReverse members of the ComputeGraph to manage the list of nodes and the mapping between node names and pointers.

Exceptions

std::bad_alloc

If memory allocation for the InputNode fails.

Note

• Ensure that the shape and data are compatible with the requirements of the InputNode constructor.
• If the name is "default", a unique name will be generated based on the node type and a reference counter.
• The time complexity of this function is O(1) for node creation and O(log m) for insertion into the nodeRoster map, where m is the number of nodes in the graph.

Warning

• Do not delete the returned pointer as the ComputeGraph takes ownership of the InputNode.

```cpp
#include <vector>
 
ComputeGraph graph;
shape_type shape = {2, 2};
InputNode* inputNode = graph.addInput(shape, {1.0f, 2.0f, 3.0f, 4.0f}, true, "my_input");
```

Definition at line 370 of file ComputeGraph.cu.

addInput() [4/5]

InputNode * nz::graph::ComputeGraph::addInput	(	const Tensor::shape_type &	shape,
		Tensor::value_type *	data,
		bool	requires_grad,
		bool	host,
		const std::string &	name = "default" )

Adds an input node to the computation graph and returns a pointer to the newly created InputNode.

Parameters

shape	A reference to the shape of the input tensor (host-to-device). This defines the dimensions of the tensor associated with the input node.
data	A pointer to the initial data for the input tensor (host-to-device). It can be nullptr if no initial data is provided.
requires_grad	A boolean indicating whether the input tensor should require gradient computation.
host	A boolean indicating whether the tensor data is stored on the host.
name	A string representing the name of the input node. If set to "default", a unique name will be generated.

Returns

A pointer to the newly created InputNode.

This function is used to add an input node to the computation graph. Memory management: It dynamically allocates memory for the InputNode using the new operator. The caller does not need to free this memory directly as the graph takes ownership of the node. When the graph is destroyed, it should be responsible for deallocating the memory of all its nodes.

Exception handling: This function does not explicitly catch exceptions. If memory allocation for the InputNode fails (new throws std::bad_alloc), the exception will propagate to the caller. Also, if there are issues with the provided shape or data, the constructor of InputNode may throw relevant exceptions.

This function interacts with the nodes, inputNodes, nodeRoster, and nodeRosterReverse members of the ComputeGraph class to manage the list of nodes and the mapping between node names and pointers.

Exceptions

std::bad_alloc

If memory allocation for the InputNode fails.

Note

• Ensure that the shape and data are valid and compatible with the requirements of the InputNode constructor.
• If the name is set to "default", a unique name will be generated based on the node type and a reference counter.
• The time complexity of this function is O(1) for the node creation and O(log m) for the insertion into the nodeRoster map, where m is the number of nodes in the graph.

Warning

• The caller should not delete the returned pointer as the graph takes ownership of the node.

```cpp
#include <vector>
 
ComputeGraph graph;
shape_type shape = {2, 2};
value_type data[] = {1.0f, 2.0f, 3.0f, 4.0f};
InputNode* inputNode = graph.addInput(shape, data, true, true, "my_input");
```

Definition at line 352 of file ComputeGraph.cu.

addInput() [5/5]

InputNode * nz::graph::ComputeGraph::addInput	(	InputNode *	input,
		const std::string &	name = "default" )

Adds an existing InputNode to the computational graph.

This method adds an already created InputNode to the ComputeGraph. The node is pushed into both the nodes and inputNodes lists. If a name is provided, the node is stored in the nodeRoster and nodeRosterReverse maps with the given name. If the name is "default", a unique name is generated for the node.

Parameters

input	A pointer to the existing InputNode to be added to the graph.
name	The name of the input node. If "default", a unique name is generated for the node.

Returns

A pointer to the added InputNode.

Note

• If the name is "default", a unique name is generated by concatenating the node's type and a reference counter.
• The node is added to both the inputNodes and nodes lists, ensuring it becomes part of the computational graph.

Usage Example:

ComputeGraph graph;
InputNode* input = new InputNode({3, 3}, true);
graph.addInput(input, "input_node");

See also

nodes::io::InputNode for more details on the input node class.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 336 of file ComputeGraph.cu.

addNode() [1/2]

template<typename... Args>

Node * nz::graph::ComputeGraph::addNode ( const std::string & type, const std::string & input1, const std::string & input2, const std::string & name = "default", Args... args )

inline

Adds a node to the computational graph based on the provided node type and inputs.

This method adds a node to the computational graph based on the specified node type (e.g., "Input", "Output", "Add", "MatMul", etc.). It also ensures that the required input nodes are present in the graph. Depending on the node type, the method creates and adds the corresponding node to the graph and returns a pointer to the added node. If the node type or input nodes are invalid, an exception will be thrown.

Parameters

type	The type of the node to be added. It can be one of the following: "Input", "Output", "Add", "MatMul", "Sub", "ReLU", "Sigmoid", "Tanh", "LeakyReLU", "Swish", "ELU", "HardSigmoid", "HardSwish", "Softmax", "MeanSquaredError", "BinaryCrossEntropy", "ScalarAdd", "ScalarSub", "ScalarMul", "ScalarDiv".
input1	The name of the first input node. The exact meaning depends on the node type.
input2	The name of the second input node (if required by the node type).
name	The name of the node to be added. If "default", a unique name is generated. The default value is "default".
args	Additional arguments required for some node types (e.g., parameters for LeakyReLU, ELU, etc.).

Returns

A pointer to the newly added node.

Exceptions

std::runtime_error

If any required input node is not found, or if an unsupported node type is provided.

Note

• For "Input" and "Scalar" nodes, the method will print a warning and return nullptr because these nodes cannot be added using this method.
• If the specified node type requires specific input nodes, the method checks if the input nodes exist in the graph before proceeding.
• The method supports variable arguments (Args... args) for nodes like LeakyReLU, ELU, and others that may require additional parameters.

Usage Example:

ComputeGraph graph;
Node* addNode = graph.addNode("Add", "input1", "input2", "add_node");
if (addNode != nullptr) {
    // Use the addNode pointer here
}

See also

nodes::Node for the base class of all node types.

nodes::io::OutputNode, nodes::calc::AddNode, nodes::calc::MatMulNode, etc., for the specific node classes that are created.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 571 of file ComputeGraph.cuh.

addNode() [2/2]

template<typename NodeType >

NodeType * nz::graph::ComputeGraph::addNode ( NodeType * node, const std::string & name = "default" )

inline

Adds a node of any type to the computational graph.

This template method allows adding a node of any type derived from the Node class to the computational graph. The node is added to the nodes list and optionally assigned a name. If the name is "default", a unique name is generated using the node's type and a reference counter.

Template Parameters

NodeType

The type of the node, which must be derived from Node. This allows the method to work with different types of nodes (e.g., InputNode, OutputNode, etc.).

Parameters

node	A pointer to the node to be added to the graph.
name	The name of the node. If "default", a unique name will be generated. The default value is "default".

Returns

A pointer to the added node.

Note

• If the name is "default", the method generates a unique name for the node by concatenating the node's type and a reference counter (e.g., Input_1, Add_2).
• The node is added to both the nodes list (for graph traversal) and the nodeRoster/nodeRosterReverse maps (for name-based access).

Usage Example:

ComputeGraph graph;
auto* node = new AddNode(&input1, &input2);
graph.addNode(node, "Add");

See also

nodes::Node for the base node class.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 511 of file ComputeGraph.cuh.

addOutput()

OutputNode * nz::graph::ComputeGraph::addOutput	(	OutputNode *	node,
		const std::string &	name = "default" )

Adds an output node to the computational graph.

This method adds an OutputNode to the computational graph. It takes an OutputNode pointer and an optional name. The node is added to both the nodes list and the outputNodes list. If the name is "default", a unique name is generated using the node's type and a reference counter.

Parameters

node	A pointer to the OutputNode to be added to the graph.
name	The name of the node. If "default", a unique name will be generated. The default value is "default".

Returns

A pointer to the added OutputNode.

Note

• If the name is "default", the method generates a unique name for the node by concatenating the node's type and a reference counter (e.g., Output_1, Output_2).
• The node is added to both the nodes list and the outputNodes list for graph traversal and output handling.
• The node's name is also stored in the nodeRoster and nodeRosterReverse maps.

Usage Example:

ComputeGraph graph;
 
// Add basic output node
auto* outputNode = new OutputNode(inputNode); // assuming inputNode is a valid InputNode pointer
graph.addOutput(outputNode, "output");
 
// Add loss function node
auto outputNode = new MeanSquaredErrorNode(inputNode1, inputNode2);
graph.addOutput(outputNode, "output");

See also

nodes::io::OutputNode for the output node class.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 389 of file ComputeGraph.cu.

backward()

void nz::graph::ComputeGraph::backward

(

)

Performs backward propagation on the computational graph.

This method performs backward propagation starting from the output node(s) in the computational graph. The method first checks if the graph is sorted. If the graph is not sorted, a runtime error is thrown. If the graph is sorted, the backward propagation is performed by iterating over the nodes in reverse topological order (i.e., from outputs to inputs). Each node’s backward() method is called to compute gradients with respect to its inputs.

Parameters

None

Returns

None

Exceptions

std::runtime_error

If the graph is not sorted or if there are multiple output nodes or no output node.

Note

• If the graph is not sorted, a runtime error is thrown because backward propagation relies on the topological order of the nodes. Sorting ensures that each node’s gradient can be computed in the correct order.
• If the graph has exactly one output node, the method proceeds with backward propagation in reverse topological order (from the output node back to the input nodes).
• If the graph has no output nodes or multiple output nodes, a runtime error is thrown.

Why Not Automatically Sort the Graph?

The backward() method does not automatically sort the graph because backward propagation must correspond to a previously completed forward pass. A forward pass determines the order of operations and ensures that the graph is in a valid state for backward propagation. Automatically sorting the graph would interfere with this flow, as backward propagation is dependent on the state of the graph after forward propagation. Hence, the graph is only processed for backward propagation if it has been sorted and the forward pass has already occurred.

Usage Example:

ComputeGraph graph;
// Assuming nodes are added and the graph is sorted...
graph.forward();  // Perform forward propagation first
graph.backward(); // Perform backward propagation after forward pass

See also

isSorted() for the method that checks if the graph is sorted.

forward() for the method that performs forward propagation.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 414 of file ComputeGraph.cu.

Here is the call graph for this function:

fill() [1/2]

void nz::graph::ComputeGraph::fill	(	const Node *	node,
		Tensor::value_type	val )

Fills the output tensor of a specified node with a given value.

This method fills the output tensor of a node, identified by its pointer, with a specified value. It checks if the node is present in the graph by searching the node pointer in the nodes list. If the node is found, it calls the fill method on the node's output tensor to set all its elements to the provided value. If the node is not found in the graph, an exception is thrown.

Parameters

node	A pointer to the node whose output tensor will be filled.
val	The value to fill the output tensor with.

Exceptions

std::runtime_error

if the node is not found in the graph.

Usage Example:

ComputeGraph graph;
// Assuming graph has nodes added and a valid node pointer "inputNode"
graph.fill(inputNode, 0.0);  // Fills the output tensor of "inputNode" with 0.0

See also

Tensor::fill() for the method that fills the tensor with a specific value.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 475 of file ComputeGraph.cu.

fill() [2/2]

void nz::graph::ComputeGraph::fill	(	const std::string &	name,
		Tensor::value_type	val )

Fills the output tensor of a specified node with a given value.

This method fills the output tensor of a node, identified by its name, with a specified value. It looks up the node by name in the nodeRoster. If the node is found, it calls the fill method on the node's output tensor, setting all its elements to the provided value. If the node is not found, an exception is thrown.

Parameters

name	The name of the node whose output tensor will be filled.
val	The value to fill the output tensor with.

Exceptions

std::runtime_error

if the node with the specified name is not found in the graph.

Usage Example:

ComputeGraph graph;
// Assuming graph has nodes added and a node named "input_1"
graph.fill("input_1", 0.0);  // Fills the output tensor of "input_1" with 0.0

See also

Tensor::fill() for the method that fills the tensor with a specific value.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 466 of file ComputeGraph.cu.

fillAll()

void nz::graph::ComputeGraph::fillAll

(

Tensor::value_type

val

)

const

Fills the output tensors of all input nodes with a given value.

This method iterates over all input nodes in the computational graph and fills their output tensors with the specified value. It calls the fill method on each input node's output tensor to set all its elements to the provided value. This operation is performed for every input node in the graph.

Parameters

val	The value to fill the output tensors of all input nodes with.

Usage Example:

ComputeGraph graph;
// Assuming graph has input nodes added
graph.fillAll(0.0);  // Fills the output tensors of all input nodes with 0.0

See also

Tensor::fill() for the method that fills the tensor of a specific node with a value.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 484 of file ComputeGraph.cu.

forward()

void nz::graph::ComputeGraph::forward

(

)

Performs forward propagation on the computational graph.

This method performs forward propagation on all nodes in the computational graph. It first ensures the graph is sorted in topological order (if not already sorted), and then propagates the data through each node in the sorted order. Each node's forward() method is called to compute its output based on its inputs.

Parameters

None

Returns

None

Note

• This method checks if the graph is sorted by calling the isSorted() method. If the graph is not sorted, it calls the topologicalSort() method to sort the nodes in topological order before performing the forward propagation.
• The nodes are processed in sorted order, ensuring that each node’s inputs are computed before the node itself.
• After calling topologicalSort(), the forward() method calls each node's forward() method to compute the node’s output and propagate the result through the graph.

Usage Example:

ComputeGraph graph;
// Assuming nodes are added to the graph...
graph.forward(); // Performs forward propagation on all nodes in the graph

See also

topologicalSort() for the method that sorts the nodes in topological order.

isSorted() for the method that checks if the graph is sorted.

backward() for the method that performs backward propagation.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 405 of file ComputeGraph.cu.

Here is the call graph for this function:

getLoss()

Tensor::value_type nz::graph::ComputeGraph::getLoss ( ) const

nodiscard

Retrieves the loss value from the first output node in the computational graph.

This method retrieves the loss value computed by the first OutputNode in the computational graph. The method assumes that there is at least one output node in the graph. If no output nodes exist, a std::runtime_error is thrown.

Returns

The loss value computed by the first output node in the graph.

Exceptions

std::runtime_error

If no output nodes are present in the graph.

Usage Example:

ComputeGraph graph;
try {
    Tensor::value_type loss = graph.getLoss();
    std::cout << "Loss: " << loss << std::endl;
} catch (const std::runtime_error& e) {
    // Handle the case when no output node is present
    std::cerr << "Error: " << e.what() << std::endl;
}

See also

nodes::io::OutputNode for the output node class.

OutputNode::getLoss() for the method in the OutputNode class that computes the loss.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 534 of file ComputeGraph.cu.

getOutput()

Tensor::value_type * nz::graph::ComputeGraph::getOutput ( ) const

nodiscard

Retrieves the output data of the first output node in the computational graph.

This method retrieves a pointer to the output data of the first OutputNode in the computational graph. The output data is stored in the output tensor of the node. It is important to note that the returned pointer points to data that resides in GPU memory.

If no output nodes exist in the graph, a std::runtime_error is thrown. The method assumes that there is at least one output node in the graph, and will not return a nullptr.

Returns

A pointer to the output data of the first output node in the graph, which is stored in GPU memory.

Exceptions

std::runtime_error

If no output nodes are present in the graph.

Usage Example:

ComputeGraph graph;
try {
    Tensor::value_type* outputData = graph.getOutput();
    // Use the outputData pointer here
} catch (const std::runtime_error& e) {
    // Handle the case when no output node is present
    std::cerr << "Error: " << e.what() << std::endl;
}

See also

nodes::io::OutputNode for the output node class.

data::Tensor for the class representing tensors and their associated operations.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 508 of file ComputeGraph.cu.

getOutputHost()

Tensor::value_type * nz::graph::ComputeGraph::getOutputHost ( ) const

nodiscard

Retrieves the output data of the first output node in the computational graph and copies it to host memory.

This method retrieves a pointer to the output data of the first OutputNode in the computational graph. It then copies the data from GPU memory to host memory. The returned pointer points to a memory block in host memory that contains the output data.

If the graph contains no output nodes, a runtime error is thrown. The method assumes that there is at least one output node in the graph; otherwise, it will throw an exception.

The returned pointer points to memory allocated in the host's memory space. The caller is responsible for freeing this memory using free() once it's done using the data.

Returns

A pointer to the output data of the first output node in the graph, stored in host memory. The memory must be freed by the caller after use.

Exceptions

std::runtime_error

If no output nodes are present in the graph.

Usage Example:

ComputeGraph graph;
try {
    Tensor::value_type* outputDataHost = graph.getOutputHost();
    // Use the outputDataHost pointer here
    // Remember to free the memory when done
    free(outputDataHost);
} catch (const std::runtime_error& e) {
    std::cerr << "Error: " << e.what() << std::endl;
}

See also

nodes::io::OutputNode for the output node class.

data::Tensor for the class representing tensors and their associated operations.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 515 of file ComputeGraph.cu.

Here is the call graph for this function:

getOutputNode()

OutputNode * nz::graph::ComputeGraph::getOutputNode ( ) const

nodiscard

Retrieves the first output node in the computational graph.

This method retrieves the first OutputNode in the computational graph. The method assumes that there is at least one output node in the graph. If no output nodes exist, a std::runtime_error is thrown.

Returns

A pointer to the first output node in the graph.

Exceptions

std::runtime_error

If no output nodes are present in the graph.

Usage Example:

ComputeGraph graph;
try {
    OutputNode* outputNode = graph.getOutputNode();
    // Use the outputNode pointer here
} catch (const std::runtime_error& e) {
    // Handle the case when no output node is present
    std::cerr << "Error: " << e.what() << std::endl;
}

See also

nodes::io::OutputNode for the output node class.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 527 of file ComputeGraph.cu.

isSorted()

bool nz::graph::ComputeGraph::isSorted ( ) const

nodiscard

Checks whether the computational graph has been topologically sorted.

This function checks if the sortedNodes vector contains the nodes in a valid topologically sorted order. It returns true if the graph is sorted, meaning that each node appears before any node that depends on it. Otherwise, it returns false, indicating that the graph is not sorted.

Returns

true if the graph is sorted, false if not.

Note

• This function does not modify the state of the graph.
• It is a helper function that can be used to verify whether a graph needs sorting before traversing.

Usage Example:

ComputeGraph graph;
// Add nodes to the graph...
if (!graph.isSorted()) {
    graph.topologicalSort();  // Sort the graph if it is not sorted
}

See also

topologicalSort() for the function that performs the sorting.

ComputeGraph for more details on the graph structure and node management.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 298 of file ComputeGraph.cu.

load()

void nz::graph::ComputeGraph::load

(

const std::string &

path

)

Loads a computational graph from a JSON file.

This method deserializes the computational graph from the provided JSON file and reconstructs the nodes, their types, names, input-output relationships, shapes, data, gradients, and other relevant information. It validates the file structure and populates the graph accordingly.

Graph Deserialization:

• Nodes: Each node's type, name, input-output connections, and other details are extracted.
• Pre and Post nodes: Lists of indices for input (pre) and output (post) nodes are read.
• Data and Gradients: Node data and gradients (if required) are read and restored into their respective tensors.

Error Handling:

• Throws a std::runtime_error if the path is empty, the graph is already loaded, or there is an issue opening the file.

Usage Example:

ComputeGraph graph;
graph.load("path_to_load_graph.json");

Parameters

path	The file path from which the graph should be loaded.

Exceptions

std::runtime_error

If the path is empty, the graph is already loaded, or file reading fails.

See also

nodes::Node for the base class of all nodes.

data::Tensor for the class representing tensors and their associated operations.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 642 of file ComputeGraph.cu.

nodesList()

std::string nz::graph::ComputeGraph::nodesList

(

)

Generates a formatted string representing the list of nodes in the ComputeGraph.

Returns

A string containing a tabular representation of node names and types.

This function iterates through the nodeRoster of the ComputeGraph to determine the maximum width required for displaying node names and types. It then constructs a formatted string using std::ostringstream to present the nodes in a tabular format.

Memory management: The function does not allocate any dynamic memory that needs to be explicitly managed. It uses local variables and the std::ostringstream which handles its own memory internally.

Exception handling: This function does not explicitly catch exceptions. Exceptions such as std::bad_alloc may be thrown if there is insufficient memory during the construction of the output string.

This function only depends on the nodeRoster member of the ComputeGraph class, which stores the mapping between node names and pointers.

Exceptions

std::bad_alloc

If there is insufficient memory to construct the output string.

Note

• The time complexity of this function is O(n), where n is the number of nodes in the nodeRoster, as it iterates through the map twice.
• The output string is formatted in a left - aligned tabular style.

```cpp
ComputeGraph graph;
// Assume some nodes are added to the graph
std::string nodesListStr = graph.nodesList();
std::cout << nodesListStr << std::endl;
```

Definition at line 817 of file ComputeGraph.cu.

operator[]()

Node * nz::graph::ComputeGraph::operator[]

(

const std::string &

name

)

Retrieves the node associated with the given name in the computational graph.

This method overloads the operator[] to provide access to nodes in the computational graph by their name. It allows for easy retrieval of nodes from the nodeRoster map. The operator returns a pointer to the node associated with the provided name.

If the node with the specified name is not found, the method will cause undefined behavior as it directly accesses the nodeRoster map without checking for the node's existence.

Parameters

name	The name of the node to retrieve.

Returns

A pointer to the node associated with the specified name.

Note

• If the node does not exist in nodeRoster, this method will cause undefined behavior because it directly accesses the map. To safely check for the existence of a node, consider using find() instead.
• This operator does not throw exceptions; it relies on the nodeRoster map's behavior when accessing an element by key.

Usage Example:

ComputeGraph graph;
Node* node = graph["node_name"];
if (node != nullptr) {
    // Use the node here
} else {
    // Handle the case when the node is not found
}

See also

nodeRoster for the map storing nodes by name.

Node for the base class of all nodes.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 813 of file ComputeGraph.cu.

print()

std::ostream & nz::graph::ComputeGraph::print

(

std::ostream &

os

)

Prints the details of the computational graph to the provided output stream.

The print method prints a detailed description of the computational graph, including each node's name, its preceding (input) nodes, following (output) nodes, data, and gradients. If the graph is not sorted, it will automatically perform a topological sort before printing the details. The method assumes that the graph contains at least one output node and prints the loss value of the first output node.

Parameters

os	The output stream where the graph details will be printed (e.g., std::cout).

Returns

The same output stream after printing the graph details, enabling method chaining.

Exceptions

std::runtime_error

if an error occurs during the process.

Note

• If the graph is not sorted, the method will automatically call topologicalSort() to sort the nodes.
• The method prints the loss value of the first output node in the graph, assuming there is at least one output node.

Example:

ComputeGraph graph;
// Add nodes and build the graph
graph.print(std::cout);  // Print the graph details to the console

See also

topologicalSort() for sorting the nodes of the graph.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 8 of file ComputeGraph.cu.

Here is the call graph for this function:

randomize() [1/2]

void nz::graph::ComputeGraph::randomize	(	const Node *	node,
		unsigned long long	seed = 0 )

Randomizes the output tensor of a specified node in the computational graph.

This method sets the values of the specified node's output tensor to random values using the provided random seed. The method first checks if the given node exists in the graph by searching for it in the list of nodes. If the node exists, it calls the randomize() method on the node’s output tensor. If the node is not found in the graph, a runtime error is thrown.

Parameters

node	A pointer to the Node whose output tensor should be randomized.
seed	The seed value for the random number generator.

Exceptions

std::runtime_error

If the node is not found in the graph.

Note

• The randomize() method is expected to be defined for the node's output tensor to set its values randomly.
• The method uses the provided seed value to ensure reproducibility of the randomization process.
• The node is searched in the nodes list to ensure it is part of the graph.

Usage Example:

ComputeGraph graph;
// Assuming "inputNode" is a valid pointer to a node in the graph
graph.randomize(inputNode, 42);  // Randomizes the output of "inputNode" using seed 42

See also

Tensor::randomize() for the method that randomizes a specific node’s output tensor.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 449 of file ComputeGraph.cu.

randomize() [2/2]

void nz::graph::ComputeGraph::randomize	(	const std::string &	name,
		unsigned long long	seed = 0 )

Randomizes the output tensor of a specified node in the computational graph.

This method sets the values of the specified node's output tensor to random values using the provided random seed. The method first checks if the node with the given name exists in the graph. If the node exists, it calls the randomize() method on the node’s output tensor. If the node is not found in the graph, a runtime error is thrown.

Parameters

name	The name of the node whose output tensor should be randomized.
seed	The seed value for the random number generator. If not provided, the seed defaults to 0.

Exceptions

std::runtime_error

If the node with the given name is not found in the graph.

Note

• The randomize() method is expected to be defined for the node's output tensor to set its values randomly.
• The method uses the provided seed value to ensure reproducibility of the randomization process.
• If the node is not found, an error is thrown to inform the user that the node is missing from the graph.

Usage Example:

ComputeGraph graph;
// Assuming "input_node" is a valid node name in the graph
graph.randomize("input_node", 42);  // Randomizes the output of "input_node" using seed 42

See also

Tensor::randomize() for the method that randomizes a tensor's data.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 440 of file ComputeGraph.cu.

randomizeAll()

void nz::graph::ComputeGraph::randomizeAll

(

)

const

Randomizes the output tensors of all input nodes in the computational graph.

This method iterates over all input nodes in the graph and randomizes the output tensor for each of them. It uses the current system time (in nanoseconds) as the seed for the random number generator. Each input node is assigned a unique seed by incrementing the base seed for each randomization.

Note

• The randomization process is applied to each input node's output tensor.
• The seed for randomization is based on the system's current time, ensuring a unique starting point.
• The seed is incremented for each input node to provide a different randomization for each one.

Usage Example:

ComputeGraph graph;
// Assuming graph has input nodes added
graph.randomizeAll();  // Randomizes the output of all input nodes

See also

Tensor::randomize() for the method that randomizes a specific node’s output tensor.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 458 of file ComputeGraph.cu.

save()

void nz::graph::ComputeGraph::save

(

const std::string &

path

)

Saves the current computational graph to a JSON file.

This method serializes the entire computational graph into a JSON file at the specified path. It traverses the nodes in the graph and stores their types, names, input-output relationships, shapes, data, gradients (if required), and other relevant information in JSON format. The serialized graph can later be loaded for further processing or visualization.

Graph Serialization:

• Nodes: Each node's type, name, input-output connections, and other details are stored.
• Pre and Post nodes: Lists of indices for input (pre) and output (post) nodes are saved.
• Data and Gradients: Node data and gradients are copied from GPU to host and serialized.

Error Handling:

• Throws a std::runtime_error if the path is empty, the graph is not sorted, or if there is any failure during file writing.

Usage Example:

ComputeGraph graph;
graph.save("path_to_save_graph.json");

Parameters

path	The file path where the graph should be saved.

Exceptions

std::runtime_error

If the path is empty, the graph is not sorted, or file writing fails.

See also

nodes::Node for the base class of all nodes.

data::Tensor for the class representing tensors and their associated operations.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 553 of file ComputeGraph.cu.

Here is the call graph for this function:

setInput() [1/2]

void nz::graph::ComputeGraph::setInput	(	const Node *	node,
		Tensor::value_type *	data )

Sets the input data for a specified node in the computational graph using a node pointer.

This method sets the input data for a node in the computational graph by copying the provided raw data into the node's output tensor. The input data is assumed to be an array of type Tensor::value_type and will be copied into the output tensor of the specified node. The shape of the output tensor will be used to determine the amount of data to copy.

Parameters

node	A pointer to the Node whose input data is to be set.
data	A pointer to the raw input data that will be copied into the node's output tensor.

Exceptions

std::runtime_error

If the node is not found in the graph.

Usage Example:

ComputeGraph graph;
Tensor::value_type inputData[] = {1.0, 2.0, 3.0};  // Example input data
graph.setInput(inputNode, inputData);  // Sets the input data for the specified node

See also

Tensor::dataInject() for the method that inject the data into the tensor.

data::Tensor for the class representing tensors and their associated operations.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 499 of file ComputeGraph.cu.

setInput() [2/2]

void nz::graph::ComputeGraph::setInput	(	const std::string &	name,
		Tensor::value_type *	data )

Sets the input data for a specified node in the computational graph.

This method sets the input data for a node in the computational graph by copying the provided raw data into the node's output tensor. The input data is assumed to be an array of type Tensor::value_type and will be copied into the output tensor of the node specified by the name. The shape of the output tensor will be used to determine the amount of data to copy.

Parameters

name	The name of the node whose input data is to be set.
data	A pointer to the raw input data that will be copied into the node's output tensor.

Exceptions

std::runtime_error

If the node with the specified name is not found in the graph.

Usage Example:

ComputeGraph graph;
Tensor::value_type inputData[] = {1.0, 2.0, 3.0};  // Example input data
graph.setInput("input_node_name", inputData);  // Sets the input data for the specified node

See also

Tensor::dataInject() for the method that inject the data into the tensor.

data::Tensor for the class representing tensors and their associated operations.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 490 of file ComputeGraph.cu.

topologicalSort()

void nz::graph::ComputeGraph::topologicalSort

(

)

Performs topological sorting on the computational graph.

This function performs topological sorting on the computational graph to order the nodes such that each node appears before any nodes that depend on it. The sorted nodes are stored in the sortedNodes vector, which allows for a correct computation order during graph traversal. It uses Kahn's algorithm for topological sorting.

Exceptions

std::runtime_error

If the graph contains a cycle, indicating that topological sorting is not possible.

This method modifies the following member variables:

• sortedNodes: A vector that stores the nodes in topologically sorted order.
• inDegree: A map that keeps track of the in-degree (number of incoming edges) for each node.
• adjList: A map that stores the adjacency list for each node, representing which nodes depend on it.

Note

• The function assumes that the graph is a Directed Acyclic Graph (DAG). If a cycle is detected during the sorting process, an exception will be thrown.
• This method is useful in scenarios like forward propagation in neural networks, where nodes need to be processed in a specific order.

Algorithm Explanation:

1. Initialize the inDegree of each node to 0.
2. Build the adjList for each node and increment the inDegree of nodes that have incoming edges.
3. Initialize a queue with all nodes that have an in-degree of 0 (i.e., no dependencies).
4. Process each node from the queue, adding it to the sortedNodes list, and decrement the inDegree of its adjacent nodes (i.e., nodes that depend on it). If any adjacent node's in-degree becomes 0, it is added to the queue.
5. If the number of nodes in sortedNodes does not match the total number of nodes in the graph, a cycle is detected and an exception is thrown.

Usage Example:

ComputeGraph graph;
// Add nodes and edges to the graph...
graph.topologicalSort();  // Perform topological sorting

See also

ComputeGraph for more details on the graph structure and node management.

Node for information on individual node types and their operations.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 258 of file ComputeGraph.cu.

update()

void nz::graph::ComputeGraph::update

(

Optimizer *

optimizer

)

const

Updates the parameters of the nodes that require gradients using the provided optimizer.

This method iterates through all the nodes in the computational graph and applies the optimizer's update step to the nodes that have their output tensor marked as requiring gradients. The update is performed by calling the step method of the provided optimizer for each node.

Parameters

optimizer

A pointer to the optimizer that will be used to update the parameters. The optimizer's step method is called for each node that requires gradients.

Exceptions

std::runtime_error

If the optimizer is a null pointer.

Usage Example:

ComputeGraph graph;
Optimizer* optimizer = new SGD(learning_rate);  // assuming an SGD optimizer
graph.update(optimizer);

See also

opt::Optimizer for the interface of the optimizer class.

nodes::Node for the node class that holds the parameters and their gradients.

data::Tensor for the tensor class associated with the node's output.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 541 of file ComputeGraph.cu.

Here is the call graph for this function:

zeroGrad()

void nz::graph::ComputeGraph::zeroGrad

(

)

const

Resets the gradients of all nodes in the computational graph.

This method iterates over all nodes in the computational graph and calls the zeroGrad() method on each node's output tensor to reset its gradient. This is useful to clear the gradients between different backward passes, ensuring that previous gradient values do not accumulate. Typically called at the beginning of each new backward pass to prepare the graph for gradient computation.

Parameters

None

Returns

None

Note

• This method assumes that each node has an associated output tensor with a zeroGrad() method to reset gradients.
• It does not perform any checks on whether the graph is sorted or whether backward propagation has been performed previously. It simply clears the gradients of all nodes in the graph.
• The method is typically used in training loops to avoid gradient accumulation across iterations.

Usage Example:

ComputeGraph graph;
graph.zeroGrad();  // Clears the gradients of all nodes in the graph

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 434 of file ComputeGraph.cu.

Friends And Related Symbol Documentation

CreateNode

DL_API void CreateNode ( ComputeGraph * graph, const std::string & type, const std::string & name, std::vector< int > pre, const Tensor::shape_type & shape, float * data, bool requires_grad, float * grad )

friend

Creates and adds a node to the computational graph based on the specified type.

This function is used to create various types of nodes in a computational graph based on the provided node type, and then adds the created node to the ComputeGraph object. The node is initialized with the specified shape, data, and gradient information if needed. It also ensures that the nodes are connected to their previous nodes as specified by the pre vector.

Parameters

graph	The ComputeGraph object to which the new node will be added.
type	A string representing the type of node to be created. Supported types include "Input", "Output", "Add", "MatMul", "Sub", "ReLU", "Sigmoid", "Tanh", "LeakyReLU", "Swish", "ELU", "HardSigmoid", "HardSwish", "Softmax", "MeanSquaredError", "BinaryCrossEntropy".
name	The name of the node to be added to the graph.
pre	A vector of integers specifying the indices of the previous nodes (input nodes) that this node depends on. The number of elements in pre and the type of node may vary.
shape	A vector representing the shape of the node's output tensor.
data	A pointer to the data to initialize the node's output tensor.
requires_grad	A boolean flag indicating whether the node requires gradients for backpropagation.
grad	A pointer to the gradient data for the node's output tensor if requires_grad is true.

Exceptions

std::runtime_error

If an unsupported node type is provided or if there is a mismatch in node dependencies.

Note

• The CreateNode function automatically handles the creation of nodes, their connection to previous nodes, and the addition of the new node to the graph.
• The pre vector is used to specify which nodes are required as inputs for the current node, and it may differ in size based on the node type.
• Some node types, such as "ScalarMul", "ScalarDiv", "ScalarAdd", and "ScalarSub", are not supported and will throw a runtime error.

Usage Example:

ComputeGraph graph;
std::vector<int> pre = {0, 1};  // Specify the input nodes for the current node
std::vector<int> shape = {3, 3};  // Specify the shape of the output tensor
float data[] = {1.0f, 2.0f, 3.0f, 4.0f, 5.0f, 6.0f, 7.0f, 8.0f, 9.0f};  // Example data
CreateNode(&graph, "Add", "add_node", pre, shape, data, true, nullptr);  // Create an "Add" node

See also

ComputeGraph for more details on graph structure and node management.

nodes::Node for information on individual node types and their operations.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 108 of file ComputeGraph.cu.

operator<<

DL_API std::ostream & operator<< ( std::ostream & os, ComputeGraph & graph )

friend

Overloads the stream insertion operator to print the details of the computational graph.

This function overloads the << operator to provide an easy and intuitive way to print the details of a ComputeGraph object. It calls the print method of ComputeGraph to output the graph's nodes, their connections, data, gradients, and loss to the provided output stream.

Parameters

os	The output stream to which the graph details will be printed (e.g., std::cout).
graph	The ComputeGraph object whose details will be printed.

Returns

The output stream after printing the graph details, enabling method chaining.

See also

ComputeGraph::print() for more information about the internal printing process.

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2024/12/09

Definition at line 56 of file ComputeGraph.cu.

---

The documentation for this class was generated from the following files:

• D:/Users/Mgepahmge/Documents/C Program/NeuZephyr/include/NeuZephyr/ComputeGraph.cuh
• D:/Users/Mgepahmge/Documents/C Program/NeuZephyr/src/ComputeGraph.cu