nz::nodes::calc::AveragePoolingNode Class Reference

Implements average pooling operation for spatial downsampling in neural networks. More...

Inheritance diagram for nz::nodes::calc::AveragePoolingNode:

Collaboration diagram for nz::nodes::calc::AveragePoolingNode:

Public Member Functions
	AveragePoolingNode (Node *input, Tensor::size_type poolSize, Tensor::size_type stride, Tensor::size_type padding)
	Constructs an AveragePoolingNode object.
void	forward () override
	Performs the backward pass of the average pooling operation.
void	backward () override
	Performs the backward pass of the average pooling operation.
Public Member Functions inherited from nz::nodes::Node
virtual void	print (std::ostream &os) const
	Prints the type, data, and gradient of the node.
void	dataInject (Tensor::value_type *data, bool grad=false) const
	Injects data into a relevant tensor object, optionally setting its gradient requirement.
template<typename Iterator >
void	dataInject (Iterator begin, Iterator end, const bool grad=false) const
	Injects data from an iterator range into the output tensor of the InputNode, optionally setting its gradient requirement.
void	dataInject (const std::initializer_list< Tensor::value_type > &data, bool grad=false) const
	Injects data from a std::initializer_list into the output tensor of the Node, optionally setting its gradient requirement.

Detailed Description

Implements average pooling operation for spatial downsampling in neural networks.

This node performs spatial averaging over sliding windows of (poolSize x poolSize) dimensions, reducing feature map resolution while maintaining channel depth. Commonly used for dimensionality reduction and translation invariance in CNNs.

Core functionality and characteristics:

• Spatial Aggregation: Computes mean values over local receptive fields.
• Dimensionality Reduction: Reduces H and W dimensions while preserving channel count.
• Gradient Propagation: Distributes gradients equally across input positions during backward pass.
• Parameter Control: Configurable window size, stride, and padding.
• CUDA Acceleration: GPU-optimized implementation for both forward and backward passes.
• Auto Shape Calculation: Computes output dimensions using pooling parameters.

Key implementation aspects:

• Forward Pass: Applies sliding window averaging with optional zero-padding.
• Backward Pass: Scatters gradients by assigning 1/(K*K) of output gradient to each input position.
• Border Handling: Supports padding for controlling output size.
• Memory Efficiency: Maintains original channel depth for memory layout consistency.

Typical use cases:

• Reducing computational complexity in deep networks.
• Creating translation-invariant feature representations.
• Smoothing feature responses in encoder-decoder architectures.
• Regularization through spatial information compression.

Critical considerations:

• Information Loss: Over-aggressive pooling may discard important spatial details.
• Dimension Calculation: Output dimensions follow: Hout = (H + 2*padding - poolSize)/stride + 1
• Parameter Validation: Requires (H + 2*padding) ≥ poolSize and similar for width.
• Zero Padding Impact: Padding values affect border region averages.

Warning

• Excessive pooling sizes may produce invalid dimensions.
• Small stride values can significantly increase computation cost.
• Non-integer output dimensions will cause runtime errors.

Note

• Output tensor shape: (N, C, Hout, Wout)
• For variable height/width pooling, use separate parameters (not supported in current implementation)
• Gradient computation requires storing input shape during forward pass

See also

MaxPoolingNode For maximum value-based pooling alternative

Usage Example:

// Create input node with batch of 16 256x256 RGB images
InputNode input({16, 3, 256, 256}, true);
 
// Configure 4x4 pooling with stride 2 and padding 1
AveragePoolingNode pool(&input, 4, 2, 1);
pool.forward();
 
// Output shape becomes (16, 3, 128, 128)
std::cout << "Pooled shape: " << pool.output->shape() << std::endl;
 
// Backpropagate through pooling operation
pool.backward();

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2023/10/19

Definition at line 4088 of file Nodes.cuh.

Constructor & Destructor Documentation

AveragePoolingNode()

nz::nodes::calc::AveragePoolingNode::AveragePoolingNode	(	Node *	input,
		Tensor::size_type	poolSize,
		Tensor::size_type	stride,
		Tensor::size_type	padding )

Constructs an AveragePoolingNode object.

Parameters

input	A pointer to the input node. The memory of this pointer is assumed to be managed externally and is used in a read - only manner within this constructor (host - to - host).
poolSize	The size of the pooling window. It is a value of type Tensor::size_type and is used to determine the dimensions of the pooling operation.
stride	The stride value for the pooling operation. It is of type Tensor::size_type and controls how the pooling window moves across the input tensor.
padding	The padding value applied to the input tensor before the pooling operation. It is of type Tensor::size_type.

Returns

None

This constructor initializes an AveragePoolingNode object. It first stores the provided input node pointer in the inputs vector. Then, it creates a new shared pointer to a Tensor object for the output member. The shape of the output tensor is calculated based on the shape of the input tensor, the poolSize, stride, and padding values using the OUTPUT_DIM macro. The requiresGrad flag of the output tensor is set to the same value as that of the input tensor's output. Finally, it sets the type member of the node to "AveragePooling".

Memory management strategy: The constructor does not allocate memory for the input node. It only stores a pointer to it. The output tensor is created using std::make_shared, which manages the memory automatically. Exception handling mechanism: There is no explicit exception handling in this constructor. If the std::make_shared call fails to allocate memory for the output tensor, it may throw a std::bad_alloc exception.

Exceptions

std::bad_alloc

If memory allocation for the output tensor fails.

Note

• Ensure that the input node pointer is valid and points to a properly initialized node.
• The performance of this constructor is mainly determined by the memory allocation for the output tensor, which has a time complexity of O(1) for the pointer management and O(m) for the tensor data allocation, where m is the number of elements in the output tensor.

```cpp
Node* inputNode = new Node(...);
Tensor::size_type poolSize = 2;
Tensor::size_type stride = 2;
Tensor::size_type padding = 0;
AveragePoolingNode avgPoolingNode(inputNode, poolSize, stride, padding);
```

Author

Mgepahmge(https://github.com/Mgepahmge)

Date

2024/07/15

Definition at line 667 of file Nodes.cu.

Member Function Documentation

backward()

void nz::nodes::calc::AveragePoolingNode::backward ( )

overridevirtual

Performs the backward pass of the average pooling operation.

Parameters

None

Returns

None

This function conducts the backward pass of the average pooling operation. It first checks if the output tensor of the input node requires gradient computation. If it does, the function calls iAveragePoolingBackward, passing the gradient tensor of the input node's output, the gradient tensor of the output, the pooling size, stride, padding, and the dimensions of the input and output tensors. The iAveragePoolingBackward function computes the gradients and propagates them back to the input.

Memory management strategy: This function does not allocate or deallocate any memory directly. It operates on the existing gradient tensors of the input and output. Exception handling mechanism: There is no explicit exception handling in this function. If the iAveragePoolingBackward function encounters an error, it may throw an exception, and the specific type of exception depends on the implementation of iAveragePoolingBackward.

Exceptions

[Exception

type from iAveragePoolingBackward] If the iAveragePoolingBackward function encounters an error during execution.

Note

• Ensure that the gradient tensors of the input node's output and the output tensor of the AveragePoolingNode are properly initialized before calling this function.
• The performance of this function depends on the implementation of the iAveragePoolingBackward function. If the iAveragePoolingBackward function has a time complexity of O(n), where n is the number of elements in the input or output gradient tensors, then this backward pass also has a time complexity of O(n).

```cpp
AveragePoolingNode avgPoolingNode(...); // Assume AveragePoolingNode is properly initialized
avgPoolingNode.backward();
```

Author

Mgepahmge(https://github.com/Mgepahmge)

Date

2024/07/15

Implements nz::nodes::Node.

Definition at line 684 of file Nodes.cu.

forward()

void nz::nodes::calc::AveragePoolingNode::forward ( )

overridevirtual

Performs the backward pass of the average pooling operation.

Parameters

None

Returns

None

This function conducts the backward pass of the average pooling operation. It first checks if the output tensor of the input node requires gradient computation. If it does, the function calls iAveragePoolingBackward, passing the gradient tensor of the input node's output, the gradient tensor of the output, the pooling size, stride, padding, and the dimensions of the input and output tensors. The iAveragePoolingBackward function computes the gradients and propagates them back to the input.

Memory management strategy: This function does not allocate or deallocate any memory directly. It operates on the existing gradient tensors of the input and output. Exception handling mechanism: There is no explicit exception handling in this function. If the iAveragePoolingBackward function encounters an error, it may throw an exception, and the specific type of exception depends on the implementation of iAveragePoolingBackward.

Exceptions

[Exception

type from iAveragePoolingBackward] If the iAveragePoolingBackward function encounters an error during execution.

Note

• Ensure that the gradient tensors of the input node's output and the output tensor of the AveragePoolingNode are properly initialized before calling this function.
• The performance of this function depends on the implementation of the iAveragePoolingBackward function. If the iAveragePoolingBackward function has a time complexity of O(n), where n is the number of elements in the input or output gradient tensors, then this backward pass also has a time complexity of O(n).

```cpp
AveragePoolingNode avgPoolingNode(...); // Assume AveragePoolingNode is properly initialized
avgPoolingNode.backward();
```

Author

Mgepahmge(https://github.com/Mgepahmge)

Date

2024/07/15

Implements nz::nodes::Node.

Definition at line 678 of file Nodes.cu.

---

The documentation for this class was generated from the following files:

• D:/Users/Mgepahmge/Documents/C Program/NeuZephyr/include/NeuZephyr/Nodes.cuh
• D:/Users/Mgepahmge/Documents/C Program/NeuZephyr/src/Nodes.cu