NeuZephyr
Simple DL Framework
nz::nodes::calc::GlobalAvgPoolNode Class Reference

Performs global average pooling operation across spatial dimensions of input tensor. More...

Inheritance diagram for nz::nodes::calc::GlobalAvgPoolNode:
Collaboration diagram for nz::nodes::calc::GlobalAvgPoolNode:

Public Member Functions

 GlobalAvgPoolNode (Node *input)
 Constructs a GlobalAvgPoolNode object.
 
void forward () override
 Performs the forward pass of the global average pooling operation.
 
void backward () override
 Performs the backward pass of the global average pooling operation.
 
- Public Member Functions inherited from nz::nodes::Node
virtual void print (std::ostream &os) const
 Prints the type, data, and gradient of the node.
 
void dataInject (Tensor::value_type *data, bool grad=false) const
 Injects data into a relevant tensor object, optionally setting its gradient requirement.
 
template<typename Iterator >
void dataInject (Iterator begin, Iterator end, const bool grad=false) const
 Injects data from an iterator range into the output tensor of the InputNode, optionally setting its gradient requirement.
 
void dataInject (const std::initializer_list< Tensor::value_type > &data, bool grad=false) const
 Injects data from a std::initializer_list into the output tensor of the Node, optionally setting its gradient requirement.
 

Detailed Description

Performs global average pooling operation across spatial dimensions of input tensor.

This node reduces each channel's spatial dimensions (H, W) to a single average value, producing output of shape (N, C, 1, 1). Commonly used for final feature aggregation before fully connected layers in CNN architectures.

Core functionality and characteristics:

  • Global Aggregation: Computes channel-wise mean over entire spatial dimensions.
  • Dimensionality Collapse: Reduces H and W dimensions to 1 while preserving channels.
  • Gradient Distribution: Evenly distributes gradients across all spatial positions during backward pass.
  • Parameter-Free: No learnable parameters required for operation.
  • CUDA Support: Optimized GPU implementation for both forward and backward passes.
  • Input Preservation: Maintains batch and channel dimensions unchanged.

Key implementation aspects:

  • Forward Pass: Computes spatial average for each channel using tensor reduction.
  • Backward Pass: Scatters gradients by assigning 1/(H*W) of output gradient to each input position.
  • Shape Handling: Automatically handles varying input sizes through dynamic shape inference.
  • Memory Efficiency: Significantly reduces memory footprint for downstream operations.

Typical use cases:

  • Final spatial information aggregation before classification layers.
  • Network-in-network architectures requiring compact feature representation.
  • Squeeze-and-Excitation modules for channel-wise attention.
  • Reducing model parameters in transition layers.

Critical considerations:

  • Irreversible Compression: Destroys all spatial information in input features.
  • Input Requirements: Expects 4D input tensor (N, C, H, W).
  • Normalization Factor: Uses 1/(H*W) for both forward average and backward gradient scaling.
Warning
  • Input tensor must have spatial dimensions (H, W) > 0.
  • Not suitable for preserving spatial relationships in features.
  • May produce non-sensical gradients if applied to single-element spatial dimensions.
Note
  • Output tensor shape: (N, C, 1, 1)
  • Often used as alternative to flattening+fully connected layers
  • Particularly effective in combination with convolutional bottlenecks
See also
AveragePoolingNode For local spatial averaging with configurable windows
FlattenNode For alternative spatial-to-vector conversion

Usage Example:

// Input features from convolutional backbone
InputNode input({16, 512, 7, 7}, true);
// Apply global average pooling
GlobalAvgPoolNode gap(&input);
gap.forward();
// Output shape becomes (16, 512, 1, 1)
std::cout << "Pooled shape: " << gap.output->shape() << std::endl;
// Backpropagate through global averaging
gap.backward();
Performs global average pooling operation across spatial dimensions of input tensor.
Definition Nodes.cuh:4269
Author
Mgepahmge (https://github.com/Mgepahmge)
Date
2023/10/20

Definition at line 4269 of file Nodes.cuh.

Constructor & Destructor Documentation

◆ GlobalAvgPoolNode()

nz::nodes::calc::GlobalAvgPoolNode::GlobalAvgPoolNode ( Node * input)

Constructs a GlobalAvgPoolNode object.

Parameters
inputA pointer to the input node. The memory of this pointer is assumed to be managed externally and is used in a read - only manner within this constructor (host - to - host).
Returns
None

This constructor initializes a GlobalAvgPoolNode object. It first adds the provided input node pointer to the inputs vector. Then, it creates a new shared pointer to a Tensor object for the output member. The shape of the output tensor is set to have the same batch size and number of channels as the input tensor's output, but with a height and width of 1. The requiresGrad flag of the output tensor is set to the same value as that of the input tensor's output. Finally, it sets the type member of the node to "GlobalAvgPool".

Memory management strategy: The constructor does not allocate memory for the input node. It only stores a pointer to it. The output tensor is created using std::make_shared, which manages the memory automatically. Exception handling mechanism: There is no explicit exception handling in this constructor. If the std::make_shared call fails to allocate memory for the output tensor, it may throw a std::bad_alloc exception.

Exceptions
std::bad_allocIf memory allocation for the output tensor fails.
Note
  • Ensure that the input node pointer is valid and points to a properly initialized node.
  • The performance of this constructor is mainly determined by the memory allocation for the output tensor, which has a time complexity of O(1) for the pointer management and O(m) for the tensor data allocation, where m is the number of elements in the output tensor (equal to the batch size times the number of channels in this case).
```cpp
Node* inputNode = new Node(...);
GlobalAvgPoolNode globalAvgPoolNode(inputNode);
```
Base class for nodes in a neural network or computational graph.
Definition Nodes.cuh:114
Author
Mgepahmge(https://github.com/Mgepahmge)
Date
2024/07/15

Definition at line 692 of file Nodes.cu.

Member Function Documentation

◆ backward()

void nz::nodes::calc::GlobalAvgPoolNode::backward ( )
overridevirtual

Performs the backward pass of the global average pooling operation.

Parameters
None
Returns
None

This function conducts the backward pass of the global average pooling operation. It first checks if the output tensor of the input node requires gradient computation. If it does, the function calls iGlobalAvgPoolBackward, passing the gradient tensor of the input node's output, the gradient tensor of the output, and the shape information of the input tensor (batch size, number of channels, height, and width). The iGlobalAvgPoolBackward function computes the gradients and propagates them back to the input.

Memory management strategy: This function does not allocate or deallocate any memory directly. It operates on the existing gradient tensors of the input and output. Exception handling mechanism: There is no explicit exception handling in this function. If the iGlobalAvgPoolBackward function encounters an error, it may throw an exception, and the specific type of exception depends on the implementation of iGlobalAvgPoolBackward.

Exceptions
[Exceptiontype from iGlobalAvgPoolBackward] If the iGlobalAvgPoolBackward function encounters an error during execution.
Note
  • Ensure that the gradient tensors of the input node's output and the output tensor of the GlobalAvgPoolNode are properly initialized before calling this function.
```cpp
GlobalAvgPoolNode globalAvgPoolNode(...); // Assume GlobalAvgPoolNode is properly initialized
globalAvgPoolNode.backward();
```
Author
Mgepahmge(https://github.com/Mgepahmge)
Date
2024/07/15

Implements nz::nodes::Node.

Definition at line 709 of file Nodes.cu.

◆ forward()

void nz::nodes::calc::GlobalAvgPoolNode::forward ( )
overridevirtual

Performs the forward pass of the global average pooling operation.

Parameters
None
Returns
None

This function conducts the forward pass of the global average pooling operation. It iterates over each sample in the batch (i loop) and each channel (j loop) of the input tensor. For each combination of batch and channel, it calculates the sum of all elements in the corresponding 2D matrix of the input tensor using the sum method. Then, it divides this sum by the total number of elements in the 2D matrix (which is the product of the height and width of the input tensor). The result is then used to fill the corresponding element in the output tensor using the fillMatrix method.

Memory management strategy: This function does not allocate or deallocate any memory directly. It operates on the existing data tensors of the input and output. Exception handling mechanism: There is no explicit exception handling in this function. If the sum or fillMatrix methods encounter an error, they may throw exceptions depending on their implementation.

Exceptions
[Exceptiontype from sum or fillMatrix] If the sum or fillMatrix methods encounter an error during execution.
Note
  • Ensure that the input and output tensors are properly initialized before calling this function.
  • The time complexity of this function is O(b * c * h * w), where b is the batch size, c is the number of channels, h is the height, and w is the width of the input tensor, because it needs to sum all elements in each 2D matrix of the input tensor.
```cpp
GlobalAvgPoolNode globalAvgPoolNode(...); // Assume GlobalAvgPoolNode is properly initialized
globalAvgPoolNode.forward();
```
Author
Mgepahmge(https://github.com/Mgepahmge)
Date
2024/07/15

Implements nz::nodes::Node.

Definition at line 700 of file Nodes.cu.


The documentation for this class was generated from the following files: