nz::nodes::calc::GlobalMaxPoolNode Class Reference

Performs global max pooling operation across spatial dimensions of input tensor. More...

Inheritance diagram for nz::nodes::calc::GlobalMaxPoolNode:

Collaboration diagram for nz::nodes::calc::GlobalMaxPoolNode:

Public Member Functions
	GlobalMaxPoolNode (Node *input)
	Constructs a GlobalMaxPoolNode object.
void	forward () override
	Performs the forward pass of the global max - pooling operation.
void	backward () override
	Performs the backward pass of the global max - pooling operation.
Public Member Functions inherited from nz::nodes::Node
virtual void	print (std::ostream &os) const
	Prints the type, data, and gradient of the node.
void	dataInject (Tensor::value_type *data, bool grad=false) const
	Injects data into a relevant tensor object, optionally setting its gradient requirement.
template<typename Iterator >
void	dataInject (Iterator begin, Iterator end, const bool grad=false) const
	Injects data from an iterator range into the output tensor of the InputNode, optionally setting its gradient requirement.
void	dataInject (const std::initializer_list< Tensor::value_type > &data, bool grad=false) const
	Injects data from a std::initializer_list into the output tensor of the Node, optionally setting its gradient requirement.

Detailed Description

Performs global max pooling operation across spatial dimensions of input tensor.

This node reduces each channel's spatial dimensions (H, W) to a single maximum value, producing output of shape (N, C, 1, 1). Used to extract the most salient spatial features while maintaining channel-wise information.

Core functionality and characteristics:

• Global Feature Selection: Captures maximum activation per channel across all spatial positions.
• Position Tracking: Implicitly records max value locations for gradient routing.
• Sparse Gradient Flow: Propagates gradients exclusively to original max positions during backward pass.
• Dimensionality Collapse: Reduces H and W dimensions to 1 while preserving channels.
• Parameter-Free: No learnable parameters required for operation.
• CUDA Support: Optimized GPU implementation for both forward and backward passes.

Key implementation aspects:

• Forward Pass: Computes channel-wise maxima using tensor reduction with argmax tracking.
• Backward Pass: Scatters full output gradient to corresponding max positions in input tensor.
• Efficient Storage: Requires O(N*C) memory for position indices compared to spatial dimensions.
• Tie Resolution: Selects first occurrence of maximum value when multiple maxima exist.

Typical use cases:

• Highlighting strongest activation patterns in feature maps
• Final feature aggregation for classification tasks
• Networks requiring spatial position invariance
• Attention mechanisms focusing on dominant features

Critical considerations:

• Feature Discard: Discards all non-maximal spatial information
• Gradient Sparsity: Only single position per channel receives gradients
• Input Requirements: Expects 4D input tensor (N, C, H, W) with H, W > 0

Warning

• Input tensor must have spatial dimensions (H, W) > 1 to be meaningful
• Not suitable for tasks requiring spatial relationship preservation
• May produce unstable gradients if input contains multiple equal maxima

Note

• Output tensor shape: (N, C, 1, 1)
• Often used as alternative to global average pooling for sharper feature selection
• Common in architectures emphasizing dominant visual patterns

See also

GlobalAvgPoolNode For smoothed spatial feature aggregation

MaxPoolingNode For local window-based max pooling

Usage Example:

// Process batch of 16 samples with 512 channels (7x7 spatial)
InputNode input({16, 512, 7, 7}, true);
 
// Apply global max pooling
GlobalMaxPoolNode gmp(&input);
gmp.forward();
 
// Output shape becomes (16, 512, 1, 1)
std::cout << "Pooled shape: " << gmp.output->shape() << std::endl;
 
// Backpropagate through max positions
gmp.backward();

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2023/10/22

Definition at line 4619 of file Nodes.cuh.

Constructor & Destructor Documentation

GlobalMaxPoolNode()

nz::nodes::calc::GlobalMaxPoolNode::GlobalMaxPoolNode

(

Node *

input

)

Constructs a GlobalMaxPoolNode object.

Parameters

input

A pointer to the input node. This pointer is assumed to be managed externally, and the constructor uses it in a read - only manner (host - to - host).

Returns

None

This constructor initializes a GlobalMaxPoolNode object. It first adds the provided input node pointer to the inputs vector. Then, it creates a new shared pointer for the output member. The shape of the output tensor is set to have the same batch size and number of channels as the input tensor's output, but with a height and width of 1. The requiresGrad flag of the output tensor is set to the same value as that of the input tensor's output. Finally, it sets the type member to the string "GlobalMaxPool".

Memory management strategy: The constructor does not allocate memory for the input node. It only stores a pointer to it. The output tensor is created using std::make_shared, which manages its memory automatically. Exception handling mechanism: There is no explicit exception handling in this constructor. If the std::make_shared call fails to allocate memory for the output tensor, it may throw a std::bad_alloc exception.

Exceptions

std::bad_alloc

If memory allocation for the output tensor fails.

Note

• Ensure that the input node pointer is valid and points to a properly initialized node.
• The performance of this constructor is mainly determined by the memory allocation for the output tensor, which has a time complexity of O(1) for the pointer management and O(m) for the tensor data allocation, where m is the number of elements in the output tensor.

```cpp
Node* inputNode = new Node(...); // Assume Node is properly initialized
GlobalMaxPoolNode globalMaxPoolNode(inputNode);
```

Author

Mgepahmge(https://github.com/Mgepahmge)

Date

2024/07/15

Definition at line 745 of file Nodes.cu.

Member Function Documentation

backward()

void nz::nodes::calc::GlobalMaxPoolNode::backward ( )

overridevirtual

Performs the backward pass of the global max - pooling operation.

Parameters

None

Returns

None

This function performs the backward pass of the global max - pooling operation. First, it checks if the output tensor of the input node requires gradient computation. If so, it retrieves the host data and gradients of the output tensor. Then, it iterates over each batch and channel of the input tensor's output. For each combination of batch index i and channel index j, it calculates an index idx and uses the find method to locate the position of the maximum value in the input tensor corresponding to the output value at idx. Finally, it sets the gradient at that position in the input tensor using the setData method.

Memory management strategy: This function does not allocate or deallocate any memory directly. It operates on the existing data and gradient tensors of the input and output. Exception handling mechanism: There is no explicit exception handling in this function. If the hostData, hostGrad, find, or setData methods encounter an error, they may throw an exception depending on their implementation.

Exceptions

[Exception

type from hostData, hostGrad, find, or setData] If the hostData, hostGrad, find, or setData methods encounter an error during execution.

Note

• Ensure that the input and output tensors and their gradients are properly initialized before calling this function.
• The time complexity of this function is O(b * c), where b is the batch size (inputs[0]->output->shape()[0]) and c is the number of channels (inputs[0]->output->shape()[1]).

```cpp
GlobalMaxPoolNode globalMaxPoolNode(...); // Assume GlobalMaxPoolNode is properly initialized
globalMaxPoolNode.backward();
```

Author

Mgepahmge(https://github.com/Mgepahmge)

Date

2024/07/15

Implements nz::nodes::Node.

Definition at line 761 of file Nodes.cu.

forward()

void nz::nodes::calc::GlobalMaxPoolNode::forward ( )

overridevirtual

Performs the forward pass of the global max - pooling operation.

Parameters

None

Returns

None

This function conducts the forward pass of the global max - pooling operation. It iterates over each batch and channel of the input tensor's output. For each combination of batch index i and channel index j, it computes the maximum value in the corresponding slice of the input tensor using the max method. Then, it fills the corresponding position in the output tensor using the fillMatrix method.

Memory management strategy: This function does not allocate or deallocate any memory directly. It operates on the existing data tensors of the input and output. Exception handling mechanism: There is no explicit exception handling in this function. If the max or fillMatrix methods encounter an error, they may throw an exception depending on their implementation.

Exceptions

[Exception

type from max or fillMatrix] If the max or fillMatrix methods encounter an error during execution.

Note

• Ensure that the input and output tensors are properly initialized before calling this function.
• The time complexity of this function is O(b * c), where b is the batch size (inputs[0]->output->shape()[0]) and c is the number of channels (inputs[0]->output->shape()[1]).

```cpp
GlobalMaxPoolNode globalMaxPoolNode(...); // Assume GlobalMaxPoolNode is properly initialized
globalMaxPoolNode.forward();
```

Author

Mgepahmge(https://github.com/Mgepahmge)

Date

2024/07/15

Implements nz::nodes::Node.

Definition at line 753 of file Nodes.cu.

---

The documentation for this class was generated from the following files:

• D:/Users/Mgepahmge/Documents/C Program/NeuZephyr/include/NeuZephyr/Nodes.cuh
• D:/Users/Mgepahmge/Documents/C Program/NeuZephyr/src/Nodes.cu