nz::nodes::calc::MaxPoolingNode Class Reference

Implements max pooling operation for spatial downsampling with feature preservation. More...

Inheritance diagram for nz::nodes::calc::MaxPoolingNode:

Collaboration diagram for nz::nodes::calc::MaxPoolingNode:

Public Member Functions
	MaxPoolingNode (Node *input, Tensor::size_type poolSize, Tensor::size_type stride, Tensor::size_type padding)
	Constructs a MaxPoolingNode object.
void	forward () override
	Performs the forward pass of the max - pooling operation.
void	backward () override
	Performs the backward pass of the max - pooling operation.
Public Member Functions inherited from nz::nodes::Node
virtual void	print (std::ostream &os) const
	Prints the type, data, and gradient of the node.
void	dataInject (Tensor::value_type *data, bool grad=false) const
	Injects data into a relevant tensor object, optionally setting its gradient requirement.
template<typename Iterator >
void	dataInject (Iterator begin, Iterator end, const bool grad=false) const
	Injects data from an iterator range into the output tensor of the InputNode, optionally setting its gradient requirement.
void	dataInject (const std::initializer_list< Tensor::value_type > &data, bool grad=false) const
	Injects data from a std::initializer_list into the output tensor of the Node, optionally setting its gradient requirement.

Detailed Description

Implements max pooling operation for spatial downsampling with feature preservation.

This node performs spatial max selection over sliding windows of (poolSize x poolSize) dimensions, maintaining the most prominent features while reducing spatial resolution. Essential for preserving sharp feature responses in CNNs.

Core functionality and characteristics:

• Feature Selection: Captures maximum activation values within local receptive fields.
• Position Tracking: Records max value locations via position tensor for gradient computation.
• Sparse Gradient Flow: Propagates gradients only to winning positions during backward pass.
• Dimensionality Control: Reduces H/W dimensions while preserving channel structure.
• Border Handling: Supports configurable padding for output size management.
• CUDA Optimization: Accelerated implementations for both forward and backward operations.
• Dynamic Shape Adaption: Automatically calculates output dimensions using pooling parameters.

Key implementation aspects:

• Forward Pass: Computes max values and stores positions using parallelized argmax operations.
• Backward Pass: Scatters gradients exclusively to original max positions with 1:1 mapping.
• Memory Tradeoff: Stores position indices (poolSize^2 elements per window) for gradient accuracy.
• Numerical Stability: Handles multiple equal maxima by selecting first occurrence.

Typical use cases:

• Dominant feature extraction in early CNN layers
• Downsampling while preserving edge/texture information
• Constructing translation-invariant representations
• Reducing computational load in deep networks

Critical considerations:

• Information Discard: Non-maximal values are irreversibly discarded during forward pass
• Dimension Constraints: Output dimensions must satisfy: Hout = (H + 2*padding - poolSize)/stride + 1
• Position Storage: Requires O(N*C*Hout*Wout) memory for position tracking

Warning

• Invalid dimension combinations will throw runtime errors
• Overly aggressive pooling (poolSize >> stride) may cause feature loss
• Position tensor initialization must match forward pass execution order

Note

• Output shape: (N, C, Hout, Wout)
• For variable pooling windows, extend with additional size parameters
• Compared to average pooling, better preserves feature sharpness

See also

AveragePoolingNode For smoothing-based spatial aggregation

GlobalAvgPoolNode For complete spatial dimension collapse

Usage Example:

// Process 32 samples of 128x128 RGB images
InputNode input({32, 3, 128, 128}, true);
 
// Create 3x3 max pooling with stride 2 and padding 1
MaxPoolingNode mpool(&input, 3, 2, 1);
mpool.forward();
 
// Resulting shape: (32, 3, 64, 64)
std::cout << "Pooled shape: " << mpool.output->shape() << std::endl;
 
// Backpropagate gradients through max positions
mpool.backward();

Author

Mgepahmge (https://github.com/Mgepahmge)

Date

2023/10/21

Definition at line 4440 of file Nodes.cuh.

Constructor & Destructor Documentation

MaxPoolingNode()

nz::nodes::calc::MaxPoolingNode::MaxPoolingNode	(	Node *	input,
		Tensor::size_type	poolSize,
		Tensor::size_type	stride,
		Tensor::size_type	padding )

Constructs a MaxPoolingNode object.

Parameters

input	A pointer to the input node. Memory of this pointer is assumed to be managed externally and used in a read - only way within the constructor (host - to - host).
poolSize	The size of the pooling window, of type Tensor::size_type. It is passed by value.
stride	The stride of the pooling operation, of type Tensor::size_type. It is passed by value.
padding	The padding applied to the input tensor, of type Tensor::size_type. It is passed by value.

Returns

None

This constructor initializes a MaxPoolingNode object. It first stores the provided poolSize, stride, and padding values. Then, it adds the input node pointer to the inputs vector. Next, it creates a new shared pointer to a Tensor object for the output member. The shape of the output tensor is calculated based on the input tensor's shape, poolSize, stride, and padding using the OUTPUT_DIM macro. The requiresGrad flag of the output tensor is set to the same value as that of the input tensor's output. Additionally, it creates a new shared pointer to a Tensor object for the position member. The shape of the position tensor is the same as that of the output tensor, and its requiresGrad flag is set to false.

Memory management strategy: The constructor does not allocate memory for the input node. It only stores a pointer to it. The output and position tensors are created using std::make_shared, which manages their memory automatically. Exception handling mechanism: There is no explicit exception handling in this constructor. If the std::make_shared calls fail to allocate memory for the output or position tensors, they may throw a std::bad_alloc exception.

Exceptions

std::bad_alloc

If memory allocation for the output or position tensors fails.

Note

• Ensure that the input node pointer is valid and points to a properly initialized node.
• The performance of this constructor is mainly determined by the memory allocation for the output and position tensors, which has a time complexity of O(1) for the pointer management and O(m) for the tensor data allocation, where m is the number of elements in the tensors.

```cpp
Node* inputNode = new Node(...); // Assume Node is properly initialized
MaxPoolingNode maxPoolingNode(inputNode, 2, 2, 0);
```

Author

Mgepahmge(https://github.com/Mgepahmge)

Date

2024/07/15

Definition at line 716 of file Nodes.cu.

Member Function Documentation

backward()

void nz::nodes::calc::MaxPoolingNode::backward ( )

overridevirtual

Performs the backward pass of the max - pooling operation.

Parameters

None

Returns

None

This function conducts the backward pass of the max - pooling operation. It first checks if the output tensor of the input node requires gradient computation. If it does, it calls the iMaxPoolingBackward function, passing the gradient pointer of the input node's output, the data pointer of the position tensor, the gradient pointer of the output tensor, the pooling parameters (poolSize, stride, padding), and the shape information of the input and output tensors. The iMaxPoolingBackward function is responsible for computing the gradients and propagating them back to the input tensor.

Memory management strategy: This function does not allocate or deallocate any memory directly. It operates on the existing gradient tensors of the input and output, as well as the data tensor of the position. Exception handling mechanism: There is no explicit exception handling in this function. If the iMaxPoolingBackward function encounters an error, it may throw an exception depending on its implementation.

Exceptions

[Exception

type from iMaxPoolingBackward] If the iMaxPoolingBackward function encounters an error during execution.

Note

• Ensure that the input, output, and position tensors' gradient and data are properly initialized before calling this function.
• The performance of this function depends on the implementation of the iMaxPoolingBackward function. The time complexity typically depends on the size of the input tensor and the pooling parameters, and may be O(b * c * h * w) in the worst - case scenario, where b is the batch size, c is the number of channels, h is the height, and w is the width of the input tensor.

```cpp
MaxPoolingNode maxPoolingNode(...); // Assume MaxPoolingNode is properly initialized
maxPoolingNode.backward();
```

Author

Mgepahmge(https://github.com/Mgepahmge)

Date

2024/07/15

Implements nz::nodes::Node.

Definition at line 737 of file Nodes.cu.

forward()

void nz::nodes::calc::MaxPoolingNode::forward ( )

overridevirtual

Performs the forward pass of the max - pooling operation.

Parameters

None

Returns

None

This function executes the forward pass of the max - pooling operation. It calls the iMaxPooling function, passing the data pointers of the output tensor, the position tensor, and the input tensor's output. It also provides the pooling parameters (poolSize, stride, padding) and the shape information of the input and output tensors. The iMaxPooling function is responsible for computing the max - pooling result and storing it in the output tensor, as well as recording the positions of the maximum values in the position tensor.

Memory management strategy: This function does not allocate or deallocate any memory directly. It operates on the existing data tensors of the input, output, and position. Exception handling mechanism: There is no explicit exception handling in this function. If the iMaxPooling function encounters an error, it may throw an exception depending on its implementation.

Exceptions

[Exception

type from iMaxPooling] If the iMaxPooling function encounters an error during execution.

Note

• Ensure that the input, output, and position tensors are properly initialized before calling this function.
• The performance of this function depends on the implementation of the iMaxPooling function. The time complexity typically depends on the size of the input tensor and the pooling parameters, and may be O(b * c * h * w) in the worst - case scenario, where b is the batch size, c is the number of channels, h is the height, and w is the width of the input tensor.

```cpp
MaxPoolingNode maxPoolingNode(...); // Assume MaxPoolingNode is properly initialized
maxPoolingNode.forward();
```

Author

Mgepahmge(https://github.com/Mgepahmge)

Date

2024/07/15

Implements nz::nodes::Node.

Definition at line 731 of file Nodes.cu.

---

The documentation for this class was generated from the following files:

• D:/Users/Mgepahmge/Documents/C Program/NeuZephyr/include/NeuZephyr/Nodes.cuh
• D:/Users/Mgepahmge/Documents/C Program/NeuZephyr/src/Nodes.cu