Correct problem statement for CNN. Stitching parts of the map

I decided to start learning neural networks by creating a bot for the game. One of the intermediate steps is to create a global map from a series of inaccurate overlapping sub-maps. This task can be solved using OpenCV, but this solution will be too limited and sensitive (in the future I intend to complicate the task and work directly with the map image, instead of binary masks).

I’ve tried the following options:

• predict the position of a new map area within the global map. (as a probability distribution)

• predict the new state of the global map from the old and new minimap.

I’ve tried a lot of options formulation of the problem of network architecture, including the idea of conjoined networks, but nothing gave any relevant results.

Some articles about solving similar problems:

• Automatic Field-of-View Expansion using Deep Features and Image Stitching

• JigsawNet: Shredded Image Reassembly using Convolutional Neural Network and Loop-based Composition

• Robust Cylindrical Panorama Stitching for Low-Texture Scenes Based on Image Alignment Using Deep Learning and Iterative Optimization

Here is an example of one of the options statement of the problem: