Remove Duplicate product Images in Magento 2

If anyone needs a full script to remove duplicate product images, here it is. This will not remove the image with any image type (small image, thumbnail image, etc.) tagged with the image.

<?php
error_reporting(E_ALL);
use MagentoFrameworkAppBootstrap;
require __DIR__ . '/app/bootstrap.php';
$bootstrap = Bootstrap::create(BP, $_SERVER);
$obj = $bootstrap->getObjectManager(); 
$app_state = $obj->get('MagentoFrameworkAppState');
$app_state->setAreaCode('global');
ini_set('memory_limit','10240M');
ini_set('max_execution_time', 0); 
set_time_limit(0);

$mediaApi = $obj->create('MagentoCatalogModelProductGalleryProcessor');
$storeManager = $obj->get('MagentoStoreModelStoreManagerInterface');
$storeManager->setCurrentStore(0);
$mediaUrl = $storeManager->getStore()->getBaseUrl(MagentoFrameworkUrlInterface::URL_TYPE_MEDIA);
$directoryList=$obj->get('MagentoFrameworkAppFilesystemDirectoryList');
$path = $directoryList->getPath('media'); 
$productCollection=$obj->create('MagentoCatalogModelResourceModelProductCollection');
$_products = $productCollection->addAttributeToSelect('*')->load();

$productRepository = $obj->get('MagentoCatalogModelProductRepository');
$i =0;
$total = count($_products);
$count = 0;
$productCount = array();
foreach($_products as $_prod) {
    $_product = $productRepository->getById($_prod->getId());
    $_product->setStoreId(0);
    $_md5_values = array();
    $base_image = $_product->getImage();
    
    if($base_image != 'no_selection') {
        $mediaUrl = $storeManager->getStore()->getBaseUrl(MagentoFrameworkUrlInterface::URL_TYPE_MEDIA);
        $filepath = $path.'/catalog/product' . $base_image ;
        if (file_exists($filepath)) {
            $_md5_values[] = md5(file_get_contents($filepath));            
        }
        $i++;

        echo "rn processing product $i of $total ";
        // Loop through product images
        $gallery = $_product->getMediaGalleryEntries();
        if ($gallery) {
            foreach ($gallery as $key => $galleryImage) {
                //protected base image
                if($galleryImage->getFile() == $base_image) {
                    continue;
                }
                $filepath = $path.'/catalog/product' .$galleryImage->getFile();

                if(file_exists($filepath)) {
                    $md5 = md5(file_get_contents($filepath));
                } else {
                    continue;
                }

                if( in_array( $md5, $_md5_values )) {
                    if (count($galleryImage->getTypes()) > 0) {
                        continue;
                    }
                    unset($gallery[$key]);
                    echo "rn removed duplicate image from ".$_product->getSku();
                    $count++;
                } else {
                    $_md5_values[] = $md5;
                }
            }
            
            $_product->setMediaGalleryEntries($gallery);
            $productRepository->save($_product);            
        }
        //$_product->save();
    }
}
die("Script Completed");

You can Buy Extension available on MP that can clean your unused Catalog Images as well as Duplicate entries You can Clean theme manually or using CRON

Make sure you are working on the admin store view:

        $this->storeManager->setCurrentStore(0);
        $product->setStoreId(0);

Use the Media Gallery 'Entries' instead:

    $gallery = $product->getMediaGalleryEntries();

Iterate with:

    foreach ($gallery as $key => $galleryImage) {

You can access the files with:

        $galleryImage->getFile()

Then when you have a duplicate:

                unset($gallery[$key]);

Then set the gallery to your revised one:

        $product->setMediaGalleryEntries($gallery);

Then save the product using the product repository:

        $this->productRepository->save($product);