How to dynamically find main rectangular image within an image?

I’d like to take a screenshot of an image, and extract the image out of the screenshot.

For example:

I’d like to dynamically extract that image out of the screenshot. However, I want to be able to dynamically detect where the image to be extracted is in the screenshot image. So for example, if I screenshotted an image on Instagram, I’d like to dynamically extract the image from the screenshot. So I feel like I just need to come up with a calculation to find where “the main subject” inside of the screenshot image is.

I’ve done some research but most of what I’ve found are people wanting to extract an image from a scanned image in which everything surrounding the subject is mostly a solid color so I don’t think that’s going to work here.

I’m using Jimp (https://www.npmjs.com/package/jimp) as an image processor as it has no native dependencies and this is going into a React Native app.

Any help would be greatly appreciated. Thanks in advance!

Answer

I never did end up finding something that already existed so I built something myself. Using my img-items node module, I could accomplish this by doing the following:

const Jimp = require('jimp')
const imgItems = require('imgItems')

Jimp.read('image.png')
  .then(image => {
    return imgItems(image)
      .then(items => {
        const largest = items.reduce((p, c) => ((p.width + p.height) > (c.width + c.height)) ? p : c)

        return image
          .crop(largest.left, largest.top, largest.width, largest.height)
          .writeAsync('largest.png')
      })
  })