How do I remove perspective view and separate the object of interest in the image?

I am working on a nonogram solver in matlab. The whole thing would be able to solve a nonogram from a digital image of the puzzle.

I have a problem with removing perspective from the image. I have already tried to apply similar algorithm as this sudoku solver: https://www.mathworks.com/matlabcentral/fileexchange/30088-video-sudoku-solver Firstly, the puzzle does not have a shape of a square or a rectangle, so finding the 4 corners is rather hard. I have tried using ‘kmeans’ (according to the link ) on the convex hull of the image and ‘quadprog’ ( link ) to get the tangent line to the hull, from which the intersection of these lines would define the corners. Quadprog would be quite good if not for the complexity of the algorithm.

Secondly, the puzzle grid has no pre-defined dimensions like sudoku (9x9), which could also help. Also the grid can be partially filled in already. How should I remove perspective from the image?