\n I recently acquired a pretty fun new hobby, solving sudokus. In\n that process, I discovered a new YouTube channel,{\" \"}\n \n Cracking the Cryptic\n \n , which is releasing two new sudoku puzzles every day. I have been\n diligently attempting these new puzzles and practicing easier puzzles on a\n sudoku book I have at home.\n
\n\n While struggling to solve another sudoku (rated as \"easy\" which is\n probably incorrect), I thought of another computer vision project:{\" \"}\n could my computer solve sudoku from a picture of one?\n
\n\n I want to be able to recognize the sudoku in an image, adjust for\n perspective, identify the numbers in the grid, and solve the sudoku. I\n then want to print the solution back into the image,{\" \"}\n adjusting for perspective. This article will cover the algorithm I\n created to process images of sudokus. It consists of the following steps:\n
\n\n This project will require OpenCV, sklearn, and numpy installed on your\n machine.\n
\nLet's start by importing our dependencies:
\n\n I will be processing images in a folder. I process images one by one,\n first resizing them to a more manageable size to help with processing. The\n code below will form the main loop of this image processing task. I will\n be processing images in a folder.\n
\n\n I am looking for a large square consisting of 81 smaller squares.{\" \"}\n This square can be taken from any perspective. In other words, I am\n looking for a quadrilateral in an image.\n
\n\n I will be using OpenCV contours to find quadrilaterals. After thresholding\n the image (using adaptive thresholding), I can generate a list of contours\n present in the image. I then need to filter these contours to only those\n that closely resemble a quadrilateral.\n
\n\n \n cv2.arcLength and cv2.approxPolyDP are used to\n detect quadrilaterals.\n \n cv2.arcLength returns the length of the perimeter of a\n contour while cv2.approxPolyDP\n returns a contour forming an approximation of the contour passed as\n argument. The approximation is controlled by a parameter,{\" \"}\n cv2.approxPolyDP is useful when trying to find contours that\n approximates a quadrilateral. If the approximate contour has four points\n while
\n To verify if I found the contour of the sudoku grid (and not the contour\n of a cell of the grid for example),{\" \"}\n I check that the contour contains 81 smaller quadrilaterals. If\n this is the case, it is very likely that I found the sudoku grid.\n
\n\n Two images of the same planar surface are related by a homography.\n If we can determine the plane of the sudoku grid, we can create a new\n image where the grid appears directly below the camera (ie. on the image\n plane). This process is also known as perspective removal.\n
\n\n To find a homography matrix, I need to find four corresponding points\n shared between the two images. I already have four points from our initial\n image when I located the sudoku grid. The corresponding points in the\n perspective corrected image are the four corners of the image.\n
\n\n OpenCV allows us to calculate a homography matrix and apply the\n perspective correction to an image.\n
\n\n I need to extract each cell from the warped image to read their value when\n creating a 2D array of the sudoku grid. I apply the same methods that I\n used to find the sudoku grid.\n
\n\n I have an image of all the 81 cells of the sudoku, corrected for\n perspective. Now I need to extract the number printed in each cell (if it\n exists). For this, I trained a k-nearest neighbours classifier on a\n subset of the cell images taken from the same sudoku book.\n
\n\n Each cell image was resized to a
\n I selected
\n I can determine which row and column each cell belongs to through a simple\n sort. Sorting cells by their
\n This logic works as I adjusted the sudoku grid for perspective in an\n earlier step. Hence, I assume that rows will remain roughly horizontal and\n columns roughly vertical.\n
\n\n I used a backtracking algorithm to solve the sudoku. After retrieving a\n list of all empty cells of the sudoku grid, I test a number for each empty\n cell one by one. I first verify that the number is a valid move for this\n cell. If so, I use recursion to solve the sudoku given the guess I made\n for that initial empty cell. If no number is valid for a particular empty\n cell, I backtrack, testing a new value for the previous empty cell.{\" \"}\n \n This article\n {\" \"}\n provides a more thorough explanation of this process.\n
\n\n Backtracking algorithms are usually equivalent to brute-force methods as\n most don't adopt any efficient heuristics. The algorithm I used is no\n different. However, for this project, it proved to be a relatively simple\n solution to solving sudoku.\n
\n\n With the sudoku solved, I can then print the values of the solved grid\n back into the perspective corrected image. I then warp the solved sudoku\n image back into its original perspective using the inverse of the\n homography matrix found in the initial step.\n
\n\n You can find all the code for this small project in this{\" \"}\n \n Gist\n \n !\n
\n \n);\n\nexport const query = graphql`\n query ($id: String) {\n javascriptFrontmatter(id: { eq: $id }) {\n frontmatter {\n author {\n email\n firstName\n name\n }\n category {\n name\n }\n meta\n subtitle\n title\n date\n }\n }\n }\n`;\n\nexport default wrapWithPostTemplate(\n Sudoku,\n