Upgrades for v2.0.0

.github/ISSUE_TEMPLATE/bug_report.md

@@ -30,5 +30,5 @@ If applicable, add screenshots to help explain your problem.
 
 **Additional context**
 Add any other context about the problem here.
-
-Error Stack trace. Sample images used, etc
+ - Error Stack trace (using --debug flag).
+ - Sample images used, etc

.gitignore

@@ -9,6 +9,7 @@ outputs/*
 
 # Misc
 **/.DS_Store
+**/__ignore__
 **/__pycache__
 venv/
 OMRChecker.wiki/

.pre-commit-config.yaml

@@ -1,17 +1,57 @@
 exclude: "__snapshots__/.*$"
 default_install_hook_types: [pre-commit, pre-push]
 repos:
+  # Convert the pngs to jpgs inside samples folder
+  - repo: local
+    hooks:
+      - id: convert-images
+        name: Convert png to jpg in samples
+        entry: scripts/convert_images_hook.py
+        args: ["--trigger-size", "150"]
+        files: ^samples.*\.(png|PNG)$
+        pass_filenames: true
+        stages: [commit]
+        language: python
+        language_version: python3
+        additional_dependencies: [pillow]
+        always_run: true
+        fail_fast: true # Wait for user to remove the png files
+
+  # Run resize image before the compress image check
+  - repo: local
+    hooks:
+      - id: resize-images
+        name: Resize images
+        entry: scripts/resize_images_hook.py
+        args: ["--max-width", "1500", "--max-height", "1500", "--trigger-size", "150"]
+        files: \.(png|jpg|jpeg|PNG|JPG|JPEG)$
+        pass_filenames: true
+        stages: [commit]
+        language: python
+        language_version: python3
+        additional_dependencies: [pillow]
+        always_run: true
+        fail_fast: false
+
+  # Run image compressor before the large files check
+  - repo: https://github.com/boidolr/pre-commit-images
+    rev: v1.5.2
+    hooks:
+      - id: optimize-png
+      - id: optimize-jpg
+        args: ["--quality", "90"]
+
   - repo: https://github.com/pre-commit/pre-commit-hooks
     rev: v4.4.0
     hooks:
       - id: check-yaml
         stages: [commit]
       - id: check-added-large-files
-        args: ['--maxkb=300']
+        args: ["--maxkb=300"]
         fail_fast: false
         stages: [commit]
       - id: pretty-format-json
-        args: ['--autofix', '--no-sort-keys']
+        args: ["--autofix", "--no-sort-keys"]
       - id: end-of-file-fixer
         exclude_types: ["csv", "json"]
         stages: [commit]
@@ -34,14 +74,23 @@ repos:
     hooks:
       - id: flake8
         args:
-          - "--ignore=E501,W503,E203,E741,F541" # Line too long, Line break occurred before a binary operator, Whitespace before ':'
+          - "--ignore=E203,E501,E741,E731,F403,F405,F541,W503"
+          # https://www.flake8rules.com/
+          # E203: Whitespace before ':'
+          # E501: Line too long
+          # E731: do not assign a lambda expression, use a def
+          # E741: Do not use variables named 'I', 'O', or 'l'
+          # F403: unable to detect undefined names
+          # F405: 'Tuple' may be undefined, or defined from star imports
+          # F541: f-string without any placeholders
+          # W503: Line break occurred before a binary operator
         fail_fast: true
         stages: [commit]
   - repo: local
     hooks:
       - id: pytest-on-commit
         name: Running single sample test
-        entry:  python3 -m pytest -rfpsxEX --disable-warnings --verbose -k sample1
+        entry: python3 -m pytest -rfpsxEX --disable-warnings --verbose -k test_run_omr_marker_mobile
         language: system
         pass_filenames: false
         always_run: true

Contributors.md

@@ -20,3 +20,4 @@
 - [aayushibansal2001](https://github.com/aayushibansal2001)
 - [ShamanthVallem](https://github.com/ShamanthVallem)
 - [rudrapsc](https://github.com/rudrapsc)
+- [palash018](https://github.com/palash018)

README.md

@@ -1,12 +1,13 @@
 # OMR Checker
 
-Read OMR sheets fast and accurately using a scanner 🖨 or your phone 🤳.
+Evaluate OMR sheets fast and accurately using a scanner 🖨 or your phone 🤳.
 
 ## What is OMR?
 
-OMR stands for Optical Mark Recognition, used to detect and interpret human-marked data on documents. OMR refers to the process of reading and evaluating OMR sheets, commonly used in exams, surveys, and other forms.
+OMR stands for Optical Mark Recognition, used to detect and interpret human-marked data on documents. OMR refers to the process of reading and evaluating OMR sheets, commonly used in exams, surveys, and other forms. The OMR sheet scanning is typically done using a scanner, but with OMRChecker it's supported from a mobile camera as well.
 
 #### **Quick Links**
+
 - [Installation](#getting-started)
 - [User Guide](https://github.com/Udayraj123/OMRChecker/wiki)
 - [Contributor Guide](https://github.com/Udayraj123/OMRChecker/blob/master/CONTRIBUTING.md)
@@ -33,9 +34,10 @@ A full-fledged OMR checking software that can read and evaluate OMR sheets scann
 | Specs <img width=200/> | ![Current_Speed](https://img.shields.io/badge/Speed-200+_OMRs/min-blue.svg?style=flat-square) ![Min Resolution](https://img.shields.io/badge/Min_Resolution-640x480-blue.svg?style=flat-square) <img width=200/> |
 | :--------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | 💯 **Accurate**        | Currently nearly 100% accurate on good quality document scans; and about 90% accurate on mobile images.                                                                                                          |
-| 💪🏿 **Robust**          | Supports low resolution, xeroxed sheets. See [**Robustness**](https://github.com/Udayraj123/OMRChecker/wiki/Robustness) for more.                                                                                |
+| 💪🏿 **Robust**          | Supports low resolution, xeroxed sheets as well as colored images. See [**Robustness**](https://github.com/Udayraj123/OMRChecker/wiki/Robustness) for more.                                                      |
 | ⏩ **Fast**            | Current processing speed without any optimization is 200 OMRs/minute.                                                                                                                                            |
 | ✅ **Customizable**    | [Easily apply](https://github.com/Udayraj123/OMRChecker/wiki/User-Guide) to custom OMR layouts, surveys, etc.                                                                                                    |
+| ✅ **Colorful**        | Supports Colored Outputs Since April 2024                                                                                                                                                                        |
 | 📊 **Visually Rich**   | [Get insights](https://github.com/Udayraj123/OMRChecker/wiki/Rich-Visuals) to configure and debug easily.                                                                                                        |
 | 🎈 **Lightweight**     | Very minimal core code size.                                                                                                                                                                                     |
 | 🏫 **Large Scale**     | Tested on a large scale at [Technothlon](https://en.wikipedia.org/wiki/Technothlon).                                                                                                                             |
@@ -248,7 +250,6 @@ Standard Quality Dataset(For ML Based methods) (3 GB)
 High Quality Dataset(For custom processing) (6 GB)
 -->
 
-
 ## FAQ
 
 <details>
@@ -328,23 +329,25 @@ Here's a snapshot of the [Android OMR Helper App (archived)](https://github.com/
 
 [![Stargazers over time](https://starchart.cc/Udayraj123/OMRChecker.svg)](https://starchart.cc/Udayraj123/OMRChecker)
 
-***
+---
+
 <h2 align="center">Made with ❤️ by Awesome Contributors</h2>
 
 <a href="https://github.com/Udayraj123/OMRChecker/graphs/contributors">
   <img src="https://contrib.rocks/image?repo=Udayraj123/OMRChecker" />
 </a>
 
-***
+---
+
 ### License
 
 [![GitHub license](https://img.shields.io/github/license/Udayraj123/OMRChecker.svg)](https://github.com/Udayraj123/OMRChecker/blob/master/LICENSE)
 
 For more details see [LICENSE](https://github.com/Udayraj123/OMRChecker/blob/master/LICENSE).
 
 ### Donate
-<a href="https://www.buymeacoffee.com/Udayraj123" target="_blank"><img src="https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png" alt="Buy Me A Coffee" style="height: 41px !important;width: 174px !important;box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;-webkit-box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;" ></a> [![paypal](https://www.paypalobjects.com/en_GB/i/btn/btn_donate_LG.gif)](https://www.paypal.me/Udayraj123/500)
 
+<a href="https://www.buymeacoffee.com/Udayraj123" target="_blank"><img src="https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png" alt="Buy Me A Coffee" style="height: 41px !important;width: 174px !important;box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;-webkit-box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;" ></a> [![paypal](https://www.paypalobjects.com/en_GB/i/btn/btn_donate_LG.gif)](https://www.paypal.me/Udayraj123/500)
 
 _Find OMRChecker on_ [**_Product Hunt_**](https://www.producthunt.com/posts/omr-checker/) **|** [**_Reddit_**](https://www.reddit.com/r/computervision/comments/ccbj6f/omrchecker_grade_exams_using_python_and_opencv/) **|** [**Discord**](https://discord.gg/qFv2Vqf) **|** [**Linkedin**](https://www.linkedin.com/pulse/open-source-talks-udayraj-udayraj-deshmukh/) **|** [**goodfirstissue.dev**](https://goodfirstissue.dev/language/python) **|** [**codepeak.tech**](https://www.codepeak.tech/) **|** [**fossoverflow.dev**](https://fossoverflow.dev/projects) **|** [**Interview on Console by CodeSee**](https://console.substack.com/p/console-140) **|** [**Open Source Hub**](https://opensourcehub.io/udayraj123/omrchecker)
 

main.py

@@ -12,7 +12,7 @@
 from pathlib import Path
 
 from src.entry import entry_point
-from src.logger import logger
+from src.utils.logger import logger
 
 
 def parse_args():
@@ -77,21 +77,28 @@ def parse_args():
     args = vars(args)
 
     if len(unknown) > 0:
-        logger.warning(f"\nError: Unknown arguments: {unknown}", unknown)
         argparser.print_help()
-        exit(11)
+        raise Exception(f"\nError: Unknown arguments: {unknown}")
     return args
 
 
 def entry_point_for_args(args):
     if args["debug"] is True:
         # Disable tracebacks
         sys.tracebacklimit = 0
+        # TODO: set log levels
     for root in args["input_paths"]:
-        entry_point(
-            Path(root),
-            args,
-        )
+        try:
+            entry_point(
+                Path(root),
+                args,
+            )
+        except Exception:
+            if args["debug"] is True:
+                logger.critical(
+                    f"OMRChecker crashed. add --debug and run again to see error details"
+                )
+            raise
 
 
 if __name__ == "__main__":

requirements.dev.txt

@@ -1,7 +1,8 @@
 -r requirements.txt
-flake8>=6.0.0
-freezegun>=1.2.2
-pre-commit>=3.3.3
-pytest-mock>=3.11.1
-pytest>=7.4.0
-syrupy>=4.0.4
+flake8>=7.0.0
+freezegun>=1.4.0
+pre-commit>=3.7.0
+pytest-mock>=3.14.0
+pytest>=8.1.1
+syrupy>=4.6.1
+Pillow>=10.3.0

requirements.txt

@@ -1,8 +1,16 @@
-deepmerge>=1.1.0
+deepmerge>=1.1.1
 dotmap>=1.3.30
-jsonschema>=4.17.3
-matplotlib>=3.7.1
-numpy>=1.25.0
-pandas>=2.0.2
-rich>=13.4.2
+jsonschema>=4.21.1
+matplotlib>=3.8.4
+numpy>=1.26.4
+pandas>=2.2.1
+rich>=13.7.1
 screeninfo>=0.8.1
+# External deps
+scipy>=1.12.0
+shapely>=2.0.3
+einops==0.7.0
+# Note: opencv is excluded from requirements.txt because the installation steps
+# may be different as per your OS. Windows/MacOS users can uncomment the following
+# opencv-python>=4.9.0.80
+# opencv-contrib-python>=4.9.0.80

samples/1-mobile-camera/config.json

@@ -0,0 +1,6 @@
+{
+  "outputs": {
+    "show_image_level": 0,
+    "show_colored_outputs": true
+  }
+}

samples/sample1/template.json → samples/1-mobile-camera/template.json

@@ -1,5 +1,5 @@
 {
-  "pageDimensions": [
+  "templateDimensions": [
     1846,
     1500
   ],
@@ -33,9 +33,38 @@
       "q9_2"
     ]
   },
+  "preProcessors": [
+    {
+      "name": "CropPage",
+      "options": {
+        "morphKernel": [
+          15,
+          15
+        ]
+      }
+    },
+    {
+      "name": "CropOnMarkers",
+      "options": {
+        "type": "FOUR_MARKERS",
+        "referenceImage": "omr_marker.jpg",
+        "tuningOptions": {
+          "marker_rescale_range": [
+            80,
+            120
+          ]
+        },
+        "markerDimensions": [
+          35,
+          35
+        ]
+      }
+    }
+  ],
   "fieldBlocks": {
     "Medium": {
       "bubblesGap": 41,
+      "fieldType": "CUSTOM",
       "bubbleValues": [
         "E",
         "H"
@@ -175,23 +204,5 @@
         1195
       ]
     }
-  },
-  "preProcessors": [
-    {
-      "name": "CropPage",
-      "options": {
-        "morphKernel": [
-          10,
-          10
-        ]
-      }
-    },
-    {
-      "name": "CropOnMarkers",
-      "options": {
-        "relativePath": "omr_marker.jpg",
-        "sheetToMarkerWidthRatio": 17
-      }
-    }
-  ]
+  }
 }

samples/sample5/README.md → samples/2-omr-marker/README.md

@@ -3,4 +3,4 @@
 This sample demonstrates multiple things, namely -
 - Running OMRChecker on images scanned using popular document scanning apps
 - Using a common template.json file for sub-folders (e.g. multiple scan batches)
-- Using evaluation.json file with custom marking (without streak-based marking)
+- Using evaluation on custom labels

samples/2-omr-marker/config.json

@@ -0,0 +1,5 @@
+{
+  "outputs": {
+    "show_image_level": 0
+  }
+}

samples/sample5/evaluation.json → samples/2-omr-marker/evaluation.json

@@ -30,8 +30,7 @@
       "D",
       "B",
       "A"
-    ],
-    "should_explain_scoring": true
+    ]
   },
   "marking_schemes": {
     "DEFAULT": {
@@ -89,5 +88,8 @@
         "unmarked": 0
       }
     }
+  },
+  "outputs_configuration": {
+    "should_explain_scoring": true
   }
 }

samples/sample5/template.json → samples/2-omr-marker/template.json

@@ -1,5 +1,5 @@
 {
-  "pageDimensions": [
+  "templateDimensions": [
     1846,
     1500
   ],
@@ -11,8 +11,12 @@
     {
       "name": "CropOnMarkers",
       "options": {
-        "relativePath": "omr_marker.jpg",
-        "sheetToMarkerWidthRatio": 17
+        "type": "FOUR_MARKERS",
+        "referenceImage": "omr_marker.jpg",
+        "markerDimensions": [
+          35,
+          35
+        ]
       }
     }
   ],
@@ -44,6 +48,7 @@
   },
   "fieldBlocks": {
     "Medium": {
+      "fieldType": "CUSTOM",
       "bubbleValues": [
         "E",
         "H"