feat: Added a decorator function, which will be used to test `explici…

…t examples` (#28251)

docs/overview/deep_dive/ivy_tests.rst

@@ -63,6 +63,7 @@ Ivy Tests
 .. _`num_positional_args`: https://github.com/unifyai/ivy/blob/e50f71e283313caa9737f3c284496022ac67b58b/ivy_tests/test_ivy/helpers/testing_helpers.py#L78
 .. _`CI Pipeline`: continuous_integration.rst
 .. _`Hypothesis docs`: https://hypothesis.readthedocs.io/en/latest/data.html#core-strategies
+.. _`this`: https://github.com/unifyai/ivy/blob/8dcc33b895240395686db165c710ac31708aa691/ivy_tests/test_ivy/test_functional/test_core/test_general.py#L1650
 
 On top of the Array API `test suite`_, which is included as a submodule mapped to the folder :code:`test_array_api`, there is also a collection of Ivy tests, located in subfolder `test_ivy`_.
 
@@ -221,7 +222,7 @@ Ivy Test Decorators
 
 - Why do we need to handle test decorators?
 
-In order to run a test, a lot of pre-processing must be done, e.g. import the function, does it support complex data type? does it run on CPU? how many parameters does it take? are they positional or keyword only, or both? and a lot of information about the function that is being tested, this allows us later to run the test efficiently and in a **complete** way. all of this happens at collecting time.
+In order to run a test, a lot of pre-processing must be done, e.g. import the function, does it support complex data type? does it run on CPU? how many parameters does it take? are they positional or keyword only, or both? and a lot of information about the function that is being tested, this allows us later to run the test efficiently and in a **complete** way. All of this happens at collecting time.
 
 - What do the handle test decorators do?
 
@@ -241,14 +242,15 @@ This is not an exhaustive list of what the :code:`handle_test` decorators actual
 
 - Why do we have multiple handle test decorators?
 
-Having multiple test decorators is mainly for efficiency, `handle_test` could do what `handle_frontend_test` does, it just handles the parameters slightly different, and this can be inferred at run time, but we choose to separate the decorator for general different usages, currently we have 4 separate decorators
+Having multiple test decorators is mainly for efficiency, `handle_test` could do what `handle_frontend_test` does, it just handles the parameters slightly different, and this can be inferred at run time, but we choose to separate the decorator for general different usages, currently we have 5 separate decorators
 
 1.  :code:`handle_test`
 2.  :code:`handle_method`
 3.  :code:`handle_frontend_test`
 4.  :code:`handle_frontend_method`
+5.  :code:`handle_example`
 
-One of the few differences between the 4 decorators is that they generate different kinds of flags, some generate more or less, but they all share the same general structure.
+One of the few differences between the 5 decorators is that they generate different kinds of flags, some generate more or less, but they all share the same general structure.
 
 - Integration
 
@@ -359,6 +361,49 @@ Let's look at the data produced by this strategy -:
 These values are then unpacked, converted to :class:`ivy.Array` class, with corresponding dtypes.
 The test then runs on the newly created arrays with specified data types.
 
+Adding Explicit Examples to tests
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+In certain cases where we'd like to test certain examples explicitly which are outliers and it isn't feasible to define them as a strategy,
+we can use the :code:`@handle_example` decorator. One such example is `this`_ where we need to test `ivy.set_item` with slice objects.
+
+Hypothesis allows us to test with an explicit example deterministically using the `@example`_ decorator. Our :code:`@handle_example` decorator is a wrapper around this.
+Which helps us to use the default values of the test_flags, method_flags allowing us to easily test explicit examples for the ivy functional tests, frontend_tests, methods, and frontend_methods.
+
+We have to pass one of the following 4 arguments as `True` to the :code:`@handle_example` decorator depending on what test we are dealing with.
+1. `test_example`
+2. `test_frontend_example`
+3. `test_method_example`
+4. `test_frontend_method_example`
+
+The following example shows, how we can use the :code:`@handle_example` decorator to test the one of the frontend functions by adding an explicit example.
+
+.. code-block:: python
+    @handle_frontend_test(
+    fn_tree="paddle.acos",
+    dtype_and_x=helpers.dtype_and_values(available_dtypes=helpers.get_dtypes("float"),),
+    )
+    @handle_example(
+        test_frontend_example=True,
+        dtype_and_x=(["float32"], [np.array(9.0, dtype=np.float32)]),
+        fn_tree="ivy.functional.frontends.paddle.acos",
+        test_flags={
+            "native_arrays": [True],
+            "with_copy": True,
+            "with_out": True,
+        },
+    )
+    def test_some_function(
+        *,
+        dtype_and_x,
+        fn_tree,
+        frontend,
+        test_flags,
+        backend_fw,
+    ):
+        pass
+
+
 Why do we need helper functions?
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 

ivy_tests/test_ivy/helpers/testing_helpers.py

@@ -7,7 +7,7 @@
 import functools
 from typing import List, Optional
 
-from hypothesis import given, strategies as st
+from hypothesis import given, strategies as st, example
 
 # local
 import ivy.functional.frontends.numpy as np_frontend
@@ -939,3 +939,114 @@ def _create_transpile_report(
     json_object = json.dumps(data, indent=6)
     with open(file_name, "w") as outfile:
         outfile.write(json_object)
+
+
+def handle_example(
+    *,
+    test_example: bool = False,
+    test_frontend_example: bool = False,
+    test_method_example: bool = False,
+    test_frontend_method_example: bool = False,
+    **given_kwargs,
+):
+    if test_example:
+        test_flags = given_kwargs.get("test_flags", {})
+        flags = pf.FunctionTestFlags(
+            ground_truth_backend=test_flags.get("ground_truth_backend", "numpy"),
+            num_positional_args=test_flags.get("num_positional_args", 0),
+            instance_method=test_flags.get("instance_method", False),
+            with_out=test_flags.get("with_out", False),
+            with_copy=test_flags.get("with_copy", False),
+            test_gradients=test_flags.get("test_gradients", False),
+            test_trace=test_flags.get("test_trace", False),
+            transpile=test_flags.get("transpile", False),
+            as_variable=test_flags.get("as_variable", [False]),
+            native_arrays=test_flags.get("native_arrays", [False]),
+            container=test_flags.get("container", [False]),
+            precision_mode=test_flags.get("precision_mode", False),
+            test_cython_wrapper=test_flags.get("test_cython_wrapper", False),
+        )
+
+        given_kwargs["test_flags"] = flags
+
+    elif test_frontend_example:
+        test_flags = given_kwargs.get("test_flags", {})
+        flags = pf.FrontendFunctionTestFlags(
+            num_positional_args=test_flags.get("num_positional_args", 0),
+            with_out=test_flags.get("with_out", False),
+            with_copy=test_flags.get("with_copy", False),
+            inplace=test_flags.get("inplace", False),
+            as_variable=test_flags.get("as_variable", [False]),
+            native_arrays=test_flags.get("native_arrays", [False]),
+            test_trace=test_flags.get("test_trace", False),
+            generate_frontend_arrays=test_flags.get("generate_frontend_arrays", False),
+            transpile=test_flags.get("transpile", False),
+            precision_mode=test_flags.get("precision_mode", False),
+        )
+
+        given_kwargs["test_flags"] = flags
+
+    elif test_method_example:
+        method_flags = given_kwargs.get("method_flags", {})
+        init_flags = given_kwargs.get("init_flags", {})
+        flags_1 = pf.MethodTestFlags(
+            num_positional_args=method_flags.get("num_positional_args", 0),
+            as_variable=method_flags.get("as_variable", [False]),
+            native_arrays=method_flags.get("native_arrays", [False]),
+            container_flags=method_flags.get("container", [False]),
+            precision_mode=method_flags.get("precision_mode", False),
+        )
+
+        flags_2 = pf.InitMethodTestFlags(
+            num_positional_args=init_flags.get("num_positional_args", 0),
+            as_variable=init_flags.get("as_variable", [False]),
+            native_arrays=init_flags.get("native_arrays", [False]),
+            precision_mode=init_flags.get("precision_mode", False),
+        )
+
+        given_kwargs["method_flags"] = flags_1
+        given_kwargs["init_flags"] = flags_2
+
+    elif test_frontend_method_example:
+        method_flags = given_kwargs.get("method_flags", {})
+        init_flags = given_kwargs.get("init_flags", {})
+        flags_1 = pf.FrontendMethodTestFlags(
+            num_positional_args=method_flags.get("num_positional_args", 0),
+            as_variable=method_flags.get("as_variable", [False]),
+            native_arrays=method_flags.get("native_arrays", [False]),
+            precision_mode=method_flags.get("precision_mode", False),
+            inplace=method_flags.get("inplace", False),
+            test_trace=method_flags.get("test_trace", False),
+            generate_frontend_arrays=method_flags.get(
+                "generate_frontend_arrays", False
+            ),
+        )
+
+        flags_2 = pf.FrontendInitTestFlags(
+            num_positional_args=init_flags.get("num_positional_args", 0),
+            as_variable=init_flags.get("as_variable", [False]),
+            native_arrays=init_flags.get("native_arrays", [False]),
+        )
+
+        given_kwargs["method_flags"] = flags_1
+        given_kwargs["init_flags"] = flags_2
+
+    def test_wrapper(test_fn):
+
+        hypothesis_test_fn = example(**given_kwargs)(test_fn)
+
+        @functools.wraps(hypothesis_test_fn)
+        def wrapped_test(*args, **kwargs):
+            try:
+                hypothesis_test_fn(*args, **kwargs)
+            except Exception as e:
+                # A string matching is used instead of actual exception due to
+                # exception object in with_backend is different from global Ivy
+                if e.__class__.__qualname__ == "IvyNotImplementedException":
+                    pytest.skip("Function not implemented in backend.")
+                else:
+                    raise e
+
+        return wrapped_test
+
+    return test_wrapper

ivy_tests/test_ivy/test_functional/test_core/test_general.py

@@ -6,7 +6,7 @@
 from types import SimpleNamespace
 
 import pytest
-from hypothesis import given, assume, example, strategies as st
+from hypothesis import given, assume, strategies as st
 import numpy as np
 from collections.abc import Sequence
 
@@ -18,7 +18,7 @@
 from ivy_tests.test_ivy.helpers import (
     handle_test,
     BackendHandler,
-    test_parameter_flags as pf,
+    handle_example,
 )
 from ivy_tests.test_ivy.helpers.assertions import assert_all_close
 from ivy_tests.test_ivy.test_functional.test_core.test_elementwise import pow_helper
@@ -1647,7 +1647,11 @@ def test_set_inplace_mode(mode):
     container_flags=st.just([False]),
     test_with_copy=st.just(True),
 )
-@example(
+@handle_example(
+    test_example=True,
+    test_flags={
+        "num_positional_args": 3,
+    },
     dtypes_x_query_val=(
         ["int32", "int32"],
         np.ones((1, 3, 3, 3)),
@@ -1656,21 +1660,6 @@ def test_set_inplace_mode(mode):
     ),
     copy=False,
     fn_name="set_item",
-    test_flags=pf.FunctionTestFlags(
-        ground_truth_backend="numpy",
-        num_positional_args=3,
-        instance_method=False,
-        with_out=False,
-        with_copy=False,
-        test_gradients=False,
-        test_trace=False,
-        transpile=False,
-        as_variable=[False],
-        native_arrays=[False],
-        container=[False],
-        precision_mode=False,
-        test_cython_wrapper=False,
-    ),
 )
 def test_set_item(
     dtypes_x_query_val,