skbase.testing.TestAllObjects

class skbase.testing.TestAllObjects

Package level tests for BaseObjects.

Attributes:

exclude_objects

excluded_tests

object_type_filter

valid_base_types

valid_tags

Methods

generator_dict()	Return dict with methods _generate_[variable] collected in a dict.
is_excluded(test_name, est)	Shorthand to check whether test test_name is excluded for object est.
object_instance(request)	object_instance fixture definition for indirect use.
pytest_generate_tests(metafunc)	Test parameterization routine for pytest.
run_tests(obj[, raise_exceptions, ...])	Run all tests on one single object.
test_class_tags_match_mro(object_class)	Check that class tags correctly reflect the full MRO.
test_clone(object_instance)	Check that clone method does not raise exceptions and results in a clone.
test_constructor(object_class)	Check that the constructor has sklearn compatible signature and behaviour.
test_create_test_instance(object_class)	Check create_test_instance logic and basic constructor functionality.
test_create_test_instances_and_names(...)	Check that create_test_instances_and_names works.
test_get_params(object_instance)	Check that get_params works correctly, against sklearn interface.
test_inheritance(object_class)	Check that object inherits from BaseObject.
test_no_between_test_case_side_effects(...)	Test that there are no side effects across instances of the same test.
test_no_cross_test_side_effects_part1(...)	Test that there are no side effects across tests, through object state.
test_no_cross_test_side_effects_part2(...)	Test that there are no side effects across tests, through object state.
test_object_tags(object_class)	Check conventions on object tags.
test_repr(object_instance)	Check that __repr__ call to instance does not raise exceptions.
test_repr_html(object_instance)	Check that _repr_html_ call to instance does not raise exceptions.
test_set_params(object_instance)	Check that set_params works correctly.
test_set_params_sklearn(object_class)	Check that set_params works correctly, mirrors sklearn check_set_params.
test_valid_object_class_tags(object_class)	Check that object class tags are in self.valid_tags.
test_valid_object_tags(object_instance)	Check that object tags are in self.valid_tags.

test_create_test_instance(object_class)

Check create_test_instance logic and basic constructor functionality.

create_test_instance and create_test_instances_and_names are the key methods used to create test instances in testing. If this test does not pass, validity of the other tests cannot be guaranteed.

Also tests inheritance and super call logic in the constructor.

Tests that:

• create_test_instance results in an instance of object_class

• __init__ calls super.__init__

• _tags_dynamic attribute for tag inspection is present after construction

test_create_test_instances_and_names(object_class)

Check that create_test_instances_and_names works.

create_test_instance and create_test_instances_and_names are the key methods used to create test instances in testing. If this test does not pass, validity of the other tests cannot be guaranteed.

Tests expected function signature of create_test_instances_and_names.

test_object_tags(object_class)

Check conventions on object tags.

test_class_tags_match_mro(object_class)

Check that class tags correctly reflect the full MRO.

Based on the bugfix for Issue #539: Python’s C3 linearisation can place mixins after BaseObject in the MRO. class flags/tags should therefore be collected from the entire MRO, excluding only object.

test_inheritance(object_class)

Check that object inherits from BaseObject.

test_no_cross_test_side_effects_part1(object_instance)

Test that there are no side effects across tests, through object state.

test_no_cross_test_side_effects_part2(object_instance)

Test that there are no side effects across tests, through object state.

test_no_between_test_case_side_effects(object_instance, a)

Test that there are no side effects across instances of the same test.

test_get_params(object_instance)

Check that get_params works correctly, against sklearn interface.

test_set_params(object_instance)

Check that set_params works correctly.

test_set_params_sklearn(object_class)

Check that set_params works correctly, mirrors sklearn check_set_params.

Instead of the “fuzz values” in sklearn’s check_set_params, we use the other test parameter settings (which are assumed valid). This guarantees settings which play along with the __init__ content.

test_clone(object_instance)

Check that clone method does not raise exceptions and results in a clone.

A clone of an object x is an object that:

• has same class and parameters as x

• is not identical with x

• is unfitted (even if x was fitted)

test_repr(object_instance)

Check that __repr__ call to instance does not raise exceptions.

test_repr_html(object_instance)

Check that _repr_html_ call to instance does not raise exceptions.

test_constructor(object_class)

Check that the constructor has sklearn compatible signature and behaviour.

Based on sklearn check_estimator testing of __init__ logic. Uses create_test_instance to create an instance. Assumes test_create_test_instance has passed and certified create_test_instance.

Tests that:

• constructor has no varargs

• tests that constructor constructs an instance of the class

• tests that all parameters are set in init to an attribute of the same name

• tests that parameter values are always copied to the attribute and not changed

• tests that default parameters are one of the following:

  None, str, int, float, bool, tuple, function, joblib memory, numpy primitive (other type parameters should be None, default handling should be by writing the default to attribute of a different name, e.g., my_param_ not my_param)

test_valid_object_class_tags(object_class)

Check that object class tags are in self.valid_tags.

test_valid_object_tags(object_instance)

Check that object tags are in self.valid_tags.

generator_dict()

Return dict with methods _generate_[variable] collected in a dict.

The returned dict is the one required by create_conditional_fixtures_and_names,

used in this _conditional_fixture plug-in to pytest_generate_tests, above.

Returns:

generator_dictdict, with keys [variable]

dictionary of methods _generate_[variable] for fixture generation where [variable] are all strings such that self has a static method, named _generate_[variable](test_name: str, **kwargs) value at [variable] is a reference to _generate_[variable]

is_excluded(test_name, est)

Shorthand to check whether test test_name is excluded for object est.

object_instance(request)

object_instance fixture definition for indirect use.

pytest_generate_tests(metafunc)

Test parameterization routine for pytest.

This uses create_conditional_fixtures_and_names and generator_dict to create the fixtures for a mark.parametrize decoration of all tests.

run_tests(obj, raise_exceptions=False, tests_to_run=None, fixtures_to_run=None, tests_to_exclude=None, fixtures_to_exclude=None, verbose=False)

Run all tests on one single object.

All tests in self are run on the following object type fixtures:

• if est is a class, then object_class = est, and object_instance loops over est.create_test_instance()

• if est is an object, then object_class = est.__class__, and object_instance = est

Compatibility with pytest fixtures:

• run_tests is compatible with pytest.mark.parametrize decoration, but currently only with multiple single variable annotations.

• the following pytest reserved fixture names are supported: tmp_path, monkeypatch, capsys, caplog

Parameters:

objsubclass of scikit-base BaseObject, or instance thereof

scikit-base object class or scikit-base object instance

raise_exceptionsbool, optional, default=False

whether to return exceptions/failures in the results dict, or raise them

• if False: returns exceptions in returned results dict

• if True: raises exceptions as they occur

tests_to_runstr or list of str, names of tests to run. default = all tests

sub-sets tests that are run to the tests given here.

fixtures_to_runstr or list of str, pytest test-fixture combination codes.

which test-fixture combinations to run. Default = run all of them. sub-sets tests and fixtures to run to the list given here. If both tests_to_run and fixtures_to_run are provided, runs the union of tests implied by both, i.e., all test-fixture combinations for tests in tests_to_run, plus all test-fixture combinations in fixtures_to_run.

tests_to_excludestr or list of str, names of tests to exclude. default = None

removes tests that should not be run, after subsetting via tests_to_run.

fixtures_to_excludestr or list of str, fixtures to exclude. default = None

removes test-fixture combinations that should not be run. This is done after subsetting via fixtures_to_run.

verboseint or bool, optional, default=0.

verbosity level for printouts from tests run.

• 0 or False (default): no printout

• 1 or True: print summary of test run, but no print from tests

• 2: print all test output, including output from within the tests

Returns:

resultsdict of results of the tests in self

keys are test/fixture strings, identical as in pytest, e.g., test[fixture] entries are the string "PASSED" if the test passed, or the exception raised if the test did not pass.

results is returned only if all tests pass, or raise_exceptions=False.

Raises:

if raise_exception=True, raises any exception produced by the tests directly

Examples

>>> from skbase.tests.mock_package.test_mock_package import CompositionDummy
>>> from skbase.testing.test_all_objects import TestAllObjects
>>> TestAllObjects().run_tests(
...     CompositionDummy,
...     tests_to_run="test_constructor"
... )
{'test_constructor[CompositionDummy]': 'PASSED'}
>>> TestAllObjects().run_tests(
...     CompositionDummy, fixtures_to_run="test_repr[CompositionDummy-1]"
... )
{'test_repr[CompositionDummy-1]': 'PASSED'}