Base Class

Prototypes of privacy safe model wrappers.

class sacroml.safemodel.safemodel.SafeModel[source]

Privacy protected model base class.

Attributes:

model_typestring: A string describing the type of model. Default is “None”.
model:: The Machine Learning Model.
saved_model:: A saved copy of the Machine Learning Model used for comparison.
ignore_itemslist: A list of items to ignore when comparing the model with the saved_model.
examine_separately_itemslist: A list of items to examine separately. These items are more complex datastructures that cannot be compared directly.
filenamestring: A filename to save the model.
researcherstring: The researcher user-id used for logging

Methods

`additional_checks`(curr_separate, saved_separate)	Perform additional posthoc checks.
`examine_seperate_items`(curr_vals, saved_vals)	Check model-specific items exist in both current and saved copies.
`get_current_and_saved_models`()	Copy self.__dict__ and split into dicts for current and saved versions.
`get_params`([deep])	Get a dictionary of parameter values restricted to those expected.
`posthoc_check`()	Check whether model has been interfered with since fit() was last run.
`preliminary_check`([verbose, apply_constraints])	Check whether current model parameters violate the safe rules.
`request_release`(path, ext[, target])	Save model and create a report for the TRE output checkers.
`run_attack`(target, attack_name[, output_dir])	Run a specified attack on the trained model and save report to file.
`save`([name])	Write model to file in appropriate format.

Examples

>>> safeRFModel = SafeRandomForestClassifier()
>>> safeRFModel.fit(X, y)
>>> safeRFModel.save(name="safe.pkl")
>>> safeRFModel.preliminary_check()
>>> safeRFModel.request_release(
...     path="safe",
...     ext="pkl",
...     target=target,
... )
WARNING: model parameters may present a disclosure risk:
- parameter min_samples_leaf = 1 identified as less than the recommended min value of 5.
Changed parameter min_samples_leaf = 5.

Model parameters are within recommended ranges.

__apply_constraints(operator: str, key: str, val: Any, cur_val: Any) → str: Apply a safe rule for a given parameter.

__check_model_param(rule: dict, apply_constraints: bool) → tuple[str, bool]

Check whether a current model parameter violates a safe rule.

Optionally fixes violations.

__check_model_param_and(rule: dict, apply_constraints: bool) → tuple[str, bool]

Check whether current model parameters violate a logical AND rule.

Optionally fixes violations.

__check_model_param_or(rule: dict) → tuple[str, bool]: Check whether current model parameters violate a logical OR rule.

__get_constraints() → dict: Get constraints relevant to the model type from the a read-only file.

__init__() → None[source]: Super class constructor, gets researcher name.

__str__() → str[source]: Return string with model description.

additional_checks(curr_separate: dict, saved_separate: dict) → tuple[str, bool][source]

Perform additional posthoc checks.

Placeholder function for additional posthoc checks e.g. keras. This version just checks that any lists have the same contents.

Parameters:

curr_separatedict
saved_separatedict

Returns:

msgstring: A message string.
disclosivebool: A boolean value to indicate whether the model is potentially disclosive.

Notes

Posthoc checking makes sure that the two dicts have the same set of keys as defined in the list self.examine_separately.

examine_seperate_items(curr_vals: dict, saved_vals: dict) → tuple[str, bool][source]: Check model-specific items exist in both current and saved copies.

get_current_and_saved_models() → tuple[dict, dict][source]: Copy self.__dict__ and split into dicts for current and saved versions.

get_params(deep: bool = True) → dict[source]: Get a dictionary of parameter values restricted to those expected.

posthoc_check() → tuple[str, bool][source]: Check whether model has been interfered with since fit() was last run.

preliminary_check(verbose: bool = True, apply_constraints: bool = False) → tuple[str, bool][source]

Check whether current model parameters violate the safe rules.

Optionally fixes violations.

Parameters:

verbosebool: A boolean value to determine increased output level.
apply_constraintsbool: A boolean to determine whether identified constraints are to be upheld and applied.

Returns:

msgstring: A message string.
disclosivebool: A boolean value indicating whether the model is potentially disclosive.

request_release(path: str, ext: str, target: Target | None = None) → None[source]

Save model and create a report for the TRE output checkers.

Parameters:

pathstring: Path to save the outputs.
extstr: File extension defining the model saved format, e.g., “pkl” or “sav”.
targetattacks.target.Target: Contains model and dataset information.

Notes

If target is not null, then worst case MIA and attribute inference attacks are called via run_attack.

run_attack(target: Target, attack_name: str, output_dir: str = 'outputs_safemodel') → dict[source]

Run a specified attack on the trained model and save report to file.

Parameters:

targetTarget: The target in the form of a Target object.
attack_namestr: Name of the attack to run.
output_dirstr: Name of the directory to store JSON and PDF reports.

Returns:

dict: Metadata results.

save(name: str = 'undefined') → None[source]

Write model to file in appropriate format.

Note this is overloaded in SafeKerasClassifer to deal with tensorflow specifics.

Parameters:

namestring: The name of the file to save.

Notes

Optimizer is deliberately excluded to prevent possible restart to training and thus possible back door into attacks.

__annotations__ = {}

__dict__ = mappingproxy({'__module__': 'sacroml.safemodel.safemodel', '__doc__': 'Privacy protected model base class.\n\n Attributes\n ----------\n model_type : string\n A string describing the type of model. Default is "None".\n model:\n The Machine Learning Model.\n saved_model:\n A saved copy of the Machine Learning Model used for comparison.\n ignore_items : list\n A list of items to ignore when comparing the model with the\n saved_model.\n examine_separately_items : list\n A list of items to examine separately. These items are more\n complex datastructures that cannot be compared directly.\n filename : string\n A filename to save the model.\n researcher : string\n The researcher user-id used for logging\n\n Examples\n --------\n >>> safeRFModel = SafeRandomForestClassifier()\n >>> safeRFModel.fit(X, y)\n >>> safeRFModel.save(name="safe.pkl")\n >>> safeRFModel.preliminary_check()\n >>> safeRFModel.request_release(\n ... path="safe",\n ... ext="pkl",\n ... target=target,\n ... )\n WARNING: model parameters may present a disclosure risk:\n - parameter min_samples_leaf = 1 identified as less than the recommended min value of 5.\n Changed parameter min_samples_leaf = 5.\n\n Model parameters are within recommended ranges.\n ', '__init__': <function SafeModel.__init__>, 'get_params': <function SafeModel.get_params>, 'save': <function SafeModel.save>, '_SafeModel__get_constraints': <function SafeModel.__get_constraints>, '_SafeModel__apply_constraints': <function SafeModel.__apply_constraints>, '_SafeModel__check_model_param': <function SafeModel.__check_model_param>, '_SafeModel__check_model_param_and': <function SafeModel.__check_model_param_and>, '_SafeModel__check_model_param_or': <function SafeModel.__check_model_param_or>, 'preliminary_check': <function SafeModel.preliminary_check>, 'get_current_and_saved_models': <function SafeModel.get_current_and_saved_models>, 'examine_seperate_items': <function SafeModel.examine_seperate_items>, 'posthoc_check': <function SafeModel.posthoc_check>, 'additional_checks': <function SafeModel.additional_checks>, 'request_release': <function SafeModel.request_release>, 'run_attack': <function SafeModel.run_attack>, '__str__': <function SafeModel.__str__>, '__dict__': <attribute '__dict__' of 'SafeModel' objects>, '__weakref__': <attribute '__weakref__' of 'SafeModel' objects>, '__annotations__': {'model_type': 'str', 'model_load_file': 'str', 'model_save_file': 'str', 'ignore_items': 'list[str]', 'examine_seperately_items': 'list[str]', 'filename': 'str', 'researcher': 'str', 'timestamp': 'str'}})

__module__ = 'sacroml.safemodel.safemodel'

sacroml.safemodel.safemodel.check_equal(key: str, val: Any, cur_val: Any) → tuple[str, bool][source]

Check equality value constraint.

Parameters:

keystring: The dictionary key to examine.
valAny Type: The expected value of the key.
cur_valAny Type: The current value of the key.

Returns:

msgstring: A message string.
disclosivebool: A boolean value indicating whether the model is potentially disclosive.

sacroml.safemodel.safemodel.check_max(key: str, val: Any, cur_val: Any) → tuple[str, bool][source]

Check maximum value constraint.

Parameters:

keystring: The dictionary key to examine.
valAny Type: The expected value of the key.
cur_valAny Type: The current value of the key.

Returns:

msgstring: A message string.
disclosivebool: A boolean value indicating whether the model is potentially disclosive.

sacroml.safemodel.safemodel.check_min(key: str, val: Any, cur_val: Any) → tuple[str, bool][source]

Check minimum value constraint.

Parameters:

keystring: The dictionary key to examine.
valAny Type: The expected value of the key.
cur_valAny Type: The current value of the key.
..

Returns:

msgstring: A message string.
disclosivebool: A boolean value indicating whether the model is potentially disclosive.

sacroml.safemodel.safemodel.check_type(key: str, val: Any, cur_val: Any) → tuple[str, bool][source]

Check the type of a value.

Parameters:

keystring: The dictionary key to examine.
valAny Type: The expected value of the key.
cur_valAny Type: The current value of the key.

Returns:

msgstring: A message string.
disclosivebool: A boolean value indicating whether the model is potentially disclosive.