Skip to content

api.types not found when using pytest #187

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
smokestacklightnin opened this issue Oct 19, 2024 · 1 comment
Open

api.types not found when using pytest #187

smokestacklightnin opened this issue Oct 19, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@smokestacklightnin
Copy link
Member

Please go to Stack Overflow for help and support:

System information

  • Have I written custom code (as opposed to using a stock example script
    provided in TensorFlow Model Analysis)    : No
  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 22.04
  • TensorFlow Model Analysis installed from (source or binary): source
  • TensorFlow Model Analysis version (use command below): 0.47.0.dev
  • Python version: 3.9, 3.10
  • Jupyter Notebook version:
  • Exact command to reproduce: $ pytest

Describe the problem

Before PR #183, running pytest yields several errors. One such repeated error is

FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testAnalyzeRawData - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'

This error is repeated for many tests (listed below).

I have included a temporary fix in #183 by including

from tensorflow_model_analysis.api import types

__all__ = [
    "types",
]

to tensorflow_model_analysis/api/__init__.py. This gets rid of the error, but not of the deeper underlying issue.

Source code / logs

The full traceback for the particular example (tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testAnalyzeRawData) above is

encoded = b'QlpoOTFBWSZTWfGpBdoACmP/8P//////////v+//qv////Zr5A4AYAAAAOAJJwfYhkBQAUkKKWWKQFOGiKek0T1T09U9NGmgCDZJ5T2qYamm0Jj1TI8o...Wo108/nlI02xnupRhNuthTljTSAfIjSd7o2REZ4IzMws1nNJkh21YUjBjs4aY3SFDGN1qTL4LmU7aWzo2d/p5Hr3GDG5YT9Oj+LL/8XckU4UJDxqQXaA=='
enable_trace = True, use_zlib = False

    def loads(encoded, enable_trace=True, use_zlib=False):
      """For internal use only; no backwards-compatibility guarantees."""
    
      c = base64.b64decode(encoded)
    
      if use_zlib:
        s = zlib.decompress(c)
      else:
        s = bz2.decompress(c)
    
      del c  # Free up some possibly large and no-longer-needed memory.
    
      with _pickle_lock:
        try:
>         return dill.loads(s)

.venv310/lib/python3.10/site-packages/apache_beam/internal/dill_pickler.py:415: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.venv310/lib/python3.10/site-packages/dill/_dill.py:275: in loads
    return load(file, ignore, **kwds)
.venv310/lib/python3.10/site-packages/dill/_dill.py:270: in load
    return Unpickler(file, ignore=ignore, **kwds).load()
.venv310/lib/python3.10/site-packages/dill/_dill.py:472: in load
    obj = StockUnpickler.load(self)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

import_name = 'tensorflow_model_analysis.api.types', safe = False

    def _import_module(import_name, safe=False):
        try:
            if '.' in import_name:
                items = import_name.split('.')
                module = '.'.join(items[:-1])
                obj = items[-1]
            else:
                return __import__(import_name)
>           return getattr(__import__(module, None, None, [obj]), obj)
E           AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'

.venv310/lib/python3.10/site-packages/dill/_dill.py:827: AttributeError

During handling of the above exception, another exception occurred:

self = <tensorflow_model_analysis.api.model_eval_lib_test.EvaluateTest testMethod=testAnalyzeRawData>

    def testAnalyzeRawData(self):
    
      # Data
      # age language  label  prediction
      # 17  english   0      0
      # 30  spanish   1      1
      dict_data = [
          {'age': 17, 'language': 'english', 'prediction': 0, 'label': 0},
          {'age': 30, 'language': 'spanish', 'prediction': 1, 'label': 1},
      ]
      df_data = pd.DataFrame(dict_data)
    
      # Expected Output
      expected_slicing_metrics = {
          (('language', 'english'),): {
              '': {
                  '': {
                      'binary_accuracy': {'doubleValue': 1.0},
                      'example_count': {'doubleValue': 1.0},
                  }
              }
          },
          (('language', 'spanish'),): {
              '': {
                  '': {
                      'binary_accuracy': {'doubleValue': 1.0},
                      'example_count': {'doubleValue': 1.0},
                  }
              }
          },
          (): {
              '': {
                  '': {
                      'binary_accuracy': {'doubleValue': 1.0},
                      'example_count': {'doubleValue': 2.0},
                  }
              }
          },
      }
    
      # Actual Output
      eval_config = text_format.Parse(
          """
        model_specs {
          label_key: 'label'
          prediction_key: 'prediction'
        }
        metrics_specs {
          metrics { class_name: "BinaryAccuracy" }
          metrics { class_name: "ExampleCount" }
        }
        slicing_specs {}
        slicing_specs {
          feature_keys: 'language'
        }
      """,
          config_pb2.EvalConfig(),
      )
>     eval_result = model_eval_lib.analyze_raw_data(df_data, eval_config)

tensorflow_model_analysis/api/model_eval_lib_test.py:1483: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tensorflow_model_analysis/api/model_eval_lib.py:1609: in analyze_raw_data
    p
.venv310/lib/python3.10/site-packages/apache_beam/pvalue.py:138: in __or__
    return self.pipeline.apply(ptransform, self)
.venv310/lib/python3.10/site-packages/apache_beam/pipeline.py:681: in apply
    return self.apply(
.venv310/lib/python3.10/site-packages/apache_beam/pipeline.py:692: in apply
    return self.apply(transform, pvalueish)
.venv310/lib/python3.10/site-packages/apache_beam/pipeline.py:754: in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
.venv310/lib/python3.10/site-packages/apache_beam/runners/runner.py:191: in apply
    return self.apply_PTransform(transform, input, options)
.venv310/lib/python3.10/site-packages/apache_beam/runners/runner.py:195: in apply_PTransform
    return transform.expand(input)
.venv310/lib/python3.10/site-packages/apache_beam/transforms/ptransform.py:1005: in expand
    return self._fn(pcoll, *args, **kwargs)
tensorflow_model_analysis/api/model_eval_lib.py:1216: in ExtractEvaluateAndWriteResults
    extracts = examples | 'BatchedInputsToExtracts' >> BatchedInputsToExtracts()
.venv310/lib/python3.10/site-packages/apache_beam/pvalue.py:138: in __or__
    return self.pipeline.apply(ptransform, self)
.venv310/lib/python3.10/site-packages/apache_beam/pipeline.py:681: in apply
    return self.apply(
.venv310/lib/python3.10/site-packages/apache_beam/pipeline.py:692: in apply
    return self.apply(transform, pvalueish)
.venv310/lib/python3.10/site-packages/apache_beam/pipeline.py:754: in apply
    pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
.venv310/lib/python3.10/site-packages/apache_beam/runners/runner.py:191: in apply
    return self.apply_PTransform(transform, input, options)
.venv310/lib/python3.10/site-packages/apache_beam/runners/runner.py:195: in apply_PTransform
    return transform.expand(input)
.venv310/lib/python3.10/site-packages/apache_beam/transforms/ptransform.py:1005: in expand
    return self._fn(pcoll, *args, **kwargs)
tensorflow_model_analysis/api/model_eval_lib.py:876: in BatchedInputsToExtracts
    return batched_inputs | 'AddArrowRecordBatchKey' >> beam.Map(to_extracts)
.venv310/lib/python3.10/site-packages/apache_beam/transforms/core.py:2109: in Map
    pardo = FlatMap(wrapper, *args, **kwargs)
.venv310/lib/python3.10/site-packages/apache_beam/transforms/core.py:2052: in FlatMap
    pardo = ParDo(CallableWrapperDoFn(fn), *args, **kwargs)
.venv310/lib/python3.10/site-packages/apache_beam/transforms/core.py:1564: in __init__
    super().__init__(fn, *args, **kwargs)
.venv310/lib/python3.10/site-packages/apache_beam/transforms/ptransform.py:870: in __init__
    self.fn = pickler.loads(pickler.dumps(self.fn))
.venv310/lib/python3.10/site-packages/apache_beam/internal/pickler.py:50: in loads
    return desired_pickle_lib.loads(
.venv310/lib/python3.10/site-packages/apache_beam/internal/dill_pickler.py:419: in loads
    return dill.loads(s)
.venv310/lib/python3.10/site-packages/dill/_dill.py:275: in loads
    return load(file, ignore, **kwds)
.venv310/lib/python3.10/site-packages/dill/_dill.py:270: in load
    return Unpickler(file, ignore=ignore, **kwds).load()
.venv310/lib/python3.10/site-packages/dill/_dill.py:472: in load
    obj = StockUnpickler.load(self)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

import_name = 'tensorflow_model_analysis.api.types', safe = False

    def _import_module(import_name, safe=False):
        try:
            if '.' in import_name:
                items = import_name.split('.')
                module = '.'.join(items[:-1])
                obj = items[-1]
            else:
                return __import__(import_name)
>           return getattr(__import__(module, None, None, [obj]), obj)
E           AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'

.venv310/lib/python3.10/site-packages/dill/_dill.py:827: AttributeError

The list of tests with this error is

FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testAnalyzeRawData - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testBytesProcessedCountForRecordBatches - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testBytesProcessedCountForSerializedExamples - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysis - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisMultiMicroAggregationNoHistogram - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisMultiMicroAggregationWithHistogram - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisWithDeterministicConfidenceIntervals - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisWithExplicitModelAgnosticPredictions - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisWithKerasModelrubber_stamp - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisWithKerasModeltf_js - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisWithKerasModeltf_keras - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisWithKerasModeltf_keras_custom_metrics - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisWithKerasModeltf_lite - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisWithKerasMultiOutputModel - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisWithQueryBasedMetrics - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisWithSchema - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/api/model_eval_lib_test.py::EvaluateTest::testRunModelAnalysisWithUncertainty - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/evaluators/metrics_plots_and_validations_evaluator_test.py::MetricsPlotsAndValidationsEvaluatorTest::testEvaluateWithAttributions - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/evaluators/metrics_plots_and_validations_evaluator_test.py::MetricsPlotsAndValidationsEvaluatorTest::testEvaluateWithJackknifeAndDiffMetrics - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/evaluators/metrics_plots_and_validations_evaluator_test.py::MetricsPlotsAndValidationsEvaluatorTest::testEvaluateWithKerasAndDiffMetrics - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/evaluators/metrics_plots_and_validations_evaluator_test.py::MetricsPlotsAndValidationsEvaluatorTest::testEvaluateWithKerasModelWithInGraphMetricscompiled_metrics - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/evaluators/metrics_plots_and_validations_evaluator_test.py::MetricsPlotsAndValidationsEvaluatorTest::testEvaluateWithKerasModelWithInGraphMetricsevaluate - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/evaluators/metrics_plots_and_validations_evaluator_test.py::MetricsPlotsAndValidationsEvaluatorTest::testMetricsSpecsCountersInModelAgnosticMode - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/counterfactual_predictions_extractor_test.py::CounterfactualPredictionsExtactorTest::test_cf_predictions_extractor_single_cf - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/counterfactual_predictions_extractor_test.py::CounterfactualPredictionsExtactorTest::test_cf_predictions_extractor_single_non_cf_multiple_cf - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/counterfactual_predictions_extractor_test.py::CounterfactualPredictionsExtactorTest::test_cf_predictions_extractor_single_non_cf_single_cf - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/example_weights_extractor_test.py::ExampleWeightsExtractorTest::testExampleWeightsExtractorMultiModel - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/example_weights_extractor_test.py::ExampleWeightsExtractorTest::testExampleWeightsExtractorMultiOutput - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/example_weights_extractor_test.py::ExampleWeightsExtractorTest::testExampleWeightsExtractorwith_example_weight - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/example_weights_extractor_test.py::ExampleWeightsExtractorTest::testExampleWeightsExtractorwithout_example_weight - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/features_extractor_test.py::FeaturesExtractorTest::test_features_extractor - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/features_extractor_test.py::FeaturesExtractorTest::test_features_extractor_no_features - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/labels_extractor_test.py::LabelsExtractorTest::testLabelsExtractorMultiModel - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/labels_extractor_test.py::LabelsExtractorTest::testLabelsExtractorMultiOutput - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/labels_extractor_test.py::LabelsExtractorTest::testLabelsExtractorwith_label - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/labels_extractor_test.py::LabelsExtractorTest::testLabelsExtractorwithout_label - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/legacy_input_extractor_test.py::InputExtractorTest::testInputExtractor - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/legacy_input_extractor_test.py::InputExtractorTest::testInputExtractorMultiModel - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/legacy_input_extractor_test.py::InputExtractorTest::testInputExtractorMultiOutput - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/materialized_predictions_extractor_test.py::MaterializedPredictionsExtractorTest::test_rekey_predictions_in_features - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/predictions_extractor_test.py::PredictionsExtractorTest::testBatchSizeLimitWithKerasModel - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/predictions_extractor_test.py::PredictionsExtractorTest::testPredictionsExtractorWithKerasModelModelSignaturesDoFnInferenceCallableModel - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/predictions_extractor_test.py::PredictionsExtractorTest::testPredictionsExtractorWithKerasModelModelSignaturesDoFnInferenceServingDefault - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/predictions_extractor_test.py::PredictionsExtractorTest::testPredictionsExtractorWithSequentialKerasModelModelSignaturesDoFnInferenceCallableModel - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/predictions_extractor_test.py::PredictionsExtractorTest::testPredictionsExtractorWithSequentialKerasModelModelSignaturesDoFnInferenceServingDefault - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/predictions_extractor_test.py::PredictionsExtractorTest::testRekeyPredictionsInFeaturesForPrematerializedPredictions - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/sql_slice_key_extractor_test.py::SqlSliceKeyExtractorTest::testSqlSliceKeyExtractor - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/sql_slice_key_extractor_test.py::SqlSliceKeyExtractorTest::testSqlSliceKeyExtractorWithCrossSlices - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/sql_slice_key_extractor_test.py::SqlSliceKeyExtractorTest::testSqlSliceKeyExtractorWithEmptySqlConfig - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/sql_slice_key_extractor_test.py::SqlSliceKeyExtractorTest::testSqlSliceKeyExtractorWithMultipleSchema - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tfjs_predict_extractor_test.py::TFJSPredictExtractorTest::testTFJSPredictExtractorWithKerasModelmulti_model_multi_output_batched_examples_batched_inputs - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tfjs_predict_extractor_test.py::TFJSPredictExtractorTest::testTFJSPredictExtractorWithKerasModelmulti_model_single_output - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tfjs_predict_extractor_test.py::TFJSPredictExtractorTest::testTFJSPredictExtractorWithKerasModelsingle_model_multi_output - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tfjs_predict_extractor_test.py::TFJSPredictExtractorTest::testTFJSPredictExtractorWithKerasModelsingle_model_single_output - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel0 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel1 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel10 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel11 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel12 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel13 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel14 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel15 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel2 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel3 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel4 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel5 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel6 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel7 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel8 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/tflite_predict_extractor_test.py::TFLitePredictExtractorTest::testTFlitePredictExtractorWithKerasModel9 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/transformed_features_extractor_test.py::TransformedFeaturesExtractorTest::testPreprocessedFeaturesExtractorkeras_custom - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/transformed_features_extractor_test.py::TransformedFeaturesExtractorTest::testPreprocessedFeaturesExtractorkeras_defaults - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/transformed_features_extractor_test.py::TransformedFeaturesExtractorTest::testPreprocessedFeaturesExtractortf_custom - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/transformed_features_extractor_test.py::TransformedFeaturesExtractorTest::testPreprocessedFeaturesExtractortf_defaults - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/unbatch_extractor_test.py::UnbatchExtractorTest::testUnbatchExtractor - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/unbatch_extractor_test.py::UnbatchExtractorTest::testUnbatchExtractorMultiModel - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/extractors/unbatch_extractor_test.py::UnbatchExtractorTest::testUnbatchExtractorMultiOutput - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/example_count_test.py::ExampleCountEnd2EndTest::testExampleCountsWithoutLabelPredictions - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_confusion_matrix_metrics_test.py::ObjectDetectionConfusionMatrixMetricsTest::testObjectDetectionMetrics_max_recall - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_confusion_matrix_metrics_test.py::ObjectDetectionConfusionMatrixMetricsTest::testObjectDetectionMetrics_precision - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_confusion_matrix_metrics_test.py::ObjectDetectionConfusionMatrixMetricsTest::testObjectDetectionMetrics_precision_at_recall - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_confusion_matrix_metrics_test.py::ObjectDetectionConfusionMatrixMetricsTest::testObjectDetectionMetrics_recall - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_confusion_matrix_metrics_test.py::ObjectDetectionConfusionMatrixMetricsTest::testObjectDetectionMetrics_threshold_at_recall - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_confusion_matrix_plot_test.py::ObjectDetectionConfusionMatrixPlotTest::testConfusionMatrixPlot - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_metrics_test.py::ObjectDetectionMetricsTest::testMetricValuesWithLargerData_average_precision_ave - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_metrics_test.py::ObjectDetectionMetricsTest::testMetricValuesWithLargerData_average_precision_iou0.5 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_metrics_test.py::ObjectDetectionMetricsTest::testMetricValuesWithLargerData_average_precision_iou0.75 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_metrics_test.py::ObjectDetectionMetricsTest::testMetricValuesWithLargerData_average_recall_arlarge - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_metrics_test.py::ObjectDetectionMetricsTest::testMetricValuesWithLargerData_average_recall_armedium - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_metrics_test.py::ObjectDetectionMetricsTest::testMetricValuesWithLargerData_average_recall_arsmall - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_metrics_test.py::ObjectDetectionMetricsTest::testMetricValuesWithLargerData_average_recall_mdet1 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_metrics_test.py::ObjectDetectionMetricsTest::testMetricValuesWithLargerData_average_recall_mdet10 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_metrics_test.py::ObjectDetectionMetricsTest::testMetricValuesWithLargerData_average_recall_mdet100 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/object_detection_metrics_test.py::ObjectDetectionMetricsTest::testMetricValuesWithSplittedData_average_precision_iou0.5 - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/score_distribution_plot_test.py::ScoreDistributionPlotTest::testScoreDistributionPlot - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/semantic_segmentation_confusion_matrix_metrics_test.py::SegmentationConfusionMatrixTest::testEncodedImage_fp_two_class_with_ignore - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/semantic_segmentation_confusion_matrix_metrics_test.py::SegmentationConfusionMatrixTest::testEncodedImage_tp_two_class_with_ignore - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/semantic_segmentation_confusion_matrix_metrics_test.py::SegmentationConfusionMatrixTest::testEncodedImage_two_class - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/semantic_segmentation_confusion_matrix_metrics_test.py::SegmentationConfusionMatrixTest::testEncodedImage_two_class_with_ignore - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/set_match_confusion_matrix_metrics_test.py::SetMatchConfusionMatrixMetricsTest::testSetMatchMetricsWithClassWeights_precision_with_class_weight - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/set_match_confusion_matrix_metrics_test.py::SetMatchConfusionMatrixMetricsTest::testSetMatchMetricsWithClassWeights_recall_with_class_weight - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/set_match_confusion_matrix_metrics_test.py::SetMatchConfusionMatrixMetricsTest::testSetMatchMetrics_precision - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/set_match_confusion_matrix_metrics_test.py::SetMatchConfusionMatrixMetricsTest::testSetMatchMetrics_precision_top_k - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/set_match_confusion_matrix_metrics_test.py::SetMatchConfusionMatrixMetricsTest::testSetMatchMetrics_recall - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/set_match_confusion_matrix_metrics_test.py::SetMatchConfusionMatrixMetricsTest::testSetMatchMetrics_recall_top_k - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/set_match_confusion_matrix_metrics_test.py::SetMatchConfusionMatrixMetricsTest::testSetMatchMetrics_recall_top_k_with_threshold_set - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/stats_test.py::MeanEnd2EndTest::testMeanEnd2End - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/metrics/stats_test.py::MeanEnd2EndTest::testMeanEnd2EndWithoutExampleWeights - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/utils/example_keras_model_test.py::ExampleModelTest::test_example_keras_model - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/writers/metrics_plots_and_validations_writer_test.py::MetricsPlotsAndValidationsWriterTest::testWriteAttributionsparquet_file_format - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/writers/metrics_plots_and_validations_writer_test.py::MetricsPlotsAndValidationsWriterTest::testWriteAttributionstfrecord_file_format - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/writers/metrics_plots_and_validations_writer_test.py::MetricsPlotsAndValidationsWriterTest::testWriteValidationResultsNoThresholdsparquet_file_format - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/writers/metrics_plots_and_validations_writer_test.py::MetricsPlotsAndValidationsWriterTest::testWriteValidationResultsNoThresholdstfrecord_file_format - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/writers/metrics_plots_and_validations_writer_test.py::MetricsPlotsAndValidationsWriterTest::testWriteValidationResultsparquet_file_format - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
FAILED tensorflow_model_analysis/writers/metrics_plots_and_validations_writer_test.py::MetricsPlotsAndValidationsWriterTest::testWriteValidationResultstfrecord_file_format - AttributeError: module 'tensorflow_model_analysis.api' has no attribute 'types'
@smokestacklightnin smokestacklightnin added the bug Something isn't working label Oct 19, 2024
@Maheshingle984
Copy link

Maheshingle984 commented Apr 2, 2025 •
edited
Loading

Possible Causes & Solutions

1. Check the TFMA Installation

Since you're installing from source, it's possible that your build is incomplete or dependencies are missing. Try reinstalling TFMA properly:

pip uninstall tensorflow_model_analysis
pip install tensorflow_model_analysis

If you want to install from source again, ensure you're following the correct steps:

git clone https://github.com/tensorflow/model-analysis.git
cd model-analysis
pip install -e .

Then, verify the installation:

import tensorflow_model_analysis as tfma
print(tfma.__version__)

2. Check Import Structure in tensorflow_model_analysis.api

Your error message suggests that tensorflow_model_analysis.api.types is missing. Normally, TFMA exposes types through tensorflow_model_analysis.api.types, but if it's not included in the __init__.py, it might not be available.

To confirm:

import tensorflow_model_analysis.api as api
print(dir(api))  # Check if 'types' exists

If types is missing, your fix (adding from tensorflow_model_analysis.api import types to __init__.py) is reasonable, but we need to understand why it’s missing.

3. Confirm if types.py Exists in the Expected Location

Go to the tensorflow_model_analysis/api/ directory and check if types.py is present:

ls tensorflow_model_analysis/api/

If it’s missing, you may need to pull the latest code from the repository:

git pull origin main

Then, rebuild and reinstall.

4. Run Tests in an Isolated Environment

Conflicting installations of TFMA might cause issues. Run tests in a clean virtual environment:

python -m venv tfma_env
source tfma_env/bin/activate  # On Windows: tfma_env\Scripts\activate
pip install -e .
pytest

If the issue persists, try installing a specific stable version of TFMA instead of the latest development version.

5. Investigate PR #183

Since you mentioned PR #183, check its details on GitHub:
🔗 [PR #183 on GitHub](#183)

If this PR introduced breaking changes, consider checking out an earlier version of the repo before PR #183:

git checkout <commit-before-PR-183>

Then, reinstall TFMA and rerun tests.

Final Debugging Questions

  1. Does tensorflow_model_analysis.api.types exist in dir(tfma.api)?
  2. Did PR Use pytest and make GitHub Workflow for tests #183 modify any core API imports?
  3. Is the issue happening in a fresh environment?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@smokestacklightnin @Maheshingle984