Import in Python 3.11 Lambda creates: No module named 'rpds.rpds'

[rupe120]

Describe the bug
I am importing sagemaker in a Python 3.11 Lambda and it produces the error No module named 'rpds.rpds'. The module rpds does appear to be available.

To reproduce
I am using a Lambda Layer with the requirements of:

sagemaker==2.246.0

I have narrowed down my Lambda code to just

import json
import boto3
import logging
import os
import uuid
import time
from datetime import datetime
from botocore.exceptions import ClientError

help('modules')

# SageMaker imports
from sagemaker import get_execution_role, Session
# from sagemaker.workflow.pipeline import Pipeline
# from sagemaker.workflow.steps import ProcessingStep
# from sagemaker.workflow.parameters import ParameterString, ParameterInteger
# from sagemaker.processing import ProcessingInput, ProcessingOutput
# from sagemaker.wrangler.processing import DataWranglerProcessor

DEFAULT_INPUT_DATA_S3_URI = os.environ.get('DEFAULT_INPUT_DATA_S3_URI', '')
DEFAULT_OUTPUT_S3_URI = os.environ.get('DEFAULT_OUTPUT_S3_URI', '')
SAGEMAKER_ROLE_ARN = os.environ.get('SAGEMAKER_ROLE_ARN', '')

if not DEFAULT_INPUT_DATA_S3_URI:
    raise ValueError("DEFAULT_INPUT_DATA_S3_URI is not set")
if not DEFAULT_OUTPUT_S3_URI:
    raise ValueError("DEFAULT_OUTPUT_S3_URI is not set")

# Configure logging
logger = logging.getLogger()
logger.setLevel(logging.INFO)

def lambda_handler(event, context):
    """
    Lambda function to create a SageMaker pipeline using Data Wrangler Processor
    Following patterns from AWS samples: sm-data-wrangler-mlops-workflows
    
    Expected event structure:
    {
        "input_data_s3_uri": "s3://bucket/path/to/dataset",
        "data_wrangler_flow_s3_uri": "s3://bucket/path/to/workflow.flow",
        "output_s3_uri": "s3://bucket/path/to/output/",
        "pipeline_name": "data-processing-pipeline",
        "role_arn": "arn:aws:iam::account:role/SageMakerRole" (optional),
        "instance_type": "ml.m5.4xlarge" (optional),
        "instance_count": 1 (optional),
        "flow_output_name": "node_id.default" (optional),
        "flow_output_content_type": "CSV" (optional)
    }
    """
    logger.info(f"Event: {json.dumps(event)}")

Expected behavior
Be able to import sagemaker without error.

Screenshots or logs
These are the reported available modules from the help('modules') call.

_multiprocess       crypt               pathlib             textwrap
_multiprocessing    csv                 pathos              this
_opcode             ctypes              pdb                 threading
_operator           curses              pickle              time
_osx_support        dataclasses         pickletools         timeit
_pickle             datetime            pip                 tkinter
_posixshmem         dateutil            pipes               token
_posixsubprocess    dbm                 pkg_resources       tokenize
_py_abc             decimal             pkgutil             tomllib
_pydecimal          difflib             platform            tqdm
_pyio               dill                platformdirs        trace
_queue              dis                 plistlib            traceback
_random             distutils           poplib              tracemalloc
_sha1               docker              posix               tty
_sha256             doctest             posixpath           turtle
_sha3               email               pox                 turtledemo
_sha512             encodings           pp                  types
_signal             ensurepip           ppft                typing
_sitebuiltins       enum                pprint              typing_extensions
_socket             errno               profile             typing_inspection
_sqlite3            fastapi             pstats              tzdata
_sre                faulthandler        psutil              unicodedata
_ssl                fcntl               pty                 unittest
_stat               filecmp             pwd                 urllib
_statistics         fileinput           py_compile          urllib3
_string             fnmatch             pyclbr              uu
_strptime           fractions           pydantic            uuid
_struct             ftplib              pydantic_core       uvicorn
_symtable           functools           pydevd_plugins      venv
_sysconfigdata__linux_x86_64-linux-gnu gc                  pydoc               warnings
_testbuffer         genericpath         pydoc_data          wave
_testcapi           getopt              pyexpat             weakref
_testclinic         getpass             pygments            webbrowser
_testimportmultiple gettext             pytz                wsgiref
_testinternalcapi   glob                queue               xdrlib
_testmultiphase     graphene            quopri              xml
_thread             graphlib            random              xmlrpc
_threading_local    graphql             re                  xxlimited
_tokenize           graphql_relay       readline            xxlimited_35
_tracemalloc        grp                 referencing         xxsubtype
_typing             gzip                reprlib             yaml
_warnings           h11                 requests            zipapp
_weakref            hashlib             resource            zipfile
_weakrefset         heapq               rich                zipimport
_xxsubinterpreters  hmac                rlcompleter         zipp
_xxtestfuzz         html                rpds                zlib
_yaml               http                runpy               zoneinfo
_zoneinfo           idlelib             runtime_client      
abc                 idna                s3transfer          
aifc                imaplib             sagemaker           

System information
A description of your system. Please provide:

• SageMaker Python SDK version: sagemaker==2.246.0
• Framework name (eg. PyTorch) or algorithm (eg. KMeans): N/A
• Framework version: N/A
• Python version: 3.11
• CPU or GPU: x86
• Custom Docker image (Y/N): N

Additional context
I was able to create a local virtual environment with Python 3.11 and the same version of the Sagemaker module without receiving this error. I'm not sure what to try next.