Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add pytest-socket library + disable network calls in tests #5315

Merged
merged 9 commits into from
Dec 2, 2024

Conversation

joeyorlando
Copy link
Contributor

@joeyorlando joeyorlando commented Dec 1, 2024

What this PR does

Inspired by this discussion. tldr; ensures that if any of our tests try making an external network call, they will fail.

Setup an example test:

def test_external_network_call():
    import requests

    response = requests.get('https://www.example.com')
    assert response.status_code == 200

and it worked (failed; example CI test run) as expected:

__________________________ test_external_network_call __________________________
    def test_external_network_call():
        import requests
    
>       response = requests.get('https://www.example.com')
requests   = <module 'requests' from '/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/requests/__init__.py'>
apps/test_joey.py:4: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/requests/api.py:73: in get
    return request("get", url, params=params, **kwargs)
        kwargs     = {}
        params     = None
        url        = 'https://www.example.com'
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/requests/api.py:59: in request
    return session.request(method=method, url=url, **kwargs)
        kwargs     = {'params': None}
        method     = 'get'
        session    = <requests.sessions.Session object at 0x7f10ebaada90>
        url        = 'https://www.example.com'
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/requests/sessions.py:589: in request
    resp = self.send(prep, **send_kwargs)
        allow_redirects = True
        auth       = None
        cert       = None
        cookies    = None
        data       = None
        files      = None
        headers    = None
        hooks      = None
        json       = None
        method     = 'get'
        params     = None
        prep       = <PreparedRequest [GET]>
        proxies    = {}
        req        = <Request [GET]>
        self       = <requests.sessions.Session object at 0x7f10ebaada90>
        send_kwargs = {'allow_redirects': True, 'cert': None, 'proxies': OrderedDict(), 'stream': False, ...}
        settings   = {'cert': None, 'proxies': OrderedDict(), 'stream': False, 'verify': True}
        stream     = None
        timeout    = None
        url        = 'https://www.example.com'
        verify     = None
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/requests/sessions.py:703: in send
    r = adapter.send(request, **kwargs)
        adapter    = <requests.adapters.HTTPAdapter object at 0x7f10ebaada30>
        allow_redirects = True
        hooks      = {'response': []}
        kwargs     = {'cert': None, 'proxies': OrderedDict(), 'stream': False, 'timeout': None, ...}
        request    = <PreparedRequest [GET]>
        self       = <requests.sessions.Session object at 0x7f10ebaada90>
        start      = 1733064371.649901
        stream     = False
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/requests/adapters.py:667: in send
    resp = conn.urlopen(
        cert       = None
        chunked    = False
        conn       = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f10ebaadd30>
        proxies    = OrderedDict()
        request    = <PreparedRequest [GET]>
        self       = <requests.adapters.HTTPAdapter object at 0x7f10ebaada30>
        stream     = False
        timeout    = Timeout(connect=None, read=None, total=None)
        url        = '/'
        verify     = True
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/urllib3/connectionpool.py:715: in urlopen
    httplib_response = self._make_request(
        assert_same_host = False
        body       = None
        body_pos   = None
        chunked    = False
        clean_exit = False
        conn       = None
        destination_scheme = None
        err        = None
        headers    = {'User-Agent': 'python-requests/2.32.3', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive'}
        http_tunnel_required = False
        is_new_proxy_conn = False
        method     = 'GET'
        parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/', query=None, fragment=None)
        pool_timeout = None
        redirect   = False
        release_conn = False
        release_this_conn = True
        response_kw = {'decode_content': False, 'preload_content': False}
        retries    = Retry(total=0, connect=None, read=False, redirect=None, status=None)
        self       = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f10ebaadd30>
        timeout    = Timeout(connect=None, read=None, total=None)
        timeout_obj = Timeout(connect=None, read=None, total=None)
        url        = '/'
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/urllib3/connectionpool.py:404: in _make_request
    self._validate_conn(conn)
        chunked    = False
        conn       = <urllib3.connection.HTTPSConnection object at 0x7f10ebaadd60>
        httplib_request_kw = {'body': None, 'headers': {'User-Agent': 'python-requests/2.32.3', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive'}}
        method     = 'GET'
        self       = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f10ebaadd30>
        timeout    = Timeout(connect=None, read=None, total=None)
        timeout_obj = Timeout(connect=None, read=None, total=None)
        url        = '/'
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/urllib3/connectionpool.py:1060: in _validate_conn
    conn.connect()
        __class__  = <class 'urllib3.connectionpool.HTTPSConnectionPool'>
        conn       = <urllib3.connection.HTTPSConnection object at 0x7f10ebaadd60>
        self       = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f10ebaadd30>
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/urllib3/connection.py:363: in connect
    self.sock = conn = self._new_conn()
        self       = <urllib3.connection.HTTPSConnection object at 0x7f10ebaadd60>
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/urllib3/connection.py:174: in _new_conn
    conn = connection.create_connection(
        extra_kw   = {'socket_options': [(6, 1, 1)]}
        self       = <urllib3.connection.HTTPSConnection object at 0x7f10ebaadd60>
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/urllib3/util/connection.py:85: in create_connection
    sock.connect(sa)
        address    = ('www.example.com', 443)
        af         = <AddressFamily.AF_INET: 2>
        canonname  = ''
        err        = None
        family     = <AddressFamily.AF_UNSPEC: 0>
        host       = 'www.example.com'
        port       = 443
        proto      = 6
        res        = (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('93.184.215.14', 443))
        sa         = ('93.184.215.14', 443)
        sock       = <socket.socket fd=12, family=2, type=1, proto=6, laddr=('0.0.0.0', 0)>
        socket_options = [(6, 1, 1)]
        socktype   = <SocketKind.SOCK_STREAM: 1>
        source_address = None
        timeout    = None
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
inst = <socket.socket fd=12, family=2, type=1, proto=6, laddr=('0.0.0.0', 0)>
args = (('93.184.215.14', 443),), host = '93.184.215.14'
    def guarded_connect(inst, *args):
        host = host_from_connect_args(args)
        if host in allowed_ip_hosts_and_hostnames or (
            _is_unix_socket(inst.family) and allow_unix_socket
        ):
            return _true_connect(inst, *args)
    
>       raise SocketConnectBlockedError(allowed_list, host)
E       pytest_socket.SocketConnectBlockedError: A test tried to use socket.socket.connect() with host "93.184.215.14" (allowed: "calendar.google.com (142.251.167.100,142.251.167.101,142.251.167.102,142.251.167.113,142.251.167.138,142.251.167.139,2607:f8b0:4004:c09::65,2607:f8b0:4004:c09::66,2607:f8b0:4004:c09::71,2607:f8b0:4004:c09::8b),localhost (127.0.0.1,::1),oncall-dev-mariadb ()").
allow_unix_socket = False
allowed_ip_hosts_and_hostnames = {'127.0.0.1', '142.251.167.100', '142.251.167.101', '142.251.167.102', '142.251.167.113', '142.251.167.138', ...}
allowed_list = ['calendar.google.com (142.251.167.100,142.251.167.101,142.251.167.102,142.251.167.113,142.251.167.138,142.251.167.139...8b0:4004:c09::66,2607:f8b0:4004:c09::71,2607:f8b0:4004:c09::8b)', 'localhost (127.0.0.1,::1)', 'oncall-dev-mariadb ()']
args       = (('93.184.215.14', 443),)
host       = '93.184.215.14'
inst       = <socket.socket fd=12, family=2, type=1, proto=6, laddr=('0.0.0.0', 0)>
/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/pytest_socket.py:252: SocketConnectBlockedError

Checklist

  • Unit, integration, and e2e (if applicable) tests updated
  • Documentation added (or pr:no public docs PR label added if not required)
  • Added the relevant release notes label (see labels prefixed w/ release:). These labels dictate how your PR will
    show up in the autogenerated release notes.

@joeyorlando joeyorlando added pr:no public docs Added to a PR that does not require public documentation updates release:ignore PR will not be added to release notes labels Dec 1, 2024
@joeyorlando joeyorlando requested a review from a team as a code owner December 1, 2024 13:49
Copy link
Contributor

@matiasb matiasb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! +1

engine/tox.ini Outdated Show resolved Hide resolved
@joeyorlando joeyorlando merged commit fa071bc into dev Dec 2, 2024
26 checks passed
@joeyorlando joeyorlando deleted the jorlando/disable-network-calls-in-tests branch December 2, 2024 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr:no public docs Added to a PR that does not require public documentation updates release:ignore PR will not be added to release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants