[#499] authenticate native and pam_password schemes using iRODS 4.3+ auth flow

Author: d-w-moore

README.md

@@ -129,51 +129,90 @@ the `encryption_*` and `ssl_*` options
 directly to the constructor as keyword arguments, even though it is
 required when they are placed in the environment file.
 
-Creating PAM or Native Credentials File (.irodsA)
--------------------------------------------------
+Creating a PAM or Native Authentication File
+--------------------------------------------
 
-Two free functions exist for creating encoded authentication files:
+The following free functions may be used to create the authentication secrets files (called
+`.irodsA` per the convention of iRODS's iCommands):
+   - `irods.client_init.write_native_irodsA_file`
+   - `irods.client_init.write_pam_irodsA_file`
+
+These functions can roughly be described as duplicating the "authentication" functionality of `iinit`,
+provided that a valid `irods_environment.json` has already been created.
+
+Each of the above functions can take a cleartext password and write an appropriately encoded
+version of it into an authentication file in the appropriate location.  That location is
+`~/.irods/.irodsA` unless the environment variable IRODS_AUTHENTICATION_FILE has been set
+in the command shell to dictate an alternative file path.
+
+As an example, here we write a native `.irodsA` file using the first of the two functions.  We
+provide the one required argument, a password string which is entered interactively at the
+terminal.
+
+```bash
+$ echo '{ "irods_user_name":"rods",
+          ... # other parameters as needed
+        }'> ~/.irods/irods_environment.json
+$ python -c "import irods.client_init, getpass
+irods.client_init.write_native_irodsA_file(getpass.getpass('Enter iRODS password -> '))"
 ```
-irods.client_init.write_native_credentials_to_secrets_file
-irods.client_init.write_pam_credentials_to_secrets_file
+
+By default, when an `.irodsA` file already exists, it will be overwritten. If however the
+`overwrite` parameter is set to `False`, an exception of type `irods.client_init.irodsA_already_exists`
+is raised to warn of any older `.irodsA` file that might otherwise have been overwritten.
+
+Equivalently to the above, we can issue the following command.
+
+```bash
+$ prc_write_irodsA.py native <<<"${MY_CURRENT_IRODS_PASSWORD}"
 ```
 
-Each takes a cleartext password and writes an appropriately processed version of it
-into an .irodsA (secrets) file in the login environment.
+The redirect may of course be left off, in which case the user is prompted for the iRODS password
+and echo of the keyboard input will be suppressed, in the style of `iinit`.  Regardless of
+which technique is used, no password will be visible on the terminal during or after input.
+
+For the `pam_password` scheme, typically SSL/TLS must first be enabled to avoid sending data related
+to the password - or even sending the raw password itself - over a network connection in the clear.
 
-Examples:
-For the `native` authentication scheme, we can use the currently set iRODS password to create the .irodsA file directly:
+Thus, for `pam_password` authentication to work well, we should first ensure, when setting up the
+client environment, to include within `irods_environment.json` the appropriate SSL/TLS connection
+parameters.  In a pinch, `iinit` can be used to verify this prerequisite is fulfilled,
+as its invocation would then create a valid `.irodsA` from merely prompting the user for their PAM password.
+
+Once again, this can also be done using the free function directly:
 
 ```python
-import irods.client_init as iinit
-iinit.write_native_credentials_to_secrets_file(irods_password)
+irods.client_init.write_pam_irodsA_file(getpass.getpass('Enter current PAM password -> '))
 ```
 
-Note, in the `pam_password` case, this involves sending the cleartext password
-to the server (SSL must be enabled!) and then writing the scrambled token that
-is returned from the transaction.
+or from the Bash command shell:
 
-If an .irodsA file exists already, it will be overwritten by default; however, if these functions'
-overwrite parameter is set to `False`, an exception of type `irods.client_init.irodsA_already_exists`
-will be raised to indicate the older .irodsA file is present.
+```bash
+$ prc_write_irodsA.py pam_password <<<"${MY_CURRENT_PAM_PASSWORD}"
+```
 
-For the `pam_password` authentication scheme, we must first ensure an `irods_environment.json` file exists in the 
-client environment (necessary for establishing SSL/TLS connection parameters as well as obtaining a PAM token from the server after connecting)
-and then make the call to write .irodsA using the Bash commands:
+As a final note, in the `pam_password` scheme, the default SSL requirement can be disabled.
+**Warning:** Disabling the SSL requirement may cause user passwords to be sent over the network
+in the clear. This should only be done for purposes of testing. Here's how to do it:
 
-```bash
-$ cat > ~/.irods/irods_environment.json << EOF
-{
-  "irods_user_name":"rods",
-  "irods_host":"server-hostname",
-  ...  [all other connection settings, including SSL parameters, needed for communication with iRODS] ...
-}
-EOF
-$ python -c "import irods.client_init as iinit; iinit.write_pam_credentials_to_secrets_file(pam_cleartext_password)"
+```python
+from irods.auth.pam_password import ENSURE_SSL_IS_ACTIVE
+
+session = irods.session.iRODSSession(host = "localhost", port = 1247,
+                                     user = "alice", password = "test123", zone="tempZone",
+                                     authentication_scheme = "pam_password")
+
+session.set_auth_option_for_scheme('pam_password', ENSURE_SSL_IS_ACTIVE, False)
+
+# Do something with the session:
+home = session.collections.get('/tempZone/home/alice')
 ```
 
-PAM logins
-----------
+Note, however, in future releases of iRODS it is possible that extra SSL checking could be
+implemented server-side, at which point the above code could not be guaranteed to work.
+
+Legacy (iRODS 4.2-compatible) PAM authentication
+------------------------------------------------
 
 Since v2.0.0, the Python iRODS Client is able to authenticate via PAM using the same file-based client environment as the
 iCommands.

irods/account.py

@@ -28,6 +28,11 @@ def __init__(
                 irods_host = v
 
         self.env_file = env_file
+
+        # The '_auth_file' attribute will be written in the call to iRODSSession.configure,
+        # if an .irodsA file from the client environment is used to load password information.
+        self._auth_file = ""
+
         tuplify = lambda _: _ if isinstance(_, (list, tuple)) else (_,)
         schemes = [_.lower() for _ in tuplify(irods_authentication_scheme)]
 

irods/api_number.py

@@ -179,4 +179,5 @@
     "REPLICA_CLOSE_APN": 20004,
     "TOUCH_APN": 20007,
     "AUTH_PLUG_REQ_AN": 1201,
+    "AUTHENTICATION_APN": 110000,
 }

irods/auth/__init__.py

@@ -1,8 +1,115 @@
+import importlib
+import logging
+import weakref
+from irods.api_number import api_number
+from irods.message import iRODSMessage, JSON_Message
+import irods.password_obfuscation as obf
+import irods.session
+
+
 __all__ = ["pam_password", "native"]
 
+
 AUTH_PLUGIN_PACKAGE = "irods.auth"
 
-import importlib
+
+# Python3 does not have types.NoneType
+_NoneType = type(None)
+
+
+class AuthStorage:
+    """A class that facilitates flexible means password storage.
+
+    Using an instance of this class, passwords may either be one of the following:
+
+        - directly placed in a member attribute (pw), or
+
+        - they may be written to / read from a specified file path in encoded
+          form, usually in an .irodsA file intended for iRODS client authentication.
+
+    Most typical of this class's utility is the transfer of password information from
+    the pam_password to the native authentication flow.  In this usage, whether the
+    password is stored in RAM or in the filesystem depends on whether it was read
+    originally as a function parameter or from an authentication file, respectively,
+    when the session was created.
+    """
+
+    @staticmethod
+    def get_env_password(filename=None):
+        options = dict(irods_authentication_file=filename) if filename else {}
+        return irods.session.iRODSSession.get_irods_password(**options)
+
+    @staticmethod
+    def get_env_password_file():
+        return irods.session.iRODSSession.get_irods_password_file()
+
+    @staticmethod
+    def set_env_password(unencoded_pw, filename=None):
+        if filename is None:
+            filename = AuthStorage.get_env_password_file()
+        from ..client_init import _open_file_for_protected_contents
+
+        with _open_file_for_protected_contents(filename, "w") as irodsA:
+            irodsA.write(obf.encode(unencoded_pw))
+        return filename
+
+    @staticmethod
+    def get_temp_pw_storage(conn):
+        """Fetch the AuthStorage instance associated with this connection object."""
+        return getattr(conn, "auth_storage", lambda: None)()
+
+    @staticmethod
+    def create_temp_pw_storage(conn):
+        """Creates an AuthStorage instance to be cached and associated with this connection object.
+
+        Called multiple times for the same connection, it will return the cached instance.
+
+        The value returned by this call should be stored by the caller into an appropriately scoped
+        variable to ensure the AuthStorage instance endures for the desired lifetime -- that is,
+        for however long we wish to keep the password information around.  This is because the
+        connection object only maintains a weak reference to said instance.
+        """
+
+        # resolve the weakly referenced AuthStorage obj for the connection if there is one.
+        weakref_to_store = getattr(conn, "auth_storage", None)
+        store = weakref_to_store and weakref_to_store()
+
+        # In absence of a persistent AuthStorage object, create one.
+        if store is None:
+            store = AuthStorage(conn)
+            # So that the connection object doesn't hold on to password data too long:
+            conn.auth_storage = weakref.ref(store)
+        return store
+
+    def __init__(self, conn):
+        self.conn = conn
+        self.pw = ""
+        self._auth_file = ""
+
+    @property
+    def auth_file(self):
+        if self._auth_file is None:
+            return ""
+        return self._auth_file or self.conn.account.derived_auth_file
+
+    def use_client_auth_file(self, auth_file):
+        """Set to None to completely suppress use of an .irodsA auth file."""
+        if isinstance(auth_file, (str, _NoneType)):
+            self._auth_file = auth_file
+        else:
+            msg = f"Invalid object in {self.__class__}._auth_file"
+            raise RuntimeError(msg)
+
+    def store_pw(self, pw):
+        if self.auth_file:
+            self.set_env_password(pw, filename=self.auth_file)
+        else:
+            self.pw = pw
+
+    def retrieve_pw(self):
+        if self.auth_file:
+            return self.get_env_password(filename=self.auth_file)
+        return self.pw
 
 
 def load_plugins(subset=set(), _reload=False):
@@ -18,9 +125,81 @@ def load_plugins(subset=set(), _reload=False):
     return dir_
 
 
-# TODO(#499): X models a class which we could define here as a base for various server or client state machines
-#             as appropriate for the various authentication types.
+class REQUEST_IS_MISSING_KEY(Exception):
+    pass
 
 
-class X:
+class ClientAuthError(Exception):
     pass
+
+
+def throw_if_request_message_is_missing_key(request, required_keys):
+    for key in required_keys:
+        if not key in request:
+            raise REQUEST_IS_MISSING_KEY(f"key = {key}")
+
+
+def _auth_api_request(conn, data):
+    message_body = JSON_Message(data, conn.server_version)
+    message = iRODSMessage(
+        "RODS_API_REQ", msg=message_body, int_info=api_number["AUTHENTICATION_APN"]
+    )
+    conn.send(message)
+    response = conn.recv()
+    return response.get_json_encoded_struct()
+
+
+__FLOW_COMPLETE__ = "authentication_flow_complete"
+__NEXT_OPERATION__ = "next_operation"
+
+
+CLIENT_GET_REQUEST_RESULT = "client_get_request_result"
+FORCE_PASSWORD_PROMPT = "force_password_prompt"
+STORE_PASSWORD_IN_MEMORY = "store_password_in_memory"
+
+
+class authentication_base:
+
+    def __init__(self, connection, scheme):
+        self.conn = connection
+        self.loggedIn = 0
+        self.scheme = scheme
+
+    def call(self, next_operation, request):
+        logging.info("next operation = %r", next_operation)
+        old_func = func = next_operation
+        # One level of indirection should be sufficient to get a callable method.
+        if not callable(func):
+            old_func, func = (func, getattr(self, func, None))
+        func = func or old_func
+        if not callable(func):
+            raise RuntimeError("client request contains no callable 'next_operation'")
+        resp = func(request)
+        logging.info("resp = %r", resp)
+        return resp
+
+    def authenticate_client(
+        self, next_operation="auth_client_start", initial_request=()
+    ):
+        if not isinstance(initial_request, dict):
+            initial_request = dict(initial_request)
+
+        to_send = initial_request.copy()
+        to_send["scheme"] = self.scheme
+
+        while True:
+            resp = self.call(next_operation, to_send)
+            if self.loggedIn:
+                break
+            next_operation = resp.get(__NEXT_OPERATION__)
+            if next_operation is None:
+                raise ClientAuthError(
+                    "next_operation key missing; cannot determine next operation"
+                )
+            if next_operation in (__FLOW_COMPLETE__, ""):
+                raise ClientAuthError(
+                    f"authentication flow stopped without success: scheme = {self.scheme}"
+                )
+            to_send = resp
+
+        logging.info("fully authenticated")