Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ExchangeOnlineManagement session not imported in runspace PodeRoute #841

Open
crackermania opened this issue Sep 25, 2021 · 27 comments
Open

Comments

@crackermania
Copy link

Hi,

I'm trying to connect to O365 environment via module ExchangeOnlineManagement (https://docs.microsoft.com/en-us/powershell/exchange/exchange-online-powershell-v2?view=exchange-ps). It's working fine if i using :

  1. Normal powershell
  2. Added inside PodeRoute as per below :

`Add-PodeRoute -Method Get -Path '/exo/mailbox/:id' -ScriptBlock {
Connect-ExchangeOnline -CertificateThumbPrint "xxxxxxxxxx" -AppID "xxxxxxxxx" -Organization "xxxx.onmicrosoft.com" -ShowBanner:$false
$id = $WebEvent.Parameters['id']

	$user = Get-Mailbox -Email $id
	$user = $user|ConvertTo-Json
    Write-PodeJsonResponse -Value $user
	Disconnect-ExchangeOnline -Confirm:$false
}`

However, this will cause every request to route to make a new connection and slightly slow for each of the request. So, I tried to establish connection (Connect-ExchangeOnline) by place it at the top of the script (before Start-PodeServer). Based on the documentation, this will import the session into each of the runspace. But it's not work. When I request to route, the console server will prompt for credential - seems like it's not have session for ExchangeOnline.

So, perhaps someone can help me how can I solve this? Sorry, newbie here and just play around with Pode around 2 weeks. But honestly I can say it's great software! Any feedback, view or reference are welcome.

Thanks guys.

@Badgerati
Copy link
Owner

Hey @crackermania,

I had a look at the Exchange module's docs and was hoping there'd be a way to get the session object it creates - as that would have made this really simple! :D sadly, from what I can see at least, it doesn't look like there is.

The session state for the route runspaces are different from the parent session, and while Pode does import most things - like modules/snapins/variables - hidden session objects is something it can't.

I do have an idea though 😄 this is just for a test, but what happens if you move the Connect-ExchangeOnline call to a middleware? like so:

Add-PodeMiddleware -Name 'ConnectExchange' -ScriptBlock {
    Connect-ExchangeOnline -CertificateThumbPrint "xxxxxxxxxx" -AppID "xxxxxxxxx" -Organization "xxxx.onmicrosoft.com" -ShowBanner:$false
    return $true
}

Add-PodeRoute -Method Get -Path '/exo/mailbox/:id' -ScriptBlock {
    $id = $WebEvent.Parameters['id']
    $user = Get-Mailbox -Email $id
    $user = $user|ConvertTo-Json
    Write-PodeJsonResponse -Value $user
    Disconnect-ExchangeOnline -Confirm:$false
}

If my idea is right, then the connect in the middleware should create the session on the runspace, and the route's Get-Mailbox should (hopefully!) work. 🤞

If it does, then I've got a way to make this work as is, and a potential better way using some new events I've got written down to implement for the next release 😄

@crackermania
Copy link
Author

Hey @Badgerati ,

Thanks for the responds.

I changed to middleware and its working. But the performance still same. I can see every request, console server will reimport the command from the module. So, from my view it's do same things but simplified if im having multiple route.

I do have 1 method to get the session, i can see and identified the session thru get-pssession. But Im not really sure how I can utilize this in Pode :

$CloudSession = Get-PSSession | Where { $_.ComputerName -like "outlook.office365*" Import-PSSession $CloudSession -AllowClobber|Out-Null

But reutilize the same session, it's 8 seconds compare to connect and disconnect 25++ seconds for each request.

Thanks.

@Badgerati
Copy link
Owner

Hey @crackermania,

It was just a test to make sure doing the connect in middleware worked :)

To speed up your current way, do Import-Module ExchangePowerShell (I think that's the module's name?) outside of Start-PodeServer and Pode will preload the module for you in runspaces. Also, remove the Disconnect-ExchangeOnline otherwise you will have to suffer recreating connections on every route call. If the Connect is using a PSSession, then hopefully it should connect once, and then never again making it quicker:

Import-Module ExchangePowerShell

# ...

Start-PodeServer {
    # ...

    Add-PodeMiddleware -Name 'ConnectExchange' -ScriptBlock {
        if (!(Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" })) {
            Connect-ExchangeOnline -CertificateThumbPrint "xxxxxxxxxx" -AppID "xxxxxxxxx" -Organization "xxxx.onmicrosoft.com" -ShowBanner:$false
        }
        return $true
    }

    Add-PodeRoute -Method Get -Path '/exo/mailbox/:id' -ScriptBlock {
        $id = $WebEvent.Parameters['id']
        $user = Get-Mailbox -Email $id
        $user = $user|ConvertTo-Json
        Write-PodeJsonResponse -Value $user
    }

    # ...
}

Or, if you Connect-ExchangeOnline outside of Start-PodeServer and get the PSSession then you might be able to use Pode's shared state and New-PSSession:

Import-Module ExchangePowerShell
Connect-ExchangeOnline -CertificateThumbPrint "xxxxxxxxxx" -AppID "xxxxxxxxx" -Organization "xxxx.onmicrosoft.com" -ShowBanner:$false

# ...

Start-PodeServer {
    # ...

    Set-PodeState -Name 'ConnectSession' -Value (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" })

    Add-PodeMiddleware -Name 'ConnectExchange' -ScriptBlock {
        if (!(Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" })) {
            $session = Get-PodeState -Name 'ConnectSession'
            $session | New-PSSession -Name $session.Name
        }
        return $true
    }

    Add-PodeRoute -Method Get -Path '/exo/mailbox/:id' -ScriptBlock {
        $id = $WebEvent.Parameters['id']
        $user = Get-Mailbox -Email $id
        $user = $user|ConvertTo-Json
        Write-PodeJsonResponse -Value $user
    }

    # ...
}

I've tested that last one using New-PSSession and Invoke-Command and it works, but not sure on Exchange though.

@crackermania
Copy link
Author

Hi @Badgerati ,

Both test failed. Here is the result :

1) For first test - failed :

  • Disconnect not inside route (so no disconnect command sent from any line)

  • Import-module already outside start-podeserver

  • I tried to add write-host to see if it's still connect the connection, and the result, it's still spawn new connection and throw "Not found: connect" for each request:
    if (!(Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" })) { Connect-ExchangeOnline -CertificateThumbPrint "xxxxxxxxxxxx" -AppID "xxxxxxx" -Organization "xxxxxxxxxx.onmicrosoft.com" -ShowBanner:$false Write-Host "Not found : connect" }

  • To confirm if the session available when using normal powershell - in others windows, i confirm can see the session thru Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" } once connected. just not sure why Pode cannot see

2) for second test :

It's prompt for username and password for the request. Seems like connect-exchangeonline cannot be seen in start-podeserver based on this test. Session looks like not appear and not exist in start-podeserver.

Thanks.

@Badgerati
Copy link
Owner

HI @crackermania,

Hmm, that is peculiar on the first 1. If it's seemingly trying to connect again each time, how long do subsequent requests take after the first one?

To see if there are any sessions, try doing adding the following around the connect:

Get-PSSession | Out-Default
Connect-ExchangeOnline -CertificateThumbPrint "xxxxxxxxxx" -AppID "xxxxxxxxx" -Organization "xxxx.onmicrosoft.com" -ShowBanner:$false
Get-PSSession | Out-Default

It should output 1 session after the connect, and then hopefully on subsequent requests 2 sessions each time (1 before and after).

For the 2nd one, I guess the New-PSSession from the existing session drops the credentials. I wonder if doing something like the following would work instead, using Invoke-Command 🤔

Import-Module ExchangePowerShell

Connect-ExchangeOnline -CertificateThumbPrint "xxxxxxxxxx" -AppID "xxxxxxxxx" -Organization "xxxx.onmicrosoft.com" -ShowBanner:$false
$session = (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" })

# ...

Start-PodeServer {
    # ...

    Set-PodeState -Name 'ConnectSession' -Value $session

    Add-PodeRoute -Method Get -Path '/exo/mailbox/:id' -ScriptBlock {
        $session = Get-PodeState -Name 'ConnectSession'

        $user = Invoke-Command -Session $session -ArgumentList $WebEvent.Parameters['id'] -ScriptBlock {
            param($id)
            return (Get-Mailbox -Email $id)
        }

        Write-PodeJsonResponse -Value $user
    }

    # ...
}

@crackermania
Copy link
Author

Hi @Badgerati ,

For 1st test, here is my code :

Import-Module ExchangePowerShell

Start-PodeServer {   

    Add-PodeMiddleware -Name 'ConnectExchange' -ScriptBlock {
        if (!(Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" })) {
            Get-PSSession | Out-Default
	    Connect-ExchangeOnline -CertificateThumbPrint "xxxxxxxxxx" -AppID "xxxxxxxxx" -Organization "xxxx.onmicrosoft.com" -ShowBanner:$false
            Get-PSSession | Out-Default
        }
        return $true
    }

    Add-PodeRoute -Method Get -Path '/exo/mailbox/:id' -ScriptBlock {
        $id = $WebEvent.Parameters['id']
        $user = Get-Mailbox -Email $id
        $user = $user|ConvertTo-Json
        Write-PodeJsonResponse -Value $user
    }


}

I can see only 1 pssession capture and each time it's different session!

 Id Name            ComputerName    ComputerType    State         ConfigurationName     Availability
-- ----            ------------    ------------    -----         -----------------     ------------
11 ExchangeOnli... outlook.offi... RemoteMachine   Opened        Microsoft.Exchange       Available

Hit new request!

Id Name            ComputerName    ComputerType    State         ConfigurationName     Availability
-- ----            ------------    ------------    -----         -----------------     ------------
12 ExchangeOnli... outlook.offi... RemoteMachine   Opened        Microsoft.Exchange       Available

Looks like previous PSSession gone missing. The time for execution for each request are same, means it still slow due to connect will execute - not utilizing old session/connection.

For second test , here is my code :

Import-Module ExchangePowerShell

#I make some changes due to I can see some of the session detect as broken
$session = (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" -and $_.State -eq "Opened" })
		if($session.count -eq 0)
		{
			Write-Host "Session 0. Proceed to connect"
			Connect-ExchangeOnline -CertificateThumbPrint "xxxxxxxxxxxxxx" -AppID "xxxxxxxxx" -Organization "xxxxxx.onmicrosoft.com" -ShowBanner:$false
			$session = (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" -and $_.State -eq "Opened" })
		}
		else
		{
			write-host "Session $($session.count). So, reuse."
		}

Start-PodeServer {   
    # bind ConnectSession to 1 open session if got multiple session connection
    Set-PodeState -Name 'ConnectSession' -Value $session[0]

    Add-PodeRoute -Method Get -Path '/exo/mailbox/:id' -ScriptBlock {

        $session = Get-PodeState -Name 'ConnectSession'
        $user = Invoke-Command -Session $session -ArgumentList $WebEvent.Parameters['id'] -ScriptBlock {
            param($id)
            return (Get-Mailbox -Email $id)
        }

        Write-PodeJsonResponse -Value $user
    }


}

However, the API return status as 500 (internal server error). I made below changes and it's looks like working fine!

Add-PodeRoute -Method Get -Path '/exo/mailbox/:id' -ScriptBlock {
	
		$session = Get-PodeState -Name 'ConnectSession'
                #import session solved the issue
		Import-PSSession -Session $session -AllowClobber|Out-Null
		
        $user = Get-Mailbox $WebEvent.Parameters['id']
        Write-PodeJsonResponse -Value $user
		
		
    }

Eventhough it's reimport the PSSession, the speed are accepted and finally the same session are utilize.

However, using this, Im still have issue when the session broken/disconnect and I'm still checking a ways to refresh the connection or maintain the session . Any comment are welcome if you can see is there any improvement that I can make.

Thanks!

@Badgerati
Copy link
Owner

Ooo, OK!

I wonder if you could check the state of the session in Pode's state, if it's broken reconnect and set back in state. Something like:

Add-PodeMiddleware -Name 'ReconnectCheck' -ScriptBlock {
    if ((Get-PodeState -Name 'ConnectSession').State -ine 'Opened') {
        Connect-ExchangeOnline -CertificateThumbPrint "xxxxxxxxxxxxxx" -AppID "xxxxxxxxx" -Organization "xxxxxx.onmicrosoft.com" -ShowBanner:$false
        $session = (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" -and $_.State -eq "Opened" })
        Set-PodeState -Name 'ConnectSession' -Value $session
    }

    return $true
}

If it works, you'll likely have a call go slow while it reconnects, but after that should be fine 🤔

If that state of the session does change like that, you could also do that check in a Timer/Schedule to do the reconnect in the background. Downside is, you could get some requests that fail if the connection is broken before the reconnect occurs.

@crackermania
Copy link
Author

@Badgerati ,

Thanks for suggestion. Sorry for late reply.. busy with others thing. I will post the result after few days monitoring.

@crackermania
Copy link
Author

Hi @Badgerati ,

Sorry.. due to I'm on vocation. It's finally working fine. Below is latest code :

Import-Module ExchangeOnlineManagement

$session = (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" -and $_.State -eq "Opened" })
if($session.count -eq 0)
{
	Connect-ExchangeOnline -CertificateThumbPrint "xxxx" -AppID "xxxx" -Organization "xx.onmicrosoft.com" -ShowBanner:$false
	$session = (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" -and $_.State -eq "Opened" })
}

Start-PodeServer {

	Set-PodeState -Name 'ConnectSession' -Value $session[0]

	Add-PodeMiddleware -Name 'ReconnectCheck' -ScriptBlock {
		if ((Get-PodeState -Name 'ConnectSession').State -ine 'Opened'){
			$session = (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" -and $_.State -eq "Opened" })
			if($session.count -eq 0)
			{					
				Connect-ExchangeOnline -CertificateThumbPrint "xxxxx" -AppID "xxxxx" -Organization "xxxxxx.onmicrosoft.com" -ShowBanner:$false
				$session = (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" -and $_.State -eq "Opened" })					
			}
			Set-PodeState -Name 'ConnectSession' -Value $session[0]
		}
		return $true
	}
	
	Add-PodeRoute -Method Get -Path '/exo/mailbox/:id' -ScriptBlock {	
		$session = Get-PodeState -Name 'ConnectSession'
		Connect-PSSession -Session $session		
        $user = Get-Mailbox $WebEvent.Parameters['id']
        Write-PodeJsonResponse -Value $user	
		
    }

}

However, I'm still stuck when I'm turn on multithread by addding -Thread switch at Start-PodeServer. Seems like the thread it's not thread safe for Exchangeonline connection. From the log, I can see the session Availability will change to Busy and causing new request failed. I'm getting random no valid session if i'm hit a lot of request at 1 time.

How can actually im utilizing lock-podeobject for my session? My planning is i will spawn 5 connection to O365 and assign PodeState for each connection and every request will using one of available (not busy session). Or any others idea for me to enable the multi thread for my case?

Thanks.

@Badgerati
Copy link
Owner

Hey @crackermania,

Really sorry for the late response on this! Got distracted with Pode.Web and forgot 😭

Not sure if you figured it out, but the following should work for Lock-PodeObject:

Import-Module ExchangeOnlineManagement

$session = (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" -and $_.State -eq "Opened" })
if($session.count -eq 0)
{
    Connect-ExchangeOnline -CertificateThumbPrint "xxxx" -AppID "xxxx" -Organization "xx.onmicrosoft.com" -ShowBanner:$false
    $session = (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" -and $_.State -eq "Opened" })
}

Start-PodeServer {
    Set-PodeState -Name 'ConnectSession' -Value $session[0]

    Add-PodeMiddleware -Name 'ReconnectCheck' -ScriptBlock {
        Lock-PodeObject -ScriptBlock {
            if ((Get-PodeState -Name 'ConnectSession').State -ine 'Opened') {
                $session = (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" -and $_.State -eq "Opened" })
                if($session.count -eq 0)
                {
                    Connect-ExchangeOnline -CertificateThumbPrint "xxxxx" -AppID "xxxxx" -Organization "xxxxxx.onmicrosoft.com" -ShowBanner:$false
                    $session = (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" -and $_.State -eq "Opened" })
                }

                Set-PodeState -Name 'ConnectSession' -Value $session[0]
            }
        }
        return $true
    }

    Add-PodeRoute -Method Get -Path '/exo/mailbox/:id' -ScriptBlock {
        $session = Get-PodeState -Name 'ConnectSession'

        $user = (Lock-PodeObject -Return -ScriptBlock {
            Connect-PSSession -Session $session | Out-Null
            return (Get-Mailbox $WebEvent.Parameters['id'])
        })

        Write-PodeJsonResponse -Value $user
    }
}

^ that should make each connection threadsafe, on connecting and reconnecting

@robinmalik
Copy link

Using the above code with a single thread for Start-PodeServer, with 2 routes, I get what appears to be 5 outputs of:

WARNING: The names of some imported commands from the module 'tmp_lqmvwxvw.nyk' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb.

... suggesting the session is being imported 5 times (though I could be wrong of course). I'm not sure why but anyway... this has been 'up' for 12 hours now with me periodically issuing get requests to /exo/mailbox/:id and it's still ok :) I've not been able to replicate a previous issue where the session had closed/terminated and it didn't reconnect. Maybe when it's used in anger more some issues will surface but for now it seems fine for me at least 😊

@Badgerati
Copy link
Owner

@robinmalik 😊 -- was the warning on server start, or from Connect-PSSession? If the latter, could be something weird it does with the module name being randomised 🤔

@robinmalik
Copy link

robinmalik commented Jan 5, 2022

@Badgerati The script starts, it connects to Exchange (using a custom wrapper function which amongst many other things, adds -ShowProgress:$false -ShowBanner:$false). Non-issue but the progress suppression doesn't work as you see the usual:
image

(this can be sped up sometimes by specifying the cmdlets you want and is recommended by MS).

With some verbose logging added:

VERBOSE: Open session count is zero. Calling first 'Connect-LUExchangeOnline'                                                                                                       VERBOSE: Calling 'Start-PodeServer'
VERBOSE: Calling Set-PodeState to store Exchange session
VERBOSE: Calling New-PodeLoggingMethod
VERBOSE: Calling Add-PodeEndpoint
VERBOSE: Calling Add-PodeMiddleware for checking Exchange Connection
VERBOSE: Adding PodeRoute for /exo/mailbox/:id
VERBOSE: Adding PodeRoute for /exo/mailboxstatistics/:id
VERBOSE: Adding PodeRoute for /exo/session
WARNING: The names of some imported commands from the module 'tmp_2kxdos2v.jix' include unapproved verbs that might make them less discoverable. To find the commands with
unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb.
WARNING: The names of some imported commands from the module 'tmp_2kxdos2v.jix' include unapproved verbs that might make them less discoverable. To find the commands with
unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb.
WARNING: The names of some imported commands from the module 'tmp_2kxdos2v.jix' include unapproved verbs that might make them less discoverable. To find the commands with
unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb.
WARNING: The names of some imported commands from the module 'tmp_2kxdos2v.jix' include unapproved verbs that might make them less discoverable. To find the commands with
unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb.
WARNING: The names of some imported commands from the module 'tmp_2kxdos2v.jix' include unapproved verbs that might make them less discoverable. To find the commands with
unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb.
Listening on the following 1 endpoint(s) [1 thread(s)]:
        - http://localhost:8099/

Given that the warning is being generated by Import-Module (and you can suppress this with the -DisableNameChecking parameter) I thought I'd try adding $PSDefaultParameterValues = @{"Import-Module:DisableNameChecking" = $True } at the top of the script but it didn't work. Interestingly, I came across an open issue that Snover himself raised which talks about nested modules not respecting the preference.

It's not really an issue, but out of curiosity, how come there are 5 import warnings? With 5 threads, I got 9 warnings 😄

EDIT: Of course, if you connect and do specify your limited cmdlets (e.g. Get-Mailbox, Get-MailboxStatistics) that confirm to the approved verb-noun structure, this goes away so it's not something that needs solving per se :)

@Badgerati
Copy link
Owner

Aaaah, riight, this makes more sense now. It'll be the .ImportPSModule call on each runspace's initialsessionstate - and annoyingly there's no way to suppress the warning there either that I've seen! The random module name will come from Pode's auto module importing.

The 5 warnings will be:

  • 2 for web (1 for your thread/runspace, another for a single runspace to dispose the listener when the server terminates)
  • 1 for logging (if any)
  • 1 for timers (if any, there are some created interally for auto-restarting when used)
  • 1 for schedules (if any - there's some created internally for sessions/auto-restarting when used)

When you bump it with -Threads 5, that's +4 to web :)

@crackermania
Copy link
Author

Hi @robinmalik ,

I saw you build something related to ExchangeOnline. Do you manage to make it's support for multi thread? Im still got issue to build multi thread for ExchangeOnline. Perhaps you can share some idea or code if you manage to take care of multi request?

From my understanding, session only can be handle 1 request each time, so, the only solution to support this is spawn more connection and handle it properly to ensure thread safe.

Thanks!

@robinmalik
Copy link

@crackermania Hi, sorry for the delayed reply. So far I've only used Pode for some proof of concept tests so I've not considered multithreading or performance really. With my code I was simply starting the Pode server with 5 threads and issuing the occasional query by myself.

If you like, you could suggest a way I might try and test for you? (although, I don't think my code will be much different to the above!)

@bodybybuddha
Copy link

Hey @crackermania & @robinmalik -

Throwing in my experience. Using Pode v2.6.0 (modified with a hotfix from another thread), I currently connect to both on-premise and online Exchange environments. I'm still using basic auth for both connections - so I'm still opening a PSSession to each environment. However, I think my implementation should work even if you use the new v2 Exchange module - with similar results.

In essence, I don't load any sessions or modules (other than ActiveDirectory) before staring Pode server. Rather, I use @Badgerati suggestion of using a middleware function to load/test for sessions. If one doesn't exist, or if it's in a broken state, it'll call a routine to tear down all sessions and recreate new ones.

Obviously, this is expensive and you take a hit. However, in my experience, it's about 8 secs (sometimes 15...) The key here is that every thread will have it's own set of Exchange sessions. So that hit only comes the first time you establish a connection or it needs to be rebuilt.

I run with four threads and I consistently have an initial hit of about 9 secs for an initial hit - 8 for the connection and 1 for the action being performed - for each thread. However, the second time I hit the thread, the connection is already there and my results come back in about a second.

Here is an example of my middleware code and a route:

Add-PodeMiddleware -Name "ExchangeSessionCheck" -Route "/exo/*" -ScriptBlock {
    
    Get-PSSession | Out-Default
    
    (Get-PSSession | Where-Object { $_.Name -eq "Sess_O365_$ThreadId" }).State | Out-Default

    if ( (Get-PSSession | Where-Object { $_.Name -eq "Sess_O365_$ThreadId" }).State -ine "Opened") {
        'getting connection' | out-default
        Connect-RemoteConnection -EmailEnv 'EXOL' -SessId $ThreadId 
        get-module | Out-Default
    }
    else {
        'connection opened' | Out-Default
    }

    return $true
}

Add-PodeRoute -Method Get -Path '/exo/mailbox/:id' -ScriptBlock {
   ("Inside exo/Mailbox. ThreadId: {0}" -f $ThreadId) | Out-Default

   'getting mailbox' | Out-Default
    $targetMBx = invoke-command -Session (Get-PSSession | Where-Object { $_.Name -eq "Sess_O365_$ThreadId" }) -scriptblock { Get-Mailbox <alias>}
           
    Get-PSSession | out-default
    
    Write-PodeJsonResponse -Value $targetMBx
    
} 

The code above is my testing code and hasn't been optimized. Also, I don't import the PSSession to the local machine. I only establish the PSSession in the Connect-RemoteConnection routine, which does the heavy lifting of establishing the connection, and I use invoke-command to get the information I need. I think the code sample should convey the idea.

Also note - I'm not using any locks as each thread should have it's own session. I do have the ability to open a number of concurrent session with both exchange environments. Your mileage may vary based on your tenant's configuration. However, you could also make your own Connect-RemoteConnection routine pull from a pool of ids. Mine actually does some gymnastics in getting a secured credentials, etc.

I'm in the process of converting all my existing code to use the new v2 Exchange cmdlets, so I should soon be in a position where I can test the above idea with code similar to what you guys are using. Unless one of you can test it before I can. 😄

@crackermania
Copy link
Author

@bodybybuddha ,

Thanks for the hint and sorry for delay reply (just have time to check back this thread). Finally able to make it's work using your logic.. Im running Pode 2.6.2 with ExchangeOnlineManagement 2.0.5 and looks like all working fine so far. Below is my code :

Import-Module ExchangeOnlineManagement

Start-PodeServer -Thread 2{
    Add-PodeEndpoint -Address * -Port 8210 -Protocol Http
    Add-PodeMiddleware -Name 'ReconnectCheck' -ScriptBlock {
        
        $temp = "O365_$ThreadId"        
        $test = Get-PodeState -Name $temp
        if($test)
        {
            (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*"})|Out-Default
            ("$temp Available!!!") | Out-Default 
            if ((Get-PodeState -Name $temp).State -eq 'Opened'){
                ("$temp Opened!!!") | Out-Default
            }
            else
            {
                ("$temp Not Opened!!! Disconnected") | Out-Default
                Disconnect-ExchangeOnline -Confirm:$false

                ("$temp reconnect!!!") | Out-Default
                Connect-ExchangeOnline -CertificateThumbPrint "xxxxxxx" -AppID "xxxxxx" -Organization "xxxxxx.onmicrosoft.com" -ShowBanner:$false
                $session = (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" -and $_.State -eq "Opened" })
                Set-PodeState -Name $temp -Value $session[0]
            }
        }
        else
        {
            ("PodeState : $temp not available. Check session") | Out-Default
            (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*"})|Out-Default
            ("PodeState : $temp not available. Create new session") | Out-Default


            Connect-ExchangeOnline -CertificateThumbPrint "xxxxxx" -AppID "xxxxxx" -Organization "xxxxxx.onmicrosoft.com" -ShowBanner:$false
            $session = (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" -and $_.State -eq "Opened" })
            Set-PodeState -Name $temp -Value $session[0]
        }
		
		
		return $true
	}

    Add-PodeRoute -Method Get -Path '/:id' -ScriptBlock {
        ("Inside exo/Mailbox. ThreadId: {0}" -f $ThreadId) | Out-Default
        $id = $WebEvent.Parameters['id']
        $user = Get-ExoMailbox $id
	$user = $user|ConvertTo-Json
        Write-PodeJsonResponse -Value $user
    }   
}

The things is I no need to import PSSession or invoke-command to choose the PSSession. The code above is not optimize yet, but so far do the jobs well after use for 2 days.

By the way, Im try to cleanup or issued Disconnect-ExchangeOnline, but not sure how to confirm it's clean the session or not.. tried below code but no output in console :

Register-PodeEvent -Type Terminate -Name 'FinalTerminate' -ScriptBlock {
        # inform a portal, write a log, etc
        Disconnect-ExchangeOnline -Confirm:$false
        ("Disconnect!!!!!!!!!!!!!!!!") | Out-Default
    }

Just to ensure the session is disconnected properly in O365.

@robinmalik
Copy link

Hi all.

Now that ExchangeOnlineManagement has moved to 2.0.6-Preview6, and all the cmdlets are REST-backed (i.e. no longer use PS sessions), it may make life a little easier for us.

It looks like [Microsoft.Exchange.Management.ExoPowershellSnapin.ConnectionContextFactory]::GetAllConnectionContexts() could be useful for monitoring the connection state.

I'm just doing some testing without the overhead of setting Pode states, locking objects etc.

@bodybybuddha
Copy link

bodybybuddha commented Aug 4, 2023

Hey @robinmalik - Did you work out how to do the connections for multiple threads with the latest ExchangeOnlineManagement module? Now that Session based connections have been removed from my tenant, I have to switch to the REST base version. I went from 2.0.6 to 3.2.0.

My existing code base only loads the Exchange cmdlets into the first thread. Other threads see no connections. I've tried a few things, but I'm at the point where I may just change the code to open and close a connection per route as opposed to keeping a "connection" open. I have a hybrid environment and so I wanted to keep a connection to both onprem and exo.

@robinmalik
Copy link

robinmalik commented Aug 7, 2023

@bodybybuddha How can we see which thread a particular request is running on? That'd be handy to know if my test works 😅
Nevermind, found that out. I have it working, but let me ensure all superfluous code is removed before I put it here :)

@robinmalik
Copy link

robinmalik commented Aug 7, 2023

Ok, I initially had a bunch of code referencing pode state, locking objects etc but I'm not sure I really understand that anyway, so I removed it all and now I can't see anything particularly special about my working approach, so here you go:

server.ps1

########################################################################################
try
{
    $ExchangeConnection = [Microsoft.Exchange.Management.ExoPowershellSnapin.ConnectionContextFactory]::GetAllConnectionContexts()
    if(!$ExchangeConnection)
    {
        $ConnectRequired = $True
    }
}
catch
{
    $ConnectRequired = $True
    $Error.Remove($Error)
}

if($ConnectRequired)
{
    Write-Verbose -Message "Calling 'Connect-LUExchangeOnline'"
    Connect-LUExchangeOnline -Tenancy removed -LimitedCmdlets @('Get-Mailbox', 'Get-MailboxStatistics', 'Get-EXOMailbox')
}

########################################################################################
Write-Verbose "Calling 'Start-PodeServer'"
Start-PodeServer {

    Write-Verbose "Calling New-PodeLoggingMethod"
    New-PodeLoggingMethod -Terminal | Enable-PodeErrorLogging -Level Error, Debug, Verbose

    Write-Verbose "Calling Add-PodeEndpoint"
    Add-PodeEndpoint -Address localhost -Port 8099 -Protocol Http

    Write-Verbose "Adding PodeRoute for /exo/mailbox/:id"
    Add-PodeRoute -Method Get -Path '/exo/mailbox/:id' -ScriptBlock {
        ("Request to: $($WebEvent.Path) with method: $($WebEvent.Route.Method). ThreadId: {0}" -f $ThreadId) | Out-Default
        $user = Get-Mailbox $WebEvent.Parameters['id']
        Write-PodeJsonResponse -Value $user
    }

} -Threads 3

Connect-LUExchangeOnline is a custom wrapper function around Connect-ExchangeOnline that exists in its own module. It has some internal logic to cater to credentials and tenancies. The connection line internally for this test is merely:
Connect-ExchangeOnline -CommandName $LimitedCmdlets -Credential $ExchangeCredential -ShowProgress:$false -ShowBanner:$false -WarningAction SilentlyContinue -Prefix $Prefix -ErrorAction Stop so as you can see there's no scoping anything to be global as you sometimes needed to do.

Testing done with (running as Administrator):

Name                           Value
----                           -----
PSVersion                      7.3.6
PSEdition                      Core
GitCommitId                    7.3.6
OS                             Microsoft Windows 10.0.22631
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

and ExchangeOnlineManagement v3.2.0.

To test I simply ran: $x = 0; do { iwr http://localhost:8099/exo/mailbox/username; $x++ } until ($x -eq 3) and it showed thread 1, 2 and 3 in the shell for server.ps1, and the mailbox object returned as JSON for the iwr.

@bodybybuddha
Copy link

@crackermania THANKS!! Looks very clean.

My latest version is a variation of the code I posted before. Instead of capturing the EXO PSSession in a global object, I'm creating a variable, "EXOLConnectionId_$ThreadId" = EXOL connection id and checking for that in function that AND checking if it's active in the function that is responsible for connecting to EXOL. In my testing, if I didn't do this, I would be creating additional EXO connections per thread, if the previous connection wasn't active/open. This ended up creating a lot of connections per thread, which ate up a lot of memory, etc.

Your code very clean and I'm going to try to refactor my code to be similar. Thanks again!!

@crackermania
Copy link
Author

@bodybybuddha i think you are quoting wrong person ?

By the way, here is my latest code that I use.. so far it's serve me well without any issue after running almost 3 month

Add-PodeMiddleware -Name 'ReconnectCheck'  -Route "api/exo/*" -ScriptBlock {

	$connectedno = (Get-ConnectionInformation|where {$_.State -eq "Connected"}).count
	if($connectedno -lt 4){
		#need to test if this thread connected or not
		#dumpLog -data "Thread $($ThreadId) : Connected EXO $connectedno .Trying to check if EXOmailbox command have issue."
		try{
			$test = get-user [email protected] -ErrorAction Stop				
		}
		catch{
			#dumpLog -data "Thread $($ThreadId)  : Error when try exomailbox. Proceed create new connection."
			Connect-ExchangeOnline -CertificateThumbPrint "xxxx" -AppID "yyyy" -Organization "zzzzz.onmicrosoft.com" -ShowBanner:$false -CommandName Get-EXOMailbox,Get-EXOMailboxStatistics| Out-Null
		
		}
	}

	else{
		#dumpLog -data "Thread $($ThreadId)  : Connected EXO $connectedno . Session connected full, no action require."
	}




	return $true
}

I'm just need to ensure connected session are equal to the Pode Thread no.. if not, i will try this thread if command can use. Im using Pode 2.8.0 and ExchangeOnlineManagement 3.1.0

@robinmalik , thanks for the code snippet. Will explore more on your method.

@robinmalik
Copy link

@crackermania I haven't run anything with anger so I can't say if it'll hold up. There's no middleware as you have, that'd check to see if a reconnect is needed, but given my example seemed to work across threads maybe connecting before declaring Start-PodeServer is sufficient? I guess @Badgerati might have a better idea!

@bodybybuddha
Copy link

@crackermania / @robinmalik LOL sorry about that! and Thank you both!

@Badgerati
Copy link
Owner

@robinmalik Hey, I know that when the ExchangeOnline module used PSSessions the middleware solution was pretty much necessary. If the module's been switch to REST based, and doesn't use PSSessions any more, then it could work similar to az or docker possibly 🤔 If so, then yeah just calling connect once before Start-PodeServer should suffice 😄

You could still use the middleware if needed, in case the connection was dropped for whatever reason while the server was running.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants