-
-
Notifications
You must be signed in to change notification settings - Fork 91
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ExchangeOnlineManagement session not imported in runspace PodeRoute #841
Comments
Hey @crackermania, I had a look at the Exchange module's docs and was hoping there'd be a way to get the session object it creates - as that would have made this really simple! :D sadly, from what I can see at least, it doesn't look like there is. The session state for the route runspaces are different from the parent session, and while Pode does import most things - like modules/snapins/variables - hidden session objects is something it can't. I do have an idea though 😄 this is just for a test, but what happens if you move the Add-PodeMiddleware -Name 'ConnectExchange' -ScriptBlock {
Connect-ExchangeOnline -CertificateThumbPrint "xxxxxxxxxx" -AppID "xxxxxxxxx" -Organization "xxxx.onmicrosoft.com" -ShowBanner:$false
return $true
}
Add-PodeRoute -Method Get -Path '/exo/mailbox/:id' -ScriptBlock {
$id = $WebEvent.Parameters['id']
$user = Get-Mailbox -Email $id
$user = $user|ConvertTo-Json
Write-PodeJsonResponse -Value $user
Disconnect-ExchangeOnline -Confirm:$false
} If my idea is right, then the connect in the middleware should create the session on the runspace, and the route's If it does, then I've got a way to make this work as is, and a potential better way using some new events I've got written down to implement for the next release 😄 |
Hey @Badgerati , Thanks for the responds. I changed to middleware and its working. But the performance still same. I can see every request, console server will reimport the command from the module. So, from my view it's do same things but simplified if im having multiple route. I do have 1 method to get the session, i can see and identified the session thru get-pssession. But Im not really sure how I can utilize this in Pode :
But reutilize the same session, it's 8 seconds compare to connect and disconnect 25++ seconds for each request. Thanks. |
Hey @crackermania, It was just a test to make sure doing the connect in middleware worked :) To speed up your current way, do Import-Module ExchangePowerShell
# ...
Start-PodeServer {
# ...
Add-PodeMiddleware -Name 'ConnectExchange' -ScriptBlock {
if (!(Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" })) {
Connect-ExchangeOnline -CertificateThumbPrint "xxxxxxxxxx" -AppID "xxxxxxxxx" -Organization "xxxx.onmicrosoft.com" -ShowBanner:$false
}
return $true
}
Add-PodeRoute -Method Get -Path '/exo/mailbox/:id' -ScriptBlock {
$id = $WebEvent.Parameters['id']
$user = Get-Mailbox -Email $id
$user = $user|ConvertTo-Json
Write-PodeJsonResponse -Value $user
}
# ...
} Or, if you Import-Module ExchangePowerShell
Connect-ExchangeOnline -CertificateThumbPrint "xxxxxxxxxx" -AppID "xxxxxxxxx" -Organization "xxxx.onmicrosoft.com" -ShowBanner:$false
# ...
Start-PodeServer {
# ...
Set-PodeState -Name 'ConnectSession' -Value (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" })
Add-PodeMiddleware -Name 'ConnectExchange' -ScriptBlock {
if (!(Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" })) {
$session = Get-PodeState -Name 'ConnectSession'
$session | New-PSSession -Name $session.Name
}
return $true
}
Add-PodeRoute -Method Get -Path '/exo/mailbox/:id' -ScriptBlock {
$id = $WebEvent.Parameters['id']
$user = Get-Mailbox -Email $id
$user = $user|ConvertTo-Json
Write-PodeJsonResponse -Value $user
}
# ...
} I've tested that last one using |
Hi @Badgerati , Both test failed. Here is the result : 1) For first test - failed :
2) for second test : It's prompt for username and password for the request. Seems like connect-exchangeonline cannot be seen in start-podeserver based on this test. Session looks like not appear and not exist in start-podeserver. Thanks. |
HI @crackermania, Hmm, that is peculiar on the first 1. If it's seemingly trying to connect again each time, how long do subsequent requests take after the first one? To see if there are any sessions, try doing adding the following around the connect: Get-PSSession | Out-Default
Connect-ExchangeOnline -CertificateThumbPrint "xxxxxxxxxx" -AppID "xxxxxxxxx" -Organization "xxxx.onmicrosoft.com" -ShowBanner:$false
Get-PSSession | Out-Default It should output 1 session after the connect, and then hopefully on subsequent requests 2 sessions each time (1 before and after). For the 2nd one, I guess the Import-Module ExchangePowerShell
Connect-ExchangeOnline -CertificateThumbPrint "xxxxxxxxxx" -AppID "xxxxxxxxx" -Organization "xxxx.onmicrosoft.com" -ShowBanner:$false
$session = (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" })
# ...
Start-PodeServer {
# ...
Set-PodeState -Name 'ConnectSession' -Value $session
Add-PodeRoute -Method Get -Path '/exo/mailbox/:id' -ScriptBlock {
$session = Get-PodeState -Name 'ConnectSession'
$user = Invoke-Command -Session $session -ArgumentList $WebEvent.Parameters['id'] -ScriptBlock {
param($id)
return (Get-Mailbox -Email $id)
}
Write-PodeJsonResponse -Value $user
}
# ...
} |
Hi @Badgerati , For 1st test, here is my code :
I can see only 1 pssession capture and each time it's different session!
Hit new request!
Looks like previous PSSession gone missing. The time for execution for each request are same, means it still slow due to connect will execute - not utilizing old session/connection. For second test , here is my code :
However, the API return status as 500 (internal server error). I made below changes and it's looks like working fine!
Eventhough it's reimport the PSSession, the speed are accepted and finally the same session are utilize. However, using this, Im still have issue when the session broken/disconnect and I'm still checking a ways to refresh the connection or maintain the session . Any comment are welcome if you can see is there any improvement that I can make. Thanks! |
Ooo, OK! I wonder if you could check the state of the session in Pode's state, if it's broken reconnect and set back in state. Something like: Add-PodeMiddleware -Name 'ReconnectCheck' -ScriptBlock {
if ((Get-PodeState -Name 'ConnectSession').State -ine 'Opened') {
Connect-ExchangeOnline -CertificateThumbPrint "xxxxxxxxxxxxxx" -AppID "xxxxxxxxx" -Organization "xxxxxx.onmicrosoft.com" -ShowBanner:$false
$session = (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" -and $_.State -eq "Opened" })
Set-PodeState -Name 'ConnectSession' -Value $session
}
return $true
} If it works, you'll likely have a call go slow while it reconnects, but after that should be fine 🤔 If that state of the session does change like that, you could also do that check in a Timer/Schedule to do the reconnect in the background. Downside is, you could get some requests that fail if the connection is broken before the reconnect occurs. |
Thanks for suggestion. Sorry for late reply.. busy with others thing. I will post the result after few days monitoring. |
Hi @Badgerati , Sorry.. due to I'm on vocation. It's finally working fine. Below is latest code :
However, I'm still stuck when I'm turn on multithread by addding -Thread switch at Start-PodeServer. Seems like the thread it's not thread safe for Exchangeonline connection. From the log, I can see the session Availability will change to Busy and causing new request failed. I'm getting random no valid session if i'm hit a lot of request at 1 time. How can actually im utilizing lock-podeobject for my session? My planning is i will spawn 5 connection to O365 and assign PodeState for each connection and every request will using one of available (not busy session). Or any others idea for me to enable the multi thread for my case? Thanks. |
Hey @crackermania, Really sorry for the late response on this! Got distracted with Pode.Web and forgot 😭 Not sure if you figured it out, but the following should work for Import-Module ExchangeOnlineManagement
$session = (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" -and $_.State -eq "Opened" })
if($session.count -eq 0)
{
Connect-ExchangeOnline -CertificateThumbPrint "xxxx" -AppID "xxxx" -Organization "xx.onmicrosoft.com" -ShowBanner:$false
$session = (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" -and $_.State -eq "Opened" })
}
Start-PodeServer {
Set-PodeState -Name 'ConnectSession' -Value $session[0]
Add-PodeMiddleware -Name 'ReconnectCheck' -ScriptBlock {
Lock-PodeObject -ScriptBlock {
if ((Get-PodeState -Name 'ConnectSession').State -ine 'Opened') {
$session = (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" -and $_.State -eq "Opened" })
if($session.count -eq 0)
{
Connect-ExchangeOnline -CertificateThumbPrint "xxxxx" -AppID "xxxxx" -Organization "xxxxxx.onmicrosoft.com" -ShowBanner:$false
$session = (Get-PSSession | Where-Object { $_.ComputerName -like "outlook.office365*" -and $_.State -eq "Opened" })
}
Set-PodeState -Name 'ConnectSession' -Value $session[0]
}
}
return $true
}
Add-PodeRoute -Method Get -Path '/exo/mailbox/:id' -ScriptBlock {
$session = Get-PodeState -Name 'ConnectSession'
$user = (Lock-PodeObject -Return -ScriptBlock {
Connect-PSSession -Session $session | Out-Null
return (Get-Mailbox $WebEvent.Parameters['id'])
})
Write-PodeJsonResponse -Value $user
}
} ^ that should make each connection threadsafe, on connecting and reconnecting |
Using the above code with a single thread for
... suggesting the session is being imported 5 times (though I could be wrong of course). I'm not sure why but anyway... this has been 'up' for 12 hours now with me periodically issuing get requests to |
@robinmalik 😊 -- was the warning on server start, or from |
@Badgerati The script starts, it connects to Exchange (using a custom wrapper function which amongst many other things, adds (this can be sped up sometimes by specifying the cmdlets you want and is recommended by MS). With some verbose logging added:
Given that the warning is being generated by It's not really an issue, but out of curiosity, how come there are 5 import warnings? With 5 threads, I got 9 warnings 😄 EDIT: Of course, if you connect and do specify your limited cmdlets (e.g. Get-Mailbox, Get-MailboxStatistics) that confirm to the approved verb-noun structure, this goes away so it's not something that needs solving per se :) |
Aaaah, riight, this makes more sense now. It'll be the The 5 warnings will be:
When you bump it with |
Hi @robinmalik , I saw you build something related to ExchangeOnline. Do you manage to make it's support for multi thread? Im still got issue to build multi thread for ExchangeOnline. Perhaps you can share some idea or code if you manage to take care of multi request? From my understanding, session only can be handle 1 request each time, so, the only solution to support this is spawn more connection and handle it properly to ensure thread safe. Thanks! |
@crackermania Hi, sorry for the delayed reply. So far I've only used Pode for some proof of concept tests so I've not considered multithreading or performance really. With my code I was simply starting the Pode server with 5 threads and issuing the occasional query by myself. If you like, you could suggest a way I might try and test for you? (although, I don't think my code will be much different to the above!) |
Hey @crackermania & @robinmalik - Throwing in my experience. Using Pode v2.6.0 (modified with a hotfix from another thread), I currently connect to both on-premise and online Exchange environments. I'm still using basic auth for both connections - so I'm still opening a PSSession to each environment. However, I think my implementation should work even if you use the new v2 Exchange module - with similar results. In essence, I don't load any sessions or modules (other than ActiveDirectory) before staring Pode server. Rather, I use @Badgerati suggestion of using a middleware function to load/test for sessions. If one doesn't exist, or if it's in a broken state, it'll call a routine to tear down all sessions and recreate new ones. Obviously, this is expensive and you take a hit. However, in my experience, it's about 8 secs (sometimes 15...) The key here is that every thread will have it's own set of Exchange sessions. So that hit only comes the first time you establish a connection or it needs to be rebuilt. I run with four threads and I consistently have an initial hit of about 9 secs for an initial hit - 8 for the connection and 1 for the action being performed - for each thread. However, the second time I hit the thread, the connection is already there and my results come back in about a second. Here is an example of my middleware code and a route: Add-PodeMiddleware -Name "ExchangeSessionCheck" -Route "/exo/*" -ScriptBlock {
Get-PSSession | Out-Default
(Get-PSSession | Where-Object { $_.Name -eq "Sess_O365_$ThreadId" }).State | Out-Default
if ( (Get-PSSession | Where-Object { $_.Name -eq "Sess_O365_$ThreadId" }).State -ine "Opened") {
'getting connection' | out-default
Connect-RemoteConnection -EmailEnv 'EXOL' -SessId $ThreadId
get-module | Out-Default
}
else {
'connection opened' | Out-Default
}
return $true
}
Add-PodeRoute -Method Get -Path '/exo/mailbox/:id' -ScriptBlock {
("Inside exo/Mailbox. ThreadId: {0}" -f $ThreadId) | Out-Default
'getting mailbox' | Out-Default
$targetMBx = invoke-command -Session (Get-PSSession | Where-Object { $_.Name -eq "Sess_O365_$ThreadId" }) -scriptblock { Get-Mailbox <alias>}
Get-PSSession | out-default
Write-PodeJsonResponse -Value $targetMBx
} The code above is my testing code and hasn't been optimized. Also, I don't import the PSSession to the local machine. I only establish the PSSession in the Connect-RemoteConnection routine, which does the heavy lifting of establishing the connection, and I use invoke-command to get the information I need. I think the code sample should convey the idea. Also note - I'm not using any locks as each thread should have it's own session. I do have the ability to open a number of concurrent session with both exchange environments. Your mileage may vary based on your tenant's configuration. However, you could also make your own Connect-RemoteConnection routine pull from a pool of ids. Mine actually does some gymnastics in getting a secured credentials, etc. I'm in the process of converting all my existing code to use the new v2 Exchange cmdlets, so I should soon be in a position where I can test the above idea with code similar to what you guys are using. Unless one of you can test it before I can. 😄 |
Thanks for the hint and sorry for delay reply (just have time to check back this thread). Finally able to make it's work using your logic.. Im running Pode 2.6.2 with ExchangeOnlineManagement 2.0.5 and looks like all working fine so far. Below is my code :
The things is I no need to import PSSession or invoke-command to choose the PSSession. The code above is not optimize yet, but so far do the jobs well after use for 2 days. By the way, Im try to cleanup or issued Disconnect-ExchangeOnline, but not sure how to confirm it's clean the session or not.. tried below code but no output in console :
Just to ensure the session is disconnected properly in O365. |
Hi all. Now that ExchangeOnlineManagement has moved to 2.0.6-Preview6, and all the cmdlets are REST-backed (i.e. no longer use PS sessions), it may make life a little easier for us. It looks like I'm just doing some testing without the overhead of setting Pode states, locking objects etc. |
Hey @robinmalik - Did you work out how to do the connections for multiple threads with the latest ExchangeOnlineManagement module? Now that Session based connections have been removed from my tenant, I have to switch to the REST base version. I went from 2.0.6 to 3.2.0. My existing code base only loads the Exchange cmdlets into the first thread. Other threads see no connections. I've tried a few things, but I'm at the point where I may just change the code to open and close a connection per route as opposed to keeping a "connection" open. I have a hybrid environment and so I wanted to keep a connection to both onprem and exo. |
@bodybybuddha |
Ok, I initially had a bunch of code referencing pode state, locking objects etc but I'm not sure I really understand that anyway, so I removed it all and now I can't see anything particularly special about my working approach, so here you go: server.ps1 ########################################################################################
try
{
$ExchangeConnection = [Microsoft.Exchange.Management.ExoPowershellSnapin.ConnectionContextFactory]::GetAllConnectionContexts()
if(!$ExchangeConnection)
{
$ConnectRequired = $True
}
}
catch
{
$ConnectRequired = $True
$Error.Remove($Error)
}
if($ConnectRequired)
{
Write-Verbose -Message "Calling 'Connect-LUExchangeOnline'"
Connect-LUExchangeOnline -Tenancy removed -LimitedCmdlets @('Get-Mailbox', 'Get-MailboxStatistics', 'Get-EXOMailbox')
}
########################################################################################
Write-Verbose "Calling 'Start-PodeServer'"
Start-PodeServer {
Write-Verbose "Calling New-PodeLoggingMethod"
New-PodeLoggingMethod -Terminal | Enable-PodeErrorLogging -Level Error, Debug, Verbose
Write-Verbose "Calling Add-PodeEndpoint"
Add-PodeEndpoint -Address localhost -Port 8099 -Protocol Http
Write-Verbose "Adding PodeRoute for /exo/mailbox/:id"
Add-PodeRoute -Method Get -Path '/exo/mailbox/:id' -ScriptBlock {
("Request to: $($WebEvent.Path) with method: $($WebEvent.Route.Method). ThreadId: {0}" -f $ThreadId) | Out-Default
$user = Get-Mailbox $WebEvent.Parameters['id']
Write-PodeJsonResponse -Value $user
}
} -Threads 3
Testing done with (running as Administrator):
and To test I simply ran: |
@crackermania THANKS!! Looks very clean. My latest version is a variation of the code I posted before. Instead of capturing the EXO PSSession in a global object, I'm creating a variable, "EXOLConnectionId_$ThreadId" = EXOL connection id and checking for that in function that AND checking if it's active in the function that is responsible for connecting to EXOL. In my testing, if I didn't do this, I would be creating additional EXO connections per thread, if the previous connection wasn't active/open. This ended up creating a lot of connections per thread, which ate up a lot of memory, etc. Your code very clean and I'm going to try to refactor my code to be similar. Thanks again!! |
@bodybybuddha i think you are quoting wrong person ? By the way, here is my latest code that I use.. so far it's serve me well without any issue after running almost 3 month
I'm just need to ensure connected session are equal to the Pode Thread no.. if not, i will try this thread if command can use. Im using Pode 2.8.0 and ExchangeOnlineManagement 3.1.0 @robinmalik , thanks for the code snippet. Will explore more on your method. |
@crackermania I haven't run anything with anger so I can't say if it'll hold up. There's no middleware as you have, that'd check to see if a reconnect is needed, but given my example seemed to work across threads maybe connecting before declaring Start-PodeServer is sufficient? I guess @Badgerati might have a better idea! |
@crackermania / @robinmalik LOL sorry about that! and Thank you both! |
@robinmalik Hey, I know that when the ExchangeOnline module used PSSessions the middleware solution was pretty much necessary. If the module's been switch to REST based, and doesn't use PSSessions any more, then it could work similar to You could still use the middleware if needed, in case the connection was dropped for whatever reason while the server was running. |
Hi,
I'm trying to connect to O365 environment via module ExchangeOnlineManagement (https://docs.microsoft.com/en-us/powershell/exchange/exchange-online-powershell-v2?view=exchange-ps). It's working fine if i using :
`Add-PodeRoute -Method Get -Path '/exo/mailbox/:id' -ScriptBlock {
Connect-ExchangeOnline -CertificateThumbPrint "xxxxxxxxxx" -AppID "xxxxxxxxx" -Organization "xxxx.onmicrosoft.com" -ShowBanner:$false
$id = $WebEvent.Parameters['id']
However, this will cause every request to route to make a new connection and slightly slow for each of the request. So, I tried to establish connection (Connect-ExchangeOnline) by place it at the top of the script (before Start-PodeServer). Based on the documentation, this will import the session into each of the runspace. But it's not work. When I request to route, the console server will prompt for credential - seems like it's not have session for ExchangeOnline.
So, perhaps someone can help me how can I solve this? Sorry, newbie here and just play around with Pode around 2 weeks. But honestly I can say it's great software! Any feedback, view or reference are welcome.
Thanks guys.
The text was updated successfully, but these errors were encountered: