Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Various problems on Windows #35

Open
TTimo opened this issue Jan 13, 2019 · 16 comments
Open

Various problems on Windows #35

TTimo opened this issue Jan 13, 2019 · 16 comments

Comments

@TTimo
Copy link

TTimo commented Jan 13, 2019

I am not sure if Windows is supported, and what functionality should be expected (all/same as on *nix?).

Here are various problems I am seeing on Windows 10:

  • Python 2.7.13:
ttimo_000@VANGUARD /c/d/R/Thespian> /cygdrive/c/Python27/python.exe setup.py test
running test
running egg_info
writing thespian.egg-info\PKG-INFO
writing top-level names to thespian.egg-info\top_level.txt
writing dependency_links to thespian.egg-info\dependency_links.txt
reading manifest file 'thespian.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.txt' under directory 'contrib'
warning: no files found matching '*.txt' under directory 'examples'
writing manifest file 'thespian.egg-info\SOURCES.txt'
running build_ext
thespian.test.test_load (unittest.loader.ModuleImportFailure) ... ERROR
thespian.test.test_runcommand (unittest.loader.ModuleImportFailure) ... ERROR

======================================================================
ERROR: thespian.test.test_load (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
ImportError: Failed to import test module: thespian.test.test_load
Traceback (most recent call last):
  File "C:\Python27\lib\unittest\loader.py", line 254, in _find_tests
    module = self._get_module_from_name(name)
  File "C:\Python27\lib\unittest\loader.py", line 232, in _get_module_from_name
    __import__(name)
  File "d:\roomored\thespian\thespian\test\test_load.py", line 141, in <module>
    'stream': open('/dev/null','w'),
IOError: [Errno 2] No such file or directory: '/dev/null'


======================================================================
ERROR: thespian.test.test_runcommand (unittest.loader.ModuleImportFailure)
----------------------------------------------------------------------
ImportError: Failed to import test module: thespian.test.test_runcommand
Traceback (most recent call last):
  File "C:\Python27\lib\unittest\loader.py", line 254, in _find_tests
    module = self._get_module_from_name(name)
  File "C:\Python27\lib\unittest\loader.py", line 232, in _get_module_from_name
    __import__(name)
  File "d:\roomored\thespian\thespian\test\test_runcommand.py", line 5, in <module>
    import thespian.runcommand
  File "d:\roomored\thespian\thespian\runcommand.py", line 18, in <module>
    import fcntl,os
ImportError: No module named fcntl


----------------------------------------------------------------------
Ran 2 tests in 0.000s

FAILED (errors=2)
Test failed: <unittest.runner.TextTestResult run=2 errors=2 failures=0>
error: Test failed: <unittest.runner.TextTestResult run=2 errors=2 failures=0>
  • Python 3.6:

Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 17:00:18) [MSC v.1900 64 bit (AMD64)]

ttimo_000@VANGUARD /c/d/R/Thespian> /cygdrive/c/Python36/python.exe setup.py test
running test
running egg_info
creating thespian.egg-info
writing thespian.egg-info\PKG-INFO
writing dependency_links to thespian.egg-info\dependency_links.txt
writing top-level names to thespian.egg-info\top_level.txt
writing manifest file 'thespian.egg-info\SOURCES.txt'
reading manifest file 'thespian.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.txt' under directory 'contrib'
warning: no files found matching '*.txt' under directory 'examples'
writing manifest file 'thespian.egg-info\SOURCES.txt'
running build_ext
Traceback (most recent call last):
  File "setup.py", line 54, in <module>
    'fault tolerant']
  File "C:\Python36\lib\site-packages\setuptools\__init__.py", line 140, in setup
    return distutils.core.setup(**attrs)
  File "C:\Python36\lib\distutils\core.py", line 148, in setup
    dist.run_commands()
  File "C:\Python36\lib\distutils\dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "C:\Python36\lib\distutils\dist.py", line 974, in run_command
    cmd_obj.run()
  File "C:\Python36\lib\site-packages\setuptools\command\test.py", line 228, in run
    self.run_tests()
  File "C:\Python36\lib\site-packages\setuptools\command\test.py", line 250, in run_tests
    exit=False,
  File "C:\Python36\lib\unittest\main.py", line 94, in __init__
    self.parseArgs(argv)
  File "C:\Python36\lib\unittest\main.py", line 118, in parseArgs
    self._do_discovery(argv[2:])
  File "C:\Python36\lib\unittest\main.py", line 229, in _do_discovery
    self.test = loader.discover(self.start, self.pattern, self.top)
  File "C:\Python36\lib\unittest\loader.py", line 341, in discover
    tests = list(self._find_tests(start_dir, pattern))
  File "C:\Python36\lib\unittest\loader.py", line 398, in _find_tests
    full_path, pattern, namespace)
  File "C:\Python36\lib\unittest\loader.py", line 475, in _find_test_path
    tests = self.loadTestsFromModule(package, pattern=pattern)
  File "C:\Python36\lib\site-packages\setuptools\command\test.py", line 54, in loadTestsFromModule
    tests.append(self.loadTestsFromName(submodule))
  File "C:\Python36\lib\unittest\loader.py", line 190, in loadTestsFromName
    return self.loadTestsFromModule(obj)
  File "C:\Python36\lib\site-packages\setuptools\command\test.py", line 54, in loadTestsFromModule
    tests.append(self.loadTestsFromName(submodule))
  File "C:\Python36\lib\unittest\loader.py", line 153, in loadTestsFromName
    module = __import__(module_name)
  File "d:\roomored\thespian\thespian\system\dictconfig.py", line 28, in <module>
    class six:
  File "d:\roomored\thespian\thespian\system\dictconfig.py", line 29, in six
    string_types = basestring  # this module only used in Python2.
NameError: name 'basestring' is not defined
  • Running the multi_system examples:
ttimo_000@VANGUARD /c/d/R/T/e/m/act1> /cygdrive/c/Python36/python.exe start.py multiprocUDPBase
ttimo_000@VANGUARD /c/d/R/T/e/m/act1> echo "Hello, World!" | /cygdrive/c/Python36/python.exe app.py multiprocUDPBase
Traceback (most recent call last):
  File "app.py", line 57, in <module>
    asys = ActorSystem((sys.argv + ['multiprocTCPBase'])[1])
  File "C:\Python36\lib\site-packages\thespian\actors.py", line 638, in __init__
    systemBase, capabilities, logDefs)
  File "C:\Python36\lib\site-packages\thespian\actors.py", line 676, in _startupActorSys
    systemBase = sbc(self, logDefs=logDefs)
  File "C:\Python36\lib\site-packages\thespian\system\multiprocUDPBase.py", line 26, in __init__
    super(ActorSystemBase, self).__init__(system, logDefs)
  File "C:\Python36\lib\site-packages\thespian\system\multiprocCommon.py", line 85, in __init__
    super(multiprocessCommon, self).__init__(system, logDefs)
  File "C:\Python36\lib\site-packages\thespian\system\systemBase.py", line 329, in __init__
    logDefs)
  File "C:\Python36\lib\site-packages\thespian\system\multiprocCommon.py", line 114, in _startAdmin
    'not a valid ActorSystem admin')
thespian.actors.InvalidActorAddress: ActorAddr-(UDP|:1029) is not a valid ActorSystem admin

(same error happens with Python 2.7)

@kquick
Copy link
Owner

kquick commented Jan 14, 2019

Hi @TTimo ,

Thanks for the reports. Windows is supported and generally has the same functionality as the Unix versions, but I don't have a Windows system available to test on, so sometimes Unix-isms creep in. In general, I know that within the last 2 years there was a lot of Windows use, so any issues should be within that timeframe.

Running the tests using Python's unittest is not really supported: you should use pytest (https://docs.pytest.org) to run them, which will ensure the testing environment is setup correctly. Please let me know if the errors you are getting running the tests still occur when running via pytest and if so I'm definitely interested in resolving those issues.

With regards to the failure to run the Act1 example, one of the common problems on Windows is that it defaults to a rather aggressive firewall configuration which can block network activity even on the local machine. You might want to check your network settings to disable this. You can also check the C:\Windows\temp\ directory to see if thespian has left a logfile there; if so you could supply that via gist or email and I can see if it will help diagnose what is going wrong.

@TTimo
Copy link
Author

TTimo commented Jan 14, 2019

Hello @kquick - thanks for the prompt reply.

Maybe let's just start with Python 2.7 and try to get that working then. I tried to use pytest:

ttimo_000@VANGUARD /c/d/R/T/b/l/thespian> /cygdrive/c/Python27/Scripts/pytest.exe
============================= test session starts =============================
platform win32 -- Python 2.7.13, pytest-3.6.0, py-1.5.3, pluggy-0.6.0
rootdir: D:\Roomored\Thespian, inifile:
collected 2189 items / 2 errors

=================================== ERRORS ====================================
____________ ERROR collecting build/lib/thespian/test/test_load.py ____________
test\test_load.py:141: in <module>
    'stream': open('/dev/null','w'),
E   IOError: [Errno 2] No such file or directory: '/dev/null'
_________ ERROR collecting build/lib/thespian/test/test_runcommand.py _________
ImportError while importing test module 'D:\Roomored\Thespian\build\lib\thespian\test\test_runcommand.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
test\test_runcommand.py:5: in <module>
    import thespian.runcommand
runcommand.py:18: in <module>
    import fcntl,os
E   ImportError: No module named fcntl
!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!
=========================== 2 error in 5.26 seconds ===========================

Now back to multi_system, I didn't see any logging to C:\Windows\Temp, I suspect permissions wouldn't allow writing there anyway, so I added local logging: https://gist.github.com/TTimo/e093b3001a17206d41283a427c7a999d

I did the same to app.py and I see two start.log and app.log now, but they are empty.

Using the default multiprocTCPBase seems to work:

ttimo_000@VANGUARD /c/d/R/T/e/m/act1> echo "Hello, World!" | /cygdrive/c/Python27/python.exe app.py
Encoder [density=1.000]  Hello, World!
Base64Encoder [density=0.650]  SGVsbG8sIFdvcmxkIQ==
MorseEncoder [density=0.283]  .... . .-.. .-.. ---      .-- --- .-. .-.. -..
Rot13Encoder [density=1.000]  Hkrru, Wuxrj!

The TCP example writes the following to the log file (not sure if relevant, including for completeness):

WARNING  Actor encoder.Encoder @ ActorAddr-(T|:49696) retryable exception on message <encoder.EncodeThis object at 0x03026770>
Traceback (most recent call last):

  File "d:\roomored\thespian\thespian\system\actorManager.py", line 156, in _handleOneMessage
    actor_result = self.actorInst.receiveMessage(msg, envelope.sender)

  File "d:\roomored\thespian\thespian\actors.py", line 838, in receiveMessage
    r = getattr(klass, methodName)(self, message, sender)

  File "D:\Roomored\Thespian\examples\multi_system\act1\encoder.py", line 32, in receiveMsg_EncodeThis
    self.send(self.analyzer,

AttributeError: 'Encoder' object has no attribute 'analyzer'

ERROR    Actor encoder.Encoder @ ActorAddr-(T|:49696) second exception on message <encoder.EncodeThis object at 0x03026770>
Traceback (most recent call last):

  File "d:\roomored\thespian\thespian\system\actorManager.py", line 171, in _handleOneMessage
    actor_result = self.actorInst.receiveMessage(copy.deepcopy(msg), envelope.sender)

  File "d:\roomored\thespian\thespian\actors.py", line 838, in receiveMessage
    r = getattr(klass, methodName)(self, message, sender)

  File "D:\Roomored\Thespian\examples\multi_system\act1\encoder.py", line 32, in receiveMsg_EncodeThis
    self.send(self.analyzer,

AttributeError: 'Encoder' object has no attribute 'analyzer'

INFO     ActorSystem Logging Shutdown

I turned the firewall off to make sure, but the previous error happens the same for multiprocUDBBase, and there is nothing in the log files:

ttimo_000@VANGUARD /c/d/R/T/e/m/act1> echo "Hello, World!" | /cygdrive/c/Python27/python.exe app.py multiprocUDPBase
Traceback (most recent call last):
  File "app.py", line 74, in <module>
    asys = ActorSystem((sys.argv + ['multiprocTCPBase'])[1], logDefs=logcfg)
  File "d:\roomored\thespian\thespian\actors.py", line 638, in __init__
    systemBase, capabilities, logDefs)
  File "d:\roomored\thespian\thespian\actors.py", line 676, in _startupActorSys
    systemBase = sbc(self, logDefs=logDefs)
  File "d:\roomored\thespian\thespian\system\multiprocUDPBase.py", line 26, in __init__
    super(ActorSystemBase, self).__init__(system, logDefs)
  File "d:\roomored\thespian\thespian\system\multiprocCommon.py", line 85, in __init__
    super(multiprocessCommon, self).__init__(system, logDefs)
  File "d:\roomored\thespian\thespian\system\systemBase.py", line 329, in __init__
    logDefs)
  File "d:\roomored\thespian\thespian\system\multiprocCommon.py", line 114, in _startAdmin
    'not a valid ActorSystem admin')
thespian.actors.InvalidActorAddress: ActorAddr-(UDP|:1029) is not a valid ActorSystem admin
ttimo_000@VANGUARD /c/d/R/T/e/m/act1> ls -1al *.log
-rwxrwxr-x+ 1 ttimo_000 ttimo_000 0 Jan 14 09:23 app.log*
-rwxrwxr-x+ 1 ttimo_000 ttimo_000 0 Jan 14 09:23 start.log*

Maybe something is written to the log but not being flushed to disk, I cannot initiate a clean shutdown due to this error, so I can't obtain it..

@kquick
Copy link
Owner

kquick commented Jan 15, 2019

The test_load.py error is strange because that particular line hasn't changed in 3 years (pretty much since the initial public release), so I don't know why I didn't encounter this bug in Windows previously. That said, the entire noLogging variable is no longer used, so you should be able to delete lines 136-146 in that file (I'll wait to commit that until you confirm there are no other issues in that file).

For the runcommand.py error, that is code that was added since the last Windows test. I believe that can be fixed by changing line 18 from import fcntl, os to just import fcntl and then adding import fcntl at line 360, just before the two uses of it:

    if can_watch:
        import fcntl
        fcntl.fcntl(...)
        fcntl.fcntl(..)

And also modify thespian/system/multiprocTCPBase.py to add an import os in the import section and change the setting of 'Thespian Watch Supported' from True to os.name != "nt". Again, let me know if this helps and if so I'll get them committed and released.

It looks like the multiprocTCPBase is running fine, which is good. The multiprocUDPBase uses port 1029 by default, so there are likely two things going on since you've ruled out the firewall: (1) there is an old Actor system manager process running on port 1900 that is not working and should be killed so that a new one can be started, or (2) there is some other service running on port 1029 that is blocking Thespian from using it.

The easiest way to fix this is to use an alternative port. This is done by providing a dictionary as the optional second argument to the ActorSystem() calls to specify system capabilities, with a key of 'Admin Port' and the value specifying the port number to use. For the multi_system examples, change all of the following calls (in app.py, start.py, and stop.py) from:

    asys = ActorSystem((sys.argv + ['multiprocTCPBase'])[1])

to add the capabilities argument:

    asys = ActorSystem((sys.argv + ['multiprocTCPBase'])[1], { 'Admin Port': 12345})

where you can use any port number > 1024 that you'd like and which isn't already in use in place of 12345.

Thanks for your patience and help in looking into these Windows issues, and let me know if they work so that I can make them permanently.

@TTimo
Copy link
Author

TTimo commented Jan 16, 2019

I sent #36 to fix the tests setup.

I will re-run them to capture the full output, but there were quite a number of failures:
149 failed, 2186 passed, 236 skipped, 3 xfailed, 1 warnings in 8564.81 seconds

Port 1900 is indeed used on Windows - by https://en.wikipedia.org/wiki/Simple_Service_Discovery_Protocol

I made the changes to bind to port 12345, unfortunately I still get the same error for the UDP multiproc.
I verified having a python.exe process listening on UDP 12345 after starting, but app.py or even stop.py still fail:

ttimo_000@VANGUARD /c/d/R/T/e/m/act1> /cygdrive/c/Python27/python.exe start.py multiprocUDPBase
[verified seeing a new python.exe process listening on UDP 12345]
ttimo_000@VANGUARD /c/d/R/T/e/m/act1> /cygdrive/c/Python27/python.exe stop.py multiprocUDPBase
Traceback (most recent call last):
  File "stop.py", line 11, in <module>
    capabilities=cap,
  File "d:\roomored\thespian\thespian\actors.py", line 638, in __init__
    systemBase, capabilities, logDefs)
  File "d:\roomored\thespian\thespian\actors.py", line 676, in _startupActorSys
    systemBase = sbc(self, logDefs=logDefs)
  File "d:\roomored\thespian\thespian\system\multiprocUDPBase.py", line 26, in __init__
    super(ActorSystemBase, self).__init__(system, logDefs)
  File "d:\roomored\thespian\thespian\system\multiprocCommon.py", line 85, in __init__
    super(multiprocessCommon, self).__init__(system, logDefs)
  File "d:\roomored\thespian\thespian\system\systemBase.py", line 329, in __init__
    logDefs)
  File "d:\roomored\thespian\thespian\system\multiprocCommon.py", line 114, in _startAdmin
    'not a valid ActorSystem admin')
thespian.actors.InvalidActorAddress: ActorAddr-(UDP|:12345) is not a valid ActorSystem admin

@kquick
Copy link
Owner

kquick commented Jan 16, 2019

Thanks for the PR! The Thespian Watch Supported change above may help with some of the failures, and if there is an overall UDP issue that may be causing a number of the other failures.

That's also good to know about port 1900; I'll change that default in an upcoming release.

To help diagnose the app.py problem, there is some internal logging that Thespian performs (i.e. it does not use the logging library). This is driven from the thespian/system/utilis.py file. By default it will log to $TMPDIR/thespian.log. You might want to adjust the logging threshold (either by setting the THESPLOG_THRESHOLD environment variable or editing the default value on line 24 of the utilis.py file. The default value is WARNING and setting it to INFO or DEBUG may give more information.

@TTimo
Copy link
Author

TTimo commented Jan 20, 2019

I applied the change to Thespian Watch Supported, although there are no obvious improvements from that.

I've reproduced the multiprocUDPBase problem with the logging enabled (debug level). I can't quite parse them for a problem though, so here goes:

ttimo_000@VANGUARD /c/d/R/T/e/m/act1> env TMPDIR=D:/tmp THESPLOG_THRESHOLD=DEBUG /cygdrive/c/Python27/python.exe start.py multiprocUDPBase
ttimo_000@VANGUARD /c/d/R/T/e/m/act1> echo "Hello, World!" | env TMPDIR=D:/tmp THESPLOG_THRESHOLD=DEBUG /cygdrive/c/Python27/python.exe start.py multiprocUDPBase
Traceback (most recent call last):
  File "start.py", line 30, in <module>
    logDefs=logcfg
  File "d:\roomored\thespian\thespian\actors.py", line 638, in __init__
    systemBase, capabilities, logDefs)
  File "d:\roomored\thespian\thespian\actors.py", line 676, in _startupActorSys
    systemBase = sbc(self, logDefs=logDefs)
  File "d:\roomored\thespian\thespian\system\multiprocUDPBase.py", line 26, in __init__
    super(ActorSystemBase, self).__init__(system, logDefs)
  File "d:\roomored\thespian\thespian\system\multiprocCommon.py", line 85, in __init__
    super(multiprocessCommon, self).__init__(system, logDefs)
  File "d:\roomored\thespian\thespian\system\systemBase.py", line 329, in __init__
    logDefs)
  File "d:\roomored\thespian\thespian\system\multiprocCommon.py", line 114, in _startAdmin
    'not a valid ActorSystem admin')
thespian.actors.InvalidActorAddress: ActorAddr-(UDP|:12345) is not a valid ActorSystem admin

And the log:

2019-01-19 21:24:02.182000 p104660 dbg  ++++ Starting Admin from d:\roomored\thespian\thespian\__init__.pyc
2019-01-19 21:24:02.186000 p104660 I    ++++ Admin started @ ActorAddr-(UDP|:12345) / gen (3, 9)
2019-01-19 21:24:03.165000 p104660 dbg  Admin of ReceiveEnvelope(from: ActorAddr-(UDP|:60402), <class 'thespian.system.messages.multiproc.LoggerConnected'> msg: <thespian.system.messages.multiproc.LoggerConnected object at 0x03141490>)
2019-01-19 21:24:03.169000 p104660 dbg  actualTransmit of TransportIntent(ActorAddr-(UDP|:56255)-pending-ExpiresIn_0:05:00-<class 'thespian.system.messages.multiproc.EndpointConnected'>-<thespian.system.messages.multiproc.EndpointConnected object at 0x02AAAD90>-quit_0:05:00)
2019-01-19 21:24:03.175000 p105984 dbg  actualTransmit of TransportIntent(ActorAddr-(UDP|:12345)-pending-ExpiresIn_0:05:00-<class 'thespian.system.messages.admin.QueryExists'>-<thespian.system.messages.admin.QueryExists object at 0x0289D870>-quit_0:05:00)
2019-01-19 21:24:03.178000 p104660 dbg  Admin of ReceiveEnvelope(from: ActorAddr-(UDP|:56255), <class 'thespian.system.messages.admin.QueryExists'> msg: <thespian.system.messages.admin.QueryExists object at 0x03141650>)
2019-01-19 21:24:03.181000 p104660 dbg  Attempting intent TransportIntent(ActorAddr-(UDP|:56255)-pending-ExpiresIn_0:05:00-<class 'thespian.system.messages.admin.QueryAck'>-<thespian.system.messages.admin.QueryAck object at 0x03141490>-quit_0:05:00)
2019-01-19 21:24:03.184000 p104660 dbg  actualTransmit of TransportIntent(ActorAddr-(UDP|:56255)-pending-ExpiresIn_0:04:59.997000-<class 'thespian.system.messages.admin.QueryAck'>-<thespian.system.messages.admin.QueryAck object at 0x03141490>-quit_0:04:59.997000)
2019-01-19 21:25:17.438000 p74744 dbg  ++++ Starting Admin from d:\roomored\thespian\thespian\__init__.pyc
2019-01-19 21:25:17.441000 p74744 I    ++++ Admin started @ ActorAddr-(UDP|:12345) / gen (3, 9)
2019-01-19 21:25:18.420000 p104660 dbg  Admin of ReceiveEnvelope(from: ActorAddr-(UDP|:56461), <class 'thespian.system.messages.multiproc.LoggerConnected'> msg: <thespian.system.messages.multiproc.LoggerConnected object at 0x03141690>)
2019-01-19 21:25:18.425000 p104660 dbg  actualTransmit of TransportIntent(ActorAddr-(UDP|:56255)-pending-ExpiresIn_0:05:00-<class 'thespian.system.messages.multiproc.EndpointConnected'>-<thespian.system.messages.multiproc.EndpointConnected object at 0x03141590>-quit_0:05:00)

@kquick
Copy link
Owner

kquick commented Jan 21, 2019

Hi @TTimo ,

Please try editing thespian/system/transport/UDPTransport.py line 166 to change the value from a 1 to a 0 so that the line reads:

    ss.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 0)

I did some local testing and think this will fix the UDP issue, but I'd like to get confirmation from your setup.

Thanks,
Kevin

@TTimo
Copy link
Author

TTimo commented Jan 21, 2019

Hey @kquick - that seems to have fixed it. I can start/stop UDP multiproc now.

@kquick
Copy link
Owner

kquick commented Jan 21, 2019

Excellent!! Thanks again for your patience and help with this. I'll get these changes committed and your PR merged today or tomorrow and put out a bugfix release.

@TTimo
Copy link
Author

TTimo commented Jan 21, 2019

That sounds good - although I guess it depends if you want to put more time into the Windows support. I am running pytest right now and it's still showing a lot of failures. And this is only with Python 2.7, early on it looked like 3.x was going to have it's own set of issues.

@kquick
Copy link
Owner

kquick commented Jan 21, 2019

I'm happy to hold off on a release until things are looking better on your end.

Also as a note, these tests are pretty heavy-weight because they are primarily functional tests (as opposed to unit tests where lots of actual functionality is mocked out). These tests create lots of processes and sockets and try to ensure that the right things happen in degenerate corner cases. As a result, they may be affected by resource constraints on your system and you may need to clean up a few processes that escaped the test cleanup. In particular, the tests usually pick a port number for each test, but if that port is already in use for something else that will trigger a collision and a test failure. I've done my best to make the tests complete, reliable, tolerant of system resource issues, hygenic, and fast (in that order) with the perspective that it's better to have Thespian handle as many of the possible scenarios as possible (instead of leaving them as pitfalls for the user to deal with) at the expense of testing being a bit of a pain.

That said, the tests should still pass (modulo system resource issues) and I'm definitely interesting in resolving any failures you are seeing.

@kquick
Copy link
Owner

kquick commented Mar 6, 2019

@TTimo , I just released Thespian 3.9.8 which contains the socketopt change above. If you still have time to work on this, I'm happy to continue investigating the issues you've been seeing.

@TTimo
Copy link
Author

TTimo commented Mar 6, 2019

Sounds good! I'll do a pass with latest.

@TTimo
Copy link
Author

TTimo commented Mar 6, 2019

ran env TMPDIR=D:/tmp winpty /cygdrive/c/Python27/Scripts/pytest.exe

 platform win32 -- Python 2.7.13, pytest-3.6.0, py-1.5.3, pluggy-0.6.0
 [..]
 205 failed, 2131 passed, 236 skipped, 3 xfailed, 1 error in 13435.93 seconds =

I don't know if the logs out of TMP are very useful, pytest is spitting a lot of things to stdout/stderr otherwise but I didn't capture it.

https://www.dropbox.com/s/ucy0fkkaj59dthr/thespian-3.9.8-python27-logs.zip?dl=0

@TTimo
Copy link
Author

TTimo commented Mar 7, 2019

I did a second run and captured the whole output:

https://www.dropbox.com/s/fu6b5oekj6banq3/thespian-3.9.8-python27-logs-2.zip?dl=0

@kquick
Copy link
Owner

kquick commented Mar 7, 2019

Thanks @TTimo. That's a lot of good info. I'm setting up some local Windows access to research those issues you are still seeing, because it's definitely not what I was expecting. I'll keep you updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants