Skip to content

PHPUnit 11 #4057

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Jun 13, 2025
Merged

PHPUnit 11 #4057

merged 11 commits into from
Jun 13, 2025

Conversation

ondrejmirtes
Copy link
Member

/cc @staabm PHPUnit crashes with:

PHP Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 146082352 bytes) in /Users/ondrej/Development/phpstan/vendor/sebastian/exporter/src/Exporter.php on line 440
Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 146082352 bytes) in /Users/ondrej/Development/phpstan/vendor/sebastian/exporter/src/Exporter.php on line 440

Maybe there's an opportunity for optimization. Could you look into it and maybe contribute a fix to PHPUnit? Thanks!

@staabm
Copy link
Contributor

staabm commented Jun 10, 2025

I might have some time later this week to look into it

@ondrejmirtes
Copy link
Member Author

These kinds of changes work: 7389a6d

So PHPUnit is once again not handling passing huge objects from data providers to test methods. Scope contains class reflection and that will be hard to serialize.

@ondrejmirtes
Copy link
Member Author

If you're going to try to improve PHPUnit, do it on one commit before that one :)

@ondrejmirtes
Copy link
Member Author

Even if I update/remove tests that cause memory exhaustion, the whole test suite is still several times slower than before 🙁 This really needs to be solved in PHPUnit.

@staabm
Copy link
Contributor

staabm commented Jun 12, 2025

is this the same problem we discussed once in phpstan/phpstan#9914 ?

Blackfire Profile of PHPUnit 11 with blackfire run vendor/bin/phpunit tests/PHPStan/Type/ObjectTypeTest.php
https://blackfire.io/profiles/38ce70d3-f409-4081-b846-8b8bd017ff14/graph

@staabm
Copy link
Contributor

staabm commented Jun 12, 2025

btw: did you try whether phpunit 12 has the same problem?

@staabm
Copy link
Contributor

staabm commented Jun 12, 2025

Changing vendor/phpunit/phpunit/src/Event/Value/Test/TestMethodBuilder.php

            $testData[] = DataFromDataProvider::from(
                $dataSetName,
-               Exporter::export($testCase->providedData()),
+               Exporter::shortenedRecursiveExport($testCase->providedData()),
                $testCase->dataSetAsStringWithData(),
            );

turns vendor/bin/phpunit tests/PHPStan/Type/ObjectTypeTest.php from

➜  phpstan-src git:(phpunit-slow) ✗ vendor/bin/phpunit tests/PHPStan/Type/ObjectTypeTest.php
PHPUnit 11.5.22 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.22
Configuration: /Users/m.staab/Documents/GitHub/phpstan-src/phpunit.xml
Random Seed:   1749737382

...............................................................  63 / 102 ( 61%)
.......................................                         102 / 102 (100%)

Time: 00:23.674, Memory: 675.19 MB

OK (102 tests, 114 assertions)

into

➜  phpstan-src git:(phpunit-slow) vendor/bin/phpunit tests/PHPStan/Type/ObjectTypeTest.php
PHPUnit 11.5.22 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.22
Configuration: /Users/m.staab/Documents/GitHub/phpstan-src/phpunit.xml
Random Seed:   1749737437

...............................................................  63 / 102 ( 61%)
.......................................                         102 / 102 (100%)

Time: 00:00.256, Memory: 88.00 MB

OK (102 tests, 114 assertions)

.. still looking into it..

@staabm
Copy link
Contributor

staabm commented Jun 12, 2025

with the above patch the test-suite is working for me again:

ddde40e with PHPUnit 11.x

$ make tests

XDEBUG_MODE=off php tests/vendor/bin/paratest --runner WrapperRunner --no-coverage
ParaTest v7.8.3 upon PHPUnit 11.5.22 by Sebastian Bergmann and contributors.

Processes:     14
Runtime:       PHP 8.3.22
Configuration: /Users/m.staab/Documents/GitHub/phpstan-src/phpunit.xml
Random Seed:   1749738159

......................SS...................................    59 / 12839 (  0%)
...........................................................   118 / 12839 (  0%)
...........................................................   177 / 12839 (  1%)
....................S..S...................................   236 / 12839 (  1%)
.................................S.........................   295 / 12839 (  2%)
..........S.......S..................SS....................   354 / 12839 (  2%)
.............S.............................S...............   413 / 12839 (  3%)
...............................................SS..........   472 / 12839 (  3%)
...........................................................   531 / 12839 (  4%)
...........................................................   590 / 12839 (  4%)
...S.......................................................   649 / 12839 (  5%)
.........................SS................................   708 / 12839 (  5%)
...........................SS..............................   767 / 12839 (  5%)
.............................S.................S...........   826 / 12839 (  6%)
......................SSS..............SSSSSSSSSSSSSSSSS...   885 / 12839 (  6%)
...................S.............I.........................   944 / 12839 (  7%)
...........................................................  1003 / 12839 (  7%)
...........................................................  1062 / 12839 (  8%)
.......................................SS..S...............  1121 / 12839 (  8%)
................................................SSSSSSSSS..  1180 / 12839 (  9%)
...........................................................  1239 / 12839 (  9%)
.......................S...................................  1298 / 12839 ( 10%)
...........................................................  1357 / 12839 ( 10%)
...........................................................  1416 / 12839 ( 11%)
..........................................................S  1475 / 12839 ( 11%)
........S..................................................  1534 / 12839 ( 11%)
...........................................................  1593 / 12839 ( 12%)
...........................................................  1652 / 12839 ( 12%)
..........................................................S  1711 / 12839 ( 13%)
...........................................................  1770 / 12839 ( 13%)
...........................................................  1829 / 12839 ( 14%)
...........................................................  1888 / 12839 ( 14%)
.....R......S..............................................  1947 / 12839 ( 15%)
..........................S................................  2006 / 12839 ( 15%)
...........................................................  2065 / 12839 ( 16%)
....S..............SSSS....................................  2124 / 12839 ( 16%)
...........................................................  2183 / 12839 ( 17%)
..................................SS.......................  2242 / 12839 ( 17%)
...........................................................  2301 / 12839 ( 17%)
...........................................................  2360 / 12839 ( 18%)
...........................................................  2419 / 12839 ( 18%)
...........................................................  2478 / 12839 ( 19%)
...........................................................  2537 / 12839 ( 19%)
...........................................................  2596 / 12839 ( 20%)
...........................................................  2655 / 12839 ( 20%)
...........................................................  2714 / 12839 ( 21%)
...........................................................  2773 / 12839 ( 21%)
....................................................S......  2832 / 12839 ( 22%)
...........................................................  2891 / 12839 ( 22%)
...............................................S...........  2950 / 12839 ( 22%)
...........................................................  3009 / 12839 ( 23%)
................................S.S........................  3068 / 12839 ( 23%)
...........................................................  3127 / 12839 ( 24%)
...........................................................  3186 / 12839 ( 24%)
...........................................................  3245 / 12839 ( 25%)
...............................................S...........  3304 / 12839 ( 25%)
..............S............................................  3363 / 12839 ( 26%)
...........................................................  3422 / 12839 ( 26%)
...........................................................  3481 / 12839 ( 27%)
...........................................................  3540 / 12839 ( 27%)
...........................................................  3599 / 12839 ( 28%)
...........................................................  3658 / 12839 ( 28%)
...........................................................  3717 / 12839 ( 28%)
...........................................................  3776 / 12839 ( 29%)
...........................................................  3835 / 12839 ( 29%)
...........................................................  3894 / 12839 ( 30%)
...........................................................  3953 / 12839 ( 30%)
...........................................................  4012 / 12839 ( 31%)
...........................................................  4071 / 12839 ( 31%)
...........................................................  4130 / 12839 ( 32%)
...........................................................  4189 / 12839 ( 32%)
...........................................................  4248 / 12839 ( 33%)
...........................................................  4307 / 12839 ( 33%)
...........................................................  4366 / 12839 ( 34%)
...........................................................  4425 / 12839 ( 34%)
...........................................................  4484 / 12839 ( 34%)
...........................................................  4543 / 12839 ( 35%)
...........................................................  4602 / 12839 ( 35%)
...........................................................  4661 / 12839 ( 36%)
...........................................................  4720 / 12839 ( 36%)
...........................................................  4779 / 12839 ( 37%)
...........................................................  4838 / 12839 ( 37%)
...........................................................  4897 / 12839 ( 38%)
...........................................................  4956 / 12839 ( 38%)
...........................................................  5015 / 12839 ( 39%)
...........................................................  5074 / 12839 ( 39%)
...S.......................................................  5133 / 12839 ( 39%)
...........................................................  5192 / 12839 ( 40%)
...........................................................  5251 / 12839 ( 40%)
...........................................................  5310 / 12839 ( 41%)
...........................................................  5369 / 12839 ( 41%)
...........................................................  5428 / 12839 ( 42%)
...........................................................  5487 / 12839 ( 42%)
...........................................................  5546 / 12839 ( 43%)
...........................................................  5605 / 12839 ( 43%)
...........................................................  5664 / 12839 ( 44%)
...........................................................  5723 / 12839 ( 44%)
...........................................................  5782 / 12839 ( 45%)
...........................................................  5841 / 12839 ( 45%)
...........................................................  5900 / 12839 ( 45%)
.........................................................S.  5959 / 12839 ( 46%)
.......................................SS..................  6018 / 12839 ( 46%)
...........................................................  6077 / 12839 ( 47%)
...........................................................  6136 / 12839 ( 47%)
..............................................SS...........  6195 / 12839 ( 48%)
...........................................................  6254 / 12839 ( 48%)
........S..................................................  6313 / 12839 ( 49%)
.................................S...S.......S.............  6372 / 12839 ( 49%)
...........................................................  6431 / 12839 ( 50%)
...........................................................  6490 / 12839 ( 50%)
......................................S..............SSSSSS  6549 / 12839 ( 51%)
SSSSSSSS.S..............................................S..  6608 / 12839 ( 51%)
...........................S...............................  6667 / 12839 ( 51%)
S.................................S........................  6726 / 12839 ( 52%)
.........................S.................................  6785 / 12839 ( 52%)
.....................................S.....................  6844 / 12839 ( 53%)
.............................S.....S.......................  6903 / 12839 ( 53%)
.........................................................SS  6962 / 12839 ( 54%)
...........................................................  7021 / 12839 ( 54%)
...........................................................  7080 / 12839 ( 55%)
S..........................................................  7139 / 12839 ( 55%)
.................SS........................................  7198 / 12839 ( 56%)
...........................................SSS.............  7257 / 12839 ( 56%)
.......................................SSSSSSSSS..........S  7316 / 12839 ( 56%)
SSSSSSS....................................................  7375 / 12839 ( 57%)
......................S....................................  7434 / 12839 ( 57%)
...........................................................  7493 / 12839 ( 58%)
...........................................................  7552 / 12839 ( 58%)
...........................................................  7611 / 12839 ( 59%)
...........................................................  7670 / 12839 ( 59%)
...........................................................  7729 / 12839 ( 60%)
...........................................................  7788 / 12839 ( 60%)
...........................................................  7847 / 12839 ( 61%)
...........................................................  7906 / 12839 ( 61%)
...........................................................  7965 / 12839 ( 62%)
...........................................................  8024 / 12839 ( 62%)
...........................................................  8083 / 12839 ( 62%)
...........................................................  8142 / 12839 ( 63%)
...........................................................  8201 / 12839 ( 63%)
...........................................................  8260 / 12839 ( 64%)
...........................................................  8319 / 12839 ( 64%)
...........................................................  8378 / 12839 ( 65%)
...........................................................  8437 / 12839 ( 65%)
...........................................................  8496 / 12839 ( 66%)
...........................................................  8555 / 12839 ( 66%)
...........................................................  8614 / 12839 ( 67%)
...........................................................  8673 / 12839 ( 67%)
...........................................................  8732 / 12839 ( 68%)
...........................................................  8791 / 12839 ( 68%)
...........................................................  8850 / 12839 ( 68%)
...........................................................  8909 / 12839 ( 69%)
...........................................................  8968 / 12839 ( 69%)
...........................................................  9027 / 12839 ( 70%)
...........................................................  9086 / 12839 ( 70%)
...........................................................  9145 / 12839 ( 71%)
...........................................................  9204 / 12839 ( 71%)
...........................................................  9263 / 12839 ( 72%)
...........................................................  9322 / 12839 ( 72%)
...........................................................  9381 / 12839 ( 73%)
...........................................................  9440 / 12839 ( 73%)
...........................................................  9499 / 12839 ( 73%)
...........................................................  9558 / 12839 ( 74%)
...........................................................  9617 / 12839 ( 74%)
...........................................................  9676 / 12839 ( 75%)
...........................................................  9735 / 12839 ( 75%)
...........................................................  9794 / 12839 ( 76%)
...........................................................  9853 / 12839 ( 76%)
...........................................................  9912 / 12839 ( 77%)
...........................................................  9971 / 12839 ( 77%)
........................................................... 10030 / 12839 ( 78%)
........................................................... 10089 / 12839 ( 78%)
........................................................... 10148 / 12839 ( 79%)
........................................................... 10207 / 12839 ( 79%)
........................................................... 10266 / 12839 ( 79%)
........................................................... 10325 / 12839 ( 80%)
........................................................... 10384 / 12839 ( 80%)
........................................................... 10443 / 12839 ( 81%)
........................................................... 10502 / 12839 ( 81%)
........................................................... 10561 / 12839 ( 82%)
........................................................... 10620 / 12839 ( 82%)
........................................................... 10679 / 12839 ( 83%)
........................................................... 10738 / 12839 ( 83%)
........................................................... 10797 / 12839 ( 84%)
........................................................... 10856 / 12839 ( 84%)
........................................................... 10915 / 12839 ( 85%)
........................................................... 10974 / 12839 ( 85%)
........................................................... 11033 / 12839 ( 85%)
........................................................... 11092 / 12839 ( 86%)
........................................................... 11151 / 12839 ( 86%)
........................................................... 11210 / 12839 ( 87%)
........................................................... 11269 / 12839 ( 87%)
........................................................... 11328 / 12839 ( 88%)
........................................................... 11387 / 12839 ( 88%)
........................................................... 11446 / 12839 ( 89%)
........................................................... 11505 / 12839 ( 89%)
........................................................... 11564 / 12839 ( 90%)
........................................................... 11623 / 12839 ( 90%)
........................................................... 11682 / 12839 ( 90%)
........................................................... 11741 / 12839 ( 91%)
........................................................... 11800 / 12839 ( 91%)
........................................................... 11859 / 12839 ( 92%)
........................................................... 11918 / 12839 ( 92%)
........................................................... 11977 / 12839 ( 93%)
........................................................... 12036 / 12839 ( 93%)
........................................................... 12095 / 12839 ( 94%)
........................................................... 12154 / 12839 ( 94%)
........................................................... 12213 / 12839 ( 95%)
........................................................... 12272 / 12839 ( 95%)
........................................................... 12331 / 12839 ( 96%)
........................................................... 12390 / 12839 ( 96%)
........................................................... 12449 / 12839 ( 96%)
........................................................... 12508 / 12839 ( 97%)
........................................................... 12567 / 12839 ( 97%)
........................................................... 12626 / 12839 ( 98%)
...........................................R............... 12685 / 12839 ( 98%)
........................................................... 12744 / 12839 ( 99%)
........................................................... 12803 / 12839 ( 99%)
....................................                        12839 / 12839 (100%)

Time: 00:44.635, Memory: 538.50 MB

There were 2 risky tests:

1) PHPStan\Analyser\AnalyserIntegrationTest::testCollectWarnings
Test code or tested code removed error handlers other than its own

/Users/m.staab/Documents/GitHub/phpstan-src/tests/PHPStan/Analyser/AnalyserIntegrationTest.php:185

2) PHPStan\Command\AnalyseApplicationIntegrationTest::testExecuteOnANonExistentPath
Test code or tested code removed error handlers other than its own

/Users/m.staab/Documents/GitHub/phpstan-src/tests/PHPStan/Command/AnalyseApplicationIntegrationTest.php:34

OK, but there were issues!
Tests: 12839, Assertions: 84526, Skipped: 132, Incomplete: 1, Risky: 2.
make: *** [tests] Error 1

vs

5b4bf80 with PHPUnit 10.x

make tests

XDEBUG_MODE=off php tests/vendor/bin/paratest --runner WrapperRunner --no-coverage
ParaTest v7.3.1 upon PHPUnit 10.5.31 by Sebastian Bergmann and contributors.

Processes:     14
Runtime:       PHP 8.3.22
Configuration: /Users/m.staab/Documents/GitHub/phpstan-src/phpunit.xml
Random Seed:   1749738298

............................................SSS............    59 / 12845 (  0%)
..............................................SS...........   118 / 12845 (  0%)
.....................SSS...................................   177 / 12845 (  1%)
..S........................................................   236 / 12845 (  1%)
.......................................................SSSS   295 / 12845 (  2%)
SSSSS..........SSSSSSSS....................................   354 / 12845 (  2%)
.......S...................................................   413 / 12845 (  3%)
...........................................................   472 / 12845 (  3%)
.........................................S.................   531 / 12845 (  4%)
....................................................SS.....   590 / 12845 (  4%)
...........................................................   649 / 12845 (  5%)
....................S......................................   708 / 12845 (  5%)
...................S.......................................   767 / 12845 (  5%)
..S..............................S.........................   826 / 12845 (  6%)
...........................................................   885 / 12845 (  6%)
..........................S...........S...S................   944 / 12845 (  7%)
...........................................................  1003 / 12845 (  7%)
...........................................................  1062 / 12845 (  8%)
..........I................................................  1121 / 12845 (  8%)
...........................................................  1180 / 12845 (  9%)
.......S...................................................  1239 / 12845 (  9%)
...........................................................  1298 / 12845 ( 10%)
......................................................SS...  1357 / 12845 ( 10%)
...........................................................  1416 / 12845 ( 11%)
...........................................................  1475 / 12845 ( 11%)
...........................................................  1534 / 12845 ( 11%)
...........................................................  1593 / 12845 ( 12%)
.........................................................S.  1652 / 12845 ( 12%)
...........................................................  1711 / 12845 ( 13%)
...........................................................  1770 / 12845 ( 13%)
...........................................................  1829 / 12845 ( 14%)
...........................................................  1888 / 12845 ( 14%)
...........................................................  1947 / 12845 ( 15%)
...........................................................  2006 / 12845 ( 15%)
...........................................................  2065 / 12845 ( 16%)
...........................................................  2124 / 12845 ( 16%)
...........................................................  2183 / 12845 ( 16%)
...........................................................  2242 / 12845 ( 17%)
...........................................................  2301 / 12845 ( 17%)
...........................................................  2360 / 12845 ( 18%)
...........................................................  2419 / 12845 ( 18%)
...........................................................  2478 / 12845 ( 19%)
...........................................................  2537 / 12845 ( 19%)
...........................................................  2596 / 12845 ( 20%)
...........................................................  2655 / 12845 ( 20%)
...........................................................  2714 / 12845 ( 21%)
...........................................................  2773 / 12845 ( 21%)
...........................................................  2832 / 12845 ( 22%)
......................S................S...................  2891 / 12845 ( 22%)
........S...............................S..................  2950 / 12845 ( 22%)
...............................S...........................  3009 / 12845 ( 23%)
......S....................................................  3068 / 12845 ( 23%)
...........................................................  3127 / 12845 ( 24%)
...........................................................  3186 / 12845 ( 24%)
...........................................................  3245 / 12845 ( 25%)
...........................................................  3304 / 12845 ( 25%)
...........................................................  3363 / 12845 ( 26%)
.........................S.................................  3422 / 12845 ( 26%)
...........................................................  3481 / 12845 ( 27%)
.........................................................SS  3540 / 12845 ( 27%)
....SSSSSS.................................................  3599 / 12845 ( 28%)
.....................S...............S.....................  3658 / 12845 ( 28%)
..........S.........S......................................  3717 / 12845 ( 28%)
...........................................................  3776 / 12845 ( 29%)
...........................................................  3835 / 12845 ( 29%)
.....................................  101 / 2705 (  3%)  202 / 2705 (  7%)  303 / 2705 ( 11%)  404 / 2705 ( 14%)  505 / 2705 ( 18%)  606 / 2705 ( 22%)  707 / 2705 ( 26%)  808 / 2705 ( 29%)  909 / 2705 ( 33%)......................  3894 / 12845 ( 30%)
...........................................................  3953 / 12845 ( 30%)
...........................................................  4012 / 12845 ( 31%)
...........................................................  4071 / 12845 ( 31%)
...........................................................  4130 / 12845 ( 32%)
...........................................................  4189 / 12845 ( 32%)
...........................................................  4248 / 12845 ( 33%)
...........................................................  4307 / 12845 ( 33%)
...........................................................  4366 / 12845 ( 33%)
...........................................................  4425 / 12845 ( 34%)
...........................................................  4484 / 12845 ( 34%)
...........................................................  4543 / 12845 ( 35%)
...........................................................  4602 / 12845 ( 35%)
...........................................................  4661 / 12845 ( 36%)
...........................................................  4720 / 12845 ( 36%)
...........................................................  4779 / 12845 ( 37%)
...........................................................  4838 / 12845 ( 37%)
...........................................................  4897 / 12845 ( 38%)
...........................................................  4956 / 12845 ( 38%)
...........................................................  5015 / 12845 ( 39%)
...........................................................  5074 / 12845 ( 39%)
...........................................................  5133 / 12845 ( 39%)
...........................................................  5192 / 12845 ( 40%)
...........................................................  5251 / 12845 ( 40%)
...........................................................  5310 / 12845 ( 41%)
...........................................................  5369 / 12845 ( 41%)
...........................................................  5428 / 12845 ( 42%)
...........................................................  5487 / 12845 ( 42%)
...........................................................  5546 / 12845 ( 43%)
...........................................................  5605 / 12845 ( 43%)
...........................................................  5664 / 12845 ( 44%)
...........................................................  5723 / 12845 ( 44%)
...........................................................  5782 / 12845 ( 45%)
...........................................................  5841 / 12845 ( 45%)
...........................................................  5900 / 12845 ( 45%)
...........................................................  5959 / 12845 ( 46%)
...........................................................  6018 / 12845 ( 46%)
...........................................................  6077 / 12845 ( 47%)
...........................................................  6136 / 12845 ( 47%)
...........................................................  6195 / 12845 ( 48%)
...........................................................  6254 / 12845 ( 48%)
...........................................................  6313 / 12845 ( 49%)
...........................................................  6372 / 12845 ( 49%)
...........................................................  6431 / 12845 ( 50%)
...........................................................  6490 / 12845 ( 50%)
...........................................................  6549 / 12845 ( 50%)
...........................................................  6608 / 12845 ( 51%)
...........................................................  6667 / 12845 ( 51%)
...........................................................  6726 / 12845 ( 52%)
...........................................................  6785 / 12845 ( 52%)
...............................................S...........  6844 / 12845 ( 53%)
..................................S............SSSSSSSSSSSS  6903 / 12845 ( 53%)
SSSSS......................................................  6962 / 12845 ( 54%)
...........................................................  7021 / 12845 ( 54%)
...........................................................  7080 / 12845 ( 55%)
...........................................................  7139 / 12845 ( 55%)
...........................................................  7198 / 12845 ( 56%)
...........................................................  7257 / 12845 ( 56%)
...........................................................  7316 / 12845 ( 56%)
...SSS..........SS.........................................  7375 / 12845 ( 57%)
.................  101 / 2565 (  3%)  202 / 2565 (  7%)  303 / 2565 ( 11%)  404 / 2565 ( 15%)  505 / 2565 ( 19%)  606 / 2565 ( 23%)  707 / 2565 ( 27%)  808 / 2565 ( 31%)  909 / 2565 ( 35%)..........................................  7434 / 12845 ( 57%)
...........................................................  7493 / 12845 ( 58%)
...........................................................  7552 / 12845 ( 58%)
...........................................................  7611 / 12845 ( 59%)
...........................................................  7670 / 12845 ( 59%)
...........................................................  7729 / 12845 ( 60%)
...........................................................  7788 / 12845 ( 60%)
...........................................................  7847 / 12845 ( 61%)
...........................................................  7906 / 12845 ( 61%)
...........................................................  7965 / 12845 ( 62%)
...........................................................  8024 / 12845 ( 62%)
...........................................................  8083 / 12845 ( 62%)
...........................................................  8142 / 12845 ( 63%)
...........................................................  8201 / 12845 ( 63%)
...........................................................  8260 / 12845 ( 64%)
...........................................................  8319 / 12845 ( 64%)
...........................................................  8378 / 12845 ( 65%)
...........................................................  8437 / 12845 ( 65%)
...........................................................  8496 / 12845 ( 66%)
...........................................................  8555 / 12845 ( 66%)
...........................................................  8614 / 12845 ( 67%)
...........................................................  8673 / 12845 ( 67%)
...........................................................  8732 / 12845 ( 67%)
...........................................................  8791 / 12845 ( 68%)
...........................................................  8850 / 12845 ( 68%)
...........................................................  8909 / 12845 ( 69%)
...........................................................  8968 / 12845 ( 69%)
...........................................................  9027 / 12845 ( 70%)
...........................................................  9086 / 12845 ( 70%)
.......................................................S...  9145 / 12845 ( 71%)
...........................................................  9204 / 12845 ( 71%)
...........................................................  9263 / 12845 ( 72%)
...........................................................  9322 / 12845 ( 72%)
...........................................................  9381 / 12845 ( 73%)
...........................................................  9440 / 12845 ( 73%)
...........................................................  9499 / 12845 ( 73%)
...........................................................  9558 / 12845 ( 74%)
...........................................................  9617 / 12845 ( 74%)
...........................................................  9676 / 12845 ( 75%)
...........................................................  9735 / 12845 ( 75%)
...........................................................  9794 / 12845 ( 76%)
...........................................................  9853 / 12845 ( 76%)
...........................................................  9912 / 12845 ( 77%)
...........................................................  9971 / 12845 ( 77%)
........................................................... 10030 / 12845 ( 78%)
.....................................SS.................... 10089 / 12845 ( 78%)
........................................................... 10148 / 12845 ( 79%)
........................................................... 10207 / 12845 ( 79%)
........................................................... 10266 / 12845 ( 79%)
..SSSSSSS.................................................. 10325 / 12845 ( 80%)
................SS........................................S 10384 / 12845 ( 80%)
..........................................................S 10443 / 12845 ( 81%)
SSSSSS.....SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS............... 10502 / 12845 ( 81%)
...........S............................................... 10561 / 12845 ( 82%)
..S........................................................ 10620 / 12845 ( 82%)
..............S............................................ 10679 / 12845 ( 83%)
........................................................... 10738 / 12845 ( 83%)
.................................S......................... 10797 / 12845 ( 84%)
...............................................S.S......... 10856 / 12845 ( 84%)
........................................................... 10915 / 12845 ( 84%)
...................................S....................... 10974 / 12845 ( 85%)
........................................................... 11033 / 12845 ( 85%)
........................................................... 11092 / 12845 ( 86%)
......................................S......S............. 11151 / 12845 ( 86%)
........................SSSSSSSSSSSSSS.S......S............ 11210 / 12845 ( 87%)
............S......................SSSS.........SS....S.... 11269 / 12845 ( 87%)
........................................................... 11328 / 12845 ( 88%)
........................................................... 11387 / 12845 ( 88%)
..........................SS............................... 11446 / 12845 ( 89%)
........................................................... 11505 / 12845 ( 89%)
.................................  101 / 1118 (  9%)  202 / 1118 ( 18%)  303 / 1118 ( 27%)  404 / 1118 ( 36%)  505 / 1118 ( 45%)  606 / 1118 ( 54%)  707 / 1118 ( 63%)  808 / 1118 ( 72%)  909 / 1118 ( 81%).......................... 11564 / 12845 ( 90%)
........................................................... 11623 / 12845 ( 90%)
........................................................... 11682 / 12845 ( 90%)
........................................................... 11741 / 12845 ( 91%)
........................................................... 11800 / 12845 ( 91%)
........................................................... 11859 / 12845 ( 92%)
........................................................... 11918 / 12845 ( 92%)
........................................................... 11977 / 12845 ( 93%)
........................................................... 12036 / 12845 ( 93%)
........................................................... 12095 / 12845 ( 94%)
........................................................... 12154 / 12845 ( 94%)
........................................................... 12213 / 12845 ( 95%)
........................................................... 12272 / 12845 ( 95%)
........................................................... 12331 / 12845 ( 95%)
........................................................... 12390 / 12845 ( 96%)
........................................................... 12449 / 12845 ( 96%)
........................................................... 12508 / 12845 ( 97%)
........................................................... 12567 / 12845 ( 97%)
........................................................... 12626 / 12845 ( 98%)
........................................................... 12685 / 12845 ( 98%)
........................................................... 12744 / 12845 ( 99%)
........................................................... 12803 / 12845 ( 99%)
..........................................                  12845 / 12845 (100%)

Time: 00:45.158, Memory: 534.50 MB

OK, but there were issues!
Tests: 12845, Assertions: 84492, Skipped: 172, Incomplete: 1.

wdyt?

@staabm
Copy link
Contributor

staabm commented Jun 13, 2025

Fix landed in sebastianbergmann/phpunit@7ff0969

For PHPUnit 11.5.x and newer

released https://github.com/sebastianbergmann/phpunit/releases/tag/11.5.23


tested locally with 11.5.23 and can confirm the fix

@ondrejmirtes
Copy link
Member Author

Thank you!

@ondrejmirtes
Copy link
Member Author

Maybe this test matrix generation needs updating? https://github.com/phpstan/phpstan-src/actions/runs/15630379293/job/44033085940?pr=4057

@staabm
Copy link
Contributor

staabm commented Jun 13, 2025

Can have a look later today

@staabm
Copy link
Contributor

staabm commented Jun 13, 2025

the xml format generated by vendor/bin/phpunit --list-tests-xml changed with sebastianbergmann/phpunit@09f8ddf#diff-001d0bba0217b0f5f5ead3d7620704c4a23e35f2b86b579ebb90f3a76004edfe

I am looking into adjusting the script


drive by observation: running blackfire run php vendor/bin/phpunit --list-tests-xml test-list.xml

has a bottleneck on AutowiredAttributeServicesExtension::processParameters

grafik

https://blackfire.io/profiles/2f4cb6e1-e2a5-4455-881f-e2eeff149dbe/graph

not sure this is expected

@staabm
Copy link
Contributor

staabm commented Jun 13, 2025

please try this fix: staabm@d024500

@ondrejmirtes
Copy link
Member Author

Trying it now. BTW you can submit PRs against (based on) other branches too, which would cause it being merged into this PR 😊

@ondrejmirtes ondrejmirtes marked this pull request as ready for review June 13, 2025 20:07
@phpstan-bot
Copy link
Collaborator

This pull request has been marked as ready for review.

@ondrejmirtes
Copy link
Member Author

On my computer:

PHPUnit 11: Time: 00:52.399, Memory: 432.00 MB
PHPUnit 10: Time: 00:51.174, Memory: 462.00 MB

Awesome! 👏

@ondrejmirtes ondrejmirtes merged commit a421da1 into 2.1.x Jun 13, 2025
418 of 421 checks passed
@ondrejmirtes ondrejmirtes deleted the phpunit-11 branch June 13, 2025 20:09
@ondrejmirtes
Copy link
Member Author

And as expected, we're not the only ones benefiting from that: https://x.com/spudley99/status/1933485562025128390

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants